Using Keen

Build and preview evidence mappings

Chapter 4 of 17

Mapping made simple

This chapter teaches the one skill that makes Keen useful: turning a real event into a rule that links evidence to controls. Start here if the words action, outcome or mapping feel abstract. The sections that follow show the full editor, then how to keep rules healthy as evidence keeps arriving.

A mapping rule says: “When an event looks like this, link it to these controls.” Begin with an event you can already see in Keen. Its fields tell you which words go in the rule.

Action is the activity name already written on the event, and Outcome is the result already written there. Filling either field in a rule asks Keen to find events with that value.

1. Open Admin → Sources & evidence mapping → New mapping. Describe what the evidence demonstrates, such as “Production build checks completed successfully.”
2. Select the source and, if relevant, its specific job, query, feed or page. Choose a recent event.
3. Choose Copy its match values. Keen fills in the system, activity and result from that event. Remove any value you want to allow to vary.
4. Read the sentence below the fields. For this example it should say: Jenkins events with activity build and result success. Keep system prod only if that is the event's recorded system.
5. Select the controls or clauses this evidence supports. One event can link to several controls and frameworks.
6. Preview the matches. Look at both the matching events and the explanations for events that did not match. Publish when those examples reflect your intention.

Diagram source (Mermaid)

flowchart TB
  E["Jenkins records a build"] --> A{"Activity equals build?"}
  A -->|Yes| O{"Result equals success?"}
  A -->|No| K["Keep the event without this rule's mapping"]
  O -->|No| K
  O -->|Yes| M["Link evidence to the selected controls"]
  M --> R["Review evidence and assess the requirement"]
  K --> X["Other rules can still match"]

Open full-size diagram

Suggested values come from recent evidence, so they are examples of what that source has recently produced. With no recent evidence, collect a sample first. The advanced Predefine option is for sources whose values you already know.

A mapping records a relationship; the judgement is yours. When you assess the requirement, weigh relevance, time period, scope and quality. A failed check can itself be useful evidence of monitoring or incident handling, with its failed result kept visible.

Configure collection and mapping with evidence definitions

The full editor is in Admin → Sources & evidence mapping. An evidence definition combines three things: a source input, the events from it that count as evidence, and the controls or clauses they support. One input can feed several definitions, and one event can support several frameworks.

Choose Create evidence definition to open the four-step editor.

1. Describe the evidence. Say what the collected evidence demonstrates, such as successful production deployments following the change process.
2. Collect and recognise. Select the source adapter and input type, then an existing query, job, feed or page, or create a collection item. Inspect a Recent event with Use this event, then narrow the definition with event fields if needed. All events from this source suits a deliberately source-wide definition.
3. Select framework controls or clauses. The list starts on the framework selected in the navigation bar. Pick a framework and a target, and use Add for each further target, including targets in other frameworks. The selected target is also included when you choose Next; check the complete list.
4. Preview and save. Choose Preview matches and inspect both matches and non-matches. Set the confidence and enabled state under Advanced rule settings, then Save rule.

Prefer source, system, actor, action, outcome, severity or label values you can see on real events, and keep regex for patterns exact values cannot express. The description-based control suggestions are candidates for you to review. Confidence is your own mapping confidence between 0 and 1; compliance is assessed through evidence and review.

Collection settings and credentials

A Jenkins job's kind becomes the event action and its label the event system. For Loki, configure the query and its normalised event fields. For BookStack, select a book and then a page, grouped by chapter, using the picker's page-loading buttons when needed; the picker uses the server's configured credentials.

Additional collection settings takes connector-specific JSON options, and Shared adapter settings holds advanced options common to an adapter. Neither is a place for secrets: connection credentials and adapter enablement are server settings for the hosting operator.

Read mapping preview results

Preview compares the rule with recent events. Green Matches entries show the event and its proposed framework targets; amber Does not match entries explain why an event was excluded. Labels accompany the colours. Adjust the conditions and preview again before saving.

While a source type is paused in the source catalogue, its existing evidence stays available for review and mapping.

Definitions before events arrive

Select Predefine this rule before evidence is available when there is no sample to preview. As soon as collection runs, preview the definition against the first real events and correct any assumption about field values or collection identity.

Build and verify an evidence definition

Three words describe the pieces. An adapter connects to a service. A collection item is one input from it: a query, job, feed, project or page. An evidence definition says which events from that input support which controls. Reuse collection items when several definitions share an input.

The checklist below is the same editor, walked through with verification in mind.

1. Choose Create evidence definition and describe the claim: for example, “Successful production deployments recorded by the approved deployment job.”
2. In Collect and recognise, select the adapter and input type and choose or create the collection item. Use All events from this source only when every input from that adapter belongs.
3. Use Recent event → Use this event to read the actual normalised fields. Keep the exact values, including distinctions such as “deploy” and “deployment”.
4. Fill the relevant source, system, actor, action, outcome, severity or label conditions. Start with exact values.
5. Add each target framework, control or clause to the list and check the references.
6. Preview a positive sample and an event that should not qualify. Make sure another collection item, a failed outcome or an unrelated environment cannot match by accident.
7. Review the enabled state, the confidence and whether to apply the rule to existing evidence. Save, and watch any background job separately from the save itself.

Historical evidence and changing rules

Rules change, and evidence already collected should follow. Apply saved rule to past evidence runs a background job that reconciles the automatic mappings this rule owns: it adds matching controls and removes links to controls you removed from the rule, including links on events that no longer match.

What stays untouched: manual and imported mappings, links still supported by another active rule, and older mappings whose rule ownership is uncertain. When the job finishes, reopen an event to review its controls.

Apply rules to existing evidence

In the rule editor, Apply this rule to previously collected evidence in the background is selected by default; review it before saving. Saving the definition and finishing the background job are separate events, and the job reports its own status. You can also run Apply saved rule to past evidence on a saved rule at any time.

Admin → Remap is the broader tool, with date, limit, source, system, action and outcome filters. By default it processes events with no mappings; Include already-mapped events adds new rule-based mappings to events already linked elsewhere. Adding mappings is a different operation from removing stale ones.

Create a mapping from an event

The quickest way to a good rule starts from the event page. Everything the rule needs is already on the screen.

1. Open an event and choose Create mapping from this event.
2. Review the suggested conditions. Field names follow the event JSON, including structured keys such as fields → process.name, fields → host.name, fields → status and fields → log.file.path.
3. Remove conditions that are too specific. A host or IP condition restricts the rule to that host or address, and event identifiers and timestamps each identify a single record.
4. Give the mapping a meaningful name and choose the framework controls or clauses. The suggestion button uses the name to find relevant controls; refine the name and try again if the suggestions miss.
5. Review and save. The existing-mapping browser lets you search, edit or delete rules, and the historical-application action maps evidence already collected.

A mapping attributes matching evidence to framework requirements. Collection inputs decide which remote data Keen fetches; a rule on its own does not configure a remote collection.