Explain the execution flow of Sensitive Data Scanner across logs and APM?

For logs, scanning happens as a stage within the log processing pipeline: after earlier processors (like the Grok Parser) have extracted structured fields, configured scanning rules are evaluated against the log's content and matching attributes, and any configured action - redact, hash, or flag - is applied before the log proceeds to indexing.

Because this happens before indexing, a redaction decision is final in the sense that the original sensitive value never becomes part of the searchable, stored record - there's no separate 'clean up afterward' step needed, which is what makes it suitable for compliance-sensitive data.

For APM, a comparable scanning capability can be applied to span tags and resource data, evaluated as spans are processed, so sensitive values that might otherwise leak into trace metadata (like a request parameter accidentally tagged onto a span) get the same treatment before that span data is stored.

Rule scope matters operationally: rules can be applied broadly across an entire org or narrowed to specific log sources, services, or namespaces, letting teams apply strict, low-false-negative rules to known higher-risk sources (like payment-related logs) while avoiding unnecessary processing overhead or false positives on unrelated telemetry.

flowchart LR
  A[Log arrives at pipeline] --> B[Grok Parser extracts fields]
  B --> C[Sensitive Data Scanner evaluates rules]
  C --> D{Match?}
  D -- Yes --> E[Apply action: redact/hash/flag]
  D -- No --> F[Pass through unchanged]
  E --> G[Proceed to indexing]
  F --> G