How does deduplication work in Google SecOps search results?

Deduplication (via the optional dedup section in a statistical query or rule) reduces the volume of duplicate or near-duplicate results returned, letting an analyst see a representative sample or count of a repeated pattern rather than being shown every single individually matching event.

match:
  $user, $source_ip over 1h

outcome:
  $event_count = count($e.metadata.id)

dedup:
  $user, $source_ip

order:
  $event_count desc

Without deduplication, a query matching a high-volume, repeated pattern — like the same user triggering the same alert type dozens of times within an hour — would return dozens of nearly identical rows, forcing an analyst to manually recognize the repetition; specifying dedup on the relevant fields collapses those into a single representative result per unique combination, with aggregate outcome values (like a count) summarizing what was collapsed.

This matters most for high-volume statistical queries and detection rules over noisy event types, where without deduplication the sheer number of individually-listed matches can bury the actually useful signal — the fact that repetition is happening, and how much of it — underneath a wall of redundant, individually uninformative rows.