Integration / Apache Pulsar Interview questions
How does topic compaction work in Pulsar?
Compaction rewrites a topic's backlog into a compacted ledger that keeps only the most recent message for each distinct key, discarding older messages with the same key - similar in spirit to Kafka's log compaction, useful for topics representing "latest state per entity."
It runs as a background job (triggered manually or automatically past a configured threshold) that reads the existing topic data and writes a new compacted ledger, without disrupting normal producer/consumer traffic on the topic in the meantime.
Consumers must explicitly opt in to reading the compacted view; by default, consumers still see the full uncompacted message history unless they request the compacted alternative.
Compaction keeps, per key:
To see the compacted view, a consumer must:
More Related questions...
