Integration / Apache Kafka Interview questions
What is the purpose of a Kafka topic's retention policy?
A retention policy determines how long Kafka keeps records in a topic before they're eligible for deletion, since a topic's log can't grow forever on finite disk space. It's configured per topic, most commonly by time (retention.ms) or by size (retention.bytes), whichever limit is hit first.
retention.ms=604800000 # 7 days retention.bytes=-1 # no size-based limit
The purpose is to balance durability against storage cost: a topic feeding real-time processing might only need a few hours or days of retention since consumers are expected to stay caught up, while an event-sourcing or audit-log topic might retain data for months or indefinitely. Retention is independent from whether a message has already been consumed — Kafka doesn't delete a record just because every consumer has read it, so multiple consumer groups can each re-read historical data within the retention window at their own pace.
More Related questions...
