Integration / Apache Kafka Interview questions
Why is Kafka better suited than a traditional message queue for high-throughput event streaming?
A traditional queue (like classic AMQP/JMS brokers) is built around individual message delivery and removal — a message is typically deleted once acknowledged, and the broker's job is mainly routing and short-term buffering. Kafka is built around a durable, replayable log instead, and that structural difference is what makes it fit high-throughput streaming better.
- Replayability — because messages aren't deleted on consumption, a new consumer can start reading from the beginning of the retention window, or an existing one can reprocess after a bug fix, without the producer resending anything.
- Multiple independent consumers — many consumer groups can read the exact same topic at their own pace, which a queue's single-delivery model doesn't naturally support without fan-out infrastructure on top.
- Partition-based parallelism — a topic scales by adding partitions and consumers, giving high aggregate throughput that a single-queue model struggles to match at the same operational simplicity.
- Batching and efficient sequential disk access — Kafka's storage model is optimized for high-throughput, append-only writes rather than per-message transactional bookkeeping.
The trade-off is that Kafka is a heavier piece of infrastructure to run than a simple queue, and its consumer model (pull-based, offset-tracked) is a different mental model than the push-based, per-message ack/nack pattern many queue systems use — for low-volume task distribution with complex per-message routing rules, a traditional queue can still be the simpler, better fit.
More Related questions...