Integration / Apache Pulsar Interview questions
Explain the internal working of Pulsar transactions?
A Pulsar transaction begins when a client requests a transaction ID from a dedicated internal component, the Transaction Coordinator, which tracks the transaction's state (open, committing, committed, aborting, aborted) throughout its life.
As the client produces messages and acknowledges consumed messages within that transaction, those operations are recorded as "pending" against the relevant topics and subscriptions - visible internally, but not yet visible to other consumers reading normally, and not yet permanently marking source messages as acknowledged.
On commit, the Transaction Coordinator writes a commit marker and drives all participating topics/subscriptions to make the pending produces visible and the pending acknowledgments permanent atomically from the consumer's perspective, even though the operations may span multiple partitions or topics.
On abort, or a coordinator/client failure before commit, the pending produced messages are marked as effectively invisible to consumers and any pending acknowledgments are rolled back, so those source messages return to being unacknowledged and eligible for normal redelivery.
This is what allows exactly-once-style, multi-topic atomic operations, like consuming from topic A and producing to topic B as one unit, that plain per-message deduplication alone can't provide, since deduplication only protects a single producer's own retries, not multi-step consume-then-produce atomicity.
More Related questions...