Integration / Apache Pulsar Interview questions
How can you optimize Pulsar for high-throughput workloads?
Enable producer-side batching and compression so many small messages are grouped and compressed into fewer, larger network writes and BookKeeper entries, dramatically reducing per-message overhead at high publish rates.
Use partitioned topics so a single logical topic's load spreads across multiple brokers, and correspondingly potentially different bookies, instead of bottlenecking on one broker's CPU/network and one ledger's write path.
Tune BookKeeper's ensemble/write-quorum/ack-quorum settings deliberately: a smaller write quorum reduces the number of bookies each entry waits on, lowering write latency, at the cost of some durability margin - the right balance depends on how much replication risk is acceptable.
On the consumption side, prefer Shared or Key_Shared subscriptions with multiple parallel consumer instances over a single Exclusive consumer wherever strict global ordering isn't required, since that's what actually lets consumption scale horizontally alongside publish throughput.
Finally, size broker and bookie hardware appropriately - especially bookie journal/ledger disks, ideally on separate fast disks such as NVMe/SSD - since BookKeeper's write path is disk-latency sensitive and undersized storage is a common throughput ceiling in practice.
More Related questions...