How can you optimize Pulsar for high-throughput workloads?

Enable producer-side batching and compression so many small messages are grouped and compressed into fewer, larger network writes and BookKeeper entries, dramatically reducing per-message overhead at high publish rates.

Use partitioned topics so a single logical topic's load spreads across multiple brokers, and correspondingly potentially different bookies, instead of bottlenecking on one broker's CPU/network and one ledger's write path.

Tune BookKeeper's ensemble/write-quorum/ack-quorum settings deliberately: a smaller write quorum reduces the number of bookies each entry waits on, lowering write latency, at the cost of some durability margin - the right balance depends on how much replication risk is acceptable.

On the consumption side, prefer Shared or Key_Shared subscriptions with multiple parallel consumer instances over a single Exclusive consumer wherever strict global ordering isn't required, since that's what actually lets consumption scale horizontally alongside publish throughput.

Finally, size broker and bookie hardware appropriately - especially bookie journal/ledger disks, ideally on separate fast disks such as NVMe/SSD - since BookKeeper's write path is disk-latency sensitive and undersized storage is a common throughput ceiling in practice.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.