Prev Next

Integration / Apache Pulsar Interview questions

How can you optimize consumer throughput with Key_Shared subscriptions?

Ensure message keys have enough distinct values and reasonably even distribution across them - if most messages share one or a few keys, that traffic still funnels to a single consumer each, capping parallelism regardless of how many consumer instances are added.

Scale the number of consumer instances to roughly match, or stay below, the practical number of distinct, active keys at any given time, since adding more consumers than there are actively-flowing keys just leaves extra consumers idle rather than increasing throughput.

Tune receiver queue size and permits per consumer appropriately - too small a queue can throttle a fast consumer waiting on network round-trips for more messages, while an oversized queue on a slow consumer can cause it to hold an unfair share of in-flight, unacknowledged key ranges.

Watch for consumer churn: every time a consumer joins or leaves a Key_Shared subscription, key-range ownership is recomputed, briefly pausing processing for the reassigned ranges - so a subscription with frequent connect/disconnect cycles, from crash-looping or aggressive autoscaling, sees lower effective throughput than the same consumer count running stably.

Throughput on Key_Shared is capped when:
Frequent consumer connect/disconnect cycles on Key_Shared cause:

Invest now in Acorns!!! 🚀 Join Acorns and get your $5 bonus!
Acorns Logo

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Earn passively and while sleeping

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Robinhood Logo

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.


Webull Logo

Webull! Receive free stock by signing up using the link: Webull signup.

More Related questions...

Explain about Apache Pulsar. What is Apache Pulsar? What is a Pulsar broker? What is Apache BookKeeper? What are tenants and namespaces in Pulsar? What is a topic in Pulsar? What is a producer in Pulsar? What is a consumer in Pulsar? What is a subscription in Pulsar? Define a partitioned topic in Pulsar? What is Pulsar's metadata store used for? What are Pulsar Functions? What is Pulsar IO? Describe geo-replication in Pulsar? What is tiered storage in Pulsar? What is a non-persistent topic in Pulsar? What are the subscription types in Pulsar? What is message retention in Pulsar? What is schema registry in Pulsar? List the core components of a Pulsar cluster? How do you create a topic in Pulsar? What is the difference between Pulsar and Kafka's storage architecture? How does Pulsar separate compute and storage? Why is Pulsar considered multi-tenant by design? What is the difference between Shared and Exclusive subscriptions? How does Key_Shared subscription maintain ordering? When should you use Failover subscription instead of Exclusive? What is the difference between a ledger and a segment in BookKeeper? How does Pulsar achieve message deduplication? Why do brokers in Pulsar not store data locally? What happens when a broker crashes in Pulsar? How does namespace bundle splitting work? What is the difference between backlog quota and retention policy? When should you use a Reader instead of a Consumer? How does topic compaction work in Pulsar? Why is ensemble size different from write quorum in BookKeeper? What is the difference between persistent and non-persistent topics? How does Pulsar handle delayed message delivery? What happens when a consumer negatively acknowledges a message? Explain the lifecycle of a message in Pulsar from produce to acknowledge? How can you optimize Pulsar for high-throughput workloads? How do you troubleshoot a growing backlog in Pulsar? Explain the execution flow of topic ownership failover in Pulsar? How can you optimize BookKeeper storage costs using tiered storage? Explain the internal working of Pulsar transactions? Which is better for exactly-once processing: idempotent producers or transactions, and why? How do you troubleshoot unbalanced load across brokers? Explain the lifecycle of a namespace bundle from creation to split? How can you optimize consumer throughput with Key_Shared subscriptions? Explain the execution flow of a Pulsar Function processing a message? How do you troubleshoot message duplication in a Pulsar producer?
Show more question and Answers...

Apache Camel Interview Questions

Comments & Discussions