How can you optimize BookKeeper storage costs using tiered storage?

Configure an offload threshold (by size or age) at the namespace or topic level so ledger segments older than the threshold are automatically moved from replicated BookKeeper disk to cheaper object storage like S3, without manual intervention once set up.

Choose the threshold deliberately based on actual access patterns: frequently read "hot" data should stay on BookKeeper's low-latency disks, while data rarely accessed except for occasional replay or compliance should be pushed to offload as early as reasonably possible to minimize the replicated-disk footprint.

Because offloaded segments remain transparently readable through the normal topic/consumer APIs, you can safely set aggressive (short) offload thresholds without breaking any existing consumer or replay workflow - the only cost is somewhat higher read latency the rare times older data actually gets accessed.

Combine this with a sensible retention policy: for data you don't need indefinitely, letting it expire and delete outright, rather than retaining and offloading forever, is cheaper still than paying even discounted object storage costs for data nobody will ever read again.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.