What is log sampling and when should you apply it?

Log sampling is the practice of recording only a fraction of log entries that match a certain pattern, rather than every single one. It is a strategy for controlling log volume and cost when some log types are emitted at very high rates and provide diminishing marginal value per entry.

The most common scenario is high-frequency success logs. If your API handles 50,000 requests per second and every successful request logs an INFO entry, you are generating 4.3 billion log lines per day — most of which are identical in structure and say everything is fine. Sampling 1 in 100 success logs while keeping 100% of warnings, errors, and slow requests reduces volume by ~99% without meaningfully hurting your ability to investigate incidents.

There are two main approaches:

Head-based (random) sampling: Log a fixed percentage of all events matching a rule. Simple to implement but may drop rare important events if they happen to fall in the unlogged fraction.

Adaptive sampling: Adjust the sampling rate dynamically based on rate — when the rate is low, log everything; when the rate spikes, increase the sampling ratio. This ensures unusual patterns (sudden traffic surges) are captured at higher fidelity.

Sampling should never be applied to error-level logs, security audit logs, or any log that is only emitted once per rare event. The critical rule: sample on volume, not on importance. Always emit 100% of high-severity events regardless of sampling configuration.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.