Tools / Monitoring and Observability Interview Questions
What is observability-driven development (ODD) and how does it shift monitoring left?
Observability-driven development (ODD) is a practice where engineers write instrumentation — metrics, logs, and trace spans — as a first-class part of feature development, not as an afterthought added after a service is deployed. The principle is "if you cannot observe it, you cannot reason about it in production", so instrumentation ships with features.
The shift-left metaphor comes from moving activities earlier in the development lifecycle. Traditional monitoring is bolted on post-deployment: ops teams add dashboards after a service is already in production and fires an incident. ODD moves this to the code review stage: observability is a requirement for merging, just like unit tests.
In practice, ODD includes:
- Instrumentation in definition of done: A feature is not "done" until it has metrics for rate, error, and duration; structured log statements at key decision points; and trace spans for every external call.
- Dashboard-first design: Engineers sketch what they want to see in production before writing the feature code, then instrument to produce those signals.
- Local observability testing: Developers run Grafana and Loki locally (via docker-compose) and verify their instrumentation works before pushing to CI. Tools like Tilt and Skaffold enable local Kubernetes observability environments.
- SLO definition at design time: The SLI and SLO for a new feature are defined before implementation, guiding what to instrument and how to alert.
ODD reduces MTTD for new features because the monitoring is ready from day one, rather than being retrofitted after the first production incident reveals it was missing.
Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!
Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
Webull! Receive free stock by signing up using the link: Webull signup.
More Related questions...
