What is distributed systems observability and what challenges does it introduce compared to monolith observability?

Distributed systems observability refers to the ability to understand the internal state and behavior of a system that consists of multiple independently deployed services communicating over a network. Unlike a monolith — where all code runs in one process and profiling, logging, and debugging are straightforward — distributed systems introduce fundamental challenges that require purpose-built tooling.

Challenge 1 — No single log stream: A request touches 10 services; their logs are in 10 different places. Log correlation requires shared request IDs injected into every service's logs. Without structured logging and log aggregation, tracing a request manually is infeasible.

Challenge 2 — Partial failures: In a monolith, a failure either crashes the process or it does not. In distributed systems, Service A can respond successfully while Service B, called internally by A, silently times out and returns a degraded result. These partial failures are invisible without distributed tracing and upstream error propagation.

Challenge 3 — Clock skew: Services run on different machines with clocks that drift. Log timestamps from different services cannot be naively sorted — a span ending at 10:00:00.001 on Service B might be recorded before a span starting at 10:00:00.000 on Service A due to clock drift. OpenTelemetry uses monotonic clocks within a single process and accepts some clock-skew inaccuracy across processes.

Challenge 4 — Attribution: When latency spikes, which of the 10 services caused it? Without traces linking spans causally, you are guessing. Distributed tracing was invented specifically to solve attribution in distributed systems.