What is Thanos and how does it extend Prometheus for large-scale deployments?

Thanos is an open-source, CNCF incubating project that extends Prometheus to provide highly available, long-term metrics storage at scale. It was created at Improbable (now Grafana Labs contributes heavily) and addresses two fundamental limitations of standalone Prometheus: single-node storage limits and multi-cluster query federation.

Thanos's architecture uses a sidecar pattern: a Thanos Sidecar runs alongside each Prometheus server. It uploads completed TSDB blocks to an object store (S3, GCS, Azure Blob) every 2 hours. This provides unlimited long-term retention without changing how Prometheus works internally — Prometheus still handles recent data (last 2 hours) locally.

The Thanos Store Gateway makes historical blocks in object storage queryable by implementing the same gRPC StoreAPI that Prometheus exposes. The Thanos Querier is a global query layer that fans out PromQL queries to multiple Prometheus instances and Thanos Store Gateways simultaneously, deduplicating results from replicated Prometheus servers (using the --deduplication.replica-label flag).

The Thanos Compactor runs in the background to downsample old blocks (5-minute and 1-hour resolution for data older than 40 days and 1 year respectively) and delete expired blocks according to retention policies, keeping object storage costs manageable.

The Thanos Ruler runs recording rules and alerting rules against the global Thanos view, enabling cross-cluster alerting rules that a single Prometheus instance cannot evaluate.