What is Prometheus and how does its pull-based scraping model work?

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud and now a CNCF graduated project. It stores all data as time-series: streams of timestamped float64 values identified by a metric name and a set of key-value labels.

What makes Prometheus distinctive is its pull-based scraping model. Instead of applications pushing metrics to a central server, Prometheus periodically sends HTTP GET requests to a /metrics endpoint on each target. The response is in Prometheus exposition format — a plain-text, line-by-line format listing metric names, label sets, and values. Prometheus stores the scraped data in its local TSDB (time-series database).

Targets are discovered through service discovery mechanisms: static configuration, Kubernetes API, Consul, EC2 tags, and many others. This means Prometheus automatically starts scraping new pods as they come up and stops when they go down — no manual registration required.

The pull model has important operational implications: Prometheus controls the scrape rate, failed targets are immediately visible as scrape failures, and there is no need for agents to know the Prometheus address. The trade-off is that short-lived jobs (batch jobs) may finish before the scrape happens — solved by the Pushgateway, which acts as an intermediary for ephemeral workloads to push metrics to.

PromQL (Prometheus Query Language) is used to query and aggregate these time-series, feeding both Grafana dashboards and Alertmanager rules.