Database / Apache Cassandra Intermediate and Advanced interview questions
How does Cassandra detect node failure?
Cassandra uses a Phi Accrual Failure Detector rather than a simple fixed heartbeat timeout to decide whether a node is up or down.
- Each node tracks the history of arrival times of heartbeats (via gossip) from every other node.
- From that history it builds a statistical distribution of expected inter-arrival times.
- Instead of a hard yes/no cutoff, it computes a suspicion level (phi) — the higher phi climbs, the more confident the detector is that the node is actually down rather than just slow.
- A configurable threshold (
phi_convict_threshold) decides when a node is finally marked down.
This approach adapts to each node's normal network conditions: a node on a congested link with naturally jittery heartbeats won't be falsely marked down as quickly as a fixed timeout would cause, while a consistently punctual node gets flagged fast if it truly goes silent.
Failure detection only marks a node down locally and informs the rest of the cluster via gossip — it does not by itself trigger any data movement; that is handled separately by hinted handoff and repair.
More Related questions...