Database / ScyllaDB Interview questions
What happens when a node becomes unavailable in a ScyllaDB cluster?
Because data is replicated across multiple nodes per the table's replication factor, a single node going down doesn't make its data unavailable - the remaining replicas for its token ranges continue serving reads and writes.
Other nodes' failure detectors (built on gossip-based heartbeat exchange) mark the node as down after missing expected heartbeats, and the cluster's topology view updates accordingly. Writes destined for the down node are handled via hinted handoff, stored on a live coordinator or replica and replayed once the node returns. Reads at consistency levels like QUORUM or LOCAL_QUORUM continue succeeding as long as enough of the remaining replicas can form the required majority; only if too many replicas for a given range are simultaneously down does an operation at that consistency level start failing. When the node comes back, it replays any hints waiting for it and, on a regular schedule, repair reconciles any data it missed while down that exceeded the hint window, bringing it fully back in sync with the rest of its replica set.
More Related questions...