Web / Apache Solr Interview questions
What happens when a Solr shard leader goes down?
Losing a shard leader triggers a well-defined recovery sequence rather than an outage, as long as at least one in-sync replica remains.
- The leader's session with ZooKeeper ends (crash, GC pause past timeout, or network partition), removing its ephemeral leader znode and its entry in
/live_nodes. - ZooKeeper's watch mechanism notifies the remaining replicas for that shard almost immediately.
- The replicas run a leader election: a replica is only eligible if it is marked as being in sync (its update log is not lagging), preventing an out-of-date replica from becoming leader and silently losing recent writes.
- The newly elected leader registers itself in ZooKeeper's
state.json, and clients/coordinating nodes start routing writes for that shard to it.
During the brief election window, writes to that specific shard may fail or be retried by the client depending on configuration, but reads from any surviving replica of that shard continue uninterrupted, since read requests don't require a leader. If every replica of a shard is down simultaneously, that shard becomes fully unavailable regardless of how many other shards are healthy, which is why replication factor matters as much as shard count for availability.
More Related questions...