Redis Cluster Architecture

Redis Cluster divides the entire keyspace into exactly 16,384 hash slots. Each key is mapped to a slot via CRC16(key) mod 16384. The choice of 16,384 (not 65,536 or more) was deliberate: the slot ownership bitmap must be exchanged in gossip messages between every node pair. At 16,384 slots, the bitmap is only 2KB — small enough to fit in a single network packet even for large clusters. With 65,536 slots, the bitmap would be 8KB, significantly increasing gossip protocol overhead. Slots are assigned to nodes during cluster creation and can be migrated between nodes for rebalancing without downtime.

Redis Cluster nodes maintain cluster state (slot ownership, node health, configuration epoch) via a gossip protocol. Every second, each node picks a random peer and sends a PING containing its view of the cluster state. The receiving node responds with a PONG containing its own view. Nodes exchange information about other nodes they know about, and state converges across the cluster in O(log N) communication rounds. Failure detection uses a combination of PING/PONG timeouts and quorum voting — a node is marked as PFAIL (possibly failed) by any node that can't reach it, and as FAIL (confirmed failed) when a majority of masters agree.

When a master node is marked as FAIL by a majority of the cluster, its replicas initiate an automatic failover. The replica with the most up-to-date replication offset starts a RAFT-inspired election: it increments the cluster configuration epoch and requests votes from all master nodes. A master grants its vote if the requesting replica's master is indeed marked as FAIL and no vote has been granted for that epoch. Once a replica receives votes from a majority of masters, it promotes itself to master status, takes ownership of the failed master's hash slots, and broadcasts the new configuration. The entire failover completes in 1–2 seconds.

Redis Cluster clients maintain a local cache of the slot-to-node mapping (the 'slot table'). When a command is sent to the wrong node (due to a stale slot table), the node returns a MOVED response indicating the correct node for that slot. The client updates its slot table and retries. During slot migration (live resharding), an ASK response indicates a temporary redirect — the slot is being moved and the client should try the target node for this specific command without updating its cached mapping. Smart clients (like redis-cluster-py, Jedis) handle MOVED/ASK transparently, making the clustering invisible to application code.

Redis Cluster executes commands atomically only when all keys in the command map to the same hash slot. Multi-key operations (MGET, transactions, Lua scripts) across different slots are not supported. To force related keys onto the same slot, Redis supports hash tags: only the substring within curly braces is hashed, so {user:1000}.name and {user:1000}.email both map to the same slot because CRC16('user:1000') is used for both. Lua scripts execute atomically on a single node and can operate on any number of keys — provided they all share the same hash slot. This constraint is the fundamental trade-off of Redis Cluster: horizontal scalability at the cost of limited cross-key atomicity.