WhatsApp Messaging at 100B Messages/Day

Erlang's BEAM virtual machine was originally built by Ericsson for telephone switches that could never go down. Each user connection maps to a lightweight Erlang process (not an OS thread) consuming only ~2KB of memory. The BEAM VM runs millions of these processes concurrently with preemptive scheduling and per-process garbage collection — no stop-the-world pauses that would freeze all connections. Hot code reloading allows WhatsApp to deploy new code to production servers without disconnecting a single user session.

WhatsApp uses Mnesia, Erlang's built-in distributed database, for user session state and routing tables. Mnesia runs inside the same BEAM VM as the application, eliminating network round-trips for metadata lookups. It supports both in-memory and on-disk tables with transparent replication across nodes. For actual message storage, WhatsApp uses a custom append-only store optimized for the write-once-read-once access pattern — messages are written sequentially when received and read exactly once when delivered, making sequential I/O the dominant pattern.

When Alice sends a message to Bob, the server stores it in a per-recipient queue. If Bob is online, the message is pushed immediately via his persistent connection. If Bob is offline, the message waits in the queue until he reconnects, at which point all queued messages are delivered in order. Bob's client sends an ACK back to the server, which deletes the message from the queue and forwards a delivery receipt to Alice. This three-way ACK chain (sent → delivered → read) provides the familiar checkmark UX and guarantees at-least-once delivery.

WhatsApp runs on FreeBSD rather than Linux because its network stack handles massive numbers of concurrent long-lived connections more efficiently. Engineers tuned kernel parameters extensively: file descriptor limits raised to 2M+, socket buffer sizes optimized for small message payloads, and TCP keepalive intervals tuned for mobile networks with variable connectivity. A single WhatsApp server handles 2 million simultaneous connections, each backed by a supervised Erlang process with its own isolated mailbox and automatic crash recovery via OTP supervision trees.

WhatsApp implements the Signal Protocol for end-to-end encryption across all messages. Each device generates a unique Curve25519 identity key pair, and message keys are ratcheted forward after every message using the Double Ratchet Algorithm, providing forward secrecy. Key exchange uses X3DH (Extended Triple Diffie-Hellman) with prekey bundles uploaded to the server, enabling encrypted session establishment even when the recipient is offline. The server handles only encrypted blobs — it can route but never read message content.