Building Scalable WebSockets for Live Streaming
Live chat, presence indicators, collaborative editing, and real-time viewer counts on a stream all rely on persistent connections. Scaling WebSockets from a demo to millions of concurrent users is where the interesting engineering begins.
Table of contents:
Why WebSockets scale differently from HTTP
A normal HTTP request is short-lived. You connect, receive a response, and disconnect. A WebSocket is a long-lived, stateful connection that a server holds open for the entire session. That changes your scaling model, because you are optimising for concurrent open connections rather than requests per second, and each connection consumes a file descriptor and a slice of memory even while idle.
The practical ceilings are operating-system limits on open file descriptors, per-connection memory, and the cost of broadcasting a message to many sockets at once. A single well-tuned node can hold tens of thousands of connections. Reaching millions means scaling horizontally and coordinating across nodes.
It is worth being honest about when you need this at all. Many features that feel real-time are served perfectly well by periodic polling or server-sent events, which are simpler to operate and scale like ordinary HTTP. Reach for WebSockets when you need low-latency, bidirectional communication, such as chat, live collaboration, multiplayer interaction or fast-moving dashboards. Choosing the right transport up front saves a great deal of operational pain later, and it keeps the architecture honest about the guarantees each feature actually requires.
Horizontal scaling and fan-out
Once you run more than one server, connections for the same chat room or document land on different nodes. When a user on node A sends a message, nodes B and C also have subscribers who need it. A backplane solves this, commonly Redis Pub/Sub, NATS or a Kafka topic, so any node can publish an event that every other node receives and forwards to its local sockets.
Sticky sessions at the load balancer keep a given client pinned to one node for the life of its connection, which simplifies per-connection state. The backplane handles everything cross-node, so no single server needs to know about every connection in the system.
Connection state and presence
Presence, meaning who is online, who is typing and who is watching, is deceptively hard because connections drop silently. Use application-level heartbeats with sensible timeouts to detect dead sockets, and store presence in a fast, expiring store such as Redis with time-to-live keys so a crashed node's users age out automatically.
- Send periodic pings and close connections that miss several responses.
- Use short time-to-live keys for presence so failures self-heal.
- Debounce noisy events such as typing indicators before broadcasting them.
- Keep only live state in memory, and push durable message history to a database or an append-only log.
Backpressure, batching and reconnection
A slow client that cannot drain messages as fast as you send them will grow an unbounded buffer and eventually take the server down. Apply backpressure by capping per-connection queues and coalescing updates for clients that fall behind. Batch high-frequency events into periodic frames rather than emitting one socket write per change.
On the client, assume the connection will drop, because networks are hostile. Implement reconnection with exponential backoff and jitter, plus a resume protocol so a user who blips offline for two seconds does not lose their place.
Securing and authenticating connections
A persistent connection needs authentication that survives its whole lifetime, not just the opening handshake. Authenticate on connect with a signed token, and re-validate periodically so a session revoked mid-stream is actually cut off rather than lingering for hours. Terminate TLS at the edge so every frame travels encrypted, and validate the Origin header to keep hostile pages from opening cross-site sockets.
Rate limiting protects the system from both abuse and honest bugs. Cap how many connections a single client or address can open, and how many messages per second each connection may send, so one misbehaving client cannot exhaust the pool for everyone else. Reject oversized frames early, before they reach application code, to blunt a whole category of denial-of-service attempts.
Authorisation applies to every action, well beyond the initial connection. A user allowed into a chat workspace still should not receive messages from a channel they never joined, so check permissions at the point of subscription and again when routing each event. Treating every inbound frame as untrusted input is the habit that keeps real-time systems safe.
Operating it in production
Instrument everything, including concurrent connections, messages per second, broadcast latency, and dropped-connection rates. Load-test with realistic connection churn rather than steady state, because the reconnect storm after a deploy is often what breaks systems. Roll deploys gradually so a million clients do not all reconnect in the same second.
Real-time infrastructure like this underpins the social platforms and streaming products we build, and we are happy to help you choose the right real-time foundation for yours. Explore our software engineering work, or start a project.