Scaling WebSockets: Handling Many Connections

The WebSocket Challenge

WebSockets are fantastic for real-time communication. Think chat apps, live dashboards, collaborative editing tools. They open up a persistent, full-duplex connection between a client and server. But what happens when you need to support thousands, or even millions, of these connections simultaneously? That’s where scaling gets tricky.

Most developers start with a single server handling all WebSocket connections. This is fine for small projects. But as traffic grows, that single server becomes a bottleneck. You start seeing increased latency, dropped connections, and eventually, outright failure. To scale, you need a plan.

Horizontal Scaling: More Servers, More Power

The most common approach is horizontal scaling. Instead of making one server more powerful (vertical scaling), you add more servers to your pool. This means distributing your incoming WebSocket connections across multiple instances.

This immediately brings up a problem: how do you ensure a client’s messages go to the right server? If a user is connected to Server A, but their message is processed by Server B, the connection is broken. This is called the “sticky session” problem. You need a way to route messages correctly.

Load Balancing and Message Brokers

This is where load balancers and message brokers come into play.

Load Balancers

A load balancer sits in front of your WebSocket servers. It intercepts incoming connections and distributes them across your available server instances. For WebSocket traffic, you’ll want a load balancer that supports sticky sessions or can intelligently route based on connection ID. Nginx and HAProxy are popular choices here. They can handle SSL termination, health checks for your servers, and basic load distribution.

However, a simple load balancer might not be enough on its own for true distributed WebSocket scaling. If Server A goes down, all its connected clients need to be reconnected. More importantly, if Server A crashes, Server B needs to know about the messages Server A would have processed. This is where message brokers shine.

Message Brokers

Message brokers, like Redis Pub/Sub, RabbitMQ, or Kafka, act as intermediaries. When a client sends a message, it doesn’t go directly to another client. Instead, it’s published to a topic on the message broker. All servers subscribed to that topic receive the message. Each server can then decide if it needs to forward that message to any of its connected clients.

Here’s a common pattern:

1. Client A connects to Server 1.
2. Client B connects to Server 2.
3. Client A sends a message. Server 1 receives it.
4. Server 1 publishes the message to a Redis Pub/Sub channel (e.g., “chat-messages”).
5. Both Server 1 and Server 2 are subscribed to “chat-messages”.
6. Both servers receive the message from Redis.
7. Server 1 checks if Client A or any other clients connected to Server 1 should receive this message and sends it.
8. Server 2 checks if Client B or any other clients connected to Server 2 should receive this message and sends it to Client B.

This decouples your servers. If Server 1 goes down, Server 2 is unaffected and can continue serving its clients. New clients will connect to the available servers, and everyone stays in sync via the message broker.

Backend Code Considerations

When implementing this, your server code needs to be aware of the distributed nature.

• Connection Management: Instead of just managing local connections, your server needs to know which clients are connected globally. This might involve storing connection metadata (user ID, connection ID, server instance ID) in a shared store like Redis.
• Message Routing: When a message arrives, your server must determine which connected clients should receive it. This often involves querying the shared store to find relevant connections and their associated server instances (if direct routing is possible, otherwise, the message broker handles distribution).
• Handling Disconnects: Implement robust logic for when a server instance fails. Clients will disconnect and need to reconnect. Your system should be able to handle these reconnections gracefully, potentially directing them to a different server instance.

Example with Node.js and Redis

Let’s look at a simplified Node.js example using ws for WebSockets and redis for pub/sub.

1
const WebSocket = require('ws');
2
const redis = require('redis');
3

4
const wss = new WebSocket.Server({ port: 8080 });
5
const redisClient = redis.createClient();
6

7
const connectedClients = new Map(); // Stores WebSocket instances by a unique ID
8
const clientToServerMap = new Map(); // Maps client ID to the server instance managing it (useful for direct message sending if not using pub/sub for all)
9

10
redisClient.on('connect', () => {
11
    console.log('Connected to Redis');
12
    redisClient.subscribe('chat-messages', (err, channel) => {
13
        if (err) console.error('Redis subscribe error:', err);
14
        console.log(`Subscribed to channel: ${channel}`);
15
    });
16
});
17

18
redisClient.on('message', (channel, message) => {
19
    console.log(`Received message from Redis channel ${channel}: ${message}`);
20
    const parsedMessage = JSON.parse(message);
21
    const senderId = parsedMessage.senderId;
22
    const payload = parsedMessage.payload;
23

24
    // Iterate through all connected clients on THIS server instance
25
    // In a real-world scenario, you'd likely have a more sophisticated way
26
    // to determine which clients should receive this message, possibly
27
    // using data stored about each client (e.g., rooms they are in).
28
    connectedClients.forEach((ws, clientId) => {
29
        if (clientId !== senderId) { // Don't send message back to sender
30
            ws.send(JSON.stringify({ from: senderId, message: payload }));
31
        }
32
    });
33
});
34

35
wss.on('connection', (ws) => {
36
    const clientId = Math.random().toString(36).substring(2, 15);
37
    console.log(`Client connected: ${clientId}`);
38
    connectedClients.set(clientId, ws);
39
    clientToServerMap.set(clientId, process.env.SERVER_ID || 'local'); // Track which server instance this client is on
40

41
    ws.on('message', (message) => {
42
        console.log(`Received message from ${clientId}: ${message}`);
43
        const parsedMessage = JSON.parse(message);
44
        // Publish the message to Redis
45
        redisClient.publish('chat-messages', JSON.stringify({
46
            senderId: clientId,
47
            payload: parsedMessage.message
48
        }));
49
    });
50

51
    ws.on('close', () => {
52
        console.log(`Client disconnected: ${clientId}`);
53
        connectedClients.delete(clientId);
54
        clientToServerMap.delete(clientId);
55
        // Optional: Notify others or clean up global state if needed
56
    });
57

58
    ws.on('error', (error) => {
59
        console.error(`WebSocket error for client ${clientId}:`, error);
60
        connectedClients.delete(clientId);
61
        clientToServerMap.delete(clientId);
62
    });
63

64
    ws.send(JSON.stringify({ message: 'Welcome!', clientId: clientId }));
65
});
66

67
redisClient.connect().catch(console.error);
68
console.log('WebSocket server started on port 8080');
69

70
// Graceful shutdown handling
71
process.on('SIGINT', async () => {
72
    console.log('SIGINT signal received: closing server');
73
    await redisClient.quit();
74
    wss.close(() => {
75
        console.log('WebSocket server closed');
76
        process.exit(0);
77
    });
78
});

This example uses Redis Pub/Sub. When a message comes in from a client, the server publishes it to a Redis channel. All other server instances (which are also subscribed to this channel) will receive the message and can then forward it to their local clients. This makes the system highly scalable.

Final Thoughts

Scaling WebSockets isn’t just about throwing more servers at the problem. It requires careful consideration of how connections are managed, how messages are routed, and how your architecture handles the distributed nature of the system. Load balancers are your first line of defense, but message brokers are crucial for true fault tolerance and scalability across multiple server instances. Get this right, and your real-time applications can handle the load.