How many WebSocket connections can a $6 server hold? 99,770, and memory is the wall

If you are building anything with live updates — a chat, a multiplayer game, a dashboard that ticks, a collaborative editor — sooner or later you need a number. How many people can be connected at once before I need a bigger server? The advice you find is remarkably unhelpful: "it depends on your workload", followed by an anecdote about someone's Erlang cluster.

So I put a plain Node WebSocket server on the cheapest machine DigitalOcean rents, one shared vCPU and 1GB of memory at $6 a month, and opened connections until it stopped accepting them.

It held 99,770 at once. The thing that eventually stopped it was memory, but not the memory you would look at.

Holding connections is nearly free

The first surprise is how little work an idle connection is. I climbed to 85,000 connections in waves of five thousand, waiting two and a half seconds after each wave so that the measurement reflects the cost of holding connections rather than the cost of opening them.

Eighty-five thousand open WebSocket connections, and the CPU is doing nothing. Zero to one per cent, the whole way up.

This is worth sitting with, because it contradicts how people reason about real-time capacity. A connection that is merely open is a file descriptor and some buffers. It does not poll, it does not wake anything, it costs no cycles. Your CPU budget is spent on two things only: completing handshakes, and moving messages. If your app has a lot of users connected and mostly quiet — which describes most chat apps, most dashboards, most multiplayer lobbies — the CPU is not what you will run out of.

The memory that does not show up in your process

Look at the two memory columns again. Node's resident memory grew by about 3.6KB per connection, which is the number you would report if you watched your own process. But free memory on the machine fell by 550MB across those 85,000 connections, which is 6.6KB each.

Half the cost is invisible from inside the application. Roughly 3KB per connection lives in kernel socket buffers — the send and receive queues the kernel keeps for every TCP connection — and it never appears in your heap snapshot or your RSS.

That gap is the whole planning story. If you size your server from what your process reports, you will be out by a factor of two, and you will discover it at the worst possible moment.

The arithmetic also predicts the wall almost exactly. At 85,000 connections there was 87MB free; at 6.6KB each, that is room for about 13,000 more, which puts the ceiling near 98,500. The measured maximum was 99,770 — within 1.3% of the prediction.

The cliff is late, sharp, and unmistakable

Up to 98,000 connections the server accepted new ones at between 3,000 and 4,400 per second. Then:

opened= 98,000   handshake rate = 3,731/s
opened= 99,000   handshake rate =    34/s
opened= 99,770   handshake rate =     0/s

A hundredfold collapse inside one thousand connections. There is no gentle degradation to warn you, which is an argument for leaving real headroom rather than running close to a limit you have measured.

What it feels like at each level

Latency, sampled from random live connections with a request id matched to its reply:

Up to 80,000 connections the box answers in about a millisecond and loses nothing. At 99,770 the median is still 2.3ms, but the tail has fallen apart and fifteen per cent of messages never came back.

So the honest capacity is not the maximum. A comfortable working number is around 80,000 connections, which still leaves a fifth of the memory free and keeps every reply inside ten milliseconds at the ninetieth percentile.

For a sense of scale: eighty thousand simultaneous live connections is a chat app with eighty thousand people in it at the same moment, on a machine that costs less per month than two coffees. Most products never see that, and the ones that do are not worrying about a $6 server.

When it is in distress

At 99,770 connections the machine was still serving its existing connections at 2.3ms, yet new arrivals mostly could not get in: four of five handshake attempts from an unrelated machine timed out at twenty seconds, and the one that succeeded took 6.17 seconds. SSH could not complete a key exchange at all, timing out during banner exchange even with a 110-second limit.

That combination is worth recognising because it is genuinely confusing in production: the application looks healthy from the inside, existing users report no problem, and you cannot get a shell on the box to find out why nobody new can connect.

What I got wrong, twice

The first run reported a p99 latency of 34,885 milliseconds — thirty-five seconds — which I nearly wrote down as a finding about tail latency under load.

It was my measuring code. I picked a random socket, sent a ping, and waited for the next message on that socket, while thousands of other handshakes were still completing. Any reply arriving on that socket stopped my stopwatch, including replies to somebody else's ping. The rewrite tags every ping with an id, ignores anything that does not match it, and enforces a hard three-second deadline, counting losses separately instead of quietly folding them into the tail. With that fixed, the real p99 at nearly 100,000 connections is 1.5 seconds. Bad, but honest, and a completely different number.

The second is simpler and more embarrassing. I restarted the server to clear it, and the client began connecting a second later, so the first wave met a server that was still coming up: 1,194 instant failures and a run that aborted with "no progress". Nothing was wrong with either machine. I had just raced my own setup.

The third mistake ran the other way, and for a while it made the server look worse than it was. Somewhere above ninety thousand connections my client stopped being able to add any, and I wrote that down as the server's ceiling. Then, mostly out of curiosity, I opened a socket to it from my laptop at home, across the public internet, while the box was still carrying its hundred thousand: the reply came back as 101 Switching Protocols after 4.6 seconds. Slow, but it worked, which meant the server had capacity my client could no longer reach. What had actually run out was a single Node process on the load generator trying to shepherd a hundred thousand sockets of its own, and I had been about to publish its limit as though it belonged to the thing I was measuring.

The short version

A $6 machine holds tens of thousands of live WebSocket connections without much drama, and the resource it runs out of is memory rather than CPU, at roughly 6.6KB of real system memory per connection rather than the 3.6KB that the process itself will own up to. That factor of two is the part worth remembering, because it is the difference between a capacity plan that works and one that fails in production while your dashboards insist everything is fine.

If your users are mostly idle, something in the region of 80,000 connections per gigabyte is a defensible planning figure, and you should halve it if they are chatty, because messages cost CPU in a way that open sockets simply do not. Watch MemAvailable on the host rather than the heap size your runtime reports. And leave yourself real headroom, because the collapse I measured arrived inside a single thousand connections with no warning at all, going from nearly four thousand handshakes a second to thirty-four.

Measured on a DigitalOcean s-1vcpu-1gb droplet in Frankfurt, with load generated from a separate s-4vcpu-8gb droplet in the same region. Both destroyed afterwards; the entire experiment cost about thirty cents.