Short version for the impatient: the number 160 in your consistent hashing config is a 2007 default that nobody re-derived, and Cloudflare just showed the formula that lets you pick your own. If you want to know why I spent a Thursday evening doing square roots, read on.
I run a very small amount of infrastructure. This blog and a couple of client apps on a Postgres box. Nothing I own needs a consistent hash ring. So when Cloudflare published Saving another 100TB of RAM with math (and Rust) on Thursday, my first reaction was that it was a story for people with data centres, and I should get back to my invoices. Then I got to the part where they changed a u32 to a u16 and saved zero bytes, and I recognised myself. I've made that exact mistake in Go. Twice.
So this post is me working through what a 100TB win at Cloudflare's scale means for someone with three servers, and which of their findings are about scale and which are about everybody.
What consistent hashing does, minus the ring diagram
Every explanation of consistent hashing draws a circle. The Cloudflare authors skip the circle and use a number line, which I found clearer, so I'll do the same.
Take a hash function that outputs a 32-bit integer. Hash each server's address and place it on the line from 0 to 4 billion. Hash each request's cache key and place it on the same line. A request belongs to the first server to its left. That's the whole algorithm. Add a server and only the keys between it and its left neighbour move. Remove one and only its keys move. That's the property you want, and it's why nginx has offered hash $key consistent in its upstream module for years.
The catch is that hashes are effectively random, so the gaps between servers are random too. One server ends up owning a huge stretch of the line and another owns a sliver. The standard fix is to give each server many points on the line instead of one. Nginx uses 160 per server. Pingora, Cloudflare's Rust proxy, copied that default. So did I, in a Go cache layer, years ago, because 160 was what everyone used and I had no argument against it.
The formula I'd never seen written down
Here's where the post earns its title. Most sources tell you that more points per server gives you a more even distribution and leave it there. The Cloudflare team derived the exact coefficient of variation (standard deviation divided by expected share) for N servers with k points each:
CV = sqrt( (N - 1) / (N * k + 1) )
For 100 servers with one point each, that's about 99%. Some server is doing double its fair share while another naps. Push k to 160 and the same formula gives about 8%. I checked their arithmetic and got the same numbers.
Then I plugged in my own. Three servers, 160 points each: 6.4%. Three servers with 16 points each: about 20%. Three servers with 1,600 points: about 2%. Each tenfold increase in k buys roughly one step down in error, and the steps get smaller. That's the diminishing-returns curve the post plots, and it holds at N=3 just as well as at N=2048.
Here's the thing I got wrong for years. I assumed 160 was tuned. It wasn't tuned for me. It was a number that made 100-ish servers look even enough for a memcached client in 2007, and every implementation since has inherited it. If you have three backends and you care about 6% imbalance, the formula says go to 1,000 points and pay a few kilobytes. If you have three backends and don't care, 32 points is fine. Either way, you can now decide instead of copying.
Why Cloudflare's ring was 6GB and yours isn't
The RAM problem in the post came from two multipliers that most of us never hit.
The first is weighting. Cloudflare's servers have different amounts of disk, so they scale each server's point count by its storage. The ketama algorithm does this by giving a server with twice the weight twice the points. The post uses a weight factor of 625 as a worked example, which turns 160 base points into 100,000 points per server.
The second is feature combinations. Not every server can take every request (compliance, caching features, and so on), so each combination of features needs its own ring. A handful of features becomes dozens of rings. Multiply 2,048 servers by 100,000 points by dozens of rings, at 8 bytes a point, and you're well into gigabytes on every machine in the fleet.
My three-server ring at 160 points is 3,840 bytes. I could store it in a tweet. So the headline number is a scale story. The two fixes underneath it are not.
The u32 to u16 change that saved nothing
Each point in the ring is a struct holding the hash and an index into the server list:
struct Point {
hash: u32,
index: u32,
}
Eight bytes. One of the engineers pointed out that no data centre has 65,000 servers, so the index fits in a u16. Change the type, save two bytes per point, ship it.
Except the struct stayed at eight bytes. Rust aligns a struct to its widest field, and the type layout rules round the total size up to a multiple of that alignment. Widest field is the 4-byte hash, so 4 + 2 rounds up to 8. The two bytes you removed came back as padding.
The workaround they shipped is to drop the typed fields entirely and store six raw bytes:
struct Point([u8; 6]);
impl Point {
fn hash(&self) -> u32 {
u32::from_ne_bytes(self.0[0..4].try_into().unwrap())
}
fn index(&self) -> u16 {
u16::from_ne_bytes(self.0[4..6].try_into().unwrap())
}
}
A byte array has alignment 1, so the struct is exactly six bytes. That's the 25% saving in the post. They mention #[repr(packed)] as the obvious alternative and decline it, and they're right to. Packed structs let you take references to misaligned fields, which is undefined behaviour on some targets, and the compiler will fight you about it. The byte-array version compiles to the same instructions without the foot-gun.
This is the part that applies to everyone. Go has the same rule. I once reordered a struct from bool, int64, bool to int64, bool, bool in a hot path and dropped it from 24 bytes to 16, after two days of being sure the allocator was the problem. Check with unsafe.Sizeof in Go or std::mem::size_of in Rust before you assume a field type change did anything.
Cutting 90% of the points, and then not flipping the switch
The formula also told them that going from 10,000 points per server to 100,000 was buying a 0.7% reduction in error. Worse, at that density the 32-bit hashes start colliding (birthday paradox), and a collision silently drops a point, which adds error the formula doesn't predict. Their simulation showed error rising, not falling, past about 10,000 points per server on a 2,048-server ring. So they cut the point count by 90%.
The bit I'd have skipped and regretted is the rollout. A new ring means keys map to different servers, which means the cache is cold everywhere at once and every origin gets hammered. They kept both rings in memory, picked per request which one to consult, and moved data centres over one group at a time while watching origin traffic. Then they deleted the old ring path.
I've written before about how little this blog's infrastructure actually costs, in Hetzner vs DigitalOcean, and one reason it's cheap is that I never rebalance anything. If I did, "run both, migrate slowly, delete the old one" is the pattern I'd copy, and it's the same pattern that works for a database column rename.
What I'm actually going to do with this
I'm not switching anything to Pingora. The pingora-ketama crate now ships the compact six-byte point and a configurable base point count behind a cargo feature, which is nice if you're in Rust, but the takeaway for me isn't the library.
It's that I had a magic number in production for years and never asked where it came from. 160 points. I'd have told you it was "the standard". It was a default from a different decade for a different fleet size.
So here's the thing to do this week. Find one constant in your infrastructure that you inherited (a virtual node count, a connection pool size, a worker count, a max_connections), and spend twenty minutes finding the formula or the benchmark that justifies it for your numbers, not the numbers of whoever wrote the default. If you can't find one, run size_of or the equivalent on your hottest struct while you're at it. One of those two will probably surprise you. If you'd rather have someone else do the surprising, that's what I do for clients.
Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.
