An HTTP framework advertised at 3 million requests per second never serves 3 million requests per second in production. The number is real, the measurement is honest, but it describes a scenario that does not exist: persistent connections, static response, no database, no TLS. The moment you add what a real service actually does, throughput collapses.
This article follows what the headline number becomes as it meets reality. Five walls, in order, with the numbers measured on HTTP Arena, the successor to the TechEmpower Framework Benchmarks archived in March 2026.
Where the headline number comes from
HTTP Arena tests 157 entries across 30 profiles, on a single Ryzen Threadripper, keeping the best of 3 runs. The famous "3M req/s" comes from the baseline profile: reused persistent HTTP connections, a static response of a few bytes, nothing else. It measures the cost of doing nothing, the overhead of the runtime and network stack at idle.
That profile is useful: it isolates the framework overhead. But nobody deploys a service that returns a constant over already-open connections. Every brick you add on top is a wall.
Wall 1: JSON serialization
First realistic addition: return an actual body. The profile that serializes 50 JSON objects drops throughput from 3M to 1.09M req/s, roughly a 64% cut.
The reason is simple: the body is no longer a constant. You have to allocate, encode, and write a payload that varies on every request. Serialization becomes the dominant cost, before you even touch encrypted networking or data. It is the first reminder that a leaderboard's "fastest framework" mostly measures the speed of producing nothing.
Wall 2: TLS
Adding encryption takes you from 1.09M to 850k req/s, about -20%. It is the cheapest wall on the list, which is counterintuitive: TLS is often imagined as a performance black hole.
In reality, the asymmetric handshake is expensive once per connection, but the symmetric encryption of the stream is hardware-accelerated on modern CPUs. On persistent connections, the amortization is good. TLS is not the problem you think it is, as long as you reuse connections.
Wall 3: the database round-trip
This is the real ceiling. A single asynchronous SELECT to Postgres drops throughput from 850k to 275k req/s, about -68%. Compared to the baseline, that is a factor of 11.
A single network round-trip to the database, even local, even indexed, even for one row, costs more than everything else combined. And it is the floor: most services make several queries per HTTP call, not one.
This is where the framework leaderboard flattens. A language leading by a factor of 31 on the static profile falls back to a factor of 6.5 the moment a database is in the path. The dominant cost is no longer yours, it is the I/O's. Optimizing the runtime without optimizing database access means shaving the 9% that remain while ignoring the 91% that matter. I detailed the data-access patterns in my article on the layered architecture of a Rust/Axum/SQLx API.
Wall 4: ephemeral connections
The first three walls assume reused connections. In practice, part of the traffic opens and closes a connection per request: mobile clients, misconfigured proxies, server-to-server calls with no pool. On ephemeral-connection profiles, throughput can drop by a further -75%.
TCP setup and teardown then dominate the processing time. This is the only wall where io_uring truly changes things: about 2.5x better than the classic socket API on this specific profile. But mind the trade-off: the same io_uring stack is about 19% slower on the static profile. You do not optimize for both worlds at once; you pick the load you serve.
Wall 5: the cloud provider throttle
The last wall is not technical. Even with a service capable of absorbing the throughput, managed infrastructure imposes its own limits.
An API Gateway often caps at 10,000 req/s by default, and at 2,500 across a good number of regions. And the cost follows the same logic: at 1 million req/s on a per-request REST tier, the bill reaches the order of 6 million dollars per month. The wall is no longer the CPU, it is the quota and the price. Many architectures that look "slow" on paper never hit their runtime: they hit the billing line first.
The realistic reference number
Once the walls are crossed, what throughput should you expect from a well-optimized entry? On the "mixed" profile (4 vCPU, 16 GB, with a database), HTTP Arena's best entries do 32,000 to 68,000 req/s.
| Profile | Throughput of best entries | Gap to baseline |
|---|---|---|
| Baseline (static, keep-alive) | 3,000,000 req/s | reference |
| JSON 50 objects | 1,090,000 req/s | -64% |
| + TLS | 850,000 req/s | -72% |
| + 1 Postgres SELECT | 275,000 req/s | -91% |
| Mixed 4 vCPU with DB | 32,000 - 68,000 req/s | -98% |
The "Gap to baseline" column is cumulative; the percentages quoted in the wall sections are relative to the previous step.
Going from the headline number to the realistic number is two orders of magnitude. Concretely, to sustain 1 million req/s on a real service, you need 15 to 32 replicas, that is 60 to 128 vCPUs. Not one magic process.
The pattern that works for 95% of cases
The practical conclusion is boring, and that is good news: horizontal scaling. Stateless replicas behind an L4 load balancer, one database access per request, a cache in front of that access. It scales linearly, it is debuggable, it ships without ceremony.
The extremely tuned single box is only justified in the 5% of cases where traffic is uniform and cacheable: ad bidding, telemetry, very high-volume static lookups. Everywhere else, traffic variability and the presence of a database make micro-optimizing the runtime marginal. That is also why I build my application engines around robustness rather than peak throughput, as in IronFlow.
When raw throughput really is not the problem
One documented case illustrates it well: Zalando's Red API served more than a million req/s, and the bottleneck was not raw throughput but a shared proxy in the hot path. Every batch of 100 requests traversed the proxy 100 times.
The fix had nothing to do with a faster framework. The team moved routing into the calling process (a consistent hashring, 100 virtual nodes per endpoint, a 30-second fade-in for new pods) and sized the load with Little's Law: concurrency = arrival rate × latency. The result: the proxy fleet went from more than 50 pods to 8, and the cost from 450 to 110 dollars per day. Throughput had never been the wall.
Latency versus throughput: the average trap
Last point, often forgotten: at high throughput, average latency lies. On HTTP Arena's TLS profile, at 1.5M req/s, mean latency is 19 ms but P99 climbs to 983 ms. As Marc Brooker (AWS) formalized in his analysis of the cost of the tail, requests above the P99 alone carry about half of the total mean latency.
Little's Law explains the mechanism: on the TLS profile, there are about 29,000 requests in flight at any instant, versus 600 on the static profile. That is 47 times more concurrency, at a lower throughput. The higher latency climbs, the more requests pile up, the more concurrency explodes, and the more the tail costs. Looking at average throughput without looking at the latency distribution means seeing only half the system.
What to take away
The framework sets the cost of doing nothing. That is worth knowing, but it is not the limiting factor of a real service. The database round-trip is the real ceiling, and it crushes the gaps between languages and frameworks. A 31x advantage on paper melts to 6.5x the moment a SQL query enters the path.
Before chasing the fastest framework in a leaderboard, ask the question that matters: how many database round-trips per request, and at what price with your provider? That is where real throughput is decided, not in the benchmark's static profile.
