Mercury 2.5 770 tok/s Benchmark: The Production Playbook [2026]

Originally published at kunalganglani.com — read it there for inline code, hero image, and live links.

Mercury 2.5 (from Inception) is getting tossed around the internet for a ~770 tokens/sec output speed. Artificial Analysis pegs it at 780.8 output tokens per second, ranking #2 out of 173 models in its class, with a 260k token context window and pricing of $0.25/M input and $0.75/M output (with a 90% cache discount) as of September 2026.

That headline number is exactly why the mercury 2.5 770 tokens per second benchmark is interesting. Not because humans can read 770 tok/s. They can’t. It’s interesting because once decode is that fast, everything you used to hand-wave becomes the thing that pages you at 2 a.m.

This is the playbook I’d use to turn “770 tok/s” into real production wins, without fooling myself with pretty benchmark charts.

What is Mercury 2.5?

Mercury 2.5 is a proprietary large language model served by Inception that Artificial Analysis lists as released in September 2026, with a 260k token context window and measured output speed of 780.8 tokens per second.

Artificial Analysis also reports an Intelligence Index score of 12 for Mercury 2.5 and notes the model produced 35M output tokens during their Intelligence Index evaluation run. That combo tells you what this thing is.

It’s not “frontier reasoning at warp speed.” It’s “very fast generation at roughly median intelligence.” Treat it like a performance tool, not a magic brain.

A clarification you need before you optimize anything:

• “770 tok/s” is decode throughput. It’s how fast the model emits tokens once it has started generating.
• It is not TTFT (time to first token).
• It is not end-to-end response time (prompt building, retrieval, network hops, client rendering, middleware, all the boring stuff).

Mental model: at ~780 tok/s, a 500-token answer is about 0.64 seconds of decode. If your user is still staring at a spinner for 2.5 seconds, the model isn’t your problem.

Speed vs latency vs end-to-end response time (and why 770 tok/s flips the bottleneck)

When teams celebrate tok/s, they’re usually celebrating the part they feel powerless over. Model decode feels like physics.

But once decode is fast, your system is the limiter. Every tax you’ve been ignoring suddenly shows up in p95.

Artificial Analysis breaks this into the three metrics you actually need: speed, latency, and end-to-end response time. That framing is right. At ~770 tok/s, it’s non-negotiable.

Here are the failure modes that routinely show up in production AI once you crank throughput:

1. Prompt assembly becomes the main event. Your “simple prompt” is usually system + policies + few-shot + tool schema + user + RAG context. Joining strings is cheap. Fetching, reranking, formatting, and validating context is not.
2. Network and TLS overhead stop being rounding errors. If you’re proxying through API gateway → auth → model router → vendor, you can burn 100–300ms before the model even sees the prompt.
3. Client rendering becomes the bottleneck. If your front end reflows the DOM for every token, you’ll drop frames. At hundreds of tok/s, “typewriter streaming” turns into a self-inflicted DoS.

This is also where benchmarks start lying by accident. If your harness is doing tokenization, JSON parsing, or log formatting on the hot path, you’re measuring your own client, not the model.

Based on the benchmark tooling I maintain at kunalganglani.com/llm-benchmarks, the most common mistake is mixing single-stream tok/s with under-load p95 end-to-end latency and labeling both as “fast.” They’re different metrics. They fail for different reasons.

How fast is “fast enough” for humans?

There’s a vibe in the Hacker News discussion around Mercury 2.5 that I agree with: cool flex. Now tell me when it matters.

A lot of people throw out “below 1000 tps feels instant.” That’s not the useful threshold.

The useful framing is:

• If you’re streaming token-by-token, you hit a UX ceiling way before 770 tok/s.
• Past a point, users prefer an instant partial answer and then an expansion, not a firehose of text.

I’m not going to play the “average reading speed” trivia game because the numbers are mushy and depend on the content. You don’t need an exact stat to see the mismatch. Even if your user reads 5–10 tokens/sec, 770 tok/s is two orders of magnitude faster than their eyes.

So if Mercury 2.5’s decode is actually that fast in your region under real concurrency, your job changes from “speed up the model” to “stop wasting speed on output nobody can consume.”

The mercury 2.5 770 tokens per second benchmark playbook: batching, streaming UX, and context packing

This is where the tradeoffs flip and a lot of teams make things worse by trying to make them better.

Batching strategy: queueing delay is your new tax

With slower models, batching feels like a free win. Better GPU utilization, higher throughput, everyone’s happy.

With ultra-fast decode, aggressive batching can absolutely murder tail latency.

If Mercury 2.5 can emit ~780 tok/s, the decode phase for short answers might be hundreds of milliseconds. Any queueing delay you introduce with micro-batches is suddenly a meaningful chunk of user-perceived latency.

My rules of thumb:

• Interactive chat: optimize TTFT and tail latency. Small micro-batches. Sometimes per-user streams if you can afford it.
• Background work (summaries, classification, agent runs): chase throughput. Continuous batching and larger batches usually win.
• Mixed workloads: separate queues. Do not let “batch-hungry background work” sit in front of user-facing traffic.

When I was building the Walmart conversational commerce chatbot at Firework (Zealsight), we hit sub-second responses at millions of queries daily. The biggest wins were almost never model-side. They were queue-shape and pipeline wins. We got more by making the context pipeline event-driven with Kafka than by obsessing over model parameters.

At 770 tok/s, that lesson gets harsh. You can buy decode speed. You can’t buy your way out of bad queueing discipline.

Streaming UX: typewriter mode breaks above ~500 tok/s

If your UI literally appends text as each token arrives, a faster model will give you a worse product.

At high throughput, you want chunked streaming:

• Buffer tokens for 50–150ms windows, then flush a chunk.
• Use adaptive pacing. If the model is producing too fast, slow the stream to a human cadence while keeping backend generation running.
• Consider “instant answer + expand.” Ask for a 1–2 sentence direct answer first (cap output around 80–120 tokens), then stream the longer explanation behind a disclosure.

This is one of those things where the boring answer is actually the right one. Your streaming protocol should serve the human, not the GPU.

Context window: 260k tokens changes how you pack prompts (and how you blow up caches)

Artificial Analysis lists Mercury 2.5 at 260k context. That’s huge. It’s also an invitation to get lazy.

What changes:

• You can pack multiple tasks into one request. That can reduce per-request overhead and boost throughput.
• You can do heavier retrieval-augmented generation with more source text, but you still need discipline. Bigger context isn’t automatically better. Irrelevant tokens are still irrelevant. You just pay for them more quietly.
• Prompt-prefix caching gets way more valuable. With a 90% cache discount listed for Mercury 2.5, repeated prefixes are basically screaming “cache me.”

What gets worse:

• Multi-tenant serving plus huge contexts can explode KV cache footprint. Even with a hosted API, you still eat the latency and throttling consequences.

If your app is RAG-heavy, connect this to your architecture decisions. I’d start with my post on retrieval-augmented generation and the practical limits in RAG context window limits. A 260k context window can reduce retrieval calls. It can also help you ship a slow, expensive mess with a lot of false confidence.

Benchmarks, cost, and eval harnesses that don’t lie at ultra-high throughput

Most teams will paste the Mercury 2.5 tok/s number into a slide and call it “done.” That’s how you end up optimizing the wrong thing and congratulating yourself for it.

Benchmarks: validate saturation curves, not a single number

If you’re evaluating Mercury 2.5 (or anything else), don’t ask “what’s the tok/s?” Ask:

• What’s tok/s at concurrency 1, 8, 32, 128?
• What happens to p95 end-to-end latency as you increase concurrency?
• Where does it saturate in your region, with your prompt shapes?

The HN thread already shows the right kind of skepticism. People are asking whether it’s a smaller model, whether it’s aggressively quantized, and whether it’s actually useful outside a demo. Keep that energy.

Also, compare fairly. Same prompt, same max_tokens, same region, same provider routing, same client, same retry policy. Otherwise you’re benchmarking your own stack, not the model.

If you want a checklist mindset for harness overhead, my related piece is How to Measure AI Coding Agent Harness Overhead [2026]. Different domain, same principle: measure the harness, or the harness will measure you.

Cost: fast models increase spend when you stop paying attention

Artificial Analysis lists Mercury 2.5 at $0.25 per 1M input tokens and $0.75 per 1M output tokens. That’s not outrageous.

The trap is behavioral.

When generation is slow, product teams naturally keep answers short. When generation feels instant, they let it ramble. You end up paying for tokens nobody reads.

Do the basic math:

• A 1,000-token output costs $0.00075 at $0.75/M output.
• At 780 tok/s, you can generate that in ~1.28 seconds.

Now scale it. If you serve 10 million responses a day and you accidentally allow an extra 300 tokens of fluff per response, that’s 3 billion extra tokens. That’s $2,250/day in output tokens. For fluff.

This is why I keep pushing teams to think in LLM cost per task, not per-token pricing. I wrote this out in more detail in Agent per-task cost calculation and AI agent cost per task.

Artificial Analysis even gives you a sanity anchor: Mercury 2.5 shows $0.06 cost per Intelligence Index task on their eval. Use it. If your “simple customer support answer” is costing you $0.06, something is broken.

Evals: what breaks when tok/s is very high

At very high throughput, eval harnesses fail in dumb ways:

• Timer resolution: if per-request decode is <1s, sloppy timing code creates huge relative error.
• Client-side overhead: tokenization, JSON parsing, logging, tracing. All of it can become the bottleneck.
• Rate limits and retries: a fast model makes it easier to slam into limits. Your p95 becomes “retry storm p95.”

If you care about evals in production, wire them into tracing and treat them like any other distributed-systems measurement problem.

I’ve been using trace-first thinking in my own tooling work on this blog, and it maps closely to the approach in MLflow LLM evaluation tracing and LLM observability metrics.

Don’t ignore safety either. Faster generation increases the velocity of harmful output. If your moderation only runs after completion, you can stream a lot of bad content before you ever check it.

At 770 tok/s, you need streaming moderation, circuit breakers, and sampling strategies that don’t drop the exact logs you’ll want during an incident. If you’re building AI security controls, start from The Complete Guide to AI Security in 2026 and the concrete patterns in LLM data leakage playbook.

My prediction: by mid-2027, “tokens per second” will matter less as a headline, and “end-to-end answers per dollar at p95 under load” will decide winners.

If you’re adopting Mercury 2.5-class throughput today, the teams that win will be the ones that treat speed as an architecture change, not a brag.

Originally published on kunalganglani.com