If you build on hosted AI APIs, two questions quietly decide your architecture: how fast is provider X from where your users actually are, and how long will the model you pinned actually live. When I went looking for measured answers, I found single-region benchmarks, vendor status pages, and numbers measured through gateways that attribute the gateway's problems to the provider. So I built an independent tracker.
How it works (boring on purpose)
- A small prober opens a real connection to each provider's official API endpoint every 5 minutes and times every phase of the handshake: DNS → TCP → TLS → time-to-first-byte. Separately, where possible, it measures inference time-to-first-token.
- It runs from 4 regions: Germany, US Central, Tokyo, São Paulo. Requests go directly to providers — no gateway, no aggregator, so nothing gets attributed to the wrong party.
- 45 providers: the usual Western labs (OpenAI, Anthropic, Google, Mistral, Groq, Together, Fireworks, Cerebras, DeepSeek…) plus Chinese ones (GLM/Zhipu, Kimi/Moonshot, Qwen, MiniMax) that rarely show up in Western benchmarks.
- Results land in SQLite; a static site regenerates every 30 minutes. 38 days in: just over 2,000,000 probes.
What the data shows so far
1. Region beats provider. Over the same 7 days, on identical requests to identical endpoints, probes from Tokyo failed 0.315% of the time vs 0.089% from Germany — a 3.5× gap. A monitor that probes from one location cannot see this, by construction.
2. Uptime numbers lie by omission of the window. Over any 24-hour window almost every provider reads ~100%. Over 7 days, 40 of 45 providers dropped at least one probe. Same fleet, different window, opposite conclusion — availability quoted without a window length is not comparable to anything.
3. Availability doesn't differentiate providers; latency does. Right now the fastest p50 TTFB in the fleet is ~99 ms and the slowest is ~1424 ms — a 14× spread between endpoints whose availability is statistically indistinguishable.
4. How much warning you get before a model is killed. Computed from the providers' own announcement and shutdown dates across 203 dated retirements — a number nobody publishes, including the providers themselves:
| Provider | Retirements tracked | Median notice | Shortest |
|---|---|---|---|
| OpenAI | 133 | 182 days | 3 days |
| Mistral | 40 | 92 days | 0 days |
| Anthropic | 19 | 63 days | 60 days |
| Cohere | 11 | 0 days | 0 days |
Cohere's announcements land the day the model dies. A pinned model version can stop working the day you read the notice — that's a planning constraint, not a judgement.
The part I care most about: agent-native by design
The way people ask questions is shifting from search boxes to answer engines and agents. If your data only exists as JavaScript-rendered HTML, an agent can't reliably consume or cite it. So every page has a markdown twin, there's llms.txt and llms-full.txt, a JSON API, and an MCP server. Two observations after five weeks:
- ~23% of all crawler fetches take the markdown layer, not the HTML.
- "A human asked their AI assistant to open the page" is now a bigger channel than browser clicks from AI answers — I can tell them apart server-side, and the agent-initiated reads outnumber referral clicks by an order of magnitude.
Honest limitations
This measures endpoint reachability and TTFB/TTFT — not model quality, not rate-limit behaviour, not incidents visible only inside a provider. Four vantage points is not the whole internet. The dataset is 38 days young, and the incident log deliberately keeps a short window because two coincidences don't make a trend.
Everything is free, no accounts, no ads: llmlatency.dev — the deprecation calendar is at /deprecations, measured uptime at /best-uptime, method and limits at /methodology. Data is CC BY 4.0.
What's missing? What would you measure next?
