I saw a chart on Twitter ranking cities by cost of living against quality of life. New York and Los Angeles were sitting near the bottom of the quality axis, below places I could barely put on a map.
That is not what you expect to see. I did not believe it, and the fastest way to settle that kind of argument is to pull the source tables and rebuild the thing yourself.
It took a lot longer than I thought. Not the chart — the chart is the easy part. Collecting the data and then checking it is where the time went, and the interesting failures are all in there.
Three sources, one circle
The rebuild uses three open sources for 303 cities.
Numbeo, mid-year 2026 release — nine metrics: safety, health care, commute time, pollution, climate, purchasing power, property price to income, and two cost-of-living indices. I deliberately took a fixed half-year snapshot instead of the live "current" numbers, so a reader coming back next month sees the same chart I described.
OpenStreetMap, August 2026 snapshot — everything Numbeo does not measure, counted inside a circle of 10 km around each city centre: museums, galleries, theatres, cinemas, bars, pubs, clubs, rail stops and tram stops, pedestrian streets, playgrounds, parks, schools, libraries, universities. The circle is identical everywhere, 314 km², which means a raw count is already a density and I never need population data.
OurAirports plus Wikipedia — the number of unique nonstop destinations from every airport within 60 km, merged and counted once.
Everything is scripted, which does not make it fast: thousands of requests, then a reconciliation pass, then the part nobody budgets for — checking that the numbers are actually about the city you think they are.
What broke
The geocoder returned an administrative centroid. For Cebu and Gdansk it handed back the centre of the administrative unit rather than the city, so the 10 km circle landed in a field somewhere and every OSM metric for those cities collapsed to almost zero. This one caught itself: a city can lose one metric plausibly, but it cannot lose all of them at once. Coordinates for those two are now pinned by hand.
Diacritics failed silently. Wikipedia page titles with accented characters broke the request without raising anything, so Prague, Krakow, Atlanta and Minneapolis each came back with zero nonstop destinations. A zero looks like data. That is what makes it dangerous — a crash you notice, a plausible number you do not.
A dataset field contained something entirely different. In OurAirports, the wikipedia_link field for Dhaka's Hazrat Shahjalal International Airport holds a link to somebody's LinkedIn profile. The parser followed it, found no article, and moved on with nothing. The fix is a fallback that searches for the airport by name — but the lesson is broader: in an open dataset that anyone can edit, a field can contain anything at all, and your code should treat "wrong type of content entirely" as a normal case rather than an impossible one.
The cache poisoned the retry. Empty results were written to the cache before the fallback path ran, so the fallback then read the empty result back and confirmed it. Caracas got permanently stuck on a disambiguation page this way. Cache successes, not failures.
Same-name cities shared a key. London, Ontario overwrote London, UK. San Jose, California overwrote San Jose, Costa Rica. The key is now city plus country, and both twins carry the country in their label.
Recounting flights afterwards changed the typical city by about 4%, but it fixed the ones that were badly wrong: Charlotte went from 15 destinations to 200, Dallas from 94 to 271.
The formula was harder than the code
The first version of the score dropped purchasing power and price-to-income entirely. Numbeo's own quality-of-life index includes both, and if you leave money on the vertical axis while cost of living is on the horizontal one, the chart mostly measures itself: rich cities score high by construction.
I have since put both back — as options, not as my decision. They do measure something the cost index does not: what a local salary actually buys, and how many years of income a flat costs. That is a different question from what a basket of groceries costs, and it belongs to whoever is reading, not to me. It is off by default and there is a warning attached to it, because it is also the softest number in the set.
Three things people asked for and did not get. Bureaucracy — I have no idea how to measure it honestly, and a number I cannot defend is worse than a missing column. City size — that one is a fair request and it is on the list. Trends over time — this is a genuinely large collection problem: the historical series are scattered, they need reconciling and normalising across sources, and I do not think the result would pay for the work. Feasible for one city, miserable for 303.
What I left broken on purpose
Every weak spot is written on the page instead of quietly smoothed over.
Numbeo's purchasing power is self-reported by whoever fills in the form, and the sample skews to expats and urban professionals. Hyderabad comes out second in the world, above Zurich, while Mumbai sits at 63 — a fourfold split inside one country is a sampling artefact, not an economy.
In France a "college" is a school for eleven-year-olds, so Paris reports 407 of them against 149 universities and French cities look over-educated. Playgrounds are mapped far more densely in Europe than in Asia, and I could not separate real difference from mapping habits. Kindergartens were dropped entirely for the same reason: the tag is barely used in the US, and Europe came out thirteen times more child-friendly than North America.
There is no correct weighting
That is the actual conclusion, and it took a few hundred comments to make me build the obvious thing.
The chart exports itself with your settings written underneath, so any version of it can be shared as an image.
Any index like this is a weighted sum, and the weights are a value judgement wearing a lab coat. One person wants clean air, another wants a bar within walking distance, a third only cares that it is warm in February. So the twelve metrics now have twelve sliders, and the vertical axis measures whatever you turned up. The whole configuration lives in the query string, which means the chart you built is a link you can send to someone.
My own setup weights culture and car-free living highest, then affordability and safety. The city that comes out on top is Tokyo — expensive, and still worth it on my weights. I did not expect that, and I would not have found it with anyone else's formula.
The chart is here: https://avgrebenkin.com/research/cities/
