The most unsettling data problem I have ever been part of was not a system that crashed. It was a dashboard that had been confidently wrong for something like two years, and had been used to make real decisions the entire time. A weekly revenue report, watched by people well above my pay grade, had a subtle flaw in how it joined two tables. It double-counted a specific category of transaction under a condition that occurred often enough to matter but rarely enough to hide. Nobody caught it, because the number always looked plausible, and plausible is the most dangerous thing a wrong number can be.
That is what makes bad data so much more corrosive than a visible outage. When a system goes down, everyone knows something is wrong and rallies to fix it. When a system produces confidently incorrect numbers, everyone believes it and calmly steers by it. The failure is silent, and worse, it is authoritative, because it arrives dressed in a clean chart with a precise figure, and precision reads as correctness even when it is precisely wrong. We had been making budget decisions on a number that had a bug in it, and the bug never once announced itself.
The root cause was mundane, which is usually the case. Two systems had slightly different definitions of what counted as a completed transaction, the report reconciled them with an assumption that had been true when it was written and had quietly stopped being true after an unrelated change, and no test guarded the output because nobody tests a dashboard the way they test code. The number was downstream of a dozen moving parts, and not one of them owned the correctness of the final figure.
Since then I treat any number that drives a decision as production code, because that is exactly what it is. It deserves validation, a clear definition of what it actually measures, and a check that screams when it drifts outside a sane range. And I have learned to be most suspicious of the reports nobody ever questions, because a figure that has been trusted for years without scrutiny is not necessarily right. It has just been wrong quietly enough that no one thought to look.
A wrong number that looks reasonable does more damage than one that looks broken, because only one of them ever gets checked.
– Serguey Shinder
