A study presented at SC 2025, the main supercomputing conference, injected more than 13 million simulated hardware faults into language model inference and found that the standard way of measuring resilience is misleading. Aggregate accuracy barely moves under random bit flips, while the actual quality of generated text degrades substantially. The paper's blunt summary: language models "are not truly resilient to random bit-flips."
Key facts
- More than 13 million fault injections across nine benchmarks and eight model configurations.
- Average accuracy degradation was in the low single digits, with a worst case around thirteen points -- while generated-text quality fell much further.
- 4-bit and 8-bit quantized models were more robust than the same model in 16-bit brain-float precision.
- Published at SC '25 by Yu Sun, Lishan Yang and colleagues at George Mason, Oregon State, Rutgers, and Pacific Northwest National Laboratory. Paper PDF, code.
A soft error is a bit that flips for no persistent reason -- a cosmic ray, electromagnetic interference, a marginal cell -- changing a stored number without breaking the hardware. On a laptop this is rare enough to ignore. On a cluster of tens of thousands of accelerators running continuously, it happens constantly, and the standard answer has been that neural networks tolerate it. That belief comes from studies of image classifiers, which do shrug off random corruption. Nobody had checked whether it holds for models that generate long text.
The method is simulated rather than physical: PyTorch hooks flip a chosen bit during a run, the output is recorded, and the bits are restored. The fault model is explicit -- single-bit computational faults, double-bit computational faults, and double-bit memory faults. The double-bit memory case is the interesting one, because error-correcting memory already catches single-bit errors; the study deliberately targets the regime ECC misses. The sweep covers Llama, Qwen, and Falcon models across benchmarks spanning knowledge, reasoning, translation, summarization, and question answering, at 100 sampled inputs per benchmark.
The headline result is a measurement problem, not a doomsday scenario. "While aggregate metrics such as accuracy may suggest resilience," the authors write, "an in-depth inspection of the generated outputs shows significant degradation in text quality." Picture a multiple-choice exam where a student's handwriting deteriorates into gibberish but they still circle the right letters. Score the letters and the student looks fine. Read the page and something is clearly wrong. Every resilience claim built on benchmark accuracy has been scoring the letters.
The mechanism behind the gap is compounding. A corrupted weight or activation propagates through an entire output column, and then the corrupted token feeds back in as input to the next step. Generation is autoregressive, so a single flip does not produce one wrong word; it produces a drift that accumulates over hundreds of tokens. That is also why memory faults hurt more than computational ones, and why reasoning-heavy generation is the most fragile category tested, with grade-school math word problems the weakest benchmark of the set. Ground Truth's explainer on how AI picks its next word covers the feedback loop that makes small errors grow.
The practical findings are more useful than the headline. Quantized models held up better than full precision: the authors compared 4-bit and 8-bit builds of a 7-billion-parameter model against 16-bit brain-float under double-bit memory faults, and the quantized versions were more robust, because a flipped bit inside a bounded, low-precision value cannot blow up the way a flipped exponent bit in a floating-point number can. Among floating-point types, 16-bit float was the most resilient and brain-float the least. Beam search survived corruption better than greedy decoding. Chain-of-thought prompting improved robustness on reasoning tasks. Mixture-of-experts models were slightly worse on multiple choice and better on generation. See quantization and mixed-precision training for the number formats involved.
Why it matters: everyone deploying inference at scale is making an implicit reliability bet, and most of them are making it on the classifier-era evidence. This paper says the bet is wrong in a specific and fixable way -- not that inference collapses, but that the instrument used to declare it safe is not measuring the thing that breaks. It also hands operators a set of levers that cost nothing: the precision format, the decoding strategy, and the prompt style all move resilience, and the cheapest option on that list, 4-bit quantization, is the one people were already adopting for unrelated reasons.
It is worth separating this from the adversarial literature it superficially resembles. The classic 2019 bit-flip attack showed that a small number of precisely chosen flips could collapse an image classifier, and PrisonBreak later showed that 5 to 25 targeted flips can jailbreak an aligned language model, with a Rowhammer-style end-to-end demonstration. Those are attacks with a chosen target. This study is about random corruption, and the threat models behave differently: random errors degrade gradually and selectively, targeted flips break things on purpose with very few bits.
The honest caveats: this is simulation, not a physically irradiated deployment, and it runs one fault per inference at 100 inputs per benchmark, which is a controlled setting rather than a production trace. The companion code is public but still lightly used, with a handful of stars. The finding that generalizes cleanly is the methodological one -- if you are certifying a model as fault-tolerant using benchmark accuracy alone, you are not measuring what you think you are.
Originally published on Ground Truth, where every claim is checked against the primary source.
