Over the past year I have been fine-tuning open vision-language models - 9B dense up to a 35B mixture-of-experts - with supervised fine-tuning and GRPO-style reinforcement learning on verifiable rewards. Most of what I learned was not about algorithms. It was about the ways a training run can look healthy while doing nothing, or crash for reasons that have nothing to do with your code.
Three failures, in increasing order of how long they fooled me.
Failure 1: the metric that measured the wrong thing (18 hours)
I ran an 18-hour supervised fine-tune that reported token accuracy climbing steadily to 99%. Looked like a textbook run. The real evaluation metric - accuracy on multiple-choice questions - never moved.
The cause was a mismatch between what I supervised and what I evaluated. The training loss was over free-text reasoning traces; the evaluation scored a single extracted answer letter. The model got extremely good at reproducing the shape of the training text - hence 99% token accuracy - without that transferring to the decision I actually cared about.
Token accuracy is a proxy, and proxies drift from the target exactly when you stop checking. The fix was structural, not a hyperparameter: supervise the thing you evaluate. If the deliverable is a constrained answer, the training signal has to reach that answer, not just the prose around it.
The general rule I took: any training metric that is not your evaluation metric is a hypothesis about correlation, and you should check that correlation before you spend GPU-days on it.
Failure 2: the crash that was two libraries disagreeing about position ids
The GRPO trainer for the 9B vision model crashed in the forward pass, deep inside rotary position embedding code. Nothing in my training code had changed.
The diagnosis took a while because the bug lived at the boundary between components: the text sequence length was derived from token-type ids, while the vision sequence length came from the image grid - and image-pad tokens ended up counted twice. Two parts of the same stack, each internally consistent, disagreeing about how long the input was.
For the 35B MoE variant of the same family, an equivalent rope bug was fixable by monkeypatching the model's position-id computation. I shipped the patch with a GPU-free regression test: a tiny script that constructs the exact failing input shape and runs just the position-id path on CPU. It runs in seconds, needs no cluster, and fails loudly if an upstream update reintroduces the bug.
Two lessons. First, when you fine-tune at the edge of a model family's tooling support, the bugs you hit are integration bugs, and the stack trace points at the victim, not the culprit. Second, every monkeypatch deserves a regression test that costs nothing to run - otherwise the next library upgrade silently un-fixes it.
Failure 3: the RL loop that was learning the opposite (the quiet one)
In a separate project I fine-tune a 9B model with reinforcement learning where the reward comes from realized real-world outcomes rather than a labelled dataset. For a long stretch the training signal was flat - not diverging, not collapsing, just flat, which is the least informative failure there is.
Two compounding problems. One was label noise in the reward pipeline: some outcomes were being attributed to the wrong decisions, which dilutes any gradient. The other was worse: a sign error meant part of the advantage signal was inverted. The model was being gently pushed away from behaviour that had worked.
Nothing crashed. Every batch processed. Every log line looked like a training run. The only symptom was the absence of learning, and the only way I found it was working backwards from "the held-out metric should have moved by now" to auditing every stage of the reward computation by hand.
After both fixes I got the first genuinely monotonic learning curve on that task. I still treat it as training signal only - the decider is held-out evaluation against the base model, and I do not report improvements that exist only in the training curve.
The harness rules I now run everything under
These came out of the failures above plus a benchmarking programme across 70+ vision-language models. They are boring, and they are the difference between numbers and noise.
Smoke test before committing compute. A five-step GRPO run with two numbers watched: the PPO-style clip ratio and the fraction of outputs that parse. If the clip ratio is degenerate or parseability is low, the full run will be garbage in a way five steps already reveals.
Runs are gated fail-closed. Nothing publishes a result unless the evaluation stage actually scored. "The eval crashed but training finished" is not a result; it is an unscored run, and unscored runs must be impossible to mistake for scored ones.
Infrastructure failures and poor performance are different columns. An unparseable output, an OOM, a crashed kernel - these are exceptions whose count must be exactly zero. A weak model produces zero exceptions and simply scores badly. If a quality threshold can absorb an infrastructure failure, a totally broken run can pass your gate.
The held-out metric is the only decider. Training curves, token accuracy, reward trends - all of it is telemetry. If the held-out number did not move, nothing happened.
None of this is novel. All of it is the difference between the runs I trust and the 18 hours I lost.
I write about ML evaluation, world models, and the ways measurement quietly fails. More at dev.to/rickeshtn.
