How a Dedup Pass Deleted My Training Curriculum

Code: Megapixel99/capture-the-flag

I built a capture-the-flag arena where language models attack and defend each other's containers. The replays then became training data for a small local model that plays in the same tournament. The bot is Qwen2.5-3B-Instruct with an MLX LoRA adapter (rank 8, 8 layers) trained on turns extracted from game logs. It does well on offense. Across the 221 logged games it played in, it captured 400 flags, more than any of the five cloud models on the same network in those same games.

Defense is the gap, and it is the one I kept trying to close. Gemini 3 Flash loses its flag 0.18 times per game; the bot loses its flag 0.61 times per game, roughly the rate of models it beats comfortably on offense. Two attempts to fix that were both rolled back. The v5 self-play loop regressed and was replaced by a curated mix (v6, still the active build). The v7 retrain was built specifically around weighting good defensive turns. It regressed offense more than it gained on defense, and went back to the v6 snapshot.

I thought I had tested two weighted curricula and found both wanting. Neither of them was ever weighted.

The weights are copies, and copies do not survive

MLX LoRA fine-tuning takes a JSONL file, one example per line, with no per-example weight field. The usual way to emphasise an example is to write it more than once, which is what the pipeline does. training/defense_training_data.py (on the custom-bot branch, not the repository's default master) scores every defender turn against the game's own vulnerability-check timeline and returns a weight: 6 for a turn that patched an unpatched vulnerability and kept the flag, 3 for one that patched but lost the flag anyway, 1 for harmless background activity, 0 for turns to drop. Attacker turns get 3 if that command captured a flag and 1 otherwise. The weight is applied by repetition:

ex = _make_example('attacker', thinking, cmd)
examples.extend([ex] * weight)

Then, 56 lines later, the same function ends like this:

# Dedup by last 200 chars
seen = set()
deduped = []
for line in examples:
    key = line[-200:]
    if key not in seen:
        seen.add(key)
        deduped.append(line)

The copies are byte-identical, because _make_example is a pure function of the role, the thinking text and the command. Every copy after the first has a key already in seen, so every copy after the first is discarded. The weighting stage and the dedup stage are the same operation with opposite signs, and dedup runs second.

I ran the extractor over all 358 session directories in logs/ (329 of them have a complete game.json). Once as written, and once with the dedup key replaced by a unique object so that nothing collapses:

38,988 lines, 41.5% of the weighted corpus, were the weights themselves. The weighted total is not an approximation of the intent; it is the intent, exactly:

Those six products sum to 94,022, which is what the unmodified extractor produces before its own dedup, so the weighting really did happen; it just did not last. The gold defensive turns were meant to be 18.3% of what the trainer saw (17,166 of 94,022). They ended up at most 5.7% (2,861 of 50,145), a smaller share of the corpus than the neutral turns they were supposed to outrank.

The Gemini exemplars are the sharpest version. They are 3,275 curated defensive chains from the strongest defender in the tournament, kept in their own file and pulled in at GEMINI_CHAIN_WEIGHT = 5. The comment above that line states the goal:

# Weight 5x (up from 2x) so these dominate the bot's own neutral defense
# habits in the training pool.

Raising that constant from 2 to 5 changed nothing at all. The file's 3,275 lines carry 3,187 distinct 200-character tails, so after dedup the exemplars contribute 3,187 lines whether the constant says 2, 5 or 500.

What survived was the filter, not the curriculum

One weight did survive, because zero copies of a line is a decision dedup cannot reverse. 27,386 defender turns scored 0 were never appended (6,602 of them for issuing commands against another machine's IP, which is an attack, not a defense). The graded curriculum degenerated into a binary include-or-exclude filter, and the difference between a 6 and a 1 became the difference between a 1 and a 1.

The tail-of-the-line key does one more thing I did not intend. Comparing distinct lines (55,034) against what dedup leaves (50,145), 4,889 lines were removed that were not copies of anything: 8.9% of the genuinely distinct examples. They collide because the key is the last 200 characters, covering the end of the truncated thinking text and the command. Two turns with different reasoning that end in the same command are therefore one example. Every repeat of a good command under different circumstances is exactly the sort of thing I was trying to teach.

training/self_play_loop.py has the same construction in two places: at the end of its extractor, and again where each generation's new data is merged into the cumulative pool. The v5 self-play loop was training on the same flattened corpus.

What I can check, and what I cannot

The chain that produced the training file is arithmetic all the way through. defense_retrain.py shuffles the deduped list and splits it 90/10. The train.jsonl it wrote has 38,448 lines and valid.jsonl has 4,272, which sum to 42,720, and int(42,720 * 0.9) is 38,448. All 38,448 training lines are unique, both as whole lines and by their last 200 characters. Nothing was emphasised. That file is not something a reader can check, because the session logs, the JSONL corpora and the adapters are all gitignored; the extractor, the retrain script and the per-generation stats are in the repository, so the mechanism is public even though my copy of the data is not.

What I cannot claim is that this explains the rollbacks. Both retrains were judged on game outcomes, and a correctly weighted corpus might have regressed too; the experiment that would settle it (same data, weights preserved, same evaluation) has not been run. Two smaller caveats belong here as well. My extraction run today covers more sessions than existed when v7 trained, so the ratios are the finding and the absolute counts are not comparable to that run. The session logs also do not record which adapter version was playing, and the match format changed partway through the tournament. The per-game table above is a pooled result across versions, not a comparison between them.

There is a weaker piece of evidence that the training signal was not doing what I thought, and it is public in the repo. The 11 self-play generations each record a validation loss beside the games those weights then played. Validation loss fell from 0.526 at generation 0 to 0.462 at generation 9 (0.377 at its best). Average score over the same span fell from 172.2 to 101.4, and flags lost per game rose from 2.7 to 4.7. Self-play scores are close to zero-sum between six copies of one bot, so I would not read that score column as a capability measure; the useful part is that the metric being optimised kept improving while the behaviour it was supposed to produce did not.

What generalises

If your weighting scheme is implemented as duplication, any later uniqueness pass is a silent reset of that scheme. It will not announce itself either: the pipeline runs, the loss curve looks healthy, the model trains and deploys. My extractor does report its own work, as a count of turns scored def_gold, def_patched, def_neutral, and every one of those counts is correct. They describe the intent rather than the delivery. The number worth printing is the one I never printed, the corpus length before and after dedup. A 94,022 next to a 50,145 would have ended this in one run.

The concept, research questions and system design of this project are mine, and much of the implementation was AI-assisted, as the repository's README states. This bug is a fair illustration of what that costs. Both halves are reasonable code. A weighting function that repeats examples is a standard trick, and deduplicating a corpus assembled from overlapping logs is ordinary hygiene. Nothing is wrong with either one until you notice they sit in the same function, 56 lines apart, undoing each other.