Flash Onyx 2.2: teaching a local model law and game feel

(I know, ANOTHER Onyx post)

Flash Onyx is the model line behind FLASH, the local-first agent shell I work on. Onyx 2 is gemma4 with a system prompt that turns it into an engineering agent: read the files, make the change, run the thing, report in as few words as the truth allows. It runs on my 16 GB laptop, fully on the GPU, at single digit tokens per second. Slow, but it is mine and nothing leaves the machine.

2.2 is where I started pushing it outside code more.

Law

Ask most local models a legal question and you get one of two bad answers. Either a wall of "consult a professional" that helps nobody, or a confident paragraph with a case citation that does not exist. The second one is worse, and it is the reason lawyers keep getting sanctioned for filings that were obviously written by a chatbot.

So the LAW section is mostly a set of refusals to guess:

• Jurisdiction first, because the same facts land differently in California, New York, and England.
• Never invent a citation. No case name, reporter cite, section number, or docket it has not actually read.
• Deadlines before the analysis. A good claim dies on a missed limitation period, so a running clock goes in the opening line.
• Keep three things apart: what the rule says, what a court will likely do with it, and what happens in practice.

There is a companion CONTRACTS AND DOCUMENTS section too, since that is what people actually have in front of them. Whose paper is it, whose interest does each clause serve, rank the findings from deal-killer down to noise, and remember that silence is a term. The missing termination right is what the argument ends up being about.

Games

Different problem. A model that writes a game usually writes something that technically runs and feels like cardboard.

The GAMES section is almost entirely about feel:

• A player decides in ten seconds of holding the controls, so movement and feedback come before content or story.
• Fixed timestep, clamped accumulator, steps near 16ms. Physics tied to frame rate runs at double speed on a 120Hz screen.
• Input is polled, not handled. keydown sets a flag, the loop reads flags.
• Fairness is small lies: 100ms of coyote time, 150ms of jump buffering, a hitbox tighter than the player and looser than the pickup.
• Nothing allocates in the loop, because a garbage pause reads as a stutter and players feel it before they can name it.

Then I cut both of them

Here is the part that surprised me. On a local model, every line of system prompt is prefill you pay for on the first token of every conversation. Writing more is not free the way it is with a hosted model.

So both sections got compressed the day after I wrote them. Law went from 629 words to 481. Games went from 401 to 312. Not by deleting rules, by deleting the second sentence that explained the first one. Every rule survived. About a quarter of the tokens did not.

That is now how I write every section.

Then I tuned the sampler

Prompt work is only half of it. The other half is sampling, and I had never actually measured mine. So I scored the model against its own rules: five short prompts, three configs, two seeds, thirty generations.

Nothing broke a hard rule at any setting, which was the reassuring part. The averages hide the interesting one. Asked for a single shell line listing the 20 largest directories, the old settings wrote the command and then a 43-word paragraph explaining du, which its own throwaway-script rule forbids. At 0.6 it wrote the command and stopped. At 0.5 it also stopped, but reached for du --max-depth=1, a GNU flag this Mac does not have, breaking a different rule in the same prompt.

So temperature went 0.7 to 0.6, and min_p 0 to 0.05.

Then context, where I had been timid. num_ctx was 32768 because 16 GB felt tight, and I had never checked the actual cost. Probes at 32k, 64k and 128k came back at 8.1 GB, 8.1 GB and 8.2 GB, all fully on the GPU. The window had been nearly free the whole time. It is 65536 now, and I stopped there because the binding limit is not memory, it is prefill: at roughly 95 tokens a second, a filled 128k window is 22 minutes before the first token.

What's next

More domains, written and then cut the same way, and measured against the prompt instead of eyeballed.

The build tooling grew up a little too. models/build.py fills the model name into the prompt from the header, and injects the repo license into every model it builds, so ollama show --license tells you what you actually have.

Onyx 2.2 is not published yet. When it is, it will be Natuworkguy/flash-onyx-2.2 in 12b and 31b.