I A/B tested my own system prompt: 24 generations, one clear win, one rule that did nothing | Flash Onyx 2.2

Flash Onyx is the model line behind FLASH, the local-first agent shell I work on. It is gemma4 plus a system prompt that turns it into an engineering agent. It runs locally, and nothing leaves the machine.

FLASH is the shell around it: a terminal chat with an Ollama model that can run commands, read images, and screenshot a page it just built so it can see a broken layout instead of guessing at it from the HTML. Onyx is the model I point at it, and the system prompt is most of what makes that work.

2.2 got two things this week. The context window doubled, 32768 to 65536, once I measured what a window actually costs instead of rationing it on a feeling. And the prompt grew five new sections, which is where the new capability lives: how it reads a request, what it names files and what it builds them in, the standard it holds finished work to, and what it offers you next when that work is done.

Last time I wrote about teaching it law and game feel. Four of these five are not about a domain at all. They are about what it does with any request you hand it.

It wrote the file, then pasted the file back

This one came from watching it, not from planning it.

Ask it to build something and it does the right thing: writes the artifact to disk, names the path. Then it pastes the entire file into the reply as well. Both halves are correct on their own. Together they cost me the same 300 lines twice: once on disk, once in the context window the rest of the session still has to fit into.

On a hosted model you would not feel it. On a local one, a file pasted back is the difference between the conversation continuing and the conversation ending. The prompt already said the artifact belongs at a path. It never said the artifact does not come back afterwards.

So, into TOOLS:

Asked to make something, make it on disk, name the path, and stop. What you just wrote does not come back in the reply: it spends the context that writing it saved, and they can open the file. Quote one line to point at it, paste the whole thing only when they asked to see it.

And a clause in the closing recap, because that block is the part a 12B actually holds onto: wrote a file, the reply names the path and never repeats the contents.

READING THE ASK

The way a small model loses to a big one is almost never knowledge. It is that it answers a question adjacent to the one you asked, or quietly drops your third sentence.

The prompt already had sections on ambiguity, on scope, and on reporting multi-part work. It had nothing about the first read of a request that is perfectly clear. That is now READING THE ASK:

• The request is the spec, and most bad answers are good answers to a nearby question.
• Answer at the altitude asked. "Is this safe to deploy" wants a yes or a no and the reason. "Walk me through the auth flow" wants the walk.
• A question about work is not an instruction to do it. "How would you handle this" gets an answer, then one line offering the move.
• The reverse costs more. "Can you fix the flaky test" is a fix request, and replying with an assessment of the flaky test is how a turn gets wasted politely.
• Do the hard part. Most requests have one piece that decides whether the whole thing works and several that are typing.
• Last pass before sending: read the reply against their words, in their order, not against the plan you made after reading them.

NAMING AND STACK

Two habits, one section.

First, the names. Ask for a web Minecraft clone and you should get minecraft_clone.html, not output.html or index2.html. The file name is the only label most files ever get, and the repo's existing convention beats the model's taste every time.

Second, the stack. If nobody named a framework, picking one is the job, not a question to hand back:

• Quick web game, toy, demo, one screen: a single .html file, canvas, plain JS, no build step, opens by double-clicking.
• Real state, routing, a dozen components: React on Vite. A framework under forty lines of vanilla is ceremony, and hand-rolled routing past that is worse.
• A production game gets a window, not a tab. Godot, Unity where the team already lives there, pygame for a Python desktop toy, raylib or Bevy for native.
• A one-off over data is Python and the stdlib. pandas earns its import when the shapes get real, never for 200 rows.
• A service is the boring answer: FastAPI or Flask, SQLite until something actually forces Postgres.

Above all of them: what the project already uses beats what the model would have picked. A second framework in one repo costs more than the better framework saves.

THE BAR

FINISHING already covers not lying about being done. Nothing covered the standard the thing itself has to meet, which is where local models stop early: they ship whatever works on the demo path.

The lines I like most from it: the last ten percent is the whole difference and it is cheap, a --help that says what the tool does, an error naming the fix, a page that survives 320px. Nothing half-wired, because one dead button teaches the reader to distrust every other one. Defaults are the product, and a setting whose job is rescuing a bad default is a bad default with a manual. Use it once the way they will, from cold, on a machine with none of your state.

The two domain ones

The morning was less interesting. WINNING ARGUMENTS: state the other side at its strongest before answering it, concede what is true, win on the single point that decides the question, then stop, because restating a point that landed reads as doubt. STOCK TRENDS: a trend is structure and not a slope on two points, volume confirms or it does not, check the higher timeframe first, never call a top, and with no live feed say so and label the last-known price as stale instead of dressing it up as today's.

What it cost

The prompt went from 12,048 words to 13,974 across 59 sections, and the fixes below pushed it to 14,429 across 60. In tokens, measured cold rather than guessed at: 16,196 before, 19,209 after. Prefill runs at about 95 tokens a second here, so the whole prompt is 201 seconds before the model says a word, against 182 for the old one. Three thousand tokens of new rules, 19 seconds added to the first token of a fresh conversation, and nothing after that, since the prefix stays cached for the rest of the session.

The sampler underneath all of this

The parameter change landed the night before, at the bottom of the same file: temperature 0.7 to 0.6, min_p 0 to 0.05, num_ctx 32768 to 65536. top_p 0.95, top_k 64, repeat_penalty 1.05 and repeat_last_n 256 all stayed where they were.

The first two came out of the scoring run in the last post. The window went up because I finally measured it instead of guessing: 32k, 64k and 128k all sat around 8.1 GB, fully on the GPU, so the context I had been rationing was nearly free the whole time. I stopped at 65536 because the binding limit is prefill time, not memory.

Every number below was generated at those settings, untouched.

Then I measured it

Six prompts, two seeds, run against both the new build and one built from the commit before the first of these sections. 24 generations, capped at 320 tokens with thinking off, sampler untouched. Each prompt aims at one of the new rules.

The win is real and it is the one I care about most. Asked how it would handle rate limiting, the old build started writing the middleware. The new one answers the question and offers the move, which is exactly the line in READING THE ASK about a question not being an instruction. Across all six cases it also got 17% shorter without dropping anything I asked for.

Three things failed, and the fixes turned out to be more interesting than the failures.

Naming did nothing. Not one of the four browser-clone replies named a file, in either build. Part of that is my harness: I am hitting the Ollama API directly, so there is no write tool for a naming rule to attach to. But the rule was also in the wrong place. It lived in NAMING AND STACK, a section about what to build things in, so it never reached the moment the model formats a code block. I moved one line of it into STYLE RULES, directly under the rule about fenced code: a block holding a whole artifact gets its file name on the line above it. Same instruction, 200 lines earlier, sitting beside the thing it governs. Five seeds out of five now open with minecraft_clone.html.

The one-off regression had a cause, and it was my fault. THROWAWAY SCRIPTS says a quick script gets no main() and no if __name__. Eighteen lines further down, PYTHON said to keep import time free of side effects and put the work behind if __name__ == "__main__":, with nothing scoping it to code that gets imported. Two rules in the same file telling it opposite things, and the sampler picked a winner per seed. I scoped the Python line to importable code, and the case went from breaking on both seeds to clean on five.

pygame for a game you sell. The stack rule already said a production game gets a window, not a tab. It kept losing to PYTHON opening with "your strongest language", so I added the tie-break: being best at Python is not a reason to start a Steam game in it. That moved it from pygame every time to Godot on four seeds of five. The fifth still talks itself into pygame as a prototype "that can be ported later".

Then I made it worse, which is the part worth writing down. I added one more sentence to the pygame rule, closing the prototype loophole. The next run came back 16 of 20 against 19 of 20 for the build before it. The sentence bought nothing on the case it was aimed at, still 4 of 5, while the one-off case fell to 3 of 5 and the closing-offer case to 4 of 5. I reverted it and reran: 18 of 20.

One sentence, in a prompt already past 14,000 words, and three unrelated rules got noisier. That is the whole argument for cutting, and it is the first time I have watched it happen instead of assuming it.

125 generations across seven rounds, all of it on the 12b.

The paste-back fix is still untested. It only fires when there is a write tool to name a path with, so it needs an eval that drives FLASH itself rather than raw completions, and that is the next harness to build.

FLASH installs in one line, if you want to try any of this on your own machine:

curl -fsSL https://flashproject.dev/install.sh | bash

The current model is Natuworkguy/flash-onyx-2 on Ollama. 2.2 is still unpublished, and I am planning to push it today in 12b and 31b.

Repo, issues, and the whole system prompt in one file: github.com/Natuworkguy/Flash.