Your Benchmark Row Never Saved the Driver Version

On 2026-08-27 I tried to attach a driver version to a benchmark row from June. I could not do it. The row does not carry one, and neither does any other row I have saved.

The short answer: Seven benchmark CSV files in my RTX 5090 notebook hold 17 rows, and none of the seven has a driver column. Two of the seven record a capture time. Three files carry a date and nothing finer. Two files carry neither a date nor a capture time. A row that did not save its own environment cannot support an environment claim later, and nvidia-smi output from 2026-08-27 is not a substitute.

Canonical URL: https://bmdpat.com/blog/local-llm-benchmark-row-missing-driver-version-2026

What did the file audit find?

I read every CSV in the RTX 5090 benchmark notebook on 2026-08-27 and checked the headers, not the numbers.

Seven files. Seventeen rows. Zero driver columns. Two files carry a captured_at field. Three files carry a date column and nothing finer. Two files carry neither. The header widths run from 9 columns to 17, so the files do not even agree on what a row is.

The engine column is the part that surprised me. One runtime appears under four labels across the seven files: Ollama 0.22.1, Ollama 0.31.1, ollama+code_verifier, and plain ollama. Two of those four hold a version. Four files use the bare word.

Why does a missing driver column matter?

I wrote earlier that a GPU driver is not a local LLM benchmark. That post argued a driver number printed in a report header does not show the driver changed model speed. This is the hole underneath it.

The report header is written when the report runs. The row is written when the test runs. On 2026-08-27 the header would say driver 610.88, because that is what nvidia-smi prints on this box right now. The June row ran on whatever driver was installed in June, and nothing recorded it.

So the header number binds to the report date and to nothing else. Copy it onto an older row and you have invented a fact.

Which version is the runtime actually running?

I ran ollama --version on the same box. It printed ollama version is 0.31.2 and then warned client version is 0.32.1.

Two answers, one machine, one command. On 2026-08-27 the server answering requests is on one version and the CLI I typed is on another. Two of my seven files did record a version, and they read Ollama 0.22.1 and Ollama 0.31.1, so at least three server builds have touched this notebook. A column that says ollama cannot tell those apart, and the difference is the kind that moves results. Runtime releases change defaults, and I keep a compatibility check between Ollama and llama.cpp for that reason.

Record the server version. The client version is the one you can read most easily and the one that did no work.

What four fields does every row need?

Four fields, written by the collector at capture time, never typed by hand later:

1. Runtime version at capture. The server build that answered the request.
2. Driver version at capture. Read from nvidia-smi in the same run, not the same week.
3. GPU name at capture. My fleet has more than one card, so the row must say which one.
4. Capture timestamp with timezone. A date is not enough when a driver updates midday.

None of these needs a new tool. Each is one command in the collector that already writes the row.

What do I do with the 17 rows I already have?

Do not backfill them. Writing 610.88 into a June row would make the file look complete and make the record false.

Mark them instead. I am adding an env_provenance field with the value unknown on every existing row. Those rows still hold their measured numbers, and they still support a narrow benchmark decision inside the file that produced them. What they cannot do is join a claim about a driver, a runtime release, or a card.

That is a real loss. Seventeen rows of measured work now sit outside every environment question I might want to ask. You pay for the missing column on the day you need it, which is always later.

How do I stop this from happening again?

Move the fields out of the writer's hands and into the collector. The script that calls the API is the only thing that knows the truth at the moment of the run.

Then fail the run when a field is empty. An empty driver column is not a small gap in a spreadsheet. It is a row that will look usable in six weeks and will not be.

Accompanying prompt

What the prompt does: It checks a benchmark file for the environment fields a row needs before that row can support any hardware or runtime claim.

Copy/paste this prompt:

Role:
You are a benchmark provenance auditor for local LLM test files.

Context:
I will give you the header and a few rows from one or more benchmark files.

Task:
1. List which environment fields each file records at capture time.
2. Name the missing fields: runtime version, driver version, GPU name, capture timestamp.
3. Flag any column whose values mix a bare tool name with a versioned tool name.
4. State which claims each file can and cannot support as it stands.

Output:
- A table of files and the environment fields each one records.
- A list of missing fields per file.
- The claims each file cannot support.
- The exact collector change needed to record each missing field.

Constraints:
- Do not fill a missing field from a current system reading.
- Do not treat a report header value as row provenance.
- Do not treat a date without a time as a capture timestamp.
- Do not delete rows that lack provenance. Mark them.

Copy the block above.

Get the next measured local AI build note from The 5090 Reports.

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=local-llm-benchmark-row-missing-driver-version-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.