A 100% Score Puts the Nvidia AI Harness Above the Model

My scrappy 16GB setup has been making the same argument for years.

Claude Opus 5 went from 30.16% to 100% on ARC-AGI-3 after Nvidia changed the wrapper around it. Same frozen weights. Much better working conditions.

Meanwhile, most of my daily work runs through a 20B local model running on a consumer GPU with 16GB of VRAM. It is aggressively quantized and assigned boring, bounded jobs. Left unsupervised, it has the attention span of a golden retriever inside an Italian salumeria.

AI breaks there too.

My local model handles routine work. Stronger API models get called when a job earns the expense. I rarely touch the frontier tier because my harness handles memory and limits. It also manages verification and routing.

Nvidia gave Claude a competent boss

Anthropic’s Claude Opus 5 scored 30.16% RHAE at high reasoning effort on ARC-AGI-3, according to Anthropic’s system card. Nvidia wrapped the same model in Agentic Variation Operators (AVO). The result was 100.00 across all 25 public environments, with all 183 levels completed.

The weights stayed frozen. The working conditions changed.

AVO keeps previous attempts in persistent memory, provides tools and feeds results back into the loop. When the primary agent stalls, a supervisor steps in. I’ve managed enough talented engineers to recognize the setup. Brilliant people also look incompetent when they have no notes or feedback, especially when nobody can say, “Luca, you tried this yesterday. It caught fire.”

Nvidia AI product vice president Adel El Hallack told TechCrunch:

“Generally speaking, the world interprets an agent almost as an API of the model,”

He then gave the fuller definition:

“It is the model. It is the scaffolding around the model, which we call the harness, i.e. the set of tools that it utilizes. It is the runtime and the associated skills and libraries that we give it access to.”

I’m keeping the champagne corked. AVO cleared ARC-AGI-3’s known public set, and benchmark creator François Chollet compared the achievement to completing a video game’s tutorial level in a response reported by Laura Martel on August 21, 2026.

Nvidia’s seven-day engineering run impressed me more. AVO explored over 500 GPU-kernel optimization directions and committed 40 versions, according to Nvidia. The final kernels ran up to 3.5% faster than cuDNN and 10.5% faster than FlashAttention-4 on DGX B200 systems.

That is a long-horizon agent working with a compiler ready to expose every stupid idea. Brutal. Useful.

My expensive model waits upstairs

My 20B model handles formatting, extraction and small code changes. It also makes tool calls, validates their output and organizes first-pass research. The harness chooses which context enters the prompt and which tools become available. It decides how many retries I’ll tolerate and what evidence proves the job is done.

The 16GB detail needs some honesty. Four-bit weights and a 64k window are what get a 20.9B model into 16GB, and I still pick workloads that suit the machine. There is no miniature data center hiding under my desk in Los Angeles, despite what the cables suggest.

Nvidia is formalizing a similar division of labor with Nemotron 3.5 Lightning. The 30B mixture-of-experts model activates 3B parameters per token. Nvidia reports 86% PinchBench accuracy while completing 10,000 tasks 30% faster than Qwen3.6-35B at comparable accuracy.

Its job is gloriously unsexy: git pull and formatting, followed by tool validation and repeated execution. Complicated plans travel up to a stronger model. Chores stay downstairs.

NeMo Switchyard makes AI model routing explicit. In one evaluation, Nvidia cut cost by 74% while sending only 7% of calls to Claude Opus 4.8, with roughly six points less accuracy. A Cognition result came within 2.8 points of Opus 5 while reducing mean cost by 28%.

Databricks CEO Ali Ghodsi gave TechCrunch the version every founder should tape above the cloud invoice:

“So you think, oh, this is an expensive model. This is a cheap model. But wait, which harness are you using? That itself can 2x your cost.”

I refuse to send JSON cleanup and routine tool checks to the AI equivalent of Massimo Bottura. Frontier intelligence deserves a reservation. It should not butter every piece of bread.

The moat grows inside the loop

I can swap a model endpoint before lunch. A good AI agent harness takes months of ugly production lessons: what survives context compaction, where spending gets capped, when a human must approve an action and how the system proves it finished.

The expensive failures usually appeared between firmware and cloud services, or between an app and a device absolutely convinced it was offline. The useful company knowledge ended up encoded in recovery behavior.

Naïve’s Vetta experiment gives us a clean AI example. With GLM-5.2-FP8 held constant, Vetta cost $0.2232 per attempt versus $0.5995 for the next-best same-latency harness. It completed 12 of 16 tasks. The alternative completed 11.

Writer found a similar effect across six frozen models. Its rebuilt orchestration reduced cost per task by 41% and token use by 38%. Median runtime fell 44%, while quality stayed roughly steady. Plenty of “model spend” is waste elsewhere in the loop wearing a fake moustache.

Memory can be embarrassingly simple. PRO-LONG stored its history in an append-only logs.txt file searchable with grep. At a matched 500-action budget, its score jumped from 24.7% without the file to 45.6% with it.

I adore this result. Zero startup perfume. The mighty memory layer is a text file; the vector database can keep its black turtleneck.

The code audit also found a latent synchronization defect and no tests. That is the annoying half of owning the operational layer. Persistent memory needs checksums. Permissions need enforcement. Every claimed improvement needs a reproducible ablation.

Prompt incense will not rescue corrupted state.

My 16GB machine still knows its place

An RTX 5060 Ti runs gpt-oss:20b — 20.9 billion parameters at MXFP4 — entirely in VRAM. Fifteen gigabytes resident, a 64k context window, pinned there permanently. Nothing spills to the CPU.

Writer’s six-model experiment found a 0.99 correlation between quality and underlying model strength. In Anubhab Banerjee’s August 2026 study of 1,920 code-agent trajectories, compile success ranged from 5.7% with Phi-4-mini to 62.0% with Qwen2.5-Coder-14B. The winner there was one of the smaller models on the list. Nvidia also needed Claude Opus 5 for its perfect AVO run.

Capability sets the floor.

The ceiling is not whether the model loads. It is what four-bit costs me and what will never fit. Shayan Shahrabi-Farahani and Dara Rahmati measured Qwen retrieval accuracy falling from 81.0% to 68.3% under heavy interference with INT4, and MXFP4 is playing the same game. I have roughly a gigabyte of headroom left. Nvidia’s reference setup for Meta’s 30B Muse Glimmer uses an RTX 5090 with 32GB — a different machine and a different invoice.

My setup works because the jobs are bounded and uncertain work escalates. “Local-first” accurately describes the architecture. “Local-only” sounds like a future support ticket.

A stronger harness also expands the blast radius. The August 2026 HarnessRisk paper tested 128 adversarial cases across 14 model-harness configurations. Attack success rates ranged from 12.6% to 80.9%, while utility stayed between 75.0% and 97.6%.

I use approvals and sandboxes. Loops are bounded, actions are logged and every tool gets the minimum permissions required. Giving a cheap model unrestricted filesystem access because one demo looked molto bene is an exciting way to rediscover backups.

For one month, I’m freezing the model. No leaderboard shopping. No emergency migration because somebody posted a heroic screenshot on X.

I’ll measure completed tasks per dollar and failed-tool spend. I’ll track escalation rates alongside human rescues. Every improvement has to come from changing memory, permissions, routing, supervision or verification.

By August 2028, serious AI companies will treat models like cloud instances: important, expensive and replaceable. Anyone with a credit card can rent the same intelligence.

They cannot rent the scar tissue from everything my system already broke.

Frequently asked questions

What did Nvidia’s AVO change in Claude Opus 5?

Nvidia’s Agentic Variation Operators kept Claude Opus 5’s weights frozen while adding persistent memory, tools, feedback loops and supervisor intervention. On ARC-AGI-3’s 25 public environments, the wrapped model improved from Anthropic’s reported 30.16% RHAE score to 100.00 and completed all 183 levels.

Can a 20B AI model run on a GPU with 16GB of VRAM?

A 20.9B model at four-bit quantization runs entirely in VRAM on a 16GB consumer GPU, with no CPU offloading, at a 64k context window. Four-bit weights carry measurable accuracy costs, and uncertain or complicated tasks should still escalate to stronger API models.

Does model strength still matter with a strong AI harness?

Model strength still sets the capability floor, even with a strong harness. Writer found a 0.99 correlation between quality and underlying model strength, while a 1,920-trajectory study reported compile success from 5.7% with Phi-4-mini to 62.0% with Qwen2.5-Coder-14B. Nvidia also used Claude Opus 5 for AVO.