We spent decades engineering software to be predictable. Now we're building systems around something that isn't.
For years, the central question in AI engineering was:
Which model should we use?
GPT vs Claude.
Reasoning models vs general-purpose models.
Bigger context vs smaller context.
One model vs another.
But as AI systems evolve from chatbots into agents that can inspect repositories, modify code, run commands, browse the web, interact with APIs, execute tests and work for hours without constant supervision, the question is changing.
The interesting problem is no longer simply how capable the model is.
It is:
How do we build a system that allows a probabilistic model to do reliable engineering work?
That is the problem space increasingly described as Harness Engineering.
The term has emerged as a way of thinking about everything surrounding the model that turns raw intelligence into an operational agent: context, tools, execution environments, permissions, state, orchestration, memory, feedback, verification and recovery.
A useful mental model is:
Agent = Model + Harness
The model provides the intelligence.
The harness determines how that intelligence is exposed to the world, constrained, observed, corrected and ultimately turned into useful work.
And that distinction is becoming increasingly important.
The Model Is Not the Agent
A language model, by itself, is remarkably limited.
It receives information and generates a response.
It doesn't inherently have:
- a persistent workspace,
- a filesystem,
- a Git repository,
- a shell,
- a browser,
- durable state,
- permissions,
- a test environment,
- access to current information,
- a mechanism for recovering from failure.
Those capabilities come from the system around the model.
A coding agent might look deceptively simple from the outside:
User
↓
Model
↓
Code
But the actual system looks more like:
┌─────────────────┐
│ Human Intent │
└────────┬────────┘
↓
┌─────────────────┐
│ Context & Rules │
└────────┬────────┘
↓
┌─────────────────┐
│ Model │
└────────┬────────┘
↓
┌─────────────────┐
│ Tools & Runtime │
└────────┬────────┘
↓
┌─────────────────┐
│ Feedback / Evals│
└────────┬────────┘
↓
┌─────────────────┐
│ Recovery / Loop │
└────────┬────────┘
│
└──────→ Continue
That entire surrounding system is the harness.
The Harness Engineering material makes this distinction explicitly: the harness includes prompts, tools, skills, infrastructure and orchestration—the parts that make the model useful as an agent.
This leads to a surprisingly important conclusion:
The model may be the most intelligent component in the system, but it isn't necessarily the component that determines system reliability.
Why Better Models Don't Make Harnesses Obsolete
It is tempting to believe that sufficiently capable models will eventually eliminate the need for all this scaffolding.
If the model can reason well enough, why do we need elaborate instructions, verification loops, context management and recovery mechanisms?
There is some truth to this.
As models improve, certain pieces of the harness become unnecessary. Anthropic has explicitly described cases where behaviors previously compensated for by the harness improved in newer models, allowing parts of the harness to be removed.
But this creates a more interesting dynamic.
The harness doesn't disappear.
The operating envelope expands.
A better model allows us to attempt harder tasks.
Harder tasks introduce new failure modes.
The harness evolves to handle those failures.
So the progression looks less like:
Better model → Less harness
and more like:
Better model
↓
More capable agent
↓
More ambitious tasks
↓
New failure modes
↓
Better harness
↓
Even more capable agent
Anthropic's recent harness work makes a similar point: every harness contains assumptions about what a model cannot reliably do, and those assumptions need to be revisited as models improve.
That is why Harness Engineering isn't simply a collection of temporary workarounds.
It is becoming an ongoing engineering discipline.
From Prompt Engineering to Context Engineering to Harness Engineering
There is an interesting evolution here.
Prompt Engineering
Initially, the focus was:
What should I tell the model?
We experimented with wording, examples, roles, instructions and few-shot prompts.
Then agents became longer-running and more capable.
The question became:
What information should the model have at this moment?
That is Context Engineering.
Anthropic describes context engineering as the broader problem of curating the information available to a model during inference—including system instructions, tools, MCP servers, external data and conversation history.
And then comes the next question:
What environment should the model operate in?
That is where Harness Engineering begins.
Now we're thinking about:
- What context is available?
- What tools exist?
- What can each tool do?
- What requires approval?
- Where does execution happen?
- What state persists?
- How is work verified?
- What happens when something fails?
- How does the agent continue?
- How do multiple agents collaborate?
- How does the system improve over time?
The progression is therefore:
Prompt → Context → Harness
Each step expands the engineering boundary.
1. Context Is a Finite Engineering Resource
One of the biggest mistakes in agent design is assuming:
More context = better results.
It doesn't.
A model may technically be able to accept a huge amount of information while becoming less effective at using it.
Anthropic describes context as a finite attention resource and recommends optimizing for the smallest set of high-signal information that maximizes the desired outcome.
This changes how we think about agent memory.
A mature agent shouldn't simply receive:
Everything we know about the project
It should receive:
What matters for this task
+ what the agent needs to find
+ what it needs to remember
That leads to several important harness primitives:
- structured repository documentation,
- persistent memory,
- plans,
- skills,
- just-in-time retrieval,
- context compaction,
- durable artifacts.
The Harness Engineering analysis makes a similar distinction between long-lived instructions, persistent memory, session continuity and temporary conversation, emphasizing that context should be governed rather than endlessly accumulated.
This is why context engineering is not simply prompt writing.
It is information architecture for an intelligent runtime.
2. The Repository Becomes Part of the Agent's Brain
This is one of the most interesting consequences of agent-first development.
Humans can rely on knowledge that exists outside the repository:
- conversations,
- meetings,
- tribal knowledge,
- someone's memory,
- old design discussions.
An agent can't.
If the relevant knowledge isn't accessible during execution, it effectively doesn't exist for the agent.
OpenAI's experience building an internal product almost entirely through Codex pushed this idea particularly far. They treated repository knowledge as the system of record and deliberately worked to make the codebase itself legible to agents.
This means the repository increasingly contains:
- architecture,
- design decisions,
- development conventions,
- plans,
- quality rules,
- operational knowledge,
- documentation,
- skills,
- historical context.
The repository is no longer just where the code lives.
It becomes part of the agent's operating environment.
And this gives us a new architectural quality:
Agent Legibility
A system should not only be understandable to humans.
It should be discoverable and understandable by agents.
That means good architecture increasingly has two audiences:
Human developers and machine collaborators.
3. Tools Turn Intelligence Into Action
A model that generates text can make a bad suggestion.
An agent with tools can make a bad change.
That is a fundamental difference.
Once an agent can:
Read
↓
Modify
↓
Execute
↓
Observe
↓
Modify again
we've moved from generation to execution.
This is why tools aren't merely features of an agent.
They are execution interfaces.
The harness needs to determine:
- which tools exist,
- what they can access,
- when they can be used,
- what permissions they require,
- whether they can be interrupted,
- how their results are recorded.
The Harness Engineering material describes tools in exactly this way: managed execution interfaces that need authorization, scheduling, interruption and proper closure, with higher-risk tools receiving stronger governance.
A filesystem read and arbitrary shell execution may both be “tools,” but treating them identically would be poor engineering.
The principle is simple:
Capability should always be accompanied by control.
4. Permissions Become Part of the Agent's Constitution
Traditional applications often treat authorization as an infrastructure concern.
For autonomous agents, permissions directly shape behavior.
Consider:
ReadFile
versus:
Bash
The first is primarily observational.
The second can potentially:
- delete files,
- modify repositories,
- install software,
- change configurations,
- access networks,
- kill processes,
- trigger external actions.
So a mature harness needs something more sophisticated than:
Tool → Execute
It needs:
Agent requests action
↓
Policy evaluation
↓
Allow / Deny / Ask
↓
Execute
↓
Observe
↓
Record state
The important insight is that the permission layer isn't merely protecting the user.
It is also protecting the agent runtime from its own mistakes.
The Harness Engineering material captures this philosophy well: permission is an organ of the system rather than an accessory feature.
5. A Real Agent Needs a Heartbeat
A chatbot can be modeled as:
Input → Model → Output
A real agent needs a loop.
Something closer to:
Input
↓
Prepare state
↓
Invoke model
↓
Consume response
↓
Execute tools
↓
Observe results
↓
Update state
↓
Recover if necessary
↓
Continue
↓
Stop when appropriate
The Harness Engineering analysis calls this the query loop—the heartbeat of the agent system.
The model invocation is therefore only one part of the runtime.
The loop owns:
- state,
- tool execution,
- streaming events,
- interruption,
- recovery,
- continuation,
- stop conditions.
This is an important maturity test.
A system that can answer a prompt is a demo.
A system that can maintain coherent progress through changing state, failures and tool calls is an agent runtime.
6. Failure Is Not an Edge Case
Traditional software architecture often starts with:
Happy path
and then adds:
Error handling.
Agent systems need a different mindset.
For long-running agents:
Failure is part of the main path.
Context limits will be reached.
Tools will fail.
Commands will return unexpected output.
Networks will break.
The model will misunderstand something.
A subagent may fail.
A hook may loop.
A recovery mechanism may itself fail.
The Harness Engineering principles explicitly argue that error paths should be designed as first-class paths, with retry limits, circuit breakers and anti-loop mechanisms established at design time.
The goal isn't to build an agent that never fails.
That's unrealistic.
The goal is to build one that can fail without losing control of the workflow.
7. Recovery Should Preserve Momentum
Imagine an agent has spent an hour:
- understanding the architecture,
- locating relevant files,
- creating a plan,
- implementing half the feature,
- running tests,
- discovering a subtle issue.
Then its context becomes unusable.
A naïve system says:
Start again.
A good harness asks:
What useful state can we preserve?
This is where durable artifacts, Git, plans, memory and structured state become extremely valuable.
Anthropic's work on long-running agents emphasizes the use of structured artifacts and task decomposition to allow work to continue coherently across sessions.
Recovery isn't simply:
“Try again.”
It is:
“Continue from the best known state.”
That distinction becomes increasingly important as agent tasks move from minutes to hours and eventually days.
8. Verification Is Where Autonomy Becomes Trustworthy
Perhaps the most dangerous sentence in agentic development is:
“The agent says it's done.”
Implementation and correctness are different things.
An agent can:
- modify the code,
- run a test,
- interpret the result incorrectly,
- explain why the solution is correct,
- and still be wrong.
This is why verification needs to be an explicit stage.
Anthropic's recent long-running coding work uses a planner, generator and evaluator architecture, with the evaluator independently examining the resulting application against concrete criteria.
The Harness Engineering analysis similarly emphasizes that verification should be independent rather than simply being the implementer's final self-check.
A useful pattern is:
Research
↓
Implementation
↓
Verification
↓
Failure?
┌─┴─┐
Yes No
↓ ↓
Fix Done
↓
Verify again
The important thing isn't necessarily having multiple agents.
It is separating responsibilities.
9. Multi-Agent Systems Are About Partitioning Uncertainty
There's a tendency to think:
More agents = more intelligence.
That's not necessarily true.
Five agents performing the same task can simply produce five different versions of the same uncertainty.
The more interesting design is:
Research
↓
Synthesis
↓
Implementation
↓
Verification
Each stage has a different responsibility.
The Harness Engineering material describes the value of multi-agent systems in exactly these terms: research, implementation, verification and synthesis become separate responsibility containers, with the coordinator responsible for reconverging distributed knowledge.
This produces an important principle:
Parallelism is useful when it creates better boundaries, not merely more output.
And synthesis becomes critical.
Research can be distributed.
Understanding must reconverge.
10. Quality Needs Both Rules and Feedback
Martin Fowler's treatment of Harness Engineering introduces another useful distinction: feedforward and feedback.
Feedforward
Tell the agent what good behavior looks like before it acts.
Examples:
- architecture rules,
- repository instructions,
- coding conventions,
- design documents,
- skills,
- examples.
Feedback
Tell the agent what happened after it acted.
Examples:
- tests,
- linters,
- type checking,
- static analysis,
- logs,
- runtime checks,
- evaluation agents.
A system relying only on feedforward assumes the instructions are sufficient.
A system relying only on feedback forces the agent to repeatedly discover the same mistakes.
A strong harness combines both:
INTENT
↓
Instructions
↓
Agent
↓
Change
↓
┌─────────┴─────────┐
↓ ↓
Computational Inferential
feedback feedback
↓ ↓
tests / types review / evals
└─────────┬─────────┘
↓
Fix
↓
Done
This is essentially a feedback-controlled engineering system.
11. Architecture Can Become an Enforcement Mechanism
One of the most powerful ideas in agent-first engineering is:
Don't just document architectural rules. Make them executable.
Humans can remember:
“This module shouldn't depend on that module.”
Agents need stronger signals.
OpenAI's agent-first engineering experiment used explicit architectural constraints and custom tooling to make structural rules enforceable rather than relying purely on documentation.
This changes how we think about architecture.
A good architecture for agentic development isn't simply:
understandable.
It is:
discoverable + testable + enforceable.
The same applies to:
- coding conventions,
- dependency rules,
- API contracts,
- naming standards,
- test requirements,
- security boundaries.
Anything important enough to repeatedly explain may be important enough to encode mechanically.
12. This Creates a New Property: Harnessability
Some codebases are simply easier for agents to work with than others.
A codebase with:
- clear module boundaries,
- strong types,
- deterministic tests,
- useful logs,
- predictable conventions,
- explicit architecture,
- good documentation
provides many more useful signals to an agent.
A chaotic codebase gives the model far more ambiguity.
Martin Fowler calls attention to these environmental properties as ambient affordances—characteristics of the environment that make it easier for an agent to navigate and operate effectively.
This suggests that we may eventually evaluate architectures partly by a new criterion:
How harnessable is this system?
Not just:
- Is it maintainable?
- Is it scalable?
- Is it performant?
But also:
Can an agent reliably understand, modify and verify it?
13. Human Engineers Move Up the Abstraction Stack
This doesn't mean engineers disappear.
It changes where engineering effort is applied.
OpenAI's Codex experiment provides a striking example. With humans intentionally not writing application code, the engineers increasingly focused on defining intent, creating scaffolding, improving feedback loops and making the environment more legible to agents.
Their phrase captures the shift:
Humans steer. Agents execute.
The engineer increasingly becomes responsible for:
- defining outcomes,
- designing architecture,
- establishing constraints,
- creating evaluation criteria,
- improving tools,
- designing feedback loops,
- identifying systemic failure modes.
The agent increasingly handles:
- implementation,
- investigation,
- testing,
- refactoring,
- documentation,
- repetitive maintenance.
This produces a powerful distinction.
Suppose an agent repeatedly makes the same mistake.
You can:
Option A
Tell the agent again.
Option B
Build a mechanism that prevents the mistake.
Maybe that's:
- a test,
- a linter,
- a skill,
- a better tool,
- architecture documentation,
- a new evaluation,
- a repository rule.
Option A fixes one interaction.
Option B improves every future interaction.
That is engineering leverage.
14. Throughput Changes the Economics of Software Engineering
There is another consequence that becomes visible once agents can produce code much faster than humans can review it.
Traditional engineering is constrained by human throughput.
Agents change that.
OpenAI's internal experiment reported roughly 1,500 pull requests merged over five months while the team scaled from three engineers to seven, with humans increasingly operating at a higher level of abstraction.
At that scale, some traditional processes become bottlenecks.
If an agent can produce ten changes while a human can carefully review one, simply adding more human review doesn't necessarily solve the problem.
The solution becomes:
Automate more of the feedback.
That means:
- stronger tests,
- automated reviews,
- architectural checks,
- CI,
- evaluators,
- repository cleanup,
- continuous verification.
This changes the economics of quality.
When generation becomes extremely cheap, verification becomes one of the scarce resources.
15. Agent-Generated Code Creates an Entropy Problem
High throughput creates another challenge:
software entropy.
If agents can generate thousands of changes, they can also generate thousands of mediocre decisions.
Over time you may accumulate:
- duplicated utilities,
- unnecessary abstractions,
- dead code,
- inconsistent patterns,
- obsolete documentation,
- architectural drift.
OpenAI describes addressing this with recurring “garbage collection” workflows in which agents inspect the repository for violations of established principles and clean them up.
This leads to an important principle:
When code generation becomes cheap, code cleanup must become continuous.
Agentic engineering therefore needs not only generation loops but maintenance loops.
16. The Most Interesting Loop Is the Harness Loop
The coding loop looks like:
Prompt
↓
Code
↓
Test
↓
Fix
Harness Engineering introduces a higher-level loop:
Agent fails
↓
Understand why
↓
Identify missing capability
↓
Improve harness
↓
Agent retries
↓
Failure becomes less likely
This is a major shift in mindset.
Instead of constantly improving outputs, we improve the system producing the outputs.
OpenAI's experience illustrates this philosophy clearly: when an agent struggled, engineers focused on identifying the missing capability and making it both legible and enforceable rather than simply asking the model to try harder.
That may be the defining idea of Harness Engineering.
Don't just fix the failure. Improve the environment that allowed the failure to happen.
17. And Eventually, the Harness Can Improve Itself
This is where the idea becomes particularly interesting.
If:
Agent = Model + Harness
then we can ask:
Can the harness itself be optimized by an agent?
The supplied Harness Evolution research proposes exactly this.
A worker agent performs a task.
An evaluator examines the result.
An evolution agent studies the failures and modifies the harness.
Then the worker runs again.
Conceptually:
WORKER
↓
OUTPUT
↓
EVALUATOR
↓
Failure analysis
↓
EVOLUTION AGENT
↓
Better Harness
↓
WORKER
↺
And then comes an even more ambitious concept:
Meta-Evolution
If the harness can evolve, why not optimize the process used to evolve harnesses?
The research describes an outer loop that evaluates harness-evolution strategies across multiple tasks and attempts to improve the evolution blueprint itself.
In other words:
Model
↓
Agent
↓
Harness
↓
Harness Evolution
↓
Meta-Harness Evolution
We move from:
engineering the agent
to:
engineering the system that improves agents.
That could become one of the most interesting areas of AI engineering over the next few years.
A Practical Harness Engineering Checklist
If you're building an AI agent today, don't begin with:
“Which model should I use?”
Start with these questions.
Context
- Does the agent have the information it actually needs?
- Is important knowledge discoverable?
- Are long-lived instructions separated from temporary conversation?
- What happens when context becomes too large?
Tools
- What can the agent actually do?
- Which tools are read-only?
- Which are destructive?
- Which require approval?
- Are tool calls observable and auditable?
State
- Where does progress live?
- Can work survive an interruption?
- Can another agent resume it?
- Are plans and decisions durable?
Verification
- How do we know the task succeeded?
- Which checks are deterministic?
- Which require semantic evaluation?
- Is verification independent from implementation?
Recovery
- What happens when the model fails?
- What happens when a tool fails?
- What happens when context overflows?
- Can the system continue without starting over?
Governance
- What actions should the agent never perform?
- Which rules can be mechanically enforced?
- Where is human judgment required?
Evolution
And finally:
When the agent repeatedly fails, how does the system itself get better?
That last question is arguably the most important one.
The Future: Engineering the Environment in Which AI Engineers
The deeper implication of Harness Engineering is that the unit of engineering is changing.
Traditional software engineering largely asked:
How do humans build software?
AI-assisted development asked:
How can humans use AI to build software faster?
Agent-first engineering asks something different:
How do we design an environment in which AI can reliably build software?
That environment includes:
HUMAN INTENT
↓
┌─────────────────┐
│ HARNESS │
│ │
│ Context │
│ Tools │
│ Permissions │
│ State │
│ Memory │
│ Orchestration │
│ Verification │
│ Recovery │
│ Governance │
└────────┬────────┘
↓
MODEL
↓
ACTION
↓
FEEDBACK
↓
CORRECTION
│
└──────────→ CONTINUE
The model remains the source of intelligence.
But the harness increasingly determines how that intelligence behaves in the world.
Conclusion
The first wave of generative AI taught us how to prompt models.
The next wave taught us how to engineer context.
The emerging wave is teaching us how to engineer the systems around models.
That's Harness Engineering.
It isn't simply a better system prompt.
It isn't adding more tools.
It isn't throwing more agents at a problem.
It is a recognition of a fundamental engineering reality:
A probabilistic component cannot simply be placed inside a deterministic workflow and expected to inherit its reliability.
Reliability has to be engineered around it.
We need context that is curated rather than dumped.
Tools that are powerful but governed.
State that survives interruptions.
Feedback that catches mistakes.
Verification that doesn't simply rubber-stamp implementation.
Recovery that preserves momentum.
Architecture that is legible and enforceable.
And organizational knowledge that is encoded into the environment rather than trapped inside individual people's heads.
The most capable agent of the future may therefore not simply be the one with the smartest model.
It may be the one with the best environment in which that model can operate.
And perhaps that is the most important shift in perspective:
We are no longer just engineering software with AI.
We are engineering the system in which AI engineers software.
That is Harness Engineering.
