AI Harnessing: AI writes code fast. Now what?

AI Harness Engineering · Prologue

derek wang (derekwang85)

In 1968, a roomful of the world's best programmers admitted something nobody wanted to say out loud: writing code was quietly outrunning the human ability to manage it. Projects slipped. Budgets blew. Quality collapsed. It wasn't the exception anymore. It was the norm.

That meeting gave the industry a name for the disease — the software crisis — and over the next half-century, discipline after discipline tamed it: structured programming, object-oriented design, test-driven development, continuous integration. Chaos, ground down into order. The field settled on a shared belief: quality, speed, and cost pull against each other, and you can't get something for nothing. Every choice is a trade-off made with your eyes open. Consumer internet optimizes for experience. Industrial control software optimizes for uptime. Resource-and-process software optimizes for how fast it can absorb change. Each picks a different priority. None of them walks away from speed.

So two philosophies emerged for buying back time. One wins by reducing rework — design it carefully up front, and you don't have to rebuild it twice (waterfall). The other wins by harvesting feedback fast — iterate quickly, and the market tells you where you're wrong before you've over-invested (agile).

When you stack requirements management (what to build), methodology (how to build it), architecture (how to wall off the boundaries), and an engineering team that can actually carry the work (who gets it done), we once believed the software crisis was a solved problem.

Then LLMs and agents showed up.

And the crisis flipped. The problem is no longer that code is written too slowly. It's that code is written too fast — faster than we can understand it, trust it, or maintain it. The software crisis is back, except this time what needs taming isn't code written by humans. It's code written by AI. And taming it needs a harness.

1. The flip side of speed: trust, and what you lose

In the past year, almost every software engineer has had that moment where the speed is unsettling. You give the AI one sentence. Forty seconds later it hands you a complete feature: coherent logic, comments in place, tests already written.

Then the doubt creeps in. You stare at the code. Why was it designed this way? Does it respect the conventions of the neighboring module? Did it handle the edge cases? Will it silently collapse on some transaction, at three in the morning, when nobody's watching?

The more I use AI coding, the more I believe managing AI-native software is less a technical problem than a management problem. When your repository fills up with AI-generated files, that old comfortable feeling — "I typed every line, I know what it does" — is gone. In its place is something harder to name. You look at a screen full of code and feel hollow: is any of this actually useful?

Drucker said it sharply, and I'll quote him directly:

There is nothing so useless as doing efficiently that which should not be done at all.

AI is doing exactly that — at absurd efficiency, it generates huge volumes of code that should never have been written in the first place. It has accelerated the writing to the limit and handed the burden of understanding, trusting, and maintaining back to humans, untouched — and doubled.

My own proof point: over the last six months I shipped several projects with AI coding. The first one, in hindsight, was a learning exercise as much as a delivery. It was an internal intelligent-ops system, upgraded with AI — codename aITMS. The original ITMS was live in production, but it never felt "smart" the way modern agents do. The goal was to internalize an OpenClaw-style interaction layer.

Five weekends of spare time (roughly ten working days), using AI agents powered by OpenClaw, produced: 18 controllers, 35 request DTOs, 38 response DTOs, 1,557 lines of bilingual i18n config, 39 architecture decision records, and file-level changes scattered across hundreds of files. In retrospect, none of that is remarkable. In early 2026, it was startling — almost frightening.

Nice numbers. But those AI-coding wins didn't turn into a disaster, and that wasn't because the models were strong. It was because I kept iterating alongside them and built a constraint system behind them.

One clarification before I go further, to avoid confusion: aITMS is the original carrier of the methodology and the practice project. Nearly everything downstream — the constraint pyramid, the gates, the maturity model — evolved from it. But the test of whether these count as a methodology isn't "it worked in aITMS." It's "will it hold up in a different project?" I'll draw that line explicitly in Section 5.

2. The paradigm shift: from writing code to taming it

Traditional software engineering assumes, from day one, one thing: code is written by humans.

Every quality tool we built takes that shape — compile checks, code review, test coverage. They all exist to catch the mistakes after a human produces the code. Underneath is an unspoken management logic: you trust the engineer's process, but you don't trust every line they write — so you back them up with process and tooling.

But once the producer of code becomes an LLM or an agent, that assumption breaks. And so does the management logic built on it.

The change isn't really about speed. It's somewhere deeper: the center of gravity moved from "managing how people write" to "managing how AI is constrained." Recall the four pillars from the opening — requirements, methodology, architecture, the engineering team. In the AI era, the first three change meaning entirely. Requirements shifted from "talking to people" to "writing files the AI can read." Methodology shifted from "how people write code" to "how you constrain AI writing code." Architecture shifted from "the design in someone's head" to "the boundaries written in files." This isn't a tweak to a technical detail. It's a shift in the entire paradigm of software-engineering management.

Embracing AI coding is a kind of democratization for non-programmers — it spreads the ability to author software. Yet ironically, the most capable software engineers are the most eager users of AI coding. If AI is the more trusted, more efficient option, why do professional engineers still insist on constraining and managing it?

Maybe because an LLM generates tokens by probability — it's fundamentally a stochastic process. And maybe because the code it produces is still bounded by its memory system (context length). AI coding today isn't perfect, and because the limits live in the base model's internals, neither algorithmic nor engineering breakthroughs will arrive cheaply in the short term. So AI coding is feasible in principle — but hiding in the details are subtler defects:

1. Data hallucination — the model "decides" a field should be named X, names it X, and it doesn't match the real column in the database.
2. Concept drift — the same concept is called A in one file, B in the next, and C on the tenth regeneration.
3. Architectural drift — each generation introduces an "innocent" little deviation, and after thirty rounds, the codebase has drifted away from your design without you ever noticing.

None of these three is a "code quality" problem in the traditional sense. They are all management problems. You didn't govern the data alignment. You didn't govern the conceptual consistency. You didn't govern its improvisation. So the output drifted from the intent.

Software is an engineering problem for two reasons: the direction has to be steered dynamically — that's a management problem — and the precision has to be measured strictly — that's a math problem. A few hundred lines for a throwaway page is a different universe from hundreds of thousands or millions of lines. The gap is the difference between digging a hole with a plastic shovel on the beach and boring through a mountain with a tunnel-boring machine.

So can I solve this new problem with the old toolbox? Can code review hold the line?

If humans do the review, it only catches a fraction. AI generation has already outrun human review — in the time you review one PR, the AI regenerates three. Trying to out-speed AI with human reviewers is a losing war of attrition, like fighting an assembly line with manual inspection. Not because the inspectors aren't trying, but because the line never gives you the window.

So I stopped chasing it. I moved the defense line forward — before the AI generates, I constrain its behavioral space so the risk never materializes. That's the real subject of this series:

When you can't hold quality by "looking fast enough," you hold it by "fencing the boundaries in advance."

3. The two faces of the harness: steering and restraint

With the defense line moved forward, you need the right tool — one that fences in the AI's behavioral space without choking its productivity. There's now a popular name for it: the AI harness.

The word harness originally means the rigging you put on a horse — the straps, the bits, the tack — that lets a small human control a far stronger animal. We developed a mature, reliable harness because we wanted to wield the speed and mobility of an animal far beyond our own strength. It took roughly six thousand years of trial and error — from taming the first wild horse to finding a structure that steers the power without hurting the animal. Generations of accumulated trial-and-error wisdom. And that taming process is exactly what today's engineers are doing for LLMs. Every constraint adjustment is a product experiment answering one question: does this actually work?

Like a horse harness, an AI harness serves you in two directions:

• Steer — channel the force: make the AI "pull" in the right — or at least roughly correct — direction.
• Restrain — bound it: keep it from bolting, keep it from throwing a tantrum.

Is restraint even good? Doesn't it strip away the model's native capabilities? The AI's power is real and large. Without a harness, it tears through your codebase, leaving a mess behind. That self-doubt from Section 1 — staring at a screen full of files, unable to tell which are useful and which are hallucinated — is precisely what happens with no harness. Riding a wild horse is more exciting, but it comes with an unpredictable temperament and an unknowable destination.

AI Harness Engineering is the engineering methodology for fitting that harness. It is not a tool. It is not a prompt trick. It is a system: file-system constraints + gates + contracts + a maturity model. It may well be the seed of a new software-engineering methodology — one day, a body of practice that helps us solve the AI-era software crisis.

The core thesis, applied to AI coding, condenses to this:

When AI writes the code, you no longer control quality by writing code. You control it by building a constraint system that guides the AI's behavioral space. In other words: writing fast isn't the point. Being able to control it is.

4. The spec system: a constraint pyramid

AI Harness Engineering is sometimes translated as something like "steering engineering," and I think that is more an aspiration than a description. The practical constraint, the one that actually shows up in projects, lives in a series of files. In the project docs you'll often see them described by different names across agents — charters, PRDs, architecture docs, methodology docs, script docs, the code itself. Don't get caught up in the naming. Spend your attention on the constraint layers instead. That's where the return is.

The core of the spec system is one structure — the constraint pyramid:

Strategy   —— README.md (1 file: boundaries / naming / topology)
   ↕
Architecture —— ADR set (39 files: one record per consequential decision)
   ↕
Contracts  —— Runbook + Skill (repeatable steps + reusable workflows)
   ↕
Gates      —— pre-submit scripts (automated checks before commit)
   ↕
Implementation —— constrained AI-generated code (N files)

Each layer answers one question. Strategy answers what and what-it-is. Architecture answers where the boundaries are. Contracts answer persistence and stability. Gates provide the quality-assurance system. Implementation realizes the goals of Strategy, Architecture, and Contracts — and submits to the gates.

These five layers weren't dreamed up. They were walked into being. In aITMS, the early README was two vague lines, and the AI's code style changed every single week. Then the README grew to more than six hundred lines, indexing the other layer files — module boundaries, naming, layering rules, pinned down one by one. The style drift nearly vanished. The pyramid's point isn't a pretty document structure. It's a fence around the AI's unconstrained improvisation. Put the fence up, and the style stabilizes.

This constraint system has an endpoint. It isn't about locking AI down. It's about getting somewhere further away — to a place where the system learns to improve itself. There's a recurring character in this series: a local knowledge-brain, a "co-brain" that shares coding capability across projects, called derekinside. It has a knowledge wing and entity nodes. It's not a static knowledge base; it's an AI-native system that remembers, retrieves, and evolves on its own. In essay nine, I'll open it up and show you what it remembers, how it retrieves, and what drives its self-evolution.

5. The series map: twelve essays

Twelve essays plus this prologue — thirteen in all. Every essay follows the same recipe: a methodology, a real-project proof, and a copyable checklist.

1. The Constraint Pyramid: files as architecture — the five-layer spec system in full; why "files" are the blueprint of the AI era
2. Strategy: charter & blueprint — what to build and what it is; the project charter and AI's behavioral floor
3. Architecture: decisions as boundaries — why "why" must live in a file; how decisions hold consistency together
4. Contracts: from spec to code — spec contracts, data contracts, and repeatable governing specs
5. Gates: let checks precede code — the only automatically-enforced constraint layer; evolution from five gates to fifteen
6. Implementation: idempotency & degradation — interface idempotency, degradation contracts; the business doesn't stop when AI is down
7. The pre-execution manifesto: rules before code — the Manifest's six questions and the order files get created
8. Multi-agent orchestration: from a single point to division of labor — swarm decisions and multi-model consensus
9. Memory & knowledge: a system that evolves itself — how a co-brain that remembers, retrieves, and grows actually works
10. Debugging & attribution: beneath the iceberg — iceberg PDCA and hypothesis-driven debugging
11. Self-growing tests: let regression run itself — full regression, failure-pattern matching, and a self-growth engine
12. The maturity model: self-diagnosis & evolution — where your project stands, and how the constraint system measures it

The methodology's experience comes from four projects on my desk: a trading system (TradeOMS), a coding-methodology framework (derekcoding-framework), a quantitative-research system (SmartQuant), and a local co-brain (derekinside). Different industries. Different maturity. But all four grew out of aITMS practice — aITMS is the origin, not the whole.

The constraint pyramid you've previewed, and the maturity model to come, were extracted from live practice. They're deep case studies, they're implementation templates, and they're the methodology thinking about itself. There is certainly room to improve them. I'm putting them out here to provoke deeper thought among my peers — a brick thrown to attract jade.

6. A self-check: five first-day questions

Before you start, here's a checklist you can run today. If your project is using AI coding, go through it item by item:

• Does a README define module boundaries and naming conventions? (Not a one-paragraph "intro." A blueprint AI must read before it generates.)
• Are architectural decisions written down — or are they living in a chat history, one refresh away from being washed away?
• Is there a pre-submit script that checks automatically — or is everything riding on manual review?
• For AI-generated interfaces: have you thought about what happens when the same request is submitted twice?
• Which parts hard-depend on AI responding instantly? If it goes dark tonight, is there a degradation path or a manual fallback?

If you can't — or aren't sure you can — answer one or two of these five, this series is for you.

Putting a harness on AI isn't about holding its power back. It's about pointing that power where you want it to go. Fast isn't fast if you can't control it.

Next up, from the first foundation stone: The Constraint Pyramid: files as architecture — why files are the real blueprint of the AI era.