Textbook: Agent Evolution — From Experience to the Closed Loop

Source version of CodeSmith: v0.5.0 (commit 3a74c82f). All paths are relative to the repo root; line numbers refer to this version.
Intended audience: readers who want to know whether an Agent that has finished ten thousand jobs is any more practiced than it was on day one.

Every organ of the series has now been covered: caching, handles, compression, tools, defenses. Before I lay down the pen, one last question remains — and it is this craft's most frontier question: after an Agent has finished ten thousand jobs, will it be any more practiced than on day one?

Today's answer is, for the most part, no. A deployed model does not change its parameters by inference; in-context adaptation (the status bar, compression, memory) holds only within the current task and dissolves the moment the session ends. The model can solve an unseen complex task zero-shot, yet after ten thousand similar tasks it will still make day one's mistakes the next day. The discipline now taking shape around this is called continual evolution: since the model itself cannot be trusted to learn, build "learning" as a system around the model — record runtime evidence, verify outcomes and process, extract what recurs across trajectories, then decide what to update.

This installment is the series' final lesson: first set up the Textbook's coordinates, then watch CodeSmith land this closed loop end to end.

Saving Experience Is Not Learning from Experience

First, the most important bucket of cold water: saving experience is not learning from experience. Put a hundred trajectories into a long context or a vector store and you help the model retrieve a particular case, but you do not automatically get cross-case comparison — which steps recur in the successful trajectories, whether a given success came from the right strategy or from environmental luck. Learning happens after the system has actively done the "evaluate, compare, generalize, verify" work — not at the moment the log hits the disk.

Then why not let the model train itself directly after each task? Because production rarely offers a clean learning signal: user satisfaction is not compliance; updating parameters directly from unverified feedback would bake wrong experience and prompt injections into the weights, there to keep amplifying across later tasks. So until models can learn continually and reliably, learning must happen at the system's periphery — that is the dividing line between continual evolution and continual learning.

Learning Signals: Three Layers of Verifiers

Evolution starts with evaluation. To judge a trajectory is, in essence, to answer three questions in turn — one verifier per layer:

The outcome layer deserves to be built first — the environment's ground truth is more reliable than the model's account of its own behavior, which is exactly the principle Article 13's "server-side database ground truth" and Article 6's "the Verifier runs tests instead of gazing" share. And the shape of a verifier's output decides whether it can become a learning signal: a single total score reflects better-or-worse but not where to change; a usable evaluation carries at least four things — whether the task was a success, a partial success, or a failure; a separate verdict for every dimension; the evidence location behind each verdict (which turn, which call); and a type label for the failure.

A verifier should also hold the right to refuse to score — rather than solidify a low-confidence conclusion into fact, exclude the case from the learning set.

Four Ways to Update: Which Layer the Experience Is Written To

With a learning signal in hand, the next question is where the change lands. Four carriers, each minding its own stretch:

The four are not mutually exclusive: one capability can be split across them — facts into the knowledge base, exception principles into a Skill, non-bypassable permissions into code, high-dimensional recognition into parameters. That spectrum table from Article 2 — dedicated tools vs Skills vs the general-purpose executor — takes on a second identity in the evolution context: it is the routing table for "which layer new experience should settle into."

Three Case Studies

Case one: distilling rules from failed trajectories (τ²-bench telecom). Swap in a weaker model to run the telecom customer-service tasks, and 19 of the 20 tasks end with "transfer to a human." Hand those 19 failed trajectories to the model to distill on its own, append the handful of executable rules it produces to the end of the policy, then rerun on tasks that took no part in the distillation: the pass rate climbs from 12.3% to 19.3%, and out-of-bounds calls and missing-parameter calls both drop by roughly eighty percent. The experiment leaves behind three lessons worth more than its numbers.

First, the material you distill from determines what can be distilled — fed only failure summaries, the model generalizes "stop calling a tool that keeps erroring"; add both sides' tool inventories, and what comes out is "network status lives on the user's device side, so guide the user through operating it themselves": the former memorizes a lesson; the latter understands a duty.

Second, the model treats observed behavior as prescribed behavior — the first draft of the rules contained, in plain sight, "after three consecutive failures, transfer to a human," because the most frequent action in the trajectories was transferring to a human, and the model took it for a legitimate fallback; distilled output must clear independent verification before release, or you are writing failure into the spec. Third, what gets fixed is usually the plainest of defects — one trajectory stuffed "please provide your phone number" into a query tool's parameters, five times in a row; only after the rules took effect did the model learn to ask for it in conversation first.

Case two: compiling trajectories into programs (browser workflows). The first time it sends an email, the Agent feels out the entire path — "compose → recipient → subject → body → send" — from pixels and the DOM. The second time it sends an email, the flow is unchanged and only the parameters differ — there is no need to spend another round of model calls rediscovering it.

The method compiles that first exploration into a small parameterized program with state checks, whose life runs through six steps: capture the trajectory → parameterize (literals become template variables) → define state checks (pre-execution check, post-execution check, final-state check — a successful action is not a successful task) → verify by independent replay (reset to a clean environment and replay in full; publish only if every check passes) → matched replay (new tasks execute it directly, no per-step LLM calls) → relearn on failure (fall back to full Agent mode when an element cannot be found).

Such programs have measured 8.5–13× end-to-end speedups on repeated tasks, but more important than the speedup is the conclusion: workflow memory must carry all three verifications at once — before the action, after the action, and in independent replay — otherwise every button will have been clicked while some field is actually empty, and the system will simply repeat its mistake faster.

Case three: making the system itself evolvable (everything is a plugin). Self-modification is at its sharpest when the thing being modified is long-lived, stateful, and full of mutual dependencies. The DeepSeek Harness (the agent runtime DeepSeek built on the Cordis framework) answers by going to the extreme: the model adapter, the tool registry, the session log, even the Agent main loop itself are all plugins — "there is no privileged kernel that only humans can maintain."

The paper underneath it splits static composition (function calls and module imports fixed at compile time) from dynamic composition (loading and unloading components at runtime) into two orthogonal dimensions: temporal composability asks "when a component is unloaded, can every change it made to the shared environment be fully undone," and spatial composability asks "can components declare and resolve their dependencies structurally." The ceiling of self-evolution is set not by how good the code the model can write is, but by how composable the system carrying it is.

Evolution itself has a price, too: the moment a plugin changes, the tool set and prompt fragments visible to the model change, and the KV Cache starts invalidating from the point of change — the frequency of evolution runs head-on into Article 4's cache economics.

The Dual Loop and Three Safety Boundaries

Continual evolution exists only when the four update carriers enter one and the same loop, and the sound structure is a dual loop: the online execution loop only completes tasks and records immutable evidence, never touching production capabilities; the offline evolution loop aggregates trajectories, diagnoses root causes, generates update proposals, and publishes new versions once they clear the verification gate.

Voyager showed its minimal complete form inside Minecraft: an automatic curriculum (pick the next moderately difficult goal for the current ability), a skill library (successful programs saved as composable code), and iterative prompting (environmental feedback carried into the next round of generation until it truly passes) — lose any one of the three and it degenerates: a skill library without a curriculum does not know what to learn next; reflection without environmental verification piles errors into the skill library.

Lift that closed loop one level up and you get the discovery loop of the scientific method: hypothesis, experiment, evaluation, feedback — for self-evolution to be more than talking to itself, it must submit to this centuries-old recipe.

The capacity to evolve is itself a new risk surface: a prompt injection summarized into experience takes effect again and again across sessions; a malicious package wrapped up as a tool spreads its reach to every later task. Hence three boundaries. Separate evidence from instructions: web pages, tool outputs, and their summaries are untrusted evidence, and claims extracted from them are never to be executed as instructions. Separate the pending-verification zone from production capabilities: new knowledge, new Skills, and new code enter a staging zone first, and serve real tasks only after sandboxing, scanning, and regression tests.

The third is the hardest: the safety machinery must not modify itself — a business Agent may change Prompts, Skills, and tools, but never the verifiers, test cases, release gates, and audit logs that approve its own updates; otherwise it need only delete two failing test cases for degeneration to disguise itself as progress. The foreshadowing Article 14 planted for bootstrapping is redeemed here.

Offline consolidation has a name with a cognitive-science flavor: sleep learning — nothing requires it to literally run at night; what the name points at is "collecting and tidying are separate." One cycle, five steps: trigger (time / volume / error-rate thresholds) → orient (read the existing capability catalog to learn the boundaries) → gather and consolidate (find new signals among already-evaluated trajectories, merge and deduplicate, flag conflicts) → verify and approve (three exam papers — a migration set, a retention set, a safety set; high-risk writes wait for human approval) → prune the index (mark long-unused or overturned capabilities as expired, keeping provenance and a rollback version).

Claude Code's MEMORY.md is its plain form: a bounded index, detail split into per-topic files, read on demand, consolidation requested as the limit nears; a more complete implementation would gate writes to memory and Skills behind approval, and add an independent Curator doing deterministic pruning in idle periods.

Landing It: CodeSmith's Evolution Loop

Parts first. The one that most resembles an "experience entry" is the Slop Ledger (Article 7). Look at its data structure (crates/agent-runtime/src/slop_ledger.rs:179):

/// A single slop ledger entry.
pub struct SlopEntry {
    /// Classification bucket.
    pub bucket: SlopBucket,
    /// How severe is this residue?
    pub severity: SlopSeverity,
    /// How confident is the assessment?
    pub confidence: SlopConfidence,
    /// Who owns cleaning this up (person, team, or "auto").
    pub owner: Option<String>,
    /// Source file paths, URLs, or line references.
    pub source_links: Vec<String>,
    /// Current lifecycle status.
    pub status: SlopEntryStatus,
    // ...... (title / description / cleanup_recommendation / timestamps / linked tasks)
}

In plain words: every entry in this ledger carries the full set of fields the textbook demands — classification, severity, confidence (doubt about the assessment itself), an owner, evidence links (experience must be traceable to its source), and a five-state lifecycle (Open → InProgress → Resolved / Accepted / WontFix). Article 7 framed it as "registering AI's technical debt in the household register"; reread from the evolution angle, it is the prototype of an experience-knowledge-base entry: a structured knowledge unit that lives outside the immutable trajectories, carrying provenance and a lifecycle.

The same family of parts also includes KoD memory recall (Article 10 — on-demand retrieval capped at 5 entries per turn, bounded in time and volume), progressive disclosure of skills (Article 2 — a Skill is the instruction layer's update carrier), and side-git snapshots (Article 7 — the rollback path, so a pending-verification version can be withdrawn the moment you want it gone).

With the parts assembled, the loop closes through three muscles — one online, two offline.

The Claim Verifier: Online Reconciliation at the End of the Turn

When the model says "tests all green," the engine replays — verbatim, through the session's own dispatcher — the verification commands it ran this turn and reconciles against the exit codes; the verdict is written back to the tail of the conversation as a four-tuple (conclusion, dimension, evidence location, failure type) and lands in the jsonl ledger at the same time — "what the model said" checked against "the facts of the world." This is the outcome layer of the three verifiers made concrete; Article 15 devoted a whole installment to taking it apart, so I will not repeat it here.

doctor's Long Tail: Every Finding Fine Alone, Sick Only in Combination

codesmith doctor is a deterministic self-check: is the API key missing, is the endpoint reachable, is the binary present. Article 14 covered its layered recipe — determinism first, the LLM as backstop. What the backstop has to catch is the long tail of the deterministic checks (crates/tui/src/doctor_llm.rs:3-8): each single finding's hint is reasonable on its own, while the real disease lives in the "combination" of two findings — a proxy swallowing the API domain, a keychain key shadowed by a stale environment variable, an alias model on a custom endpoint. Take the items one by one and every Hint looks healthy — the four classical examinations all check out, but nobody took the CT scan.

The backstop layer's method (doctor_llm.rs:10-14): once the deterministic checks finish, serialize the collected warnings/errors into JSON and hand them to a single advisory LLM call — preferring [utility_model] (the designated cheap brain), falling back to the main client if it is unconfigured — asking it for root-cause hypotheses and one concrete next step per finding.

The design contract has three clauses, each barbed (doctor_llm.rs:16-31). First, deterministic results always win: the analysis is rendered only as an advice section after "All checks complete!", with no reordering, no rewriting, no swallowing the checks above; --json mode does not move a single byte — machine-readable, CI-safe.

Second, analyze only, never execute: what the model receives and what it returns are both text — no running commands, no changing configuration, no going near the approval surface. The module documentation says it plainly (doctor_llm.rs:26-28) — a verifier that can act on its own verdicts is a verifier you cannot trust.

Third, fail silently and never get in the way: if the client cannot parse it, the response is empty, transport errors, or the 30-second timeout hits, skip the section entirely and leave just one · line. The original words: "with an LLM added, doctor is not allowed to become more unreliable than it was before."

The system prompt sent to the model extends the same restraint (doctor_llm.rs:173-186): skip outright the findings whose built-in hints already suffice; prefer explanations that can string several findings together; at most one next action per finding; "you cannot execute anything, and you must never claim you have run, fixed, or verified anything"; at most 15 lines. The budgets are nailed down as well: each finding truncated to 400 characters, at most 40 findings, replies capped at 1600 tokens.

Sleep Learning: Online May Only Record, Only Offline May Rewrite

The third muscle gives memory the offline loop of "tidying the bed while asleep": codesmith memory consolidate, which the documentation itself calls "sleep learning."

The design's starting point lies in the memory system's discipline. The online loop has exactly one action available for memory — append: remember deduplicates fully identical lines, but nothing prunes (crates/agent-runtime/src/knowledge/curator.rs:3-5). As the days go by, the MEMORY.md entry file accumulates fully duplicated pointer lines, dead pointers to deleted topic files, and bloat that quietly crosses the budget.

Why may the online loop not prune? Because rewriting mid-session is itself an act of self-modification — Article 9's append-only discipline and the dual loop's separation both point to the same iron law: the online loop only records; rewriting the index is the privilege of the offline evolution loop, and the offline loop must be started explicitly by a human.

So consolidate's first layer is deterministic tidying (curator.rs:11-16) — no LLM, no judgment: drop fully duplicated pointer lines (keeping the first); delete dead pointers to files that no longer exist; orphaned topic files (on disk, unindexed) are only reported, never deleted — deleting content is a semantic decision, left to humans; and finally a checkup against the entry budget (200 lines / 25 KB, knowledge/budget.rs:17-20).

The second layer is the LLM merge proposal: hand the tidied index plus the frontmatter of every topic file to a single call (a 2000-token budget, a 60-second timeout), and ask it to rewrite only the index — sharpen overlapping descriptions, group them, exactly one pointer per file. The prompt fills in the prohibitions: do not invent files, do not lose files, do not touch file contents (crates/tui/src/memory_consolidate.rs:29-40).

The prompt is a request; the verifier is the law. validate_proposed_index (curator.rs:169-224) rules item by item: references a file that does not exist on disk — rejected; the same file pointed at twice — rejected; every pointer dropped — rejected; any existing topic file left without a pointer — rejected; over budget — rejected.

The module documentation's closing line is worth copying out (curator.rs:23-24) — "a verifier that no proposal can buy its way past is the whole point; sleep learning without a gate is just an unattended rewrite."

Writing to disk passes three more gates (memory_consolidate.rs:14-18): dry run by default, printing only a unified diff; only --apply writes, and it backs up MEMORY.md.bak first; topic file contents are never touched. Throughout the flow, the deterministic result is the floor that cannot be broken — if the LLM proposal fails to parse, errors, or is rejected by the verifier, what lands on disk is the deterministic tidying's result.

Coda: The Last Line of the Catalog of Distrust

Pull the final lesson together:

1. Saving experience is not learning; learning happens after evaluation, comparison, generalization, and verification.
2. Learning signals come from three layers of verifiers (outcome / process / quality), and their output must carry dimensions, evidence locations, and failure labels.
3. Updates have four carriers (knowledge base / Prompts and Skills / programs / parameters), routed by the representational nature of the capability.
4. A dual loop (record online, evolve offline) plus three safety boundaries — the hardest being that the safety machinery may not modify itself.
5. CodeSmith lands the closed loop as three muscles: the claim verifier reconciling online, doctor's analysis diagnosing offline, sleep learning tidying offline — all three share one safety grammar: the verifier does not execute, the analysis does not touch disk, a proposal voids itself unless it passes the verifier, and the claim-match rate from /verify stats turns the loop's true quality into a number you can keep an eye on.

Three features and their prices:

1. Dual-loop separation: online only ever appends — the price is that memory's bloat waits for a human to remember to run consolidate; there is no automatic tidying.
2. Determinism as the floor: the LLM layer can only ever add the icing — the price is that the ceiling for diagnosing combination-type root causes is that one 15-line advisory call.
3. A verifier that cannot be bought: proposals must be lossless — the price is that even when the model wants to merge two genuinely redundant topic-file pointers, it may only rearrange descriptions, not touch contents.

Now tally the global account: the evolution system's own complexity is a new maintenance burden, and a broken verifier is more dangerous than no verifier — it will keep approving degeneration disguised as progress; on open-ended tasks (research, planning) the evaluation signal arrives slowly, and humans still have to take part in defining "what is good."

Article 1 said that a good harness is, in essence, a clearly written catalog of distrust — distrust the model's output, so strip it out; distrust the model's judgment, so whitelist; distrust the model's memory, so "Memory is never a command." The evolution installment adds the last line to that catalog, and the only line that points at the future: distrust tomorrow's self — so the gate that approves today's me is one I am never authorized to modify.

The evolution installment draws to a close, and with it the series' main line reaches its end.

Evolution still needs a mirror — another harness (ZCode's Agent runtime) happens to fit: the comparison piece (Chinese) lines the two up organ by organ — six points of convergence, a four-vs-four with wins on both sides, and four places where the hypotheses fork. Thank you for reading.