What building a contract-cancellation agent taught me about memory, deterministic gates, and auditable autonomy
I created Reaper and wrote this article for the purposes of entering the All Things Agentic Hackathon.
"Have we seen this before?"
It's one of the most useful questions an experienced person can ask. A lawyer remembers a clause that caused trouble. An accountant remembers a vendor that kept billing after cancellation. An engineer remembers the deployment that failed under eerily similar conditions.
So when I added memory to Reaper — an agent that carries auto-renewal cancellations to completion — I assumed retrieval would be the easy part.
Embed the current clause. Search for similar ones. Show the closest matches. That took a morning.
Deciding what memory was allowed to do took the rest of the week.
Because an agent that remembers past decisions can also inherit past mistakes. Worse, it can quietly convert resemblance into authority:
This clause looks 97% like one we blocked before, so I blocked this one too.
That sounds like judgment. In a system that acts on contracts, it's closer to superstition.
The fix wasn't better prompting. It was architecture.
The expensive difference between "similar" and "the same"
A contract costs $540 a month and renews for another year unless written notice arrives 60 days before the term ends. Miss that window and a small administrative failure becomes a $6,480 mistake.
Reaper's job is to prevent it: read the renewal clause, compute the deadline, sleep until the notice window opens, ask for one human signature, send the notice, then check whether billing actually stopped.
But the model is not permitted to schedule anything just because its answer sounds confident.
Gemini proposes a deadline. A separate deterministic engine — regex and calendar arithmetic, no language model inside — derives the date independently from the same text. The decision is deliberately boring:
model_deadline = model.read(clause)
engine_deadline = date_engine.derive(clause)
if engine_deadline is ambiguous:
block("the clause cannot be derived safely")
elif model_deadline != engine_deadline:
block("the two readings disagree")
else:
schedule(engine_deadline)
This gate has already caught a real one-day error from the model, live. In contract work, "only one day" is another year of fees.
It also refuses clauses like this one:
written notice no later than sixty (90) days before renewal
A model can invent a plausible reading of that. Reaper can't. The contradiction becomes the result.
That was the original trust boundary: the model may propose, deterministic code decides whether the proposal is safe to act on.
Then I added precedent memory and nearly broke it.
Why useful memory can make an agent less trustworthy
The memory stores clause shapes and what happened to them:
- Did the deterministic gate accept or block the clause?
- Did the vendor stop billing?
- Was a dispute opened?
- Did the contract require email, registered post, certified mail, or a portal?
- Was this a real obligation, or a labelled test fixture?
Each clause is embedded with gemini-embedding-001 into a 768-dimension vector in BigQuery. At intake, native VECTOR_SEARCH surfaces clauses with similar meaning even when the wording differs — a blocked contradiction, a clean cancellation the vendor honoured, a valid notice the vendor ignored, a contract where email was only ever a courtesy copy.
That's genuinely valuable context. It's also perfectly capable of contaminating the current decision.
Suppose the clause in front of you cleanly says "sixty (60) days," but a highly similar prior clause said "sixty (90) days" and was blocked. Feed that precedent to the model before the gate rules, and the old contradiction anchors the new interpretation. The system starts reasoning by analogy when it should be reading the document in front of it.
So Reaper enforces an ordering rule:
CURRENT CONTRACT
↓
DETERMINISTIC GATE
↓
VERDICT IS FIXED ← nothing after this line can change it
↓
PRECEDENT LOOKUP
↓
ADVISORY CONTEXT FOR THE HUMAN
Memory arrives after the verdict. And a test makes that boundary executable rather than aspirational: a 97%-similar blocked precedent is deliberately returned for a clean clause, and the clean clause must still schedule.
The warning is allowed to say:
A near-identical clause was blocked before because its words disagreed with its numerals. This is prior history, not a verdict.
That last sentence isn't interface copy. It's the permission model, written down.
Store outcomes, not just documents
A vector database full of text answers what looks similar?
An institutional memory has to answer what happened next?
That distinction reshaped the BigQuery schema. Every row keeps the clause and its structured outcome: gate verdict, notice method, final status, whether billing stopped, whether a dispute opened, and the terminal receipt hash where one exists.
The embedding finds the neighbours. The structured fields explain why those neighbours matter — which is how the system tells apart two contracts with nearly identical renewal language:
- one was cancelled, and billing stopped;
- the other was cancelled correctly, and the vendor charged anyway.
The text is similar. The operational lesson is the opposite.
This generalises. Building memory for a support agent? Store whether the resolution actually worked. A deployment agent? Store whether the rollout was rolled back. A collections agent? Store whether the promised payment arrived.
Retrieval without outcomes is recollection without learning.
Provenance has to survive retrieval
Reaper ships with a labelled corpus of synthetic clause shapes so the memory system can be tested reproducibly. Those rows must never masquerade as customer history.
So every precedent carries its source — fixture, or a real ledger-backed obligation. A seeded example may demonstrate that registered post is required; the product must never imply a real customer previously hit that vendor.
This sounds like metadata housekeeping. It's a trust boundary. Once retrieved text is condensed into a sentence, provenance evaporates — and "a similar contract was blocked before" means something very different from "a synthetic test clause of this shape is expected to be blocked."
If your agent mixes fixtures, simulations, generated data and production history, provenance has to travel with the memory, not live in a README nobody opens.
A missing memory is also a result
Here's a tempting three lines:
try:
matches = search_memory(clause)
except Exception:
matches = []
Now "BigQuery was unavailable" and "no similar precedent exists" are indistinguishable. They are not the same fact.
Reaper's lookup fails open: if the table is disabled, expired, unreachable, or the embedding call fails, the contractual verdict is untouched. But the evidence chain records that memory was unavailable, and why. The system distinguishes four states:
- memory was consulted, nothing passed the threshold;
- memory was disabled;
- the embedding could not be created;
- BigQuery could not be reached.
An advisory system going down should never block a valid cancellation. It should also never quietly pretend history was checked.
Failure transparency is part of memory design.
The evidence should include restraint
Every material action lands in an append-only SHA-256 hash chain: extraction, precedent consultation, the gate verdict, unprompted wakes, approval, delivery, invoice verification, dispute creation.
But the more interesting receipts describe what the agent didn't do.
The mailbox pipeline reads headers first. Messages that fail its admission rules stay unopened — and those refusals are recorded. When the system declines to inspect a bank statement from an unrelated sender, the refusal becomes evidence of restraint.
The same discipline applies to memory:
- memory cannot change the gate;
- a failed lookup cannot become "no history";
- fixture data cannot become customer history;
- raw vectors never enter the operational evidence ledger.
A trustworthy agent should be able to prove not just what it did, but which powers it deliberately declined to use.
Long-running agents spend most of their lives doing nothing
Cancellation exposed a less glamorous problem: waiting.
The agent may read a document today and need to act six months from now. It must not burn tokens, hold a process open, or depend on someone remembering to restart a chat.
So Reaper persists the obligation and sleeps. When the notice window opens it wakes and writes a receipt showing that calendar time — not a human pressing continue — triggered the action. It then pauses at the single point where authority is genuinely required: signing and sending.
That pause survives a process restart, because the invocation and its resume pointer live in persistent state rather than RAM. Killing the server doesn't manufacture a new approval or lose the pending one.
And then production breaks the whole mechanism.
Reaper runs on Cloud Run, which stops the container between requests. An in-process timer works beautifully on my laptop and does precisely nothing on a service that has scaled to zero. An agent whose autonomy depends on a container happening to still be alive is not autonomous.
So the heartbeat comes from outside: Cloud Scheduler calls a /tick endpoint once a minute, and that endpoint runs the same function the local loop runs. Not a second implementation — a separate "scheduled" code path drifts from the one you demo, and then the thing you showed isn't the thing that ships. The alternative was paying for an always-on instance to preserve an illusion the architecture didn't need.
Which leads to a rule I'd carry anywhere:
Human approval should be a durable state transition, not a conversational moment.
If an approval disappears when the process dies, it was never part of the workflow. It was part of the demo.
Five rules I'd reuse in any high-consequence agent
1. Separate proposals from permissions.
The model can recommend an action without holding the authority to execute it.
2. Run deterministic checks before advisory retrieval.
Memory should explain a decision or warn a human — never rewrite facts already established from the current input.
3. Store outcomes with memories.
Similar text is only useful paired with what actually happened.
4. Preserve provenance and failure states.
"Fixture," "production history," "no match," and "search unavailable" are four different facts.
5. Test the boundaries, not the happy path.
The important test isn't whether memory returns a match. It's whether an alarming match can improperly change the verdict.
The agent I trust is the one that distrusts itself
We usually describe capable agents by listing what they can do. I've become more interested in the opposite list:
- The model cannot schedule its own date.
- A contradictory clause cannot be resolved by confidence.
- Similar history cannot overrule current evidence.
- A missing lookup cannot pretend to be an empty result.
- A process restart cannot erase a pending human decision.
- A scaled-to-zero container cannot excuse a missed deadline.
- An unread message cannot quietly become a read one.
That negative space is where trust comes from.
Memory makes an agent more experienced. Boundaries stop that experience from hardening into prejudice.
Determinism is not the opposite of agency. It is what makes agency defensible.
Reaper is my solo entry for the All Things Agentic Hackathon.
Built with Google ADK · Gemini 3.5 Flash · Gemma 4 · gemini-embedding-001 · gemini-3.5-transcribe · Cloud Run · Firestore · Cloud SQL · BigQuery · Cloud Scheduler
