I built this project and wrote this post for the All Things Agentic Hackathon (Google Cloud + Devpost). #AllThingsAgenticHackathon
In Colombia, public money for housing and schools expires unspent. Not metaphorically — Decreto 413 de 2025 sat at 8.6% execution while funding calls worth over COP $100,000 million opened and closed in weeks. Small municipalities rarely lose these on merit. They lose them on attention: nobody in a five-person planning office is watching the procurement portal at 2 a.m., cross-checking a new call against the town's formulated projects, requirement by requirement, against a closing date.
So we built a background agent that does exactly that. It watches official sources on a schedule, checks each find against the municipality's structured project data, compiles what's missing with owners and calendar time, drafts the filing, and messages the planning secretary overnight. A human reviews, decides and files — the agent never does.
That's the product. This post is about the part that was hard, and that I think generalizes: making an agent that is structurally incapable of telling you what you want to hear.
The worst possible bug
Think about what failure looks like here. A municipal secretary reads "you meet 7 of 9 requirements for this COP $2,000 million call." She mobilizes her team, spends two weeks assembling the filing, submits it — and it bounces, because requirement 4 never actually held. The agent had matched a project attribute that resembled the requirement.
That's not a bad UX moment. That's a small town losing a housing program and an official losing credibility she'll need for the next one.
For most AI products, the worst bug is a wrong answer. For this one, the worst bug is a confident wrong answer that looks like every right answer. Which means the usual approach — write careful prompts, tell the model to be rigorous, add a "be honest" instruction — is not a mitigation. It's a hope.
The pattern
We ended up with an architecture we call Evidence-Gated Agency. Four properties, all enforced by code and tests rather than by prompting.
1. The model is constitutionally forbidden from producing figures
This is the load-bearing one. In our system a language model may never emit a number, a date, an amount or a verdict. It may only point at where one lives.
Concretely: the model reads a source document and returns a verbatim quote — the exact span of text that declares the amount. Deterministic code then verifies that quote exists literally in the source. Only then does code — never the model — parse the number out of it.
If the quote isn't in the document, that's not a retry. It's a caught hallucination, and it goes into a log the UI displays with a running count.
This inverts the usual arrangement. Instead of asking the model for an answer and then checking it, we ask the model for a pointer and let code do the reading. A model that cannot produce a figure cannot fabricate one.
2. Four gates before any "requirement met"
Before the system will say a requirement is satisfied, four things must hold:
- the requirement is evaluable (not a known gap in our own data);
- the supporting quote is verified literally against the source;
- the attribute has a real value — not a placeholder, not a null dressed as a zero;
- a subtractive adversary — a second model pass — fails to refute it.
Gate 4 exists because gates 1–3 stop fabricated evidence but not lax reasoning. We learned this from a real case: the system accepted zona_estrategica: B as proof that "an open call exists for the region." The quote was real. The value was real. The reasoning was garbage.
So we added a pass whose only power is to remove. It cannot add a match, confirm one, or soften a rejection. Asymmetric authority — the adversary can subtract, never add — turns out to be a surprisingly clean way to make a second opinion trustworthy. There's no incentive gradient toward agreement, because agreement isn't an action it can take.
3. The honest zero is a first-class output
Most products treat "no results" as an empty state to be apologized for. We treat it as an answer.
"You currently qualify for nothing. Here's why, requirement by requirement, with the missing pieces, who owns each one, and how long each takes" — that is useful. It's the answer the secretary can act on. What she cannot act on is a padded list of instruments she doesn't actually qualify for.
This has consequences all the way down. An instrument with no declared amount contributes tasks, not pesos, and the plan says so. A source that's down is shown as down, with the real error, never as "nothing new" — silence is the one lie a watchdog cannot afford. A missing value renders as its own chip with an owner, never as 0 and never as blank.
The system once downgraded its own score from 1/8 to 0/8 when the adversary refuted its only match. We shipped the zero.
4. Everything is epistemically typed
Every value in the system travels as {value, tag, source, vintage} — the number, how much we trust it (VALIDATED / ESTIMATED / HYPOTHESIS / TO INVESTIGATE / DEMO DATA), where it came from, and when we read it.
The tag is not decoration; it changes what may be asserted. A published decree amount can be cited as VALIDATED. The rate a development bank would offer this municipality doesn't exist anywhere until it's offered in writing, so it can only ever be ESTIMATED. A poisoned fixture in the test suite fails the build if anyone ever tries to promote one to the other.
The counterproof
Here's the thing about honesty claims: they're unfalsifiable in a demo. Anyone can say "our agent is rigorous."
So we built a button that turns the rigour off.
The UI has a semantic-precision panel: 14 stored cases with known-correct verdicts, 8 of them deliberately loose reasoning that must be rejected. Press one button, and it runs against the live model. Press the button beside it, and it runs the same 14 cases with one gate loosened — and the false "requirement met" verdicts reappear on screen, by name.
zona-como-convocatoria. estructura-como-presupuesto-aprobado. municipio-como-aval-departamental.
That's the argument. Not "we're careful," but here is what happens when we aren't — running live, in front of you, thirty seconds apart. If you're building something whose value proposition is restraint, build the switch that removes it. It's the only demo that proves anything.
Two moments where the pattern earned itself
The agent argued against its own expansion. When the watchdog finds a new instrument, the system doesn't ask "shall I add it?" It computes cost and benefit — capital, coverage points, calendar days added to the close, steps and who owns them — and returns one of five verdicts. One of them is ADDS NO CAPITAL: this find gives you work and zero pesos, don't take it. A test fails the build if a zero-capital, twelve-step instrument ever comes back as a recommendation.
It also refuses to net money against days. Different units. It shows both sides and says why it won't collapse them into one convenient score. Every product instinct I have wanted a single number there. The single number would have been a lie.
The pattern blocked its own development. Near the end of the build I needed a seeded instrument that would produce a match, so the demo video would have a satisfying moment. I wrote the task with a rule at the top: if no real instrument crosses after an honest attempt, stop and report — don't loosen a gate.
It stopped. It came back with a measured explanation: 19 candidate instruments, all with no declared closing date; the one call with a real published amount closed in February; the PDFs in the repo are individual administrative acts, not calls. Making one cross would have required weakening a gate.
I lost the shot. I'd rather have the report — it's the same argument the product makes to its users, made to me, about my own convenience. That's the test of whether a constitution is real: it costs you something at least once.
What I'd tell someone building this
- Move the trust boundary, don't reinforce it. Don't ask a model for a number and validate it harder. Arrange things so the model never handles the number at all.
- Give your second opinion asymmetric authority. A reviewer that can only remove is far more trustworthy than one that can approve.
- Make the honest zero a designed output, with the same care as a success state. It's the answer you'll return most often in any domain worth automating.
- Encode the constitution as tests, not prompts. Ours is 729 assertions, including poisoned fixtures whose entire job is to fail the build when a guard is bypassed. A constitution that lives only in a system prompt is a suggestion.
- Build the counterproof. If you can't switch off the thing that makes you trustworthy and show what breaks, you haven't demonstrated it — you've asserted it.
Disclosure: RESOLVE was built for the All Things Agentic Hackathon on Cloud Run, Pub/Sub, Cloud Scheduler and Firestore, with Gemini 3.5 Flash via the Google GenAI SDK. Parts of the underlying agent patterns and the structured project dataset pre-exist in our C2X platform, which is separately submitted to XPRIZE Build with Gemini; what is new here is the watchdog and orchestration service, the evidence-gated matcher, the unblocking-route compiler, the cost/benefit scale, the multimodal document reader and the dashboard. Demo projects are seeded and labelled as demo data in the interface; the housing-deficit figures are real (DANE, CNPV 2018).
Try it: the live dashboard — no login. The Detection engine tab has the round trigger, the precision panel and the counterproof.
