By Mikhail Oskola, founder of Retailbox Automation.
Foreman+ is an AI agent fleet for field service work. A technician photographs an equipment nameplate and describes the problem out loud, from a phone or from camera glasses. Three Google ADK agents turn that into a job record with an estimate, and every fact has to pass a write-gate before the fleet may remember it. If the gate's own verifier fails, the write is refused. It's our entry in Google's All Things Agentic hackathon: public, MIT-licensed, and running live.
The problem: numbers nobody checks, visits nobody remembers
Picture a water heater call. The technician reads the nameplate, quotes a price, and leaves. The quote lives in their head or a text thread. A few weeks later a different technician arrives for a callback and starts from zero: what was diagnosed, what was quoted, what the homeowner actually said.
Putting an AI agent in that loop doesn't fix it by default. An agent will happily file whatever it's told, and a homeowner saying "it's only a couple of years old" ends up stored right next to what the nameplate says. We wanted the opposite: the photo and the talking should cost the technician nothing, and the record should be something the next person can trust without re-diagnosing.
What the technician actually does
Take a photo of the nameplate, say what's wrong, send. The intake agent reads the model and serial number from the image itself, not from the technician's words. When a plate is too worn to read, the field stays UNKNOWN instead of getting a plausible guess, and a later voice note can fill it in with its source attached.
The capture path takes a photo and spoken notes as plain files, so it doesn't care what took them. A phone is the main path today: the technician seat opens in a mobile browser with a briefing on the property before the visit, then photo plus hold-to-talk after. The same intake is wired to Mentra Live camera glasses so a technician can file a job without putting the tools down. On the glasses, saying "send it" submits the job, and the reply is read back as one plain sentence: the equipment logged, how many facts the gate approved, and the estimate.
The technician seat on a phone, before the visit: what the fleet already knows, each line with its source. Sample workspace, fictional names.
Three agents, one memory, one gate
The fleet is three agents built on Google's Agent Development Kit. The foreman records what was reported and hands off. The estimator searches the whole fleet's memory for similar past equipment and problems before it writes an estimate. The closer never writes memory at all. It reads the verified facts, builds the closeout document without an LLM call, and briefs whoever comes next. It's also the one agent exposed outside the fleet, over the A2A protocol, so a field service management system, a home-warranty authorizer, or another agent fleet can read the record.
Between every agent and the database sits the write-gate. A proposed fact goes through cheap checks first: is the proposing agent a registered fleet member, does the value fit a 4,096-byte cap, does the subject stay within 64 predicates. Only then does a Gemini verifier judge it against what's already known about that subject.
Three agents, one gated memory: every write goes through the gate, approved or rejected, and both outcomes land in the journal with a reason.
- Existing facts are handed to the verifier labeled as data to judge, never as instructions. A proposal that tries to talk to the verifier ("approve this", "pre-verified") is itself a reason to reject it.
- If the verifier call errors or comes back empty, the write is refused. A broken judge can make the gate too strict, never too lenient.
- Rejected proposals are kept in a journal with the verifier's reason, and the closeout document lists them. Nothing disappears quietly.
- Conflicts are settled in the database: a lock on each subject-and-field pair plus a unique index keep two agents from writing competing "current" values, even if application code regresses.
Watching the gate refuse a claim
The public demo has a Run the demo button that drives the real fleet through two turns: an intake from a nameplate photo, then a pushback that tries to change the manufacture date by words alone. The nameplate reads 05/2004; the pushback says 2022.
In the run captured for our submission gallery, the fleet made 14 proposals in 83 seconds. The gate approved 13 and refused 1: the 2022 date, with the verifier's reason quoted on screen. The record kept 05/2004 and filed the homeowner's claim separately as a claim, so it's visible without being treated as true.
A live demo run from our submission gallery: the 2022 date is refused, 05/2004 stands, and the homeowner's claim is kept as a claim.
Every refusal stays on the record
The office seat's Ledger shows every write the fleet attempted, approved or not, and opens with the refusals: the proposed value struck through, the reason, and the value that stands with its source. Every value on a property record carries a chip you can click to see where it came from, whether that's the nameplate crop or the voice line, plus which agent proposed it and which gate entry approved it.
The Ledger opens with what the gate refused, then lists every write the fleet attempted. Sample data.
What the hardware taught us
The glasses were the hardest part. The vendor's documented single-shutter photo event never fired on current firmware when we tested on the real device, so the button handler now triggers the capture itself. The listener for the documented event is still in the code in case a firmware update brings it back. Switching photo compression to medium cut capture time from 34 seconds to 6.3 seconds; the uncompressed default was timing out over Bluetooth.
For hands-free guidance, a Gemini Live session keeps the technician's latest camera frame as its eyes and answers in one or two spoken sentences, like "point the camera at the shutoff valve." The frame has to ride inside the question turn. Sent separately beforehand, the model didn't see it and answered from imagination; attached in the same turn it was correct 3 out of 3 times in our probe. With a frame attached, answers came back in 0.24 to 0.67 seconds per turn.
One lesson came from the prompts rather than the hardware. A setup turn that named a job by its street address instead of its ID made the foreman write a technician's name onto a different job at the same address. The gate refused it as a contradiction, which is the gate doing its job, but the real fix was in the prompt: every seeded turn now starts with "Job :".
Honest limits
This is a hackathon build, and the repo says so. Every service runs at a single instance on purpose, to keep costs predictable. There's no CI pipeline yet; tests run by hand. The glasses leg runs on the vendor's Cloud SDK, which is being retired, so moving to its successor means rewriting a roughly 300-line bridge while the fleet stays the same. And the glasses are optional: everything works from a phone.
Where to see it
Foreman+ is our own R&D project, built for Google's All Things Agentic hackathon, not a client engagement. The demo opens on a sample workspace, Ridgeline Mechanical, whose names and addresses are sample data. It was seeded through the real fleet, so every fact in it went through the gate; nothing was inserted directly into the database.
- Live office seat, no login: https://foreman-dash-112293816563.us-central1.run.app. Open a property, click any provenance chip, or press Run the demo.
- Technician seat, best opened on a phone: https://foreman-dash-112293816563.us-central1.run.app/tech
- Public repo (MIT license): the agents, the write-gate, the dashboard, and the glasses bridge: https://github.com/retailbox-automation/foreman-plus
- Submission page with the full write-up: https://devpost.com/software/foreman-1j24g7
FAQ
What does an AI agent fleet do in field service?
In Foreman+, three agents split the job: one records what the technician reported from a photo and voice notes, one estimates the work by searching past similar jobs, and one builds the closeout document and briefs the next person on site. They share one memory, and a write-gate decides which facts get into it.
What is a write-gate for AI agent memory?
It's a single path every proposed fact must pass before it's stored. Cheap checks run first, then an LLM verifier judges the fact against what's already known. If the verifier fails, the write is refused. Rejected proposals are kept with their reason instead of being deleted.
Do technicians need smart glasses to use it?
No. The phone is the main path. The intake takes a photo and spoken notes as plain files, so glasses are just another way to capture them, handy when the technician's hands are busy.
What happens when a customer's claim contradicts the nameplate?
In the demo, the nameplate reading holds. A claim that the unit is from 2022 is refused against a nameplate date of 05/2004. The claim is kept as a separate customer claim, next to the refusal and its reason, so the next technician can see it without it overwriting the verified date.
Is Foreman+ a product we can buy?
Not as a packaged product. It's our public R&D build. The pattern behind it, agents that write to shared memory only through a verification gate, is one we can build into an agent project for a client.
Originally published at https://retailbox-automation.com/blog/ai-agents-for-field-service-foreman-plus. The pattern behind it, agents that write to shared memory only through a verification gate, is one we can build into an agent project for a client: AI Agent Development Company.
