I built OG Foreman so one construction update can do more than sit in a chat

Hello devposters, I'm Daries, CS student, smart contract developer and AI Engineer based in Accra - Ghana.

I created this article for the purpose of entering OG Foreman in the #AllThingsAgentic Hackathon.

The problem

A construction-site update is usually short.

A foreman might say:

First-floor blockwork is complete. The electrician did not come today.
We have 10 bags of cement left. Plastering starts tomorrow.

But that one update creates several jobs. Someone must update progress, record the missed electrician, check whether plastering is now at risk, calculate the
cement shortage, prepare a request, update the daily report, and remember to
follow up.

On many sites, those details are spread across calls, chat messages, spreadsheets, and individual memory. The message is delivered, but the project record does not always catch up.

I built OG Foreman to close that gap.

What OG Foreman does

OG Foreman is an AI construction site coordinator. A site team can report what
happened using text, voice, or photos. OG connects that update to the actual tasks, materials, dependencies, and records in the project.

For the example above, OG can:

mark blockwork complete;
record the electrician's absence as a blocker, not as completed work;
update cement stock to 10 bags;
compare that stock with the cement required for plastering;
prepare a material request for the shortage;
refresh the Daily Log and activity history; and
bring the purchase decision back to a human for approval.

The last point matters. OG does not place an order or make a supplier commitment by itself. It prepares the action, explains why it is needed, and waits for an authorized person to approve or reject it.

OG also supports project setup. A project manager can upload an existing schedule, scope, or material plan. Gemini extracts a draft, normal application code validates it, and the manager reviews it before anything becomes project truth.

How I built it

The system has four clear responsibilities:

Gemini 3.6 Flash reads project sources and interprets authorized site evidence. It returns structured information, but it cannot write directly to the project database.
Google Agent Development Kit (ADK) coordinates the operational workflows,including the point where a consequential action must pause for approval.
Typed tools and application services check identity, permissions, project references, safety rules, and duplicate requests before changing anything.
Cloud Firestore stores the project truth and the audit history.

The web app uses Next.js, React, TypeScript, Firebase Authentication, and Firebase Hosting. The FastAPI API and private worker run on Cloud Run. Pub/Sub moves durable project events to the worker, and Cloud Storage holds private media. Cloud Logging and Cloud Trace connect a visible project result to the workflow that produced it.

This separation is deliberate: the model can understand language, but it does not get unchecked authority over the construction project.

The hardest parts

Understanding what did not happen

Construction updates are short and contextual. A sentence such as “the electrician did not come” must create a blocker. It must never mark electrical work complete just because the sentence contains “electrician.”

I handled this by combining structured model output with deterministic checks,known project IDs, aliases, and explicit rules for ambiguous or negative language.

Making approval survive a retry

An approval is more than a button. The system has to remember the exact action, the version that was reviewed, the person who decided, and the workflow that was paused. If a message is delivered twice or a worker restarts, the system must not create two requests or repeat an earlier action.

Firestore transactions, stable event IDs, persisted claims, and idempotent tools make that behavior testable.

What I am proud of

OG Foreman does not stop at producing a summary. One update can create linked, auditable follow-through across progress, blockers, materials, approvals, and reporting.

I am also proud that the safety boundary is easy to explain: routine record keeping can move automatically, but purchases, external commitments, major schedule changes, cancellations, and safety-critical actions remain under human control.

What I learned

The most important part of an agent is not how confidently it talks. It is how clearly the system defines what the agent may do, what normal code must verify, and when a person must decide.

I also learned that reliability is part of the user experience. An impressive demo is not enough if a retry creates a duplicate material request or a restart forgets an approval. Durable state and an audit trail are product features.

What is next

The next step for OG Foreman is a broader public beta with more repeatable live evaluations, stronger multilingual site updates, manager-reviewed corrections, and clearer media-retention controls. Any new external integration will keep the same approval and audit rules.