MUSTER: Building Safer Enterprise AI Agents That Ask Less and Retry Less

AI agents are getting good enough to do real work inside companies.

They can read documents, understand images, call tools, make decisions, and trigger actions.

But once an agent is allowed to do something important, two problems become dangerous very quickly:

How much private data should the agent be allowed to see?
What happens if an action succeeds, but the agent loses the response and does not know whether it happened?

I built MUSTER to explore those two problems.

The basic idea is simple:

Before acting, ask only for evidence that can still change the result.

After acting, if the result is uncertain, check what happened before trying again.

A Simple Example

Imagine an employee named Ravi says his Saturday pay is wrong.

To resolve the case, different systems hold different pieces of information.

The payroll system knows whether Ravi was scheduled.

The site system knows whether he was present and how long he was on site.

A simple AI-agent architecture could give one central agent access to all of those systems.

I did not want MUSTER to work that way.

Instead, each source keeps control of its own evidence.

The payroll agent works with payroll data.

The site agent works with site-access evidence.

The central MUSTER Control Plane does not get blanket access to the raw source data.

In the Google Cloud deployment, this boundary is enforced with IAM. The Site Agent can read its protected Cloud Storage evidence, while the tested central Control Plane identity receives an actual access denial.

The Site Agent can send the needed evidence content to Gemini for interpretation. After that, only validated and signed narrow facts are returned to MUSTER.

So the model can help interpret evidence, but interpretation alone does not give it authority.

The way I think about it is:

Models interpret. Sources attest. Deterministic MUSTER authorizes.

Why Signed Facts Matter

Suppose Gemini interprets a site attendance image and a gate log.

That model output does not automatically become truth inside MUSTER.

The source first validates the candidate observation and signs the resulting attestation.

MUSTER then checks whether that source is actually allowed to attest that type of fact.

For example:

Payroll can attest that Ravi was scheduled.
Site Access can attest that Ravi was present.
Site Access can attest information about Ravi's on-site duration.

Ravi saying, "I was there," is different.

His own claim can enter the case, but it has no institutional authority.

In the demo, I use Gemma 4 for this low-trust worker claim intake.

The claim can help start the process, but it cannot authorize the final action.

Asking Less Data

This is the part of MUSTER I found most interesting while building it.

The payroll policy requires Ravi to have worked at least 240 minutes.

The admitted site evidence establishes that his on-site duration was at least 508 minutes.

At this point, MUSTER still does not have an exact duration value.

A normal workflow might keep asking for more evidence until every unknown is resolved.

But while working on this case, I started asking a different question:

Can any remaining uncertainty still change the action?

For Ravi's case, the answer is no.

Every remaining possible duration that matches the admitted evidence leads to the same result.

So MUSTER does not need to collect the exact duration just for completeness.

The result is:

PAY RAVI INR 5,100.00

This is the corrected weekly payroll instruction for the worked example.

MUSTER is not claiming that it independently proved Ravi worked.

It is saying that under the pinned policy and the authorized facts admitted into the case, the consequence is now the same across the remaining possibilities.

In simpler words:

If more private data cannot change the result, stop asking for it.

The Second Problem: What If an Action Already Happened?

Evidence is only half the problem.

The other problem appears after an agent decides to do something irreversible.

Imagine MUSTER sends an external payment-like sandbox action.

The external system accepts it.

But before MUSTER receives the answer, the response is lost.

Now the system has a dangerous question:

Did the action happen or not?

This was one of the parts I spent the most time thinking about.

Retrying feels like the natural thing to do.

But if the first request actually succeeded, the retry itself can create the problem.

So MUSTER records this situation as UNCERTAIN.

It does not pretend to know what happened.

In the verified Google Cloud sandbox proof, the sequence is:

The synthetic external action is accepted once.
MUSTER deliberately loses the answer.
MUSTER records the execution as uncertain.
An independent read of the external sandbox shows that one effect exists.
A fresh Cloud Run process starts.
Instead of dispatching again, it inspects the external state.
It reconciles the execution to confirmed.
Redispatch count remains zero.
A final independent read still shows exactly one effect.

No real funds are involved. This is a sandbox proof.

The important idea is:

MUSTER does not retry an irreversible action just because it lost the answer.

It checks first.

Or more simply:

Reconcile — don't retry.

Why Deterministic Authorization?

LLMs are very useful in MUSTER, but I deliberately do not let them make the final authorization decision.

Gemini and Gemma help with interpretation.

But once structured evidence reaches the decision boundary, MUSTER uses deterministic code for things such as:

evidence admission
source authority checks
policy evaluation
consequence analysis
action binding
execution-state handling

This separation lets the models do what they are good at without making the model the final authority.

The architecture can be summarized simply:

Models interpret → sources attest → deterministic controls authorize.

Google Technologies Used

MUSTER uses Google technologies across the agent and cloud layers.

The project includes:

Google ADK for the agent runtime
Gemini 3.7 Flash for institutional evidence interpretation
Gemma 4 for low-trust worker claim intake
Vertex AI
Cloud Run
Cloud Storage
Cloud SQL
Google Cloud IAM boundaries

The hosted judge experience is a read-only verified replay, not live telemetry.

That replay exists so the important cloud proofs can be inspected reliably, including the IAM isolation and the uncertain-action reconciliation sequence.

The project also includes a live local agent path for the worked Ravi case.

What I Learned Building MUSTER

The biggest lesson for me was that making safer AI agents is not only about making the model smarter or more accurate.

Some of the important problems have to be solved around the model.

When I first started building MUSTER, I was mostly thinking about authorization.

But while working through the Ravi case, I noticed another question:

Why should the system keep collecting evidence after that evidence can no longer change the result?

The same thing happened with execution retries.

At first, retrying an action after a timeout sounds reasonable.

But if the first request already succeeded, retrying is exactly what can create a duplicate action.

That led to the two ideas at the center of MUSTER.

Before the action

Ask only for evidence that can still change the consequence.

After the action

If the result is uncertain, reconcile before retrying.

I think these ideas can apply beyond payroll too.

They can be useful in procurement, approvals, financial operations, compliance workflows, and other enterprise agent systems where sensitive evidence is distributed across different institutions.