What 100% Test Coverage Missed: State Across Google ADK A2A Boundaries

I created this article for the purpose of entering the All Things Agentic Hackathon.

TL;DR — An ADK output_key writes into the session of the agent that declares it. In-process that session is shared, so it looks like state flows. Across a RemoteA2aAgent hop it is the worker's session, and it never comes back. Nothing raises. Nothing warns. Every local run and every CI job exercises the working topology, so the failure is invisible to an offline test suite by construction — including at 100% coverage.

The system that passed

Bastion is a three-agent access-governance fleet built with Google ADK and A2A. An Orchestrator owns investigation state, an Access Auditor reads production IAM through a read-only identity, and a model-free Escalation Agent delivers validated count-only reviews.

The local graph passed its configured core statement and branch coverage gate. Every branch, every seam.

Then the same graph was split across deployed A2A workers, and an assumption that looked natural in-process became false.

The boundary we had not modeled

In-process, the previous step's result is simply there:

# The Auditor declares output_key; the Orchestrator reads it back.
report = ctx.session.state.get(AUDIT_FINDINGS_KEY)

Deploy the same sequence and only the construction changes. The graph is identical:

RemoteA2aAgent(
    name="access_auditor",
    agent_card=card_url(auditor, "access_auditor"),
    description="Reads the live IAM policy and flags anomalies. Read-only.",
    httpx_client=private_a2a_client(auditor),
    a2a_request_meta_provider=_forward_investigation,
)

output_key still writes. It writes into the worker's session, which never crosses back. The deployed Orchestrator saw an empty state key while every local run and every test saw a populated one.

Observed 2026-08-22: the Auditor completed a full sub-trail, and the next step then refused with "returned no structured report." No exception at the boundary. No warning at construction. The run still reported completed.

Two directions are easy to confuse:

A remote worker writes its output_key into its own session without making that state available to the caller.
A caller-side event whose useful result exists only as state contributes no outgoing A2A content for the next remote step.

Neither requires a crash. Plausible older content can remain in the caller and make the workflow look successful while the new state never crossed. That is far more dangerous than an obvious exception — a silent wrong answer only beats a loud failure in the sense that it survives longer.

The production fix

Bastion stopped treating remote session state as transport.

The authoritative findings now cross the boundary as structured, validated message content. Reading the reply is sound rather than a workaround because of output_schema: the Auditor's final content is validated JSON, not prose the caller has to interpret. Anything that fails to parse is skipped rather than guessed at, and an empty result still fails closed downstream.

The caller then verifies that content field by field against the deterministic tool output — IDs, categories, departments, scores, and required routing must all match. A model may supply bounded rationale. It cannot author or alter the finding set.

Correlation uses durable request metadata rather than model-visible prose. invocation_id groups one ADK run and stops at the A2A boundary; investigation_id is carried as request metadata and joins the complete distributed flow, so no model reads or restates it.

Completion is explicit too: the Orchestrator cannot mark an investigation done until every required department delivery has a validated receipt.

The result is not a workaround that copies more state. It is a service-boundary contract.

What we reported upstream

The deployed reproducer and both loss directions are in google/adk-python#6854, which is open.

A community change, #6859 by Sylvester Kaczmarek, addresses the state-only outbound loss and the rejected inbound remote-state-delta case. It is open and unmerged.

A second Google collaborator later added that the output_key half is expected behaviour — remote sessions are per-agent by design. That is true, and it was never the complaint. The report asked for the silent boundary to become observable, not for the semantics to change. They also asked for a reproduction to test #6859, which I supplied the same day.

I also opened #6862, adding a construction-time warning for the adjacent LlmAgent(output_key) -> RemoteA2aAgent pattern. A Google maintainer closed it unmerged on 2026-08-24, and the reply is the most useful artifact this whole exercise produced:

The session state boundary across A2A is a general transport property rather than an issue specific to SequentialAgent. It applies equally to Workflow, custom agents, and standalone runs. This is already being addressed centrally at the RemoteA2aAgent converter and documentation layer in #6859.

Two further reasons were given: SequentialAgent is deprecated in favour of Workflow, so new domain-specific validation should not be added to it; and my sys.modules.get reflection check was import-order sensitive and could silently skip validation. The last one is simply a correct review comment.

So the honest scoreboard: the boundary is confirmed by a maintainer and being addressed centrally, the issue is open, and my patch was closed as the wrong place to fix it.

That is a better outcome than a merged patch in a deprecated component — and it is not the same thing as "Google accepted our fix." Nothing here is merged. Check the live state before you repeat any of it.

The general rule

Treat remote agent state exactly like state across any other service boundary:

decide which service owns it;
serialize only the fields the next service needs;
validate the schema and the semantics on receipt;
carry correlation separately from model content;
make completion depend on acknowledged side effects;
test the deployed topology, not only an in-process imitation.

One hundred percent configured coverage can prove every local branch ran. It cannot prove the deployment topology preserved an assumption the local topology made invisible.

That distinction is the whole article. Coverage measures what your tests execute, not what your architecture assumes.

Why this matters for institutional agents

Long-running agents remember approvals, retries, leases, exceptions, and delivery receipts. If that context crosses departments or identities implicitly, the system cannot explain who owned a decision or whether it arrived.

For an access-governance fleet that is not a bug class, it is a compliance failure: "the state was there locally" is not an audit trail.

An explicit state boundary makes the fleet easier to secure, audit, recover, and extend — and it is the difference between an agent system you can deploy and one you can only demo.

Bastion — repository, evidence, and demo: