How teams move from "one API key per runtime" to "one gateway, any runtime"
The Fragmentation Problem (July 2026)
You've got a squad of coding agents now. Some teams use Claude Managed Agents. Others prefer Cursor. Finance is piloting Bedrock AgentCore. Engineering wants to stay self-hosted.
This sounds flexible. It's actually a disaster.
Here's what happens:
- Agent A (Claude Managed Agents team) stores API credentials in Anthropic console. No one else on the company can invoke it.
- Agent B (Cursor team) lives in a separate dashboard. Finance can't call it without direct Cursor access.
- Agent C (Bedrock, self-hosted) requires AWS IAM roles. Now you need to explain why the Marketing team can't use Bedrock.
- Agent D (N8N) sits in a workflow tool. No one knows it exists. No one can audit what it does.
The models are powerful. The infrastructure is a mess. Each runtime has its own authentication, its own API surface, its own session model. The agents themselves are stateless—the infrastructure that manages them is fragmented.
This is the multi-runtime agent problem of July 2026. Not a model problem. An infrastructure problem.
The Control Plane Solution
The answer is not "pick one runtime." The answer is: separate the agent control plane from the agent runtime.
The pattern:
- Runtimes (Claude Managed Agents, Cursor, Bedrock, N8N, custom) are where agents execute.
- Control Plane (LiteLLM Agent Platform, Microsoft Foundry, or equivalent) is where agents are governed.
The control plane sits between teams and runtimes. It's a unified gateway that:
- Normalizes invocation across runtimes — One API to call any agent, regardless of where it runs.
- Centralizes credentials — Teams don't need console access to each provider.
- Enforces policy uniformly — budgets, rate limits, tool authorization, audit trails — all in one place.
- Manages session continuity — If an agent crashes mid-execution, the session survives because Postgres is the backing store, not the runtime's ephemeral memory.
- Makes agents shareable — An agent built by one team on one runtime can be discovered and invoked by any other team through the control plane.
How This Works in Practice
Let's build it:
Step 1: Register Your Runtimes
Your control plane starts by connecting to each runtime. One YAML config:
runtimes:
claude_managed:
type: anthropic
api_key: env/ANTHROPIC_API_KEY
cursor:
type: cursor
api_key: env/CURSOR_API_KEY
bedrock:
type: aws_bedrock
region: us-west-2
credentials: env/AWS_ROLE
self_hosted:
type: custom
base_url: https://internal-agent-api.company.com
The control plane normalizes these. A single POST request to the control plane invokes any runtime:
curl -X POST https://control-plane.company.com/agents/pr-babysitter/invoke \
-H "Authorization: Bearer $CONTROL_PLANE_KEY" \
-d '{
"runtime_hint": "any", # Fallback: try best available
"prompt": "Review this PR for security issues",
"tools": ["git", "github_api"]
}'
The control plane picks a runtime (round-robin, cost-aware, or explicit), translates the request, executes, and returns a unified response format.
Step 2: Decouple Credentials from Teams
No developer touches provider consoles directly. Instead:
- Secrets stored in one vault (HashiCorp Vault, AWS Secrets Manager, etc.)
- Control plane injects credentials at invocation time
- Every call is logged with who called what, when, and why
credentials:
anthropic_shared:
provider: anthropic
api_key: vault://secrets/anthropic-prod
rotated: 2026-07-20
last_used: 2026-07-24
cursor_shared:
provider: cursor
api_key: vault://secrets/cursor-prod
last_used: 2026-07-24
Teams request agent invocation. The control plane handles credential binding. No sprawl. Auditable.
Step 3: Enforce Unified Policy
Tools, budgets, rate limits, and approvals live in the control plane—not scattered across four provider consoles.
agents:
pr_babysitter:
owner: engineering
runtime: any # Will use cheapest/fastest available
tools:
- github: read
- linear: read
budgets:
daily_usd: 50
monthly_usd: 1000
rate_limits:
calls_per_minute: 10
concurrent_sessions: 5
approval_rules:
- if_tool: github_push
requires: human_approval
- if_cost_exceeds_usd: 100
requires: human_approval
Policy changes propagate instantly. No redeploy. No framework fighting.
Step 4: Manage Session Continuity
Sessions live in the control plane (Postgres-backed), not in the runtime's memory. An agent running on Cursor can crash, and the control plane resumes it on Bedrock without losing state:
session:
id: sess-pr-babysitter-82946
agent: pr_babysitter
runtime: cursor # Started here
created_at: 2026-07-24T14:30:00Z
steps:
- turn: 1
tool_call: github_list_prs
result: ["PR #1024", "PR #1025"]
cost_usd: 0.12
- turn: 2
tool_call: github_get_diff PR#1024
result: "[diff content]"
cost_usd: 0.08
status: executing
last_activity: 2026-07-24T14:32:15Z
If the Cursor runtime crashes at turn 3, the control plane can:
- Resume on Bedrock (same cost model, similar capability)
- Inject the session history automatically
- Continue from turn 3 without re-running turns 1-2
Step 5: Discovery and Sharing
Control plane maintains a registry of all agents. Teams discover them:
# List all available agents
curl https://control-plane.company.com/agents \
-H "Authorization: Bearer $KEY"
# Response:
{
"agents": [
{
"id": "pr_babysitter",
"owner": "engineering",
"runtime": "any",
"description": "Reviews PRs for security issues",
"calls_this_month": 4829,
"avg_cost_per_call": 0.23
},
{
"id": "compliance_check",
"owner": "legal",
"runtime": "any",
"description": "Audits contracts for risk clauses",
"calls_this_month": 312,
"avg_cost_per_call": 1.42
}
]
}
Finance calls the PR babysitter. Marketing uses the compliance checker. All through the same control plane.
Why This Matters
Before (fragmented):
- Four teams = four ways to invoke agents = four auth systems = four audit trails
- New runtime = new integrations everywhere
- Moving an agent from Cursor to Bedrock = rewrite invocation code in every consumer
- Cost tracking scattered across provider dashboards
After (control plane):
- One control plane = one invocation API = one audit trail
- New runtime = register once, works everywhere
- Agent portability across runtimes = zero consumer code changes
- Cost tracked per agent, per team, per runtime in one place
The Production Pattern
This is not theoretical. LiteLLM Agent Platform is a Rust-based AI Gateway and Agent Control Plane whose goal is to let teams register, invoke, observe, and govern agents across multiple runtimes, with LiteLLM treating this as an experimental project to learn quickly.
LiteLLM Agent Platform manages unified API across runtimes—one API to create and run agents, regardless of the runtime underneath, with developers creating and running agents here without requiring Bedrock or Anthropic console access.
The five-question evaluation framework:
- Can I invoke any agent through one API? (Not five provider SDKs)
- Can I change policies without redeploying agents? (Budgets, approvals, tool auth in control plane)
- Can agents move between runtimes without rewriting consumers? (Session portability)
- Do I get unified audit trails? (All calls logged in one system)
- Can new teams discover and use agents without console access? (Agent registry, centralized secrets)
If the answer is "no" to any of these, you're still in the fragmented phase.
Next: The Maturity Curve
July 2026 moment: Teams are shipping agents on multiple runtimes. The ones that don't solve the fragmentation problem hit walls around month 3:
- Credential sprawl (who has access to what?)
- Cost visibility (which runtime is cheaper?)
- Policy inconsistency (why does one agent have budget approval, the other doesn't?)
- Session loss (agent crashes mid-execution, work lost)
The ones that build a control plane first move faster. Fewer meetings about credential management. Fewer surprises on the bill. Fewer post-mortems about lost work.
The pattern is converging. Agent infrastructure is separating into three layers: models, harnesses, and runtimes, with a fourth layer emerging as the unified agent control plane to allow calling agents living in different agent runtimes all from 1 place, because companies will not run every agent on one runtime.
This is the infrastructure layer that will define who wins in multi-agent 2026. Not the smartest models. Not the flashiest frameworks. The teams with the clearest control plane.
Paul Twist is an AI infrastructure engineer based in Berlin. This post covers the multi-runtime control plane pattern emerging as table-stakes for production agent deployments in July 2026.
