The other day an agent I was testing kept hitting production. Not on purpose, not because it was being clever. It was reading the OpenAPI spec we handed it, picking the first entry in the servers array, and going. That entry was https://api.example.com. We wanted it to hit https://api.staging.example.internal. It did not care.
This sounds like a one-line prompt fix. Add "use staging" to the system message, done. It is not a one-line fix, and it does not stay done.
Why the prompt route rots
The agent picks the baseUrl from the spec because that is what the spec says to do. The prompt override fights that default. The fight has three failure modes.
First, the override is invisible in the tool definition. You see POST /orders in the tools list. You do not see that the agent was told to swap the host. Six weeks later, someone runs the agent in a fresh shell without that system prompt and prod gets hit again.
Second, different agents resolve the conflict differently. Cursor honors the last instruction in the context. Cline tends to follow the spec over the prompt. Claude Code splits the difference when it is unsure. You end up tuning per-agent prompt wording, which is its own maintenance burden.
Third, when the spec adds a new server (a new region, a preview environment), the prompt does not know. You are back to editing system messages per environment.
Where the baseUrl actually belongs
The gateway sits between the agent and the upstream. It is the one component that already knows which environment it is serving. Pin the baseUrl there, per route or per deployment, and the agent never sees the question.
Concretely:
- The OpenAPI spec keeps
serversas documentation of where the API can live. It is the contract, not the runtime config. - The gateway has a routing table:
POST /orders->https://api.staging.example.internalin dev,https://api.example.comin prod. The mapping is deployed with the environment, not shipped in the prompt. - The MCP tool the agent exposes still says
POST /ordersagainst whatever URL the gateway presents. The agent never has to reason about hosts.
This is the same argument as storing the upstream token in the gateway rather than in the MCP client config. Anything the agent does not need to know, the agent should not be able to read. If the baseUrl is in the prompt, the agent can (and will) change it. If it is in the gateway routing table, it cannot.
The failure this prevents
The most expensive version of this bug is not a wrong GET. It is an agent that, told to "test the checkout flow," creates real orders against prod because the prompt said staging but the spec said prod and the agent chose the spec. Refunds, webhooks fired to real customers, inventory decremented. None of that is recoverable by re-running the test.
The cheap version is quieter: the agent "verifies" its code against prod responses, declares success, and the staging-only bug ships. You do not notice until the next deploy.
Both go away when the gateway is the only thing that knows the upstream host. The agent is just a caller. The spec is just a shape. The environment lives where environments belong.
What we do in Powerduck
This is the reason the gateway in Powerduck keeps the baseUrl override on the route, not on the spec. The OpenAPI file stays the source of truth for what the API is; the gateway decides where it goes for the current deployment. When you point a local MCP client at the gateway, it gets one URL, one key, and one set of tools. It does not get to pick the environment.
If you have been patching this with system-prompt lines, try moving the override down one layer. The prompt gets shorter, the agent stops arguing with itself, and the next person on call does not have to read your chat history to figure out where staging actually lives.
