On July 22 I typed a one-line prompt on a test app: a widget built on a well-known REST API, the kind that authenticates with an X-Api-Key header. The model wrote a clean widget. It also wrote my key into a request header, in the page's JavaScript. In the declaration that comes with the code, it had marked the API as not secret.
It was my own key, on my own test app. At GoodBarber we are piloting a builder in which an app owner describes an extension and a model generates its code. Nothing had been released yet, and a review before release is the cheapest place to find a key in the wrong spot.
What we changed is the answer to the question in the title. We do not scan generated code for secrets, and we do not let the model decide whether a key is secret. The model declares every key an API needs and states one fact about it: how the key is sent. Code does the classification: a key is secret unless it belongs to a family its provider documents as public. The value goes from a masked field to a server-side proxy. It never reaches the model, the page or the bundle.
A key that looked public
The model was not being careless. The keys it treated as secret without being told were the famous ones, AI providers and payment secret keys. A forty-character string with no prefix, sent in a custom header to a REST API, looks like what front-end tutorials do all day. And that reading is not absurd, because some keys really are public.
I have written on the company blog that security is the best-documented wall between a demo and production. This is what that wall looks like from the inside, on a Wednesday.
Some keys are public by design
Stripe lists its publishable key as safe to expose. Firebase documents its API keys as public by design. Supabase documents its publishable key as safe to expose online, and Algolia its search-only key as safe for production front-end code. These keys identify a project. Authorization happens elsewhere: security rules, row-level policies, restricted scopes.
So "never put a key in the page" is wrong, and a platform that enforced it would break every one of those integrations. The real question is classification. A model will get that call right many times and wrong some times, and one wrong call publishes a key.
Why we did not scan the code
The reflex is to search the generated JavaScript for things that look like keys. Two things argued against it.
We had already lived through pattern-matching on generated code. An earlier validator matched unsafe JavaScript constructs with a list of regular expressions. It produced enough false positives to be switched off, and I deleted it on July 16. The design notes for the secrets work say it in one line: the secret guard must not reintroduce pattern-matching on generated code.
And most secrets do not look like anything. GitHub redesigned its token format in 2021 because its old tokens were hard to tell from a SHA hash; with identifiable prefixes, it expected the false positive rate of secret scanning to fall to 0.5 %. That works when the provider cooperates. GitGuardian's 2025 report counts 23,770,171 new hardcoded secrets in public GitHub commits in 2024, and classifies 58 % of the leaks it detected as generic credentials, the ones no provider-specific pattern describes. The key in my widget was one of those. Forty characters, no prefix.
A scanner would have been blind to the exact key it was built for, and noisy about everything else.
A decision with a tripwire
Here is the part I am glad we wrote down. In the plan, dated July 13, the model's secret flag was trusted, and a stricter guard, secret unless proven public, was considered and deferred. Deferred with a condition, in writing: revisit if a leak by under-classification is observed.
It was observed on July 22, on my test app. The stricter guard was committed the same afternoon, at 15:01.
A deferred safeguard without a written trigger is a safeguard that "later" never schedules. The trigger turned a debate into a lookup.
Default to secret, decided by code
The fix has two layers, and neither reads generated code. Both read the declaration: the structured list of APIs and keys that the model must produce alongside the widget.
The first layer is the instruction given to the model. Its header says "DEFAULT TO SECRET when unsure", and its core is one rule: a key is secret unless the provider explicitly documents it as public, publishable, anon, browser or search-only, and a key that authenticates your account is secret no matter how it is sent, a custom header and a query parameter included.
The second layer is the one that matters, because an instruction is a request. A validator checks the declaration. An API declared as not secret, whose key does not belong to a family that providers document as browser-safe, is rejected. The rejection feeds the retry loop we already had, with a message that tells the model to route the call through the proxy helper. The exemption reads the structured fields of the declaration, never its prose: a description saying "get it from the public dashboard" must not make a key public.
The model can still raise a key to secret. It can no longer lower one. Only evidence can.
Ask the model for facts, not for judgment
The next morning showed that the rule was not enough. Reclassifying a key as secret is useless if nobody knows how to send it. To wire a server-side connector, the platform needs to know whether the key travels as a bearer token, a named header, a query parameter or a path segment. The model had been asked for that only when it judged the key secret. A key it judged public arrived with a name and a label, and nothing to build a connector from.
The instruction now asks for the transport of every key, always, whatever the model thinks of its secrecy. The prompt says why in its own words: it is "a FACTUAL property of the API, not a consequence of the secret-vs-public call".
This is the distinction I would keep if I had to drop everything else. Where does this API expect its key, a header called X-Api-Key, a query parameter called appid? That is documented, and models are good at documented facts. Is this key dangerous in a browser? That is a judgment about consequences, and the model's prior is a web full of inlined keys. So the contract asks for the fact unconditionally, and code makes the judgment.
The same part of the prompt carries the inverse rule: never invent a fact. When an API authenticates with an OAuth2 token exchange or per-request signing, none of the declared transports describes it. The model is told to say so plainly and leave the API out, rather than declare a static bearer token that would pass validation, collect a key, and fail with a 401 on every call.
The same rule at every door
One more place needed the rule. Our generation runs in two phases: a plan the owner approves, then the code. The key is requested when the plan is approved, before any code exists. A guard that runs on generated code arrives too late, because the question "is this a secret?" has already been answered by then. The classification now runs at the single point where all four completion paths collect what must be asked of the owner.
A classification rule has to live at every boundary the value can cross. Otherwise the earliest boundary wins.
There is a second net where the value itself enters, the consent form and the credentials screen. If a pasted value carries an unambiguous vendor signature, sk-, ghp_, xoxb-, AKIA, a PEM header, it is routed to the proxy even when its declaration said public. That net is deliberately narrow, because it must never match a publishable key. Ambiguity is handled upstream, by the default.
Twenty-five out of fifty-one
A default that leans towards "secret" can overcorrect. Two weeks later one of ours did, in a neighbouring rule, for one day. It is the part of this story I find most useful, because it is the opposite failure.
Some APIs require a non-secret header on every call: a version header for Anthropic or Notion, a host header for RapidAPI. We let the model declare those as constants. Constants are stored in clear, so a constant must never carry credential material, and the failure to guard against is the model splitting a pair like X-RapidAPI-Host and X-RapidAPI-Key the wrong way round.
The first guard, written on August 4, refused any constant whose name contained one of ten markers, key, token, secret and auth among them. It looked strict and safe. The next day a review pass ran it against 51 header names taken from real API documentation, and it refused 25 of them. X-Auth-Email, which Cloudflare pairs with X-Auth-Key, is an e-mail address. X-Auth-Client, X-Auth-User, X-Oauth-Version, Authority: all legitimate, all refused.
A false positive here is not a harmless excess of caution. A refused constant means a rejected declaration, a retry loop that burns its attempts, and an API the owner legitimately wants that cannot be integrated at all. The same day, a sibling rule turned out to forbid Authorization as the place where a key is injected, which is exactly where some APIs expect it. Two over-blocks in one family, fixed the same morning.
In the names we tested, the last segment carried the meaning. X-Auth-Key is a secret. X-Auth-Email is not. The guard now splits the name on its separators and reads the last segment only, against a slightly wider list of markers. Rerun the same day: 21 declaration patterns from real APIs accepted, then 12 more, and 7 out of 7 secret-carrying names still refused.
That is an observation about those families of headers, not a law of HTTP. Idempotency-Key ends in key and is not a secret. As a constant it is still refused, and I can live with that: an idempotency key changes with every request and has no business being a constant. A heuristic earns its place by being measured on real names, in both directions.
Twenty-five out of fifty-one. A guard that refuses half of the legitimate world is not strict. It is wrong in the other direction, and people route around guards that are wrong.
Choose which way to be wrong
The errors now fall on the cheap side. A public key classified as secret costs one proxied call instead of a direct one: slower, and no key is disclosed. A secret key classified as public costs the key. When a classifier has to be wrong sometimes, choose which way, then measure the other direction too.
If a model writes code that calls paid APIs on behalf of your users, where does the decision "this key is secret" live in your system: in a prompt, in a scanner, or in code that can say no?
