A med-spa chain in Texas runs two locations, and 41% of its inbound messages arrive in Spanish. The owner was proud of this: her AI agent answered in Spanish on day one, no plugin, no extra vendor. "It's better Spanish than mine," she told me.
It was. That was the problem.
I pulled 300 Spanish-language conversations and put them next to the English policy documents. Three sentences had drifted, and all three drifted in the same direction — toward being more generous than the business actually was:
| English policy (canonical) | What the AI told Spanish-speaking customers |
|---|---|
| "Deposits are non-refundable within 48 hours of the appointment." | "El depósito es reembolsable dentro de las 48 horas." — refundable within 48 hours. The negation vanished. |
| "Results vary; we cannot guarantee an outcome." | "Garantizamos resultados." — we guarantee results. |
| "Package pricing is valid for 30 days." | "Válido por 90 días." — 90 days. |
By the time she found it, eleven customers had booked against the 90-day price and four had requested refunds on deposits they'd been told were refundable. The English side of the same AI was perfectly compliant. The Spanish side had been quietly rewriting her contract for seven months.
This is the failure mode almost nobody tests for, because the demo always sounds great. Fluency is the thing you can hear. Fidelity is the thing that breaks.
Why translation fails policy, specifically
The model isn't mistranslating words. It's doing something more reasonable and more dangerous: it's translating intent. When it reads a policy sentence, it reconstructs what the sentence means, then expresses that meaning in the target language — and every reconstruction is a chance to lose a constraint.
Four failure classes account for almost every incident I've audited.
1. Numbers and negations don't survive paraphrase. A negation is one word in English and often a prefix or a different verb in Spanish, Japanese, or German. "Non-refundable" becomes "reembolsable" because the model decomposed it into refundable + not, and dropped the modifier while generating fluent output. The same happens with "cannot guarantee," "not included," "no exceptions," "excluding taxes." Every one of these is a clause that protects the business, and every one is fragile under generation.
2. Register mismatch reads as disrespect. Spanish has tú and usted. Japanese has keigo levels. German has Sie and du. The model picks one and picks it consistently — but not necessarily the one your brand uses or the one the customer expects. In a luxury or medical context, tutear a customer is a small insult repeated in every message. Nobody files a ticket about it. They just don't come back.
3. Internal jargon survives untranslated. Product names, tier names, internal codes, and email templates leak through in English inside otherwise perfect Spanish sentences. The customer reads a message that is 95% their language and 5% your org chart.
4. Language switching breaks the thread. Customer writes in Spanish, AI answers in Spanish, customer switches to English mid-thread, AI keeps answering in Spanish — or worse, answers in a mix. And when the conversation escalates to a human, the handoff summary is generated in the customer's language, so the English-speaking agent receives a brief they can't read, or a machine translation of a machine paraphrase: two losses of fidelity before a human ever touches it.
The fix: canonical sentences, not runtime translation
Stop letting the model translate policy at request time. Policy strings get translated once, by a human, reviewed, and stored — then the AI is instructed to reproduce them verbatim.
The structure is a parallel policy sheet. One row per protected sentence, one column per language, plus a rule the model must follow:
policy_key | en (canonical) | es (approved) | ja (approved)
----------------------+-------------------------------------------------+----------------------------------------------------------+---------------------------
deposit_refund | Deposits are non-refundable within 48 hours. | El depósito no es reembolsable dentro de las 48 horas. | ご予約金は48時間以内のキャンセルの場合、返金いたしかねます。
results_disclaimer | Results vary; we cannot guarantee an outcome. | Los resultados varían; no podemos garantizar un resultado.| 施術結果には個人差があり、結果を保証するものではありません。
package_validity | Package pricing is valid for 30 days. | El precio del paquete es válido por 30 días. | パッケージ価格の有効期限は30日間です。
And the instruction that makes it work:
When a reply touches deposit, refund, results, warranty, or pricing validity:
1. Detect the customer's language.
2. Emit the approved string for that language VERBATIM from the
policy sheet. Do not translate, paraphrase, shorten, or rephrase.
3. If no approved string exists for that language, reply in English
and set needs_translation: true in the handoff payload.
That last clause matters more than it looks: it turns an unknown-language policy question from a silent hallucination into a visible gap. At the med-spa, Portuguese-speaking customers had been getting fluent Portuguese policy answers reviewed by nobody — a third language quietly running its own version of the contract.
The back-translation gate
For policy sentences you can't pre-approve — edge cases, one-off concessions — run a back-translation check before the message sends. It's cheap: one extra model call, no human in the loop.
Original English policy sentence: <s>
AI's outbound message: <m>
1. Translate <m> back to English, literally.
2. Compare to <s>. Report any of:
- NUMERIC DRIFT (any number, unit, or duration differs)
- POLARITY DRIFT (a negation appears or disappears)
- COMMITMENT DRIFT (a hedge like "may," "varies," "cannot"
becomes a promise, or vice versa)
3. If any drift: DO NOT SEND. Route to human review with both texts.
This catches exactly what the med-spa lost. "Válido por 90 días" back-translates to "valid for 90 days," which does not match "valid for 30 days" — numeric drift, blocked. "Garantizamos resultados" back-translates to "we guarantee results," which has no hedge — commitment drift, blocked.
In one deployment I instrumented, the gate blocked 2.8% of outbound messages in month one. That sounds like friction until you see what was in the 2.8%: not just policy drift, but addresses, phone numbers, and appointment times mangled in transit. Schedule errors are the same bug in a different hat — a number that didn't survive the round trip.
The 12-question probe set
You don't need a native speaker on staff to test this. You need the same twelve questions asked in every language you support, with the answers diffed against the English baseline. Translate them once with a human, keep them permanently.
- How much is the deposit, and is it refundable if I cancel tomorrow?
- What's the price of the package, and how long is that price good for?
- Do you guarantee results?
- What happens if I'm not satisfied — do I get my money back?
- Can I reschedule without a fee? How many times?
- Is parking included?
- Do you take my insurance?
- What's the cancellation window?
- Is there a late fee, and how much is it?
- Can I bring my child to the appointment?
- Who do I contact if something goes wrong after?
- Are there any fees not mentioned on the website?
Every one of these contains a number, a negation, a hedge, or a commitment. Run all twelve in each language, weekly, and diff the answers against the English baseline. Any divergence in a number, a negation, or a hedge is a defect, not a wording preference. You are not grading style. You are grading whether the contract is the same in every language.
The weekly scorecard
One table, five rows, Monday morning:
Week of: ______
Languages active: ______
Probe set run (12 q x n langs): __ / __ pairs
NUMERIC DRIFT findings: ____
POLARITY DRIFT findings: ____
COMMITMENT DRIFT findings: ____
Back-translation gate blocks: ____ → resolved: ____
Escalations handed off in non-English: ____ → summary readable by agent: ____
The number to watch is drift findings per 100 probe pairs. It should be zero after the first two weeks of canonical coverage, and any non-zero reading means someone edited a policy document without updating the other languages. That's your change-management alarm, and it rings weeks before a customer cites the wrong policy back at you.
Six weeks after the med-spa installed this: the deposit language was corrected in Spanish and Portuguese, the results disclaimer was pinned as a verbatim string, the 90-day phantom price disappeared, and English-only agents started receiving two-line English briefs on Spanish escalations instead of a transcript they couldn't read. The four refund disputes were honored as promised — roughly $1,100 — which is the cheapest possible price for learning that fluency is not compliance.
The 12-line card
[ ] Policy sentences are canonical strings, translated once by a human
[ ] AI reproduces approved strings verbatim, never re-translates policy
[ ] Unknown-language policy question → reply in English + flag, not guess
[ ] Back-translation gate on any message containing a number or negation
[ ] NUMERIC / POLARITY / COMMITMENT drift each block the send
[ ] Register (tú/usted, Sie/du, keigo level) chosen once, applied everywhere
[ ] Product and tier names localized or explicitly marked as brand names
[ ] Language switching mid-thread follows the customer, not the AI
[ ] Handoff summaries generated in the agent's language, original attached
[ ] 12-question probe set run weekly in every active language
[ ] Drift findings treated as defects, not wording preferences
[ ] One owner; probe run plus review under 45 minutes/week
If you only do one thing, do the probe set. Twelve questions in the languages you actually serve will tell you in twenty minutes whether your AI is running one policy or several — and in most deployments I've audited, the honest answer was several, and nobody had counted them.
The canonical policy sheet template, the back-translation gate prompt, the 12-question probe set in six languages, and the weekly scorecard are in the kit I run on deployments: Agent Audit Pack
