A user reached out to us this week. Their generated newsletter contained the word delve. Twice.
This immediaimmediately shot alarm spikes through the team because that word has been on our banned list since version one. The system prompt in lib/prompts/anti-ai.ts tells the model never to use it. Gemini 3 used it anyway, and this was a big issue.
This is the documentation of everything we did: the architecture we shipped to fix it, and the latency numbers from the first 48 hours in production. If you ship LLM output to end users, you probably need this layer too.
Does Better Prompting Make AI Output Better?
Short answer: No.
Prompts alone stop AI slop in roughly 80% of generations. The remaining 20% is where production reputation lives. Our fix for this is a code-side validator that scans every draft against a structured banned lexicon, runs four detection passes (vocabulary, phrases, openers, regex structures), and triggers one bounded repaired retry on slop. Worst case is that the latency (time-to-output) goes from N seconds to roughly 2N. The average latency is unchanged, and the user gets a draft that does not read like ChatGPT.
What is "AI slop" and why does it slip past prompt rules?
AI slop is low-quality, formulaic, machine-sounding text that an LLM produces by default: bloated paragraphs, corporate buzzwords, predictable cadences, and a recurring set of vocabulary crutches like delve, tapestry, robust, and crucially. The term entered general use in 2024 and was named American Dialect Society's word of the year for 2024. Wikipedia tracks the broader phenomenon.
The reason it slips past prompt rules is structural, and not a bug. Three things break the prompt-as-contract assumption in production:
- Attention dilution. The longer your system prompt grows, the less weight any single rule carries during decoding. By the time any LLM is generating token 1,800 of a long-form article, the rule "do not use the word delve" is competing with several thousand other instructions and the entire user input. Anthropic's own prompt engineering guidance acknowledges that instruction following degrades over long contexts.
- Regression to the training mean. LLMs are predictive engines. When a sentence is half-built and the next likely token is a high-probability buzzword that appeared millions of times in the training corpus, the model picks it. A negative instruction in the prompt is a soft constraint. The training data is a hard prior.
- No inference-time ground truth. The model has no way to verify it complied. It cannot self-check the same way a TypeScript compiler can. Whatever rolls out of the final softmax is what ships.
We have written about why standard prompting alone is not enough in our Banned Lexicon deep dive and the System Personas deep dive. The TL;DR is that soft instructions only carry you to about 80% reliability. Production needs an enforcement layer on top.
This video goes into more detail.
What is a banned lexicon, and how is it different from a safety filter?
A banned lexicon is a curated list of words, phrases, sentence openers, and structural patterns that signal AI-generated text. It is a quality filter, not a safety filter. Safety filters block harmful content. A banned lexicon blocks bland content.
At Ozigi, the lexicon contains six categories: vocabulary tells (delve, tapestry, robust), corporate fluff (cutting-edge, game-changer, thought leadership), AI tells (at its core, plays a significant role, in today's fast-paced), Gemini affirmation tells (Certainly!, Here is, Let's explore), engagement-bait closers (Tag someone who needs this), and structural patterns (the bold-colon paragraph prefix **Term:**, double-hyphen em-dash substitutes, contrast structures like "It's not X. It's Y.").
Until last week, that lexicon lived only inside the prompt. The fix was to also live inside the code path.
The two-layer architecture
The full surface looks like this:
┌─────────────────────────────────────────────┐
│ lib/prompts/anti-ai.ts │
│ ───────────────────── │
│ ANTI_AI_RULES ← prose for the LLM │
│ BANNED_WORDS ← code-side │
│ BANNED_PHRASES ← code-side │
│ BANNED_OPENERS ← code-side │
│ BANNED_CLOSERS ← code-side │
│ BANNED_REGEX_PATTERNS ← code-side │
└─────────────────────────────────────────────┘
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────────┐
│ lib/prompts.ts │ │ lib/prompts/long-form.ts │
│ Social engine │ │ Long-form engine │
│ (X · LI · DC · EM) │ │ (blog · newsletter) │
└──────────────────────┘ └──────────────────────────┘
│ │
▼ ▼
LLM call LLM call
│ │
▼ ▼
┌──────────────────────────────────────────────┐
│ lib/prompts/lexicon-validator.ts │
│ validateText / validateCampaign / repair │
└──────────────────────────────────────────────┘
│
▼
slop? → one bounded retry → keep cleaner output
clean? → ship
anti-ai.ts is the single source of truth. It exports both the prose rulebook the model sees and the structured arrays the validator scans against. A dev-mode drift guard warns if anything drifts between the two so the rulebook can never silently disagree with the validator.
// lib/prompts/anti-ai.ts (excerpt)
export const BANNED_WORDS: readonly string[] = [
'delve', 'delving', 'tapestry', 'realm', 'paradigm',
'robust', 'seamlessly', 'underscore', 'pivotal',
/* ...several hundred more */
];
export const BANNED_REGEX_PATTERNS: readonly {
label: string;
pattern: RegExp;
kind: 'banned-structure' | 'banned-contrast' | 'banned-cadence';
}[] = [
{
label: 'bold-colon paragraph prefix (**Term:**)',
kind: 'banned-structure',
pattern: /\*\*[^*\n]{1,40}:\*\*/g,
},
{
label: 'contrast: "It is not X. It is Y."',
kind: 'banned-contrast',
pattern: /\bit\s+is\s+not\s+[\w\s,'-]{1,40}\.\s+it\s+is\b/gi,
},
// …seven more contrast patterns from §5 of the prose rules
];
if (process.env.NODE_ENV !== 'production') {
// Drift guard — warn if structured entries are missing from prose
const proseLower = ANTI_AI_RULES.toLowerCase();
for (const w of BANNED_WORDS) {
if (!proseLower.includes(w.toLowerCase())) {
console.warn(`[anti-ai] structured entry "${w}" missing from prose rules`);
}
}
}
What does the validator actually scan for?
lib/prompts/lexicon-validator.ts runs four passes on every parsed draft:
Pass 1: vocabulary. Word-bounded, case-insensitive match against BANNED_WORDS. Hits return { kind: 'banned-word', term, snippet, location }.
Pass 2: phrases. Whole-token-sequence match against BANNED_PHRASES. Catches multi-word slop like navigate the complexities or gain valuable insights.
Pass 3: openers and closers. Position-aware. An opener match only fires if the term appears at the start of a sentence, paragraph, or post — not mid-sentence where it might be legitimate.
Pass 4: regex patterns. The structural tells. The bold-colon prefix Gemini loves (**Architecture:**), double-hyphen em-dash substitutes, and seven variants of the "It's not X. It's Y." contrast structure.
Two details matter for precision. First, code-block sanitization. Engineering content includes JSON, shell commands, and inline code where words like delve might legitimately appear (or never appear, but you don't want a regex false-positive on a JSON field name). The validator strips fenced blocks, inline code, and URL targets before scanning:
function sanitize(text: string): string {
return text
.replace(/```
{% endraw %}
[\s\S]*?
{% raw %}
```/g, '') // fenced code
.replace(/`[^`\n]+`/g, '') // inline code
.replace(/!?\[[^\]]*\]\([^)]+\)/g, ''); // markdown links + images
}
Second, same-opener cadence detection. For Gemini, its signature tell is starting three or more consecutive sentences with the same word or short phrase. The validator splits on sentence boundaries and reports a banned-cadence violation when it sees three+ consecutive sentences sharing a leading word.
The output is a typed ValidationReport:
export interface ValidationReport {
violations: Violation[];
slopScore: number; // weighted total
clean: boolean;
}
Slop score is weighted: a banned-structure hit counts triple a banned-word hit because structural tells are harder to miss as a reader. Word-level slips are forgivable; bold-colon prefixes are not.
How do you repair a bad AI draft without making it worse?
The naive answer is to retry until clean. That naive answer is wrong.
LLMs regress to the mean on every call. A second attempt usually fixes the obvious tells. By the third attempt, the LLM starts introducing different tells. By the fourth attempt you are inventing slop that was not there before. Worst, you have spent four times the latency budget for diminishing returns.
We cap our regenerations at one retry. This repair directive is the key:
export function buildRepairDirective(report: ValidationReport): string {
const offenders = [...new Set(report.violations.map(v => v.term))]
.slice(0, 25)
.map(t => ` - ${t}`)
.join('\n');
return `## REPAIR DIRECTIVE
Your previous output failed the banned-lexicon check. The following exact
terms or patterns appeared and must be removed:
${offenders}
Do NOT paraphrase the rejected output. Re-read the source material and
write a fresh draft from scratch. Paraphrasing keeps the underlying
cadence and structural tells. Rewriting from source breaks them.`;
}
The "do not paraphrase, rewrite from source" instruction is the most useful line in the whole pipeline. Paraphrase prompts cause the model to keep the same paragraph skeleton and only swap synonyms — which keeps every cadence tell intact. Forcing a rewrite from source forces a different sentence-shape distribution.
After the retry, the validator runs again. We keep whichever response has the lower slop score, even if neither is fully clean. The user always gets the best of two attempts, plus a lexiconWarnings payload so the UI can surface a small "regenerate?" badge if anything still slipped through.
How much does post-generation validation slow things down?
Here are the numbers from our first 48 hours of telemetry, captured via PostHog:
| Scenario | Frequency | Extra latency |
|---|---|---|
| Validator scan, draft is clean | ~88% | < 5 ms |
| Validator scan, retry triggered, succeeds | ~10% | + 1 LLM call (3–8 s social, 15–40 s long-form) |
| Validator scan, retry fails to clean | ~2% | + 1 LLM call, ship cleaner of two |
Worst case is roughly 2× generation time. Not 4×, not 5×. The validator scan itself is regex over a few KB of text, sub-5ms even on a 2,000-word article.
We picked a single retry deliberately:
- LLM regression-to-the-mean makes additional retries unreliable
- Long-form is already 15–40 seconds; users abandon at 2 minutes
- Every retry is a billable Gemini call
- Bounded retries make worst-case latency predictable for loading states
Why we tell users about the delay
Our product reviewer asked whether we should hide the latency to make the product feel faster. We came to the conclusion that hiding it would make the wait feel arbitrary. Surfacing it makes the wait feel earned.
The pre-generation tip on the long-form page now reads:
Every draft runs through the slop validator. If AI tells slip through, we regenerate once before showing it to you.
The mid-generation loader cycles through honest steps: Running the slop filter… Scanning for AI tells… Re-running if any slop slipped through…
This is the same principle we explored in our LinkedIn post 2026 piece, when you charge a price (in money or time), name what the user is buying. Otherwise the price feels like a tax.
Should you use a humanizer API For your content instead?
Someone on the team suggested the alternative path: pipe every LLM output through a third-party humanizer API, then run a "tuning" pass on the humanized output to recover any meaning lost in humanization. So the chain becomes LLM → humanizer → re-tune → ship.
The short answer is no, with one caveat. Here is the longer answer.
Cost stack. A humanizer call adds at least one round trip, often two (the rewrite + the meaning-recovery pass). For long-form, that is +5–15 seconds on top of an already long generation. For social, it can double the entire request. The validator we shipped pays this cost only on the ~12% of drafts that need it. A humanizer pays the cost on 100%.
Detection arms race. Humanizer APIs are trained to fool AI-detector models like GPTZero or Originality.AI. That is a different goal from sounding like a person. Many humanizers degrade prose to win the detector benchmark. They introduce typos, fragmented sentences, and odd punctuation patterns that score "human" on a classifier but read worse to a real reader. Pangram's research on detector bypass is the right place to start if you want the academic version.
Meaning loss. The "tuning" pass exists in the proposed chain because humanizers regularly invert sentences, drop technical specificity, or mistranslate domain jargon. A re-tune pass on top of that adds a third LLM call where the model is now reasoning about an already-mangled draft. Each round trip introduces noise. By round three, you are far from the source material.
Ownership. A humanizer is a black box. Our banned lexicon is a TypeScript file. When a user complains about the word delve, we add it to the array, the dev-mode drift guard catches the prose-vs-code mismatch, and the next generation is fixed. With a humanizer, every fix is an outside vendor's roadmap.
The caveat. A small humanizer pass can help in one specific scenario: when you do not control the prompt. If you are wrapping a black-box API or showing third-party AI output, you have no banned-lexicon hook into the model's instruction. In that case, a constrained humanizer (one tuned for paraphrase quality, not detector bypass) is a reasonable last resort. If you control the prompt, controlling the prompt is always cheaper, faster, and more honest.
For Ozigi specifically, we give the model the rules and verify the rules were followed. That is the contract our users understand.
How we keep the lexicon updated
Two feedback loops keep the lexicon current:
Drift guard. The dev-mode block at the bottom of anti-ai.ts walks every entry in the structured arrays and verifies it appears in the prose rulebook. If a developer adds paradigm to BANNED_WORDS but forgets to add it to the §1A list in the prose, the dev console warns on next reload. CI promotes the warning to an error.
Telemetry. Every generation logs three properties to PostHog: lexiconViolations, lexiconSlopScore, lexiconRetried. We chart these weekly. When a new term starts trending in the violation feed (Gemini picked up crucial in week two — caught it in 31 generations before adding to the list), we promote it.
The result is a lexicon that grows from real production data instead of guesswork. We have written before about why production telemetry beats theoretical evals in our RAG architecture post. The same logic applies here.
What this approach does not catch
Three categories sit outside what regex can detect:
- Statistical rhythm. LLMs default to even sentence lengths. Regex cannot measure that. A future LLM-judge pass with perplexity-style scoring will. The work to add a small judge model — likely Gemini 3 Flash on a sampled fraction of drafts — is on the Q3 roadmap.
- Paragraph balance. AI defaults to roughly equal paragraph weights. Real engineering writing is uneven by design. A one-line punchline after a long technical explanation is the entire point. Detecting balance violations needs a structural pass we have not built yet.
- Tone drift. A draft can be lexicon-clean and still feel off, too formal for the user's persona, too casual for a B2B audience. Tone is what Ozigi Personas handle on the prompt side, and what manual review still owns. We have a piece on the human-in-the-loop principle that explains the 90/10 rule we follow.
Every system has gaps. The honest thing is to name them.
How to apply this in your own stack
If you ship LLM output and want a similar layer, the minimum viable version is four files:
-
anti-ai.ts: your prose rules + structured arrays. Start with the English-language buzzword list from the Pangram paper plus anything specific to your domain. -
lexicon-validator.ts: the four scan passes. Less than 200 lines of TypeScript. -
repair-directive.ts: the "rewrite from source, do not paraphrase" prompt builder. - API-route hook: call validator → check threshold → optionally retry → return final draft + warnings.
If you want the full TypeScript, the Ozigi changelog tracks the architecture as it grows. Our deep dives hub covers the surrounding pieces: multimodal ingestion, system personas, human-in-the-loop. And if you are thinking about content quality more broadly, this GEO and AEO guide explains why this work matters for AI search ranking, not just reader trust.
Related reading on Ozigi:
- The Banned Lexicon: Curing AI-Speak — the philosophy behind the word list
- System Personas — why we use editorial briefs instead of soft prompts
- Multimodal Ingestion — the input side of the same pipeline
- Human-in-the-Loop — the 90/10 rule for collaborative content
- Gemini 2.5 vs Claude 3.7 in production — the model trade-offs that informed this work
- Your launch post got 4 likes — why generic AI content fails on launch day
