Most AI projects fail somewhere between demo works and production ships. The gap is rarely the model. It's the absence of the controls that turn a one-shot prompt into a system you can run, audit, and iterate on without setting fire to the budget.
I made the chart above as the one-page version of the controls I would put on any AI team's first production sprint. Eight of them, organized by which side of the model they shape: Input, Reasoning, Output, Operations. Below is the why-each-matters and where teams typically get them wrong.
Input Control: shape what goes in
1. Few-shot prompting
Show the model two to five high-quality input/output examples instead of writing long instructions. The model picks up format, edge cases, and tone from examples in a way it does not from imperative prose. Five good examples beat five hundred words of "make sure to handle X, also Y, also Z."
The mistake teams make is treating few-shot as a fallback when the system prompt isn't working. It's the opposite. For classification, extraction, structured rewriting — most of the work that LLM apps actually do — few-shot is the primary mechanism. Long instructions are the fallback.
2. Role-specific prompting
Senior credit risk analyst, fifteen years commercial lending outperforms Act as a financial analyst by a margin that surprises people the first time they measure it. The specific role is doing real work: it constrains vocabulary, narrows the latent distribution, and gives the model permission to refuse questions that fall outside the domain.
Generic personas — helpful assistant, senior engineer, expert — don't constrain anything. They optimize for nothing. Use roles that name the years, the domain, and the seniority. The more specific, the better the calibration.
Reasoning Control: shape how it thinks
3. Chain-of-thought prompting
Force step-by-step reasoning before the final answer. The model arrives at better conclusions when the reasoning is exposed in the output, because the next-token-prediction is conditioned on the reasoning it just generated rather than on a leap to the conclusion.
For step-by-step legal, financial, or compliance-adjacent workflows, CoT is a default, not an optimization. The cost is more output tokens. The benefit is fewer wrong answers on the kinds of problems where wrong answers are expensive.
4. Extended thinking / reasoning models
For genuinely hard problems — multi-step analysis, math, code review, planning — use the provider's native reasoning mode rather than prompted CoT. Claude's extended thinking and OpenAI's o-series both expose a separate token budget for the model to think before answering. The reasoning token budget is configurable. The output token budget is separate.
Prompted CoT and native reasoning solve overlapping problems but are not interchangeable. Native reasoning is more reliable on hard problems and roughly equivalent or worse on easy ones. The default rule: use prompted CoT for routine workflows, switch to native reasoning when the failure mode is "the model jumped to a wrong conclusion despite being asked to think."
Output Control: shape what comes out
5. Structured outputs and tool use
Use the provider's native structured output feature, not prose-described JSON. Schema is enforced by the API, not requested in the prompt. The provider guarantees the output parses; your code does not have to retry-with-jq.
The mistake is asking for JSON in the prompt and then writing a tolerant parser to handle the cases where the model returns Sure! Here's the JSON: {...}. Native structured outputs and tool-use schemas remove the entire class of "the model added an apologetic preamble" failures. For any LLM call whose output feeds a downstream system or API, structured outputs are not an optimization; they are the API contract.
6. Negative prompting and output filters
Tell the model what not to do, and filter the output before it ships. Belt and suspenders. Negative prompting works in the prompt; output filters work in code, after the response. They cover different failure modes — the prompt handles the model's bias toward certain phrasings, the filter handles the cases where the prompt didn't.
This is where PII handling, tone control, and regulated-content workflows live. The control is uninteresting until the day a model paraphrases something it should have refused, and then it is the most interesting control on the list.
Operations: make it durable in production
7. Evals
Versioned test suites with pass/fail thresholds. No prompt change ships without an eval run. This is the artifact that turns prompt engineering from a vibe into an engineering discipline.
Evals belong to the same family of artifacts as test suites, lint configurations, and the append-only mistake logs I wrote about yesterday. Triggered by a change. Append-only by design. Read by the deployment pipeline, not by humans except when something fails. They are the loop that keeps the prompt from rotting.
8. Prompt caching
Cache stable system prompts and context. Anthropic's prompt caching and the equivalent on other providers cut up to 90% off the cost of repeat calls and substantially reduce latency. For high-volume agents, long-context applications, and RAG against stable corpora, prompt caching is the difference between a unit-economics-viable product and a money-losing demo.
The mistake teams make is leaving caching off because they think their workload doesn't repeat. It almost always does. The system prompt repeats on every call. The few-shot examples repeat on every call. The retrieved corpus often repeats across user sessions. Turn it on and measure; the cost reduction shows up immediately.
What sits on top
The footer of the chart names the next layer: audit logging, rate limiting, jailbreak detection, human-in-the-loop on high-stakes actions. Those are enterprise risk controls. They are necessary, they are domain-specific, and they vary by company and by regulator.
The eight controls above are not enterprise controls. They are universal — they apply to every team shipping LLMs to production, regardless of industry, scale, or risk profile. Get these right first; the enterprise layer is what you build on top once they are in place.
The thing that makes the difference between teams that ship LLM features and teams that demo them is rarely the prompt and almost never the model. It is whether these eight controls are wired into the system that ships, or living in someone's head.
