That "be helpful and friendly" paragraph in your system prompt costs $708 a year

Every chatbot tutorial starts the same way: write a system prompt. So you write one. Then a teammate adds a paragraph about tone, someone pastes in the company history, and three months later the prompt reads like a README.

Nobody prices that README, because it never shows up as a line item. It shows up as input tokens on every single call, forever.

Two words first. A token is roughly ¾ of an English word, and APIs bill per million of them. The system prompt is the instruction block you send before the user's message — and the API is stateless, so it doesn't remember it between calls. You send it again with every request, and you pay for it again with every request.

What real traffic looks like

Before the arithmetic, some background from our own gateway. Across 5,546 successful chat calls from 16 customer accounts (July 18 – October 15, 2026), the median request carried 1,280 input tokens and got back 108 output tokens. Requests are about 12 times bigger than answers — and in a typical bot, most of that input is the same fixed instruction text, sent again and again. That is what a system prompt looks like on an invoice.

Counting a typical prompt

I wrote a realistic support-bot prompt for a made-up electronics store: role, return policies, answer format, escalation rules, four few-shot examples (sample dialogues that show the model the expected style), and a line with today's date. Then I counted it with tiktoken and o200k_base, the tokenizer behind current GPT models.

import glob, tiktoken  # pip install tiktoken
enc = tiktoken.get_encoding("o200k_base")
PRICE_IN = 2.00        # $ per 1M input tokens
CALLS_PER_DAY = 10_000

for path in sorted(glob.glob("prompt/*.txt")):
    n = len(enc.encode(open(path).read()))
    per_year = n * CALLS_PER_DAY * 365 / 1e6 * PRICE_IN
    print(f"{path:28} {n:5} tokens  ${per_year:8,.0f}/year")

The whole prompt came to 946 tokens. The policies took 234, the examples 192, the answer format 127. And one paragraph took 97: "Remember that you are a helpful, friendly and professional assistant. Always do your best to give accurate, complete and well-structured answers…" — eighty-one words that every modern model already does by default.

The price of one paragraph

Here is what those 97 tokens cost per year, at official input prices (October 2026), depending on the model and how many calls you make a day:

The formula is one line: tokens × calls_per_day × 365 / 1,000,000 × input_price. On a cheap model, the paragraph is a rounding error. On a mid-tier model at moderate traffic, it is a monthly subscription to something you don't use. On a flagship at scale, it is a salary.

"But input is cheap, output is the expensive part." Per token, yes. Per request, no: with a 1,280-token request and a 108-token answer, input outweighs output even when output costs five times more.

Auditing the prompt: cut, then cache

Go through the prompt section by section and ask one question: would the answers change if this were gone?

Cut what the model does anyway or never uses. In my example that's the "be helpful" paragraph (97 tokens), the company history and mission statement (106 — the bot never needs the founding year to process a return), and two rules the format section repeated word for word (23). Total: 226 tokens, 24% of the prompt, about $1,650 a year on Sonnet 5.5 at 10K calls a day. Run your eval set after cutting; if quality holds, the tokens were decoration.

Cache what is long and stable. Prompt caching means the provider keeps the beginning of your request and bills repeated reads of it at a fraction of the normal input price. The catch: it only works on an identical prefix, byte for byte. My original prompt had today's date and the customer's name at the top — one changing line up there makes the whole prompt uncacheable. Move everything dynamic to the end:

messages = [
    {"role": "system", "content": ROLE + POLICIES + FORMAT + EXAMPLES},  # never changes
    {"role": "system", "content": f"Today: {today}. Customer: {name}."}, # changes
    {"role": "user",   "content": user_message},
]

Cutting and caching are not alternatives. Cut first, because a token you deleted costs nothing, while a cached token still costs something on every call and the full price whenever the cache expires.

What this doesn't solve

Caching has a break-even point: writes cost more than normal input, so at a low hit rate it can raise the bill. Some providers also only cache prefixes above a minimum length — check the docs before counting on it. My token counts come from OpenAI's tokenizer; Anthropic and Google split text differently, so expect your numbers to drift by some percent. And our own usage records don't split out cached tokens yet, so I can't show you real cache hit rates from our traffic.

The prices above are the vendors' official ones. We resell the same models below them — Claude Sonnet 5.5 at $1.70 per million input tokens against the official $2.00 — which takes that $708 paragraph down to $602. That's a fixed percentage off; it doesn't remove the paragraph. A discount on tokens you don't need is still paying for tokens you don't need.

Open your system prompt, run the script above on it, and find your most expensive paragraph. It's probably the one nobody remembers writing.