I measured the viral "talk like a caveman" token saver. A two-word prompt got most of it.

There is a repo doing the rounds called caveman. Its pitch, verbatim from its own GitHub description:

🪨 why use many token when few token do trick. Viral skill + proxy for coding agents that cuts 65% of tokens by talking like a caveman.

It's one of the most-starred repos on GitHub. The idea is funny enough that it spread on the joke alone, which is exactly why it's worth measuring rather than sharing.

So I did. I broke a real C# program, asked four versions of the same agent to fix it, counted every token with the project's own tokenizer, and then compiled every answer to check that shorter hadn't meant worse.

Three findings surprised me, and one of them isn't documented anywhere.

First: the headline number contradicts its own docs

The description says 65%, from talking like a caveman.

The project's documentation page says something different in its own subtitle:

Install the skill for shorter answers, or the CLI proxy to cut input tokens by 33.2%.

Those aren't the same claim. Caveman is actually two products:

• A skill — writing rules that make the agent's answers shorter. It changes what the model says (output tokens).
• A proxy — a local program that compresses files and tool output before the model reads them. It changes what the model sees (input tokens).

The 33.2% figure belongs to the proxy. The description credits the saving to "talking like a caveman," which is the skill. It's selling the wrong half of its own product.

To its credit, the docs are far more honest than the marketing — including this concession, which turned into my control test:

"be concise" already captures much of the gain.

Second: it is not a pile of prompt files

My assumption was that this was a clever paragraph in a markdown file that got lucky. I was wrong. Pulling the repo tree and grouping blobs by extension:

caveman repo: 1785 files, 21.2 MB total

Go  (proxy+engine)      640 files    6.6 MB   30.9%
JS / TS                 450 files    5.6 MB   26.5%
Python                  103 files    1.3 MB    6.3%
Markdown (all docs)     179 files    1.1 MB    5.2%

skills/  THE CAVEMAN     64 files    0.3 MB    1.3%

31% of it is Go — a real proxy and a real compression engine, 398 and 165 files respectively. The famous "talk like a caveman" instructions are 1.3% of the repository.

The meme is on the front door. The thing behind it is a compiler.

The test: a real bug, not a toy prompt

Token benchmarks are easy to fudge with soft questions, so I used a bug with a verifiable pass/fail. Here's a C# shopping cart that silently loses an item:

public class Product
{
    public string Sku { get; set; }
    public string Name { get; set; }

    public override bool Equals(object obj) =>
        obj is Product p && p.Sku == Sku;

    // the hash is built from a field anyone can change
    public override int GetHashCode() => Sku.GetHashCode();
}

var widget = new Product { Sku = "W-1", Name = "Widget" };
var cart = new Cart();
cart.Add(widget, 3);

// the warehouse renumbers the SKU, after it is already a key
widget.Sku = "W-1-NEW";

var found = cart.QuantityOf(widget);
Console.WriteLine(found == 3 ? "PASS" : $"FAIL: expected 3, got {found}");

FAIL: expected 3, got 0

A Dictionary hashes a key once, at insert. Mutating Sku afterwards changes what GetHashCode() returns, so the lookup probes a different bucket, finds nothing, and Equals is never even reached. The entry isn't gone — it's unreachable.

The prompt (276 tokens) was identical every time: explain the failure, return a corrected class. Four variants:

Three runs each, tokens counted with o200k_base — the encoding caveman's own docs specify.

Result: the 65% is real, but it's a different mode

variant      r1    r2    r3     mean   vs plain
------------------------------------------------
plain        896   951   688    845       -
concise      541   669   598    603  -28.7%
caveman      475   577   395    482  -42.9%
ultracave    314   317   293    308  -63.6%

/ultracave hits −63.6%. So the headline number is achievable — but from the maximum compression mode, not the caveman voice the description advertises. The voice itself lands at about −43%.

And "Be concise." gets −28.7% for three tokens and zero installation.

I'm not the first person here: Max Taylor benchmarked caveman against "Be brief" across 24 prompts with a second model grading quality, and found the two-word prompt matched caveman on both tokens and quality. My results differ on /ultracave — in my runs it beat the two-word prompt clearly, in his it didn't — but we agree on the part that matters: most of this saving is free.

A detail nobody mentions: compression also cuts variance

Look at the spread. plain ranged 688–951 (±15%). /ultracave ranged 293–317 (±4%). The terse modes aren't just smaller, they're markedly more predictable — which matters more than the mean if you're budgeting a long agent run.

The skill is not free

Those rules have to be sent to the model every session. Measured with the same tokenizer:

  1158 tokens   skills/caveman/SKILL.md
   608 tokens   skills/ultracave/SKILL.md

It costs 1,158 tokens to teach a model to use fewer tokens.

Against the free two-word prompt, /caveman saves ~120 tokens per answer. So:

break-even vs "Be concise.":
  caveman     1158 / 120  ≈  10 answers
  ultracave    608 / 295  ≈   2 answers

/ultracave is the better deal on both axes — half the overhead, more than double the saving. The default mode needs about ten answers in a session before it's even.

One caveat the project raises itself: if those rules sit in a cached prompt prefix, the overhead amortises. They haven't measured that. Neither have I.

Proof: did the answers get worse?

This is the part most token benchmarks skip. Tokens saved are worthless if quality drops, so I extracted the corrected class from every answer, compiled it against the original Cart, and ran it.

All four compiled. All four printed PASS.

Here's /ultracave's entire answer — 11 lines, and correct:

public class Product
{
    public string Sku { get; init; }
    public string Name { get; set; }

    public override bool Equals(object obj) =>
        obj is Product p && string.Equals(p.Sku, Sku, StringComparison.Ordinal);

    public override int GetHashCode() =>
        Sku is null ? 0 : StringComparer.Ordinal.GetHashCode(Sku);
}

init makes the identity field immutable, so the line that caused the bug (widget.Sku = "W-1-NEW";) becomes a compile error instead of silent corruption. That's the right fix.

Nothing was lost by compressing. The /caveman answer even caught a second bug the plain answer never mentioned: Sku.GetHashCode() throws on a null SKU. Meanwhile plain opened with "Didn't get permission to probe for a .NET SDK…" — precisely the filler these rules delete.

The finding that isn't documented: it compresses shapes, not formats

Now the proxy — the part the description forgot to credit, and the part nobody shares.

One thing first: the npm package is only a front end. The compression engine is a separate signed binary. Running compress without it is honest about that:

{"bytes_in":3801,"bytes_out":3801,"ratio":0,
 "engine":"missing",
 "note":"caveman-engine not installed — 0% compression, input passed through unchanged."}

With the engine in place, I fed it four real files:

file                      type      before   after    saved
-------------------------------------------------------------
build.log  (.NET build)  code         945     945       0%
repo.json  (nested API)  json        1965    1965       0%
records.json (400 recs)  json       19994    2509    87.5%
export.csv (400 rows)    tabular     8409     314    96.3%

The docs advertise ~98–99% on "CSV, logs, YAML, test output, JSON." On real data:

• Uniform tabular data: enormous. 96.3% on CSV, 87.5% on 400 JSON records. The claim holds for this shape.
• A build log and a nested API response: exactly 0%.

And it wasn't misdetection — asking the engine directly, it correctly identified them as code and json. It simply has nothing to do with them.

The rule, which isn't in the README: it compresses shapes, not formats. Rows of repeating records collapse beautifully. Prose-like logs and deeply nested objects don't. If your agent's context is mostly build output and API responses, this part does close to nothing for you.

Lossy to the model, lossless on disk

Every compressed result carries a flag that looks alarming:

"method":"elision", "lossless_to_model":false

The model genuinely does not see the original — parts are removed. That would be disqualifying, except every compression returns a recovery handle. So I took it, asked for the original back, and compared:

46,886 bytes out, 46,886 bytes in, cmp = IDENTICAL

Byte-exact. Lossy to the model, lossless on disk. That's a sound design, and it deserves stating as loudly as the warning does.

Verdict: should you install it?

The answer splits, so here's both halves.

The skill: skip it. Type be concise. Two words, three tokens, no install, and an independent benchmark found it matches. The caveman voice buys you roughly 14 extra points over that, for 1,158 tokens of overhead and about ten answers before break-even.

The proxy: genuinely good — conditionally. If your agent spends its day reading CSVs, exports and record dumps, 96% is real and you will feel it. If it reads source code, build logs and nested API responses, you've installed a proxy for almost nothing.

Not hype, but badly labelled. The engineering is better than the meme suggests — a verifiable signed supply chain, and an engine that refuses to write its recovery store into a loosely-permissioned directory. But the caveman is the marketing; the Go code is the product, and the part being shared the most is the part worth the least.

The broader lesson costs nothing: before you install something to save tokens, try asking for less. My entire control test was two words long.

Methodology: n=3 per variant on a single task with one model, so read these as "on this task", not "in general". Output-token measurements here are not comparable to the project's 33.2% input-token figure — different claims. Tested with caveman 4.0.1 on .NET 10, tokens counted with o200k_base.

This is **Naze Lab* — we break things to find out why.*

Sources: JuliusBrussee/caveman · caveman docs · Max Taylor's "Be brief" benchmark