The highest-value use of Claude Code is rarely the one in the demos, and it isn't writing code. It's answering a question like "where does the webhook retry backoff actually get configured?" in a Rails app you've been on for three weeks — and getting it right, pointing at a Sidekiq worker's sidekiq_retry_in block that four different grep attempts missed because the constant was named SPACING.
That's the shape of the real productivity gain. Below is what holds up under daily use across the kinds of codebases these tools get pointed at — a Python service, a Go CLI, a legacy Rails monolith — and the parts that reliably waste time.
Where it genuinely changes the day
Reading code you didn't write. This is the biggest one and it gets undersold because it doesn't produce a diff. Dropping into an unfamiliar module and asking "trace what happens when a POST /v2/orders comes in, list every file involved" gets you a map in under a minute. It's sometimes wrong at the edges, but it's wrong in the way a helpful colleague is wrong — close enough to orient you, and cheap to verify by opening the files it named. Onboarding to a new repo shifts from "read for two days" to "read for two hours with a search assistant."
Mechanical migrations with a clear pattern. Converting 60 test files from unittest to pytest fixtures. Swapping a deprecated moment call for date-fns across a frontend. Adding context.Context as the first parameter to 200 Go functions and threading it through. These are jobs where the transformation is obvious and the tedium is the whole problem. aider is particularly good here because it commits after each file, so git log --oneline gives a reviewable trail and git revert is per-file.
Commit messages and PR descriptions. Small, but it removes a real friction point. aider writes them from the diff by default. Outside an agent session, git diff --cached | claude -p "write a conventional commit message" does the same job. The messages are reliably better than the ones written by hand at 6pm.
The first draft of tests for code that already works. Characterization tests specifically — pinning existing behaviour before a refactor. More on this below, including where it bites.
Debugging as a search problem. Paste a stack trace plus the relevant file and ask "what are three things that could cause this?" The model isn't smarter than the developer; it's just faster at enumerating hypotheses, and it doesn't get attached to the first one.
A worked example: the characterization test loop
A pricing function nobody wants to touch:
# pricing.py
def apply_discounts(cart, customer, now):
total = sum(i.price * i.qty for i in cart.items)
if customer.tier == "gold" and total > 100:
total *= 0.9
if now.weekday() == 2: # Wednesday promo, added 2019, never removed
total -= min(5.0, total * 0.05)
return round(total, 2)
The loop, with aider:
aider --model sonnet pricing.py tests/test_pricing.py
> Write characterization tests for apply_discounts. Do not change pricing.py.
> Cover: gold tier above and below the 100 threshold, Wednesday vs non-Wednesday,
> and the interaction of both. Use explicit datetime values, no freezegun.
> Assert exact expected floats.
Then, in the same session:
> /run python -m pytest tests/test_pricing.py -q
aider feeds the failures back in and iterates. Two rounds is typically enough to go green — six tests, all passing.
Then the important step, which the tool will not do for you. Green tests prove nothing until they've been proven capable of going red. Break the function on purpose:
- if now.weekday() == 2:
+ if now.weekday() == 3:
Four of the six tests still pass. Two of them look like this:
def test_gold_customer_gets_discount(mock_pricing_client):
result = apply_discounts(cart, gold_customer, datetime(2024, 5, 1))
assert result < 200
assert mock_pricing_client.called_once
Two problems, both classic. assert result < 200 is a tautology given the inputs. And mock.called_once is not a real attribute — MagicMock happily returns a new child mock, which is truthy, so that line can never fail. The real spelling is assert_called_once(). The called_once mistake is common enough that it turns up in generated tests across unrelated repos.
The fix is to make "can this test fail?" part of the loop, not a thing anyone remembers to do. For Python:
pip install mutmut
mutmut run --paths-to-mutate pricing.py --tests-dir tests/
mutmut results
Any surviving mutant is a test that isn't testing. For a lighter version, git stash the implementation change and confirm red before confirming green. On the Go side, go test -run TestPricing -count=1 plus deliberately inverting a boolean gets 80% of the same signal in ten seconds.
That loop — generate, run, deliberately break, re-run — is the difference between AI-written tests being useful and being coverage theatre.
Where it costs you time
Hallucinated APIs, and specifically plausible ones. Not foo.bar_baz_quux() — that gets caught. It's boto3 calls with a MaxResults kwarg that the particular client doesn't take, or a pandas DataFrame.explode(ignore_index=True) on a version that predates the kwarg, or a Django QuerySet method that exists on the manager but not the queryset. The failure is a TypeError at runtime, in a code path that runs monthly.
Confident-but-wrong refactors. The pattern: you ask for a small change, the agent decides three adjacent things are also wrong, and returns a 400-line diff where the requested change is 12 lines. The other 388 lines look fine. They compile. One of them dropped a nil check. This is the argument for running agents with a hard scope instruction and reviewing every hunk — and why aider's per-change commits matter more than they sound.
Review fatigue. This is the sleeper cost. Reading a diff you didn't write is slower and less reliable than reading one you did, because you don't have the model of why in your head. Generate enough code and you become a full-time reviewer of a colleague who never learns. The common failure is approving an agent's work at the depth normally reserved for a Dependabot bump. That's how the subtle ones get in.
Anything requiring taste. Where the module boundary goes. Whether this should be a queue or a cron. Whether the right fix is to delete the feature. Current models will produce a competent implementation of a bad idea, enthusiastically.
Tool notes, briefly
Copilot's inline completion is still the highest value-per-keystroke thing on the list — it's autocomplete that understands the file, and it costs nothing when it's wrong because you just keep typing. Cursor's agent mode is strong when pointed at exact files and weak when left to search. Claude Code and Codex are better at multi-file work and much better at reading. aider wins on git hygiene. continue.dev with a local model via Ollama is the only option when code can't leave the building; quality is noticeably lower, and it's still worth it for the read-and-explain use case.
What to actually measure
Vendor dashboards will show "suggestion acceptance rate." Ignore it. Accepting a suggestion is not a business outcome; it's a keystroke.
Things that would actually tell you something:
- PR size distribution. If median diff size is climbing, review quality is falling, whether or not anyone says so.
- Time from PR open to first substantive review comment. Generation is cheap now; review is the bottleneck. If this number is growing, the constraint has moved, not disappeared.
- Change failure rate and revert rate, from your deploy tooling. This is the one that catches confident-but-wrong.
- Mutation score on new test files, if you can afford to run it in CI weekly. Line coverage will look great and mean nothing.
- Time-to-first-merged-PR for new joiners. The onboarding gain is real and this is where it shows up.
And run it as an actual comparison. Pick two similar teams, give one the tooling for a quarter, look at the numbers above. That's not a rigorous trial, but it beats the alternative, which is everyone agreeing it feels faster while the revert rate quietly doubles.
