The refactor looked clean. The agent took a module with three custom error classes — NotFoundError, ValidationError, RateLimitError — and replaced them with a single generic Error carrying a code string field. Fewer classes. Less surface area. The diff looked like simplification.
It compiled. The unit tests passed. The PR description said "consolidated error types." It looked like a win.
Three days later, a handler in the billing service started swallowing RateLimitErrors. That handler matched on instanceof RateLimitError to back off and retry. The new generic error didn't match. The request fell through to a generic 500 path, retried immediately upstream, and turned a throttle into a small incident.
Why the agent does this
Custom error classes are invisible to the compiler in the ways that matter. TypeScript checks the type on the throw side. It does not check that every instanceof caller still matches after a refactor. The agent's linter and type checker both see a module that returns Error — which is what the new shape returns — and call it done.
The failure is behavioral, not syntactic. Caller code that switched on the concrete class still exists, but it is no longer reachable. The agent has no way to know those callers matter unless it reads them, and reading every caller of every error class across a repo is exactly the kind of audit it tends to skip unless you ask for it explicitly.
The cheap guard
Before letting an agent consolidate error types, ask it to do one thing first: list every instanceof check, every catch (e) that narrows on a class name, and every error.code === string match across the repo. Print the list. Read it. Then decide whether the consolidation is actually safe.
Most of the time it isn't. The instanceof matches were there for a reason — some upstream code path relied on the concrete type to branch. Replacing the class with a string field moves that contract from the type system into a convention that nobody checks.
The same shape drift, one layer up
This is the same failure mode you see at the API boundary. A handler used to return { type: 'rate_limited', retry_after: 60 }. The agent refactors it to return a generic { error: 'too many requests' }. The contract — what clients can rely on — quietly narrows. The OpenAPI spec still says retry_after is present, because nobody updated it. The live endpoint stopped sending it.
That gap between the documented shape and the live shape is the thing you eventually have to catch by hand. We ended up doing that check locally in Powerduck — re-run the spec against the live endpoint, let the drift surface as a diff, and only then decide which side is the source of truth.
What to ask for instead
If you want cleaner error handling, ask the agent for a different refactor: keep the classes, but give each one a stable code string. That way instanceof keeps working for callers that depend on it, and the code field gives you a stable contract to match on the wire.
The result is more types, not fewer — but the types mean something. The diff looks less elegant in the PR. The system stays correct.
What to watch for
Next time you see a PR titled "consolidated errors," "simplified exception hierarchy," or "unified error handling," read it as a contract change. The code may compile. The tests may pass. The callers that branched on the old shape will not tell you they are broken until production does.
