Two weeks ago a patch I reviewed bumped a shared package from 2.1.0 to 2.2.0. Every test in the repo was green, the CI badge was green, and the changelog claimed the renamed formatting helper was exported as formatBudget. The first consumer to upgrade hit formatBudget is not a function at runtime, on a path our own tests never touched. The agent that made the change had moved the helper between files, fixed the in-file import, watched the suite stay green, and called it done. The built dist/ barrel still pointed at the old location, so the published tarball never carried the symbol under the documented name.
The reason nobody caught it before release is structural: in a monorepo with a path alias, the test runner resolves formatBudget straight to the edited TypeScript file. A missing re-export in the package barrel is invisible in that resolution. The suite validates that the modules inside the repo agree with each other. It says nothing about the tarball a consumer downloads.
Source-run tests test the source, not the ship
The package boundary only exists after build. A dev compiler and path alias resolve imports to the working files on disk. The boundary a consumer crosses — built output, the published barrel, tree-shaken exports, declared dependencies — does not exist while the tests run against source.
Green means internal agreement, not published correctness. A passing suite proves the edited file and the test that imports it agree. It cannot prove the built artifact re-exports the symbol, that the barrel lists it, or that a clean install finds it.
The agent has no reason to suspect the gap. The tool it trusts most — the test runner — returns confidence, and nothing in its loop asks it to prove the result against the artifact it is about to ship. So it reports green and moves on.
Monorepo hoisting hides the rest. A package that forgot to declare a dependency can still import it locally because another package in the workspace already hoisted it to the root node_modules. The broken declaration surfaces only on a clean install.
The cheap gate: consume what you publish
Build before you check the surface. Run the package build first, then import from dist — or from a packed tarball — in a short smoke check that imports every public export the documentation lists. Place this check after the build step, not before; running it against source reproduces the exact blind spot.
A 20-line barrel-import check catches the common failures. It surfaces a missing re-export, a renamed export left behind in the old file, an accidental removal of a documented symbol, and a bad declaration that hoisting masked. None of these appear in ordinary unit tests.
Verify on a clean install. Install the packed tarball in an empty directory and import from there, instead of relying on the workspace's hoisted dependencies. A clean install is the environment a consumer actually has.
Ask for the right report after a public-surface change. After a rename, move, or removal of a documented export, "tests green" is incomplete. The acceptance answer should state that the built artifact exists, that every documented export imports from dist, and that a clean-install import succeeded.
The mirror gap on the API side
The same blind spot appears without a build step. The handler compiles, unit tests pass, and the code looks correct, but the live response renamed or dropped a field because nobody diffed the actual response against the contract. Source-level confidence cannot see what a deployed endpoint returns to a client. The reliable check compares the live response against the spec rather than trusting the handler that compiled. That local-first loop is the one we built into Powerduck: the contract stays the source of truth, and a green build never counts as proof that the shipped surface matches it.
What I check before a public release
- Did this change touch a public export or a documented symbol, or only internal logic?
- Did any check run against
distor the packed tarball after build, or only against source? - Did we import every documented export from the built artifact in a clean-install context?
- On the API side, did anyone diff the live response against the spec, or only read the handler code?
A green suite measures agreement inside the repo. The consumer lives outside it. Until you verify against the built artifact — or, for an API, against the live response — green means the source looks right, not that the shipped thing works.
