The flaky test was the kind that everyone on the team had learned to rerun.
it('replays the event into the read model', async () => {
await bus.publish(event);
const view = await readmodel.byId(event.aggregateId);
expect(view.count).toBe(1);
});
It failed maybe one run in twenty on CI. Locally it never failed at all. The usual ritual was: hit retry, move on.
When the agent got assigned to it, it found the gap in about two minutes. The publish resolves before the consumer has flushed to the read model, so by the time the assertion runs, the row isn't there yet. The patch it produced was three lines:
await bus.publish(event);
+ await new Promise(r => setTimeout(r, 200));
const view = await readmodel.byId(event.aggregateId);
CI went green. The flaky test stopped flaking. The PR merged. We moved on to the next thing.
Why this felt like a win
For about two weeks it was a win. The test was stable. Nobody had to rerun the build. The agent had diagnosed a real ordering problem and inserted something that made the ordering deterministic, at least on the machine where CI ran.
The problem is that a sleep is not a synchronization primitive. It's a bet about how slow the system on the other side is going to be. On a quiet runner, 200ms is generous. On a loaded runner, or in a container with a noisy neighbor, or once the read model got bigger and the flush slowed down, 200ms stopped being generous. It started being exactly the wrong number.
The first symptom was not a test failure. It was a support ticket on a Tuesday: the dashboard shows zero orders right after a purchase. A refresh fixed it. The read model had been written; the UI had just read it a few hundred milliseconds too early. That is the shape of a sleep bug in production. The system is correct on average, and wrong at the worst possible moment.
What the agent should have done instead
There are three honest answers to how you know the consumer has processed the event, and none of them is a magic number.
1. Wait on the thing you actually care about. If the read model is the assertion, wait on the read model:
await eventually(() => readmodel.byId(event.aggregateId), {
timeout: 2000,
interval: 25,
});
eventually polls until the row exists or the budget runs out. On a fast machine it resolves in one tick. On a slow machine it waits the real time it needs. It fails loudly when the budget runs out, instead of silently passing and hoping.
2. Make the consumer emit a signal. In tests, use an in-memory bus that resolves its publish promise only after every handler has finished. This is the cleanest version: the test controls the timing, the production code stays the same, and you never depend on wall clock.
3. Stop racing in production. A sleep in a test is one problem. A sleep in application code is another. If the read path has to wait 200ms for a write to land, you have a consistency problem, and the fix is a stronger read (read-your-writes, a follow-up query on the source of truth, or an actual outbox) — not a delay.
The review habit this earns you
I now read any PR that contains setTimeout, sleep, wait(200), or delay as a smell, regardless of who wrote it. The question is not whether it passes CI. It is what that number is protecting against, and what happens when the thing it protects against is slower than the number.
That question is much easier to ask when the diff is three lines generated by an agent, because the agent has no stake in the number it picked. It pulled 200ms out of a distribution of numbers people use on the internet, and it happened to be right on the machines we tested on. It is not a synchronization strategy.
This is also where having a contract for the boundary between services pays off. If the read model is an HTTP endpoint, the cheapest regression safety net is to record real responses in tests and re-run them against the new code — the kind of local, spec-first loop we built into Powerduck specifically because an agent silently renaming a field is the cousin of this same class of bug.
The line I'd leave you with
A sleep makes the test deterministic on your laptop. A real wait makes the system deterministic in the world. When the agent reaches for setTimeout, that is the moment to ask it what the test would look like if we removed the wall clock entirely. The answer is usually twenty lines longer than the sleep, and it is the version that won't surface as a Tuesday support ticket.
