Factory.ai defines a software factory as a nine-stage loop: signals, triage, plan, build, test, review, secure, ship, monitor. Seven of those stages describe what happens before code merges and only two describe everything after. Mastra, Port.io, and Pulumi have similar explainers.
Mastra's is the most concrete, with seven named agents, seven named workflows, and half a dozen blocks of working TypeScript. It spends roughly 2,300 words on orchestration, context, tools, and storage, and roughly 265 on production. "Rollback when needed" appears one time inside a list of things the release agent owns. Pulumi's seven rules never mention feature flags or progressive delivery. Port.io's model assigns a role to governance and a role to measurement, and none to what happens after the merge.
Throughout this piece, I use "build side" for the first seven, pre-merge stages: the part of the factory that turns a signal into a reviewed and secured artifact. I use "run side" for the system that governs a change after merge, that is, how it reaches users, how its effects are observed, and how it can be stopped or reversed. In the published models, the run side is represented by ship and monitor; my argument is that those are not merely two final steps but a control layer that spans both. I refer to the Factory.ai, Mastra, Port, and Pulumi diagrams collectively as reference architectures or explainers.
Reviewers like AI code more than production does
New Relic's 2026 State of AI Coding report surveyed 200 US-based technology decision-makers and 94% of them rate AI-generated code as higher quality than human-authored code at review. 78% report more incidents after that code reaches production. 82% had hit at least one production failure in the previous six months that they could attribute to AI-generated code. New Relic calls this "the central contradiction of the AI-assisted era."
That survey has limits: 200 self-reported responses, nobody below manager level, and a sponsor that sells observability (and therefore has a clear motivation to emphasize a problem more observability could solve). That was enough to send me looking for better evidence.
DORA's State of AI-assisted Software Development 2025 uses more than 100 hours of qualitative data and survey responses from 4,867 technology professionals. Its summary is better than anything I would write:
"AI adoption now improves software delivery throughput, a key shift from last year. However, it still increases delivery instability. This suggests that while teams are adapting for speed, their underlying systems have not yet evolved to safely manage AI-accelerated development."
According to DORA, individual effectiveness, code quality, team performance, and organizational performance all showed negative associations with AI adoption in 2024, but changed to positive associations in 2025. But delivery instability stayed negative.
Our research, captured in the 2026 AI Control Gap report, says the same thing. The report surveyed 767 engineering and DevOps professionals. 94% say AI has increased the pace of code generation. 91% believe AI-generated code is equally or more likely to introduce production issues than human-written code. And 91% say their teams have become more cautious about pushing changes to production.
Mirco Hering, managing director for AI delivery at Accenture, has a line for this that I keep coming back to: "To make a mistake is human, but to create a catastrophe, you need automation."
The suggestions for factory automation get vague at deploy because they assumed you would be there
There is a reason all four factory explainers terminate in the same place, and it's not carelessness. Every factory is built assuming a human downstream will make a decision about it. Orchestration assumes somebody reviews the PR, context engineering assumes somebody exercises judgment about what came back and CI gates assume somebody reads the failure and decides what it means.
None of those layers break when the human steps out. They stop being gates, because the gate was the person, not the layer.
Hering raised this in the first episode of our software factory series, pointing at recent research on how many pull requests now get through with nobody looking at them. The New Relic survey reports that 62% of technology leaders say their teams often trust AI-generated code enough to ship it without manual verification.
So the last step did not get harder. It became the only one with a gate in it, and none of these explainer architectures describe how that gate is supposed to work.
Hering reaches for Toyota to explain this. On a Toyota assembly line, any worker who spots a defect can pull the andon cord and halt the whole line. His point is that the cord has to be installed early and on purpose: "if you don't build it in, the system might build a way that it will avoid you being able to build it in later." An agent-driven pipeline needs the same mechanism, and it cannot be a person watching an error-rate graph. That mechanism has to fire on a signal, without anyone choosing to pull it. All four of these explainer architectures describe the assembly line. None of them describe what can make the assembly line stop.
Two colleagues have made versions of this argument publicly. Cameron Etezadi, CTO of LaunchDarkly, wrote in July that "the factory is the delivery mechanism, and control is the safety mechanism." Jonathan Nolen, our SVP of Product, names the consequence: "You can review a change before it ships; you can't review an agent's next thousand decisions."
Five things a finished run side needs
These five requirements apply regardless of what you're building and what stack you're on.
Something has to know what shipped
When the error-rate graph starts to rise, the most important thing to know is which service handled the breaking requests. A merge log does not answer that. Neither does a successful deploy job.
Source control records a commit. A build turns that commit into an artifact. A deployment system records where it tried to put that artifact. None of those three confirm what actually ran, which is why the confirmation has to come from the runtime reporting its own identity. A deployment is not the same as a release: the LaunchDarkly docs draw that line precisely, because code entering an environment says nothing about which code paths reached which customers.
Without that, an incident starts when four services change in the same ten-minute window with no way to prioritize them. Somebody is left manually scrolling through commit history while the error rate keeps climbing.
The record has to outlast the change it describes
Knowing what shipped helps narrow down what went wrong during the incident. The same question arrives again months later from a regulator, an auditor, or a review that has to explain not one change but the combination of them: which flag was on, at what percentage, against which model version, for which segment, at 14:07 on a Tuesday in March.
Reconstructing a disaster requires more than establishing what happened. It requires interleaving multiple closely timestamped, unaltered records and asking why repeatedly until the root cause becomes visible.
Now these two properties are in tension. The record has to be immutable, and it has to be kept long enough to be useful. Highly regulated industries can require records to be preserved for years. Most of a run side is built for the opposite situation. Monitoring tools hold recent state cheaply, aggregate it after a week, and drop it after a month. To prevent record loss, whatever the run side records about exposure decisions has to be exported into something that will preserve it for the long term.
Exposure has to be decided by policy, not by a person
An agent opens a pull request overnight. The tests pass, the review agent approves it, and by morning the change is ready to ship. The pipeline still does not know whether to send it to 5% of traffic or 100%, whether the ramp should take ten minutes or two days, or which metrics should stop it.
A person used to answer those questions using their own judgment, rather than a policy. After agents take over the earlier stages, that unwritten decision becomes a missed opportunity to block harmful changes.
So the policy has to be written down as a default. Specify which environments a change moves through, how quickly exposure can grow, which guardrails matter, and how much evidence is enough to continue. It also has to attach to the pipeline. A policy that depends on somebody remembering to apply it has not removed the human step.
The unit of reversal has to shrink from the deploy to the change
"Rollback when needed" is where these architectures waiver hardest. Needed according to which signal, measured over what window, compared against what baseline, and decided by whom?
The rollback they imply stops working exactly where it used to be fine. A single stateless service can still revert to its last image But when a deploy carries a dozen unrelated changes, a database migration, and dependencies on three other services that also moved that week, reverting it is a guess about which piece caused the problem and whether undoing all of them is safe. The last known-good version of that component was tested against a system that no longer exists.
So reversal has to happen to a smaller unit. One change goes back while everything around it keeps moving forward. That mandates a precondition: the change has to be exposed separately in the first place behind something you can switch off independently of the deployment. Like Toyota's andon cords, exposing the change separately has to be a decision you make before you ship.
Our own numbers say how routinely rolling back occurs: 69% of teams roll back or hotfix at least once a week, and only 12% resolve a production issue in under an hour. Most take between four and twelve hours. Rollback is not the exception.
Hering raises the obvious objection: flag hygiene was already hard when humans were creating the flags, so what happens when agents create them?
Something has to scrub what the agents generate
Gating and ramping a release leaves residue. A flag, a branch in the code, a conditional that outlives the decision it was protecting. Cleanup is the stage that removes them, and almost nobody plans for it.
GitClear analyzed 623 million changes between 2023 and 2026. Refactoring line moves fell 70%, from 21% of changes in 2022 to 3.8% so far in 2026. Duplicated blocks rose 81%. Maintenance of older code dropped 74%. Developers are now roughly five times more likely to copy code than to refactor it, inverting a 2022 pattern where refactoring led two to one. This implies releases are happening faster and easier than before. Cleanup also has the worst incentive profile of the five requirements: it never causes an incident, so it loses every prioritization argument to the five above it, and then it compounds quietly.
How we solve it
Marek Poliks, who leads AI at LaunchDarkly, puts the premise more bluntly: "if you're working with agents, you're experimenting in production. Period."
I work at LaunchDarkly and the products below are ours, so read this as one implementation rather than a survey of the market.
Flags change the shape of the first requirement. If a change ships behind a flag, the flag is the unit of change: independently identifiable and independently reversible, whichever deploy carried it into the environment. The SDK reports its own applicationId and applicationVersion through application metadata, so a version is attributable to the code that produced it. Good deploy attribution still helps. You need less of it when the thing you would reverse is already gated.
The record is change history, which is where the audit log went after LaunchDarkly renamed it: every flag and resource change in an environment, who made it, and the version you can restore to. Retention is a plan question rather than a product claim, since unlimited change history is on select plans and Developer plans keep 30 days, so anything that has to survive an audit cycle should leave LaunchDarkly for a store you control. The change history events hook, webhooks, and the audit log API all exist for that. This is the requirement where I would argue against our own default: a run side that answers to a regulator should treat export as part of setup, not as something added after the first request arrives.
Exposure by policy lives in release policies: project-level defaults that attach guardrail metrics to every guarded release in scope, so nobody has to remember to add them. Set the standard once and every release inherits it.
Reversal per change is a guarded rollout, and there is no threshold to configure. LaunchDarkly computes the absolute difference between variations along with a confidence interval and applies sequential testing, and a regression fires when that interval falls entirely on the side of worse performance. Those checks run multiple times per minute. Traffic climbs to 100%, but no monitored step goes above 50%, because the original and new variations have to stay evenly split for the comparison to mean anything. It also rolls back on a sample ratio mismatch, or when the new variation never reaches enough contexts to judge, whether or not you switched automatic rollback on. The same machinery runs on AgentControl configs, so a judge score can be the guardrail that reverts a prompt or a model change.
Cleanup is Hering's question answered. It runs off an explicit flag lifecycle plus code references, and scheduled cleanup is itself an agent: it confirms the flag has served a single variation across every critical environment, then opens a pull request that takes the flag out of the code and resolves the branch it was gating. It reads your AGENTS.md for repository-specific instructions, and it stops rather than guessing when multiple variations are still live. Agents create the flags; an agent removes them.
None of this is the definition of a run-side layer. It is one implementation, and I would expect two teams working from the same four requirements to arrive at different answers.
What I would add to the diagram
If I were redrawing that nine-stage loop, ship and monitor would stop being two stages at the end and become one layer holding five requirements at once: knowing what shipped, keeping the record of it, controlling exposure, reversing one change, cleaning up after. The cord runs underneath all five.
A two-row diagram. The top row shows nine stages: signals, triage, plan, build, test, review, secure, ship, monitor, with the last two marked as the entire run side. The bottom row shows one run-side layer covering five requirements: know what shipped, evidence that survives, exposure by policy, reversal per change, and cleanup as a stage, with a single line across all five labelled as an automatic stop wired to a signal.
The ordering is the part I would get wrong. Cleanup looks like the lowest-priority of the five, and by GitClear's numbers it compounds fastest. I would probably still build it last.
The part I am least certain about is coverage. Getting one service to report its identity and gate its own release is a solved problem. Getting every service, region, and manual deploy path in a large estate to do it consistently is the work most teams have not finished, and it is the difference between a run side a person supervises and one an agent can be trusted to drive. If you have that working across a mixed estate, I would like to hear how.
