Rehearse the Revert Before You Merge: A Review Gate for Agent-Generated PRs

An agent opened a pull request. CI is green, the diff is 400 lines of plausible code, and your calendar says you have fifteen minutes. The instinct is to read it top to bottom and ask whether it is correct.

That is the second most useful question. The first one is cheaper: if this is wrong at 02:00, what does undo cost?

This week's front page had several posts about AI tooling improving while the surrounding software did not. Review capacity is part of that surrounding software. It did not grow. The count of machine-written diffs per reviewer did. A process that only measures correctness will fall behind quietly, and the failure mode is not a bad merge — it is a merge nobody can back out.

Revert cost is a review metric, not an incident detail

Revert cost has three parts, and only the first is about git.

1. Structural cost: after main has moved, does reverting this commit apply cleanly, or does it collide with everything that landed since?
2. State cost: what did the change write outside the repository? Rows, objects, published artifacts, webhooks already delivered, caches keyed on the old shape.
3. Blast cost: how many callers depend on the new behavior, and did any of them ship in the same week?

Add those three and you have a revert budget. The rule I use: if the revert budget is larger than the cost of fixing forward, the PR must ship with its own undo, written and tested, in the same diff. Not a plan in a comment. A command someone can run at 02:00.

Assume the test-diff and retry-policy gates already ran. This is the gate that runs after them.

Three buckets for agent diffs

Merge-now

Pure additions: new function, new tests, no writes outside the process, no new dependency. If the classifier below files the diff here and the tests fail before the change and pass after, there is nothing left to rehearse.

Rehearse-first

Additive but stateful: schema changes, lockfile bumps, CI workflow edits, environment or secret surface, feature-flag defaults. These usually merge, but only after someone has run the undo once. The rehearsal is the review.

Human-only

Destructive or externally visible: backfills, DELETE or DROP, anything that publishes to a registry, sends a webhook, or changes billing and permissions. Agent-generated diffs land here more often than their tone suggests, because models write down-migrations far less reliably than up-migrations. Do not merge these on a green pipeline alone.

The rehearsal loop

Two rehearsals, both cheap. The first models git. The second models state.

Step 1: structural rehearsal

Does the PR still revert after the rest of the team merged their work? Replay the PR on its merge base, replay what landed since, then try to undo just the PR.

#!/usr/bin/env bash
# revert-rehearsal.sh - does this PR still undo cleanly after main moved?
set -euo pipefail

PR_COMMIT="${1:?usage: revert-rehearsal.sh <pr-commit> <merge-base>}"
BASE="${2:?usage: revert-rehearsal.sh <pr-commit> <merge-base>}"
SCRATCH="$(mktemp -d)"

git worktree add --detach "$SCRATCH" "$BASE"
cd "$SCRATCH"

git cherry-pick -x "$PR_COMMIT"
PR_REPLAY="$(git rev-parse HEAD)"

# replay whatever landed on main after the PR branch point
git cherry-pick -x "$BASE"..origin/main

if git revert --no-commit "$PR_REPLAY"; then
  echo "STRUCTURAL: clean revert"
  git revert --quit
else
  echo "STRUCTURAL: conflict, the undo needs a human"
  git revert --abort
fi

cd - >/dev/null
git worktree remove --force "$SCRATCH"

Two outputs matter. A conflict during the second cherry-pick means the PR is stale against main, which is its own review finding. A conflict during the revert means the undo is a manual job, so the merge should be scheduled, not squeezed in.

Step 2: state rehearsal

Run the down-path on a scratch host with a scratch database, then compare shape before and after. The example below is Alembic, but any migration tool with a down command fits.

set -euo pipefail

up()   { alembic upgrade head; }
down() { alembic downgrade -1; }

up
psql -tAc 'select count(*) from users' > after_up.txt

down
psql -tAc 'select count(*) from users' > after_down.txt

if diff -q after_up.txt after_down.txt >/dev/null; then
  echo "STATE: shape restored"
else
  echo "STATE: down-path changed the shape, review the migration"
fi

This proves shape, not identity. A down-path can restore the column count and still lose every value in it. If the migration writes real data, the rehearsal belongs on a snapshot, and no snapshot means no merge.

Step 3: triage the diff before you read it

A regex classifier will not understand your code, and that is fine. It only has to tell you which of the three buckets to open first.

#!/usr/bin/env python3
"""Heuristic revert-friction classifier. Triage aid, not an approval gate."""
import re
import subprocess
import sys

RULES = [
    ('human-only', '^[+-].*(DROP|TRUNCATE|DELETE FROM)', 'destructive SQL'),
    ('human-only', '^[+].*(npm publish|docker push|put_object|send_webhook)', 'publishes outside the repo'),
    ('rehearse-first', '^[+].*(CREATE TABLE|ALTER TABLE|CREATE INDEX)', 'schema change'),
    ('rehearse-first', '^[+].*(workflows/|Dockerfile|package-lock.json|poetry.lock)', 'build or CI surface'),
    ('rehearse-first', '^[+].*(os.environ|secrets.)', 'env or secret surface'),
    ('merge-now', '^[+].*(def test_|test_|.spec.ts)', 'test-only addition'),
]

def classify(patch):
    buckets = {}
    for line in patch.splitlines():
        for bucket, pattern, label in RULES:
            if re.search(pattern, line):
                buckets.setdefault(bucket, []).append(label)
    return buckets

if __name__ == '__main__':
    diff = subprocess.run(
        ['git', 'diff', '--no-color', sys.argv[1]],
        capture_output=True, text=True, check=True,
    ).stdout
    for bucket, labels in classify(diff).items():
        print(f'{bucket}: {sorted(set(labels))}')

A line can land in more than one bucket, and the human-only rules win when it does. Run it as python classify.py origin/main...HEAD on a copied diff, never as a merge gate.

A decision table you can argue with

Print that table, tape it to the review checklist, and delete any row that stops being true for your stack.

Where the scratch environment comes from

The state rehearsal needs a host you are willing to break. Running a half-applied migration on a shared staging box is a worse incident than the bug you were reviewing, and requesting a sandbox from a platform team is usually the step that makes people skip the rehearsal entirely.

I ran the rehearsal above on a throwaway host from MonkeyCode's free server option and used its free model access for the agent half of the loop. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two things I can state, because the operator's current announcement says so: there is a free token allowance listed at 10 million tokens and a free server option, and both were enough for this workflow, where the jobs are small and bursty — one up/down migration cycle, one reverted test run. Two things I will not state: hardware, uptime, or retention, or that the numbers hold next month. Check the current terms before you put anything real on a free tier.

Where the free tier genuinely helps is the cost of trying. A rehearsal host you have to ask permission for is a rehearsal you skip. Where it does not help: compliance-scoped data, customer PII, or anything you would not put on a shared machine. And a free model will write you a confident down-migration; treat that as a draft, not as evidence. Run it.

If you want to try the loop before wiring a paid runner, the free tier is there, and nothing above depends on it.

Limitations, and who should skip this

• The classifier is regex over diff lines. It misses DDL generated at runtime by an ORM, infrastructure state changes, and writes performed by a queue consumer in another repository.
• The structural rehearsal models git conflicts, not runtime behavior. Caches, feature flags, and eventually consistent readers mean a clean revert can still be wrong for an hour.
• The state rehearsal proves shape, not fidelity. Column counts and nullability can return while the values inside them do not.
• Reverting is sometimes the wrong answer. If a forward fix is ten minutes and a down-migration plus re-up is two hours, fix forward and write down why.

Skip this approach if your migrations are owned by a platform team you cannot run against a scratch database, if the workload touches regulated data, or if you have no scratch environment at all. In that last case, build the sandbox first — the rehearsal is worth little without somewhere safe to fail.

You do not need all three steps on every PR. Run the classifier on every agent diff, run the structural rehearsal when the diff touches shared files, and run the state rehearsal whenever the diff writes anything. The review question is not only whether the change is right. It is whether the change is right, and how cheap being wrong is.