Characterize Before You Review: A 30-Minute Loop for AI Patches

Every AI patch lands in a repo you barely understand. Reviewing it blind is guessing with confidence. Characterization tests make hidden behavior visible and reviewable. Record first, verify probes, then approve the smallest safe change. That is the whole loop.

Here is a thirty-minute version that works on messy repos. A free model drafts the probe inputs. A local recorder freezes current behavior into golden values. No CI required, no golden-master infrastructure, no existing test suite. One script, one function, one diff.

Why characterization is review work now

AI promoted every developer to reviewer. Nobody tested the reviewer. The patch looks clean and the diff is small. The behavior underneath is still unknown.

Characterization closes that gap. It records what the code does today, bugs included. That record becomes your review baseline. You compare the patch against recorded behavior, not against intuition.

The six-step loop

Each step has exactly one output. If any step fails, stop and investigate. Do not continue to the next step.

1. Pick one function. Choose the function the patch touches. Prefer pure functions with visible inputs and outputs.
2. Draft probes. Ask a free model for edge cases and real call-site values. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access makes this drafting step low-friction. The model proposes; it does not decide.
3. Record current behavior. Run the probes against the current code. Save the outputs as golden values. This freezes today's behavior, bugs included.
4. Prove the probes are alive. Change one constant or flip one branch. Re-run the verification. A live probe fails; a dead probe passes. Delete every dead probe.
5. Apply the smallest safe change. One transform only. Rename, extract, invert, reorder. Nothing else in the same step.
6. Re-run the probes. An empty diff means the change is safe. A non-empty diff means stop and inspect the behavior shift.

The whole loop fits in thirty minutes. Most of that time goes to step one and step four.

The artifact: a probe recorder

The script below records and verifies behavior. It is deliberately small. No framework, no test runner, no dependencies beyond the standard library.

#!/usr/bin/env python3
"""probe.py - record and verify a legacy function's current behavior."""
import importlib
import json
import sys

MODULE = sys.argv[1]
FUNC = sys.argv[2]
MODE = sys.argv[3] if len(sys.argv) > 3 else "record"
GOLDEN = f"{MODULE}.{FUNC}.golden.json"

PROBES = [
    {"name": "no_code", "args": [100, ""]},
    {"name": "save10", "args": [100, "SAVE10"]},
    {"name": "save20", "args": [100, "SAVE20"]},
    {"name": "unknown_code", "args": [100, "FRIENDS"]},
    {"name": "zero_price", "args": [0, "SAVE10"]},
    {"name": "negative_price", "args": [-50, "SAVE10"]},
]


def serialize(value):
    if value is None or isinstance(value, (bool, int, float, str)):
        return value
    if isinstance(value, (list, tuple)):
        return [serialize(item) for item in value]
    if isinstance(value, dict):
        return {str(key): serialize(item) for key, item in value.items()}
    return repr(value)


def run_probes():
    fn = getattr(importlib.import_module(MODULE), FUNC)
    results = {}
    for probe in PROBES:
        try:
            results[probe["name"]] = {
                "ok": True,
                "value": serialize(fn(*probe["args"])),
            }
        except Exception as exc:
            results[probe["name"]] = {
                "ok": False,
                "error": type(exc).__name__,
            }
    return results


def main():
    results = run_probes()
    if MODE == "record":
        with open(GOLDEN, "w") as handle:
            json.dump(results, handle, indent=2, sort_keys=True)
        print(f"Recorded {len(results)} probes to {GOLDEN}")
        return

    with open(GOLDEN) as handle:
        golden = json.load(handle)
    changed = [
        name for name, expected in golden.items()
        if results.get(name) != expected
    ]
    if changed:
        print(f"FAIL: behavior changed for {', '.join(changed)}")
        sys.exit(1)
    print(f"PASS: {len(results)} probes match golden behavior")


if __name__ == "__main__":
    main()

Save it as probe.py. Replace the PROBES list with inputs from real call sites. Add edge cases that the patch might break. Record the current behavior:

python probe.py legacy discount record
# Recorded 6 probes to legacy.discount.golden.json

Verify after any change:

python probe.py legacy discount verify
# PASS: 6 probes match golden behavior

Prove the probes are alive

Golden values are only useful if the probes can fail. Test that before you trust them. Edit legacy.py and change 0.9 to 0.95.

python probe.py legacy discount verify
# FAIL: behavior changed for save10

That failure is proof of life. Revert the edit. A probe that never fails is dead weight. Mutation-check every probe before you rely on it.

Which function to characterize first

Not every function deserves probes. Use this priority table.

Start with priority one. If the patch touches nothing pure, pick the smallest impure function. Mock its dependencies and snapshot its state.

The smallest safe change, demonstrated

Here is a legacy discount function with magic numbers and magic strings.

# legacy.py
def discount(price, code):
    if code == "SAVE10":
        return price * 0.9
    if code == "SAVE20":
        return price * 0.8
    return price

Record its behavior first. Then apply one transform: extract the rates into constants.

SAVE10_RATE = 0.9
SAVE20_RATE = 0.8

def discount(price, code):
    if code == "SAVE10":
        return price * SAVE10_RATE
    if code == "SAVE20":
        return price * SAVE20_RATE
    return price

Verify. Empty diff. The change is safe. Now apply the next transform: replace the string branches with a mapping.

RATES = {"SAVE10": 0.9, "SAVE20": 0.8}

def discount(price, code):
    rate = RATES.get(code)
    return price * rate if rate is not None else price

Verify again. Empty diff. Two small changes, two verified steps, zero big-bang rewrites.

Where the free model fits

The model drafts probes. It does not review or verify anything. MonkeyCode's free model access turns probe drafting into a quick prompt. Its free server option means you can run the loop without hosting your own model endpoint.

The recorder decides. The golden file decides. The model only proposes. That separation keeps the review honest.

Limitations

Golden values freeze current behavior. If the function is wrong today, the probes bless the wrongness. That is correct for refactoring and wrong for fixing bugs.

Side effects are invisible to the recorder. File writes, database calls, and global state escape the return value. Mock them or snapshot them separately.

Non-deterministic functions will flake. Timestamps, random values, and hashes produce unstable goldens. Do not use this loop on them.

Who should skip this

Teams with full test coverage do not need this loop. Existing tests already pin the behavior. Codebases where behavior must change do not need it. You want a spec, not a snapshot.

Reviewers who trust patches without evidence should also skip it. This loop is for everyone else.

Run it once

Start with the ugliest function in the diff. Record its behavior and prove the probes are alive. Then review the patch against evidence. Thirty minutes from now you will know what the code actually does. That knowledge beats any clean diff.