Your AI Refactor Looks Right: Verify It With a Behavior Fingerprint

Your AI refactors look plausible. Plausible is not equivalent.

A clean diff can still change behavior. A reviewer's intuition misses arithmetic-order bugs. The model's confidence hides them. You need a machine to check equivalence. Not a deeper review. A repeatable behavior fingerprint.

The failure mode

Consider a small discount function. Two conditionals. One log records every branch.

def apply_discount(order, log):
    subtotal = order['total']

    if subtotal > 1000:
        subtotal *= 0.9
        log.append(('discount', order['id']))

    if order.get('loyalty') and subtotal > 100:
        subtotal -= 10
        log.append(('loyalty', order['id']))

    return subtotal

An assistant sees a cleanup. Move the loyalty check first. Reduce the branches.

def apply_discount(order, log):
    subtotal = order['total']

    if order.get('loyalty') and subtotal > 100:
        subtotal -= 10
        log.append(('loyalty', order['id']))

    if subtotal > 1000:
        subtotal *= 0.9
        log.append(('discount', order['id']))

    return subtotal

For a loyal customer with a 1200 order, the math drifts:

• Before: 1200 → 1080 → 1070. Log: discount, loyalty.
• After: 1200 → 1190 → 1071. Log: loyalty, discount.

The diff looks clean. The behavior changed. Reviewers approve this daily. Code review tests plausibility, not arithmetic.

Behavior fingerprints

A behavior fingerprint records observable effects. Same function. Same probe inputs. Same side-effect ledger. Different hash means different behavior.

A golden file captures bytes. A differential test needs a second implementation. A fingerprint only needs a probe set. Run it before the refactor. Run it after. Compare what changed.

# fingerprint.py
import hashlib
import json

PROBES = [
    ('large_plain', {'id': 'A1', 'total': 1200, 'loyalty': False}),
    ('small_loyal', {'id': 'B2', 'total': 150,  'loyalty': True}),
    ('tiny_loyal',  {'id': 'C3', 'total': 90,   'loyalty': True}),
    ('large_loyal', {'id': 'D4', 'total': 1200, 'loyalty': True}),
]

def fingerprint(apply_fn, probes=PROBES):
    rows = []
    for label, order in probes:
        log = []
        try:
            value = apply_fn(order, log)
            state = 'ok'
        except Exception as exc:
            value = None
            state = type(exc).__name__
        rows.append({
            'case': label,
            'state': state,
            'value': value,
            'log': log,
        })
    raw = json.dumps(rows, sort_keys=True).encode()
    return hashlib.sha256(raw).hexdigest(), rows

The payload records three signals per case. The return value. The exception class. The log sequence. Sorting keeps the hash stable. Stable hashes make comparisons honest.

Run it against the legacy module.

python -c "from legacy import apply_discount; from fingerprint import fingerprint; print(fingerprint(apply_discount)[0])"

Save that hash. Apply the proposed refactor. Run it again.

python -c "from refactored import apply_discount; from fingerprint import fingerprint; print(fingerprint(apply_discount)[0])"

Now compare the two hashes. A match means your probes saw the same behavior. A mismatch means something moved. Diff the rows to name the exact probe. Review has a target instead of a vibe.

Building the probe matrix

Probe matrices need four input families. Normal, boundary, failure, and side-effect-sensitive.

Normal inputs prove the happy path. Boundary inputs catch threshold moves. Failure inputs catch exception-order changes. Side-effect inputs catch logging and mutation shifts. The example above uses four probes. A payment reconciler may need forty. Add probes whenever a bug escapes.

The workflow

Six steps. No ceremony.

1. Freeze the interface. Change the body only. Keep the signature untouched.
2. Build the probe matrix. Cover happy paths, boundaries, failures, and side effects.
3. Generate the baseline fingerprint. Save the hash with the refactor branch.
4. Ask a free model for a refactor draft. MonkeyCode's free model access fits this step. Paste the legacy function and the probe list.
5. Run the fingerprint against the draft. Compare hashes.
6. Inspect only mismatched probes. Decide whether each shift is a bug or an accepted change.

Step six keeps humans in charge. The hash points at the problem. Reviewers judge the trade-off.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Where MonkeyCode fits

The probe set works without MonkeyCode. The script is plain Python. Python 3 is the only dependency.

MonkeyCode adds two practical pieces. Its free model access drafts the refactor. Its free server option can host the fingerprint script. Shared hosting keeps the team on one probe set. No drift between laptops. No works on my machine fingerprints. The method does not depend on it.

Choosing the right verification

Different refactors need different evidence. Use the right tool.

Fingerprints are cheap. They fit AI-assisted cleanup best. Golden files remain better for serialization code. Differential tests remain better when both implementations survive.

Limits

A fingerprint is evidence. It is not proof.

Matching hashes only cover the probes you wrote. Unknown inputs stay outside the evidence. Nondeterministic code breaks the method. Random values and timestamps add noise. Fix seeds first. Or move randomness outside the probed function.

Who should not use this:

• Teams without stable probe inputs. UI gestures and third-party APIs resist fingerprinting.
• Safety-critical systems. A hash is a floor, not a ceiling. Keep formal verification.
• One-off micro-changes. The setup cost may exceed the payoff.

The real gate

Code review answers one question. Does the diff look reasonable?

It never answers the second question. Does the program behave the same? A probe-based fingerprint answers that. Cheap. Repeatable. Hostile to plausible errors.

Run it before the reviewer reads anything. Your next AI refactor deserves a witness.