Characterize First, Refactor Second: A Safe Workflow for Messy Repos

A messy repo is not a code problem. It is an information problem. You cannot refactor what you do not understand. Characterization tests lock current behavior into executable assertions. Then the smallest safe change becomes possible.

AI changed the reviewer role. Every developer now reviews model output. Few test the reviewer itself. Refactoring has the same gap. A model can propose a cleaner version of your worst function. Without characterization tests, you cannot tell a safe change from a silent break.

The reviewer bottleneck is real. A model can write a patch in seconds. A human verifies it in minutes. The verification step decides the outcome. Characterization tests make that step fast and objective.

This walkthrough uses a legacy config parser with three hidden quirks. The workflow has five steps. A free model proposes candidates. Characterization tests decide which candidates survive.

The Messy Unit

Here is the function. It grew organically over two years. It returns inconsistent types for similar inputs.

def parse_config(raw):
    cfg = {}
    for key, value in raw.items():
        if key == "timeout":
            cfg["timeout_ms"] = int(float(value) * 1000)
        elif key == "retries":
            cfg["retries"] = int(value) if value else 3
        else:
            cfg[key] = value
    if "timeout_ms" not in cfg:
        cfg["timeout_ms"] = 30000
    return cfg

Three quirks hide in plain sight. Timeout arrives in seconds but leaves in milliseconds. An empty retries string becomes three. Unknown keys pass through untouched. Each quirk is a contract.

Step 1: Lock Behavior With Characterization Tests

Run the function. Capture the output. Assert what actually happens, not what should happen. Characterization tests document reality. They do not judge it.

def test_parse_config_keeps_unknown_keys():
    raw = {"retries": "0", "vendor_extra": {"x": 1}}
    result = parse_config(raw)
    assert result["vendor_extra"] == {"x": 1}

def test_parse_config_defaults_timeout():
    assert parse_config({})["timeout_ms"] == 30000

def test_parse_config_converts_seconds_to_ms():
    assert parse_config({"timeout": "1.5"})["timeout_ms"] == 1500

def test_parse_config_empty_retries_becomes_three():
    assert parse_config({"retries": ""})["retries"] == 3

Four tests. No fixtures. No mocks. Every quirk now has a witness.

Characterization tests have three useful properties. They are executable, so CI can run them. They are objective, so review arguments stop. They are permanent, so future refactors stay honest.

Step 2: Ask the Model for a Candidate, Not a Verdict

The model should propose, not decide. Give it the function and the characterization tests. Ask for a behavior-preserving refactor. Demand a list of assumptions.

I used MonkeyCode's free model access for this step. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free server option handled execution without a paid plan. The whole experiment cost nothing.

Prompt template:

Refactor parse_config to remove duplicated logic.
Keep every observable behavior identical.
List every behavioral assumption you made.

Include the tests in the prompt. The model sees the contract before it writes code. Ask for a diff, not a new file. A diff is reviewable. A new file is a rewrite.

The candidate arrived in one pass. It extracted the timeout conversion. It kept the empty-string retries quirk. It flagged the default timeout as a separate decision. Good candidates name their assumptions.

Step 3: Diff the Candidate Against the Tests

Run the characterization tests against the candidate. Green means behavior preserved for covered cases. Red means the candidate changed something.

pytest test_characterization.py -q

The first candidate failed one test. It converted an empty retries string to zero instead of three. That failure was invisible in a code review. The test made it loud.

When a test fails, read the diff. Decide whether the new behavior is the bug you wanted to fix. If yes, write a regression test for the desired behavior. If no, reject the candidate.

Step 4: Apply the Smallest Safe Change

The smallest safe change is one extracted helper. Nothing else moves.

def _timeout_to_ms(value):
    return int(float(value) * 1000)

def parse_config(raw):
    cfg = {}
    for key, value in raw.items():
        if key == "timeout":
            cfg["timeout_ms"] = _timeout_to_ms(value)
        elif key == "retries":
            cfg["retries"] = int(value) if value else 3
        else:
            cfg[key] = value
    if "timeout_ms" not in cfg:
        cfg["timeout_ms"] = 30000
    return cfg

One helper. Zero behavior changes. The characterization tests stay as the permanent guard.

Why the smallest change? Because each change is a separate verification. A large refactor bundles ten changes into one diff. One failure hides nine successes. Small changes isolate risk. They also make reverts trivial.

Step 5: Measure the Delta

Use a decision table. It keeps the review objective.

Run the suite before the change. Run it after the change. Compare the two runs. The delta is the truth. The table replaces gut feeling. That is the point of the workflow.

Limitations

Characterization tests lock current behavior. If the behavior is a bug, write a regression test for the desired behavior instead. Coverage is the ceiling. Tests only protect what they exercise. This workflow does not fix architecture. It makes small changes safe.

Do not use this for greenfield code. Do not use it for performance-critical paths where the quirk is load-bearing. Do not use it for code you plan to delete anyway. A free model can propose plausible refactors. It cannot see your production traffic. You can.

The Takeaway

Next time a model offers a refactor, run it through characterization tests first. The model proposes. The tests dispose. The smallest safe change is the only change you should merge.