Don't Characterize the Whole Messy Repo: Triage Files by Churn, Complexity, and Risk First

Most characterization-testing effort is wasted. You write probes for files nobody touches. You protect code that will be deleted next quarter.

Triage first. Test only the files that deserve protection. This article gives you a 20-line script that scores every file in a messy repo. It shows which files need characterization tests before a refactor. It shows which files to leave alone.

The trap

A messy repo has no tests. You want to refactor. The standard advice is clear: write characterization tests first.

The standard advice is also incomplete. Characterization tests are expensive. Each probe needs review. Each assertion needs validation. A 40,000-line repo can burn weeks.

Most of that time is wasted. Dead code does not need protection. Stable code does not need protection. Only hot, risky, untested code does.

The triage script

Save this as triage.sh. It uses git log for churn. It uses wc -l for size. It counts branch keywords for rough complexity.

#!/usr/bin/env bash
# triage.sh — score every file in a messy repo
set -euo pipefail

REPO="${1:-.}"
cd "$REPO"

git ls-files '*.py' '*.js' '*.ts' '*.go' | while read -r f; do
  churn=$(git log --oneline -- "$f" | wc -l | tr -d ' ')
  size=$(wc -l < "$f" | tr -d ' ')
  complexity=$(grep -cE '\b(if|for|while|case|catch)\b' "$f" || true)
  base=$(basename "$f")
  has_test="no"
  if find . -type f \( -name "*_test.py" -o -name "*_test.js" -o -name "*_test.ts" -o -name "*_test.go" \) 2>/dev/null | grep -q "$base"; then
    has_test="yes"
  fi
  printf "%s\t%s\t%s\t%s\t%s\n" "$churn" "$size" "$complexity" "$has_test" "$f"
done | sort -rn | column -t -s $'\t'

Run it:

bash triage.sh /path/to/messy-repo

Sample output:

47  812  63  no   src/payments/engine.py
39  214  41  no   src/payments/currency.py
31  98   12  no   src/legacy/import_runner.py
2   1200 80  no   src/legacy/parser.py
0   45   3   no   src/legacy/dead_util.py

Churn is the first column. It is the strongest signal. A churn threshold of 20 works for most repos. Adjust it to your history. The goal is a short list, not a perfect ranking.

The decision matrix

Do not characterize everything. Use this table.

High churn means the file changes often. High complexity means changes are risky. No tests means you are blind. That combination is the only one that needs characterization tests.

Low churn with no tests is dead or stable code. Leave it. You will not touch it during the refactor.

In the sample output, engine.py and currency.py are targets. import_runner.py has high churn but low complexity. Rewrite it directly. parser.py is stable despite its size. Do not touch it. dead_util.py is dead. Delete it later, not during this refactor.

The smallest safe change

Now you have one file. Here is the workflow.

1. Pick the top file from the matrix.
2. Draft characterization probes for that file only.
3. Review every probe before running it.
4. Run the probes. Record the baseline.
5. Make the smallest possible change.
6. Re-run the probes. Compare outputs.
7. If outputs differ, revert and investigate.

Step 5 is the discipline. One function, one branch, one constant. Then verify.

The baseline run must be green. If a probe fails before the refactor, fix the probe first. A failing baseline tells you nothing about your change.

A sample probe for engine.py:

# probe_engine.py
from payments.engine import calculate_fee

def test_calculate_fee_zero():
    assert calculate_fee(0) == calculate_fee(0)  # baseline only

def test_calculate_fee_known_input():
    result = calculate_fee(100)
    print(f"fee for 100: {result}")

These probes do not assert expected values yet. They record current behavior. After the change, the outputs must match exactly.

What the diff should look like

The whole refactor diff should fit on one screen. If it does not fit, the change is too large. Split it and re-run the probes.

-    if user.plan == "pro" or user.plan == "business":
+    if user.plan in ("pro", "business"):

A one-line diff is easy to verify. A ten-line diff is not. Keep the surface small.

Where a free model fits

Drafting probes is mechanical. You read a function, list its inputs, and write a call for each. A free coding model can draft these quickly.

MonkeyCode's free model access can draft the probes for engine.py in this workflow. Review each one before running. Expect wrong argument orders. Expect calls to private methods. Review is not optional.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server option runs the probes on every push. That adds a second verification layer. The probes run in a clean environment, separate from your laptop.

Verify current limits yourself. Free tiers change. Quotas and durations are not permanent.

Who should not use this

This workflow is for messy, untested, legacy repos. It is wrong for other situations.

• Greenfield projects. Write real tests from day one.
• Small repos under 5,000 lines. Just read the code.
• Repos with good coverage. Your tests already protect you.

If your repo has none of those problems, skip the triage. You do not need it.

The measurable outcome

In the sample output, two of five files needed characterization tests. The other three stayed untouched. That is a 60% smaller surface.

The number that matters is not probes written. It is files touched. Fewer files touched means fewer risks. Fewer risks means a safer refactor.

Start with the script. Score your repo. Protect only what changes often and breaks easily.