Cutover Diary: The Leftovers When You Move an AI Workflow to Free Infrastructure

Moving an AI workflow to free infrastructure is a cutover with a rollback plan, not a one-line configuration swap. The part nobody warns you about is the leftovers: the old endpoint, the retry logic, and the prompt quirks that surface under real traffic. This diary walks through a migration of a small AI review bot from a paid endpoint and a rented VPS.

The destination is the open-source MonkeyCode project's free model access and free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. You get the exact cutover plan, a shadow-mode script, and the leftovers I deliberately left behind.

Why the bill forced the move

An AI review bot is cheap until the diff volume grows. My bot ran on a paid model endpoint and a small VPS, and the monthly cost crept upward with every repository that adopted it. When I saw the free model access and the free server option, the arithmetic looked irresistible: zero infrastructure cost and a 10-million-token allowance on the model side. The migration was not simple, because a free tier is a different runtime rather than a cheaper copy of the old one.

The model endpoint behaves differently under load, the server has its own scheduling limits, and your tuned prompt template may not survive the move. You need a plan that treats the free path as a candidate until it earns the traffic. A shadow comparison tests the whole harness, not just the model, which is why it catches leftovers that a single benchmark would miss.

Inventory before the cutover

Before touching anything, I wrote down every dependency the old pipeline had. The list looked like this:

1. The model endpoint and the API key that authenticated against it.
2. The VPS that ran the cron job and the webhook receiver.
3. The prompt template that turned a git diff into a review comment.
4. The retry policy that handled rate limits and transient failures.
5. The webhook secret that verified incoming requests from the git host.

The inventory became the migration checklist, and every item ended up in one of three buckets: migrate, rework, or delete. The endpoint and the server went into migrate, the prompt template went into rework, and the webhook secret stayed on the old box until the new one proved itself. That bucket list is the real plan, and the cutover steps below only execute it.

The cutover plan, step by step

The plan had five steps, and the order matters because each one gives you a way back:

1. Freeze the old configuration and snapshot the prompt, the retry policy, and the server setup.
2. Run shadow mode: send every diff to both pipelines and compare the outputs.
3. Flip traffic for one repository only, not the whole organization.
4. Keep the old server alive for seven days as a rollback target.
5. Delete the old endpoint only after the new path survives a full week of real reviews.

Shadow mode is the step that most people skip, and it is also the one that catches the leftovers. I wrote a small script that ran both pipelines side by side and printed every difference between their outputs. The script below is a reproducible template, so adapt the command names to your own pipeline.

# shadow_review.py — run the old and new review pipelines side by side
import json
import subprocess

DIFF = subprocess.check_output(['git', 'diff', 'HEAD~1', 'HEAD']).decode()

def run_pipeline(name, command):
    result = subprocess.run(command, shell=True, capture_output=True, text=True, input=DIFF)
    if result.returncode != 0:
        return {'error': result.stderr.strip()}
    return json.loads(result.stdout)

old = run_pipeline('old', 'python review_old.py')
new = run_pipeline('new', 'python review_new.py')

for key in sorted(set(old) | set(new)):
    if old.get(key) != new.get(key):
        print(f'{key}: old={old.get(key)!r} new={new.get(key)!r}')

Run it against the last twenty merged pull requests, and you will see exactly what changes when the runtime changes. Save the output to a file so you can review it without scrolling: python shadow_review.py > shadow.log. The output is your rework list, and you should not flip traffic until that list is empty or explicitly accepted. In my case, most differences were prompt formatting rather than real review quality, which was a relief and a warning at the same time.

The leftovers that almost broke the cutover

Three things did not migrate cleanly, and each one is a lesson about free infrastructure. The first leftover was the retry logic: the old client retried rate limits with exponential backoff, while the free tier needs a budget-aware retry that stops after a few attempts instead of hammering the allowance. The second leftover was the prompt template, because the new model expected a shorter system prompt and ignored the old one's formatting instructions. The third leftover was the cron schedule, since the free server has its own scheduler and my assumption about local time silently broke the nightly review run.

Here is the decision table I used to sort every leftover:

The pattern is obvious in hindsight: everything that depended on my old runtime's assumptions needed rework, while everything that was just configuration migrated cleanly. Budget the rework time before you start, because the leftovers are the actual migration cost. The table also became the rollback checklist, because each reworked item had a known old behavior to restore.

Limitations and who should not do this

Free infrastructure is a trade, not a gift, and you should be honest about what you are trading. If your team needs data residency guarantees, a written SLA, or a pinned model version for compliance, keep the paid path and skip this migration entirely. Free tiers also change their terms, so verify the current allowance and server details in the project repository before you commit a workflow to them. The 10-million-token allowance and the free server option were accurate for this project at the time of writing, but you should treat any free tier as a moving target.

The approach also assumes your workload can tolerate occasional unavailability and slower cold starts. A review bot that runs nightly is a perfect candidate, while a customer-facing chat endpoint is not. If your pipeline blocks a deploy or a merge, do not put it on free infrastructure without a fallback path.

What I would do differently

I would start shadow mode a week earlier and keep the old endpoint alive for two weeks instead of one. The extra overlap costs almost nothing, and it gives you a full cycle of weekly reviews to compare instead of a single quiet weekend. I would also write the decision table before the inventory, because the table forced me to name the rework items instead of hoping they would disappear.

The migration worked because I treated the free tier as a candidate rather than a promise, and the leftovers were the real lesson. If you are planning a similar move, start with shadow mode, keep your old endpoint alive for a week, and let the diff decide when you cut over. The project repository has the current details on the free model access and the free server option, so check those before you start your own diary, and feel free to try the free tier with a single repository first.