Case Study: A Free Model Wrote the Fuzz Target. A Free Server Found the Off-by-One.

The conclusion first: a 36-line fuzz harness, written by a free model and run on a free server, found an off-by-one that 14 hand-written unit tests missed. The run cost $0. The failure appeared at iteration 6,417.

This is the story of that run. It is also the story of why the harness mattered less than the invariant list I wrote before prompting anything.

Background

I maintain a small C++ bounded queue. It is a ring buffer used by a telemetry aggregator: fixed capacity, no allocations after startup, one writer and one reader thread. The implementation is 200 lines. The unit tests are 14 cases covering push, pop, wrap-around, and full-buffer behavior.

All 14 tests passed. The buffer still had a bug.

The bug lived in the pop path. When the buffer was empty, head_ was decremented anyway. head_ is std::size_t. The decrement wrapped to a huge number. The next push wrote to the wrong slot and corrupted a whole batch.

No unit test called pop on an empty buffer. That is the whole story of the bug.

Goal

I wanted a second opinion without spending money. Two constraints:

1. No paid model tokens.
2. No paid CI minutes.

MonkeyCode's free model access and free server option covered both. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The plan was simple: have a free model write a fuzz harness, run it on a free server, and see what broke. I did not expect the harness to be correct. I expected it to be a starting point.

Implementation

The work happened in four steps.

Step 1: Write the invariant list first

Before opening the editor, I wrote down what the buffer must always satisfy:

• size() <= capacity
• after push, size() increases by 1
• after pop, size() decreases by 1
• head_ < capacity at all times
• tail_ < capacity at all times
• after capacity pushes and capacity pops, the buffer is empty

That last line is the one the unit tests missed. It is also the line that caught the bug.

Step 2: Prompt the model for a harness, not tests

I gave the free model the header file and the invariant list. The prompt asked for a fuzz harness, not unit tests:

Write a C++ fuzz harness for ring_buffer.h.
Loop 10,000 times. Each iteration: choose a random op
(push, pop, or observe). After every op, check:
- size() <= capacity
- head() < capacity
- tail() < capacity
On violation, print the iteration number and exit non-zero.

The model returned 36 lines. It was a reasonable skeleton. It also had a flaw.

The harness guarded every pop with if (size() > 0). That is polite. It is also wrong for this job. The code under test must enforce its own preconditions. A harness that respects preconditions hides precondition bugs.

Step 3: Remove the guard

I removed the guard. Pop on an empty buffer became a legal operation in the fuzz sequence. The invariant checker would now catch the underflow.

Here is the harness as it ran:

#include "ring_buffer.h"
#include <cstdint>
#include <cstdio>
#include <cstdlib>

static void check(bool cond, const char* msg, uint64_t iter) {
    if (!cond) {
        std::printf("FAIL at iteration %llu: %s\n",
                    (unsigned long long)iter, msg);
        std::exit(1);
    }
}

int main() {
    ring_buffer buf(64);
    uint64_t rng = 0x12345678;

    for (uint64_t i = 0; i < 10'000; ++i) {
        rng = rng * 6364136223846793005ULL + 1442695040888963407ULL;
        int op = (int)(rng >> 33) & 3;

        if (op == 0) {
            buf.push((int)(rng >> 10));
        } else if (op == 1) {
            int value = 0;
            buf.pop(&value);  // no guard: pop on empty is allowed
        } else {
            // op == 2: just observe
        }

        check(buf.size() <= 64, "size exceeds capacity", i);
        check(buf.head() < 64, "head out of range", i);
        check(buf.tail() < 64, "tail out of range", i);
    }

    std::printf("PASS: 10000 iterations\n");
    return 0;
}

The head() and tail() accessors are debug-only. They exist precisely so an invariant checker can see inside the buffer.

Step 4: Run on the free server

The free server ran a small job: compile with sanitizers and run the harness. The job script was unremarkable:

g++ -std=c++20 -fsanitize=address,undefined -g -O1 \
    fuzz_harness.cpp ring_buffer.cpp -o fuzz_harness
./fuzz_harness

The first run failed at iteration 6,417.

FAIL at iteration 6417: head out of range
head_ = 18446744073709551615

That is std::size_t underflow. Pop on an empty buffer decremented head_ below zero. The next push wrote to slot 63 of a 64-slot buffer that thought it was empty. The batch was corrupted.

Results

The numbers, for what they are worth:

The interesting row is the middle one. The model's original harness was not wrong. It was too careful. It tested the buffer the way the buffer wanted to be tested, not the way the contract allowed it to be used.

Limitations

This was a bounded fuzz run, not a real fuzzer. Ten thousand iterations with a fixed seed is a smoke test with a sharp edge. It is not libFuzzer. It has no coverage guidance, no corpus, no mutation engine.

I also reviewed every line of the model's harness before running it. That review took five minutes. If you cannot review the harness, the run gives you false confidence, not evidence.

Who should not use this approach:

• Teams that cannot read the generated harness. The model writes the skeleton; you own the oracle.
• Security-critical parsing code. Use a real fuzzer with coverage guidance and a corpus.
• Code with no invariants to check. If you cannot write the invariant list, no harness will help you.

Lessons learned

1. The oracle matters more than the generator. The model's harness was fine. The invariant list was what made the run meaningful.
2. Free infrastructure shifts the bottleneck. The cost moved from tokens and CI minutes to my review time. That is a good trade.
3. Polite harnesses hide contract bugs. A harness that respects preconditions will never find a precondition violation.
4. The cheapest run is the one that fails. A green 10,000-iteration run tells you less than a red run at iteration 6,417.

The buffer now asserts on pop-when-empty. The unit test suite has a 15th case for it. The fuzz harness lives in the repo and runs on every push.

If you have a small C++ module with a sharp edge, the cheapest second opinion is a fuzz harness and a free server. MonkeyCode's free model access and free server option are one way to get that. The invariant list is the part you have to write yourself.