Case Study: A Gated Loop on Free Infrastructure — 3 Rounds to a Correct C++ Header Parser

The short version

I asked a free model endpoint to implement a small C++ parser for Content-Disposition header values. Every output went through a three-stage gate on a free server: warnings-as-errors compile, ASan/UBSan, and differential testing against a hand-written reference oracle. Round 1 failed 12 of 5,000 generated cases. Round 2 failed 7. Round 3 passed, including a 20,000-case re-run with a different seed.

The failures were more interesting than the green result. All of them sat at grammar boundaries: end of input, escaped quotes, empty quoted strings, and obs-text bytes. None would have shown up in a typical "does it compile and handle this example" review.

Background: why a header parser

Content-Disposition looks simple: a type token, then semicolon-separated parameters like filename="report.pdf". The trap is the quoted-string grammar from RFC 9110 combined with RFC 6266. Inside quotes you can have spaces, tabs, obs-text bytes (0x80-0xFF), and backslash escapes. An escaped quote does not terminate the string. An unterminated string is a syntax error.

That grammar is small enough to test thoroughly by differential testing, and tricky enough that generated code will miss cases. It is the right size for a one-session case study.

Goal and gate contract

The goal was not a perfect parser. It was a parser that earns production access by passing a reproducible gate: model as author, gate as reviewer.

The gate had three stages, run in order:

1. Compile with -Wall -Wextra -Werror and sanitizers enabled.
2. Build a differential harness that compares the model's parser against a reference oracle.
3. Generate 5,000 grammar-based random inputs, feed them to both parsers, and fail on any mismatch in success/failure or parsed output.

I used MonkeyCode's free model endpoint for codegen and its free server option to run the gate. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Implementation

The contract is one function:

#pragma once
#include <string>
#include <string_view>
#include <utility>
#include <vector>

struct Disposition {
    std::string type;
    std::vector<std::pair<std::string, std::string>> params;
};

// Parses a Content-Disposition header value (RFC 6266).
// Returns false on any syntax error. Quoted-string escapes are decoded.
bool parse_disposition(std::string_view value, Disposition& out);

The reference oracle is a hand-written scanner of about 120 lines. Its core is the quoted-string rule:

static bool parse_quoted(std::string_view v, size_t& i, std::string& out) {
    ++i; // skip opening quote
    while (i < v.size()) {
        char c = v[i];
        if (c == 0x22) { ++i; return true; }   // closing quote
        if (c == 0x5C) {                       // backslash
            if (i + 1 >= v.size()) return false;   // dangling escape
            char n = v[i + 1];
            unsigned char nu = static_cast<unsigned char>(n);
            if (n != 0x09 && n != 0x20 && !(nu >= 0x21 && nu <= 0x7E) && nu < 0x80)
                return false;                       // invalid quoted-pair
            out.push_back(n);
            i += 2;
            continue;
        }
        unsigned char u = static_cast<unsigned char>(c);
        bool ok = (c == 0x09 || c == 0x20 || c == 0x21 ||
                   (u >= 0x23 && u <= 0x5B) || (u >= 0x5D && u <= 0x7E) || u >= 0x80);
        if (!ok) return false;
        out.push_back(c);
        ++i;
    }
    return false; // unterminated
}

The generator produces random values from the same grammar: a type token, zero to three parameters, and each value is either a token or a quoted string with random escapes and obs-text. The differential harness reads lines from stdin and compares both parsers:

while (std::getline(std::cin, line)) {
    Disposition a, b;
    bool ok_a = parse_disposition(line, a);
    bool ok_b = parse_disposition_oracle(line, b);
    if (ok_a != ok_b || (ok_a && !(a.type == b.type && a.params == b.params))) {
        std::cout << "MISMATCH: " << line << std::endl;
        ++mismatches;
    }
}

The gate script ties it together:

#!/usr/bin/env bash
set -euo pipefail
g++ -std=c++17 -Wall -Wextra -Werror -fsanitize=address,undefined -c model.cpp -o model.o
g++ -std=c++17 -fsanitize=address,undefined diff_main.cpp oracle.cpp model.o -o diff_gate
python3 gen_cases.py 42 5000 | ./diff_gate

The three rounds

Round 1: compile passed, 12 mismatches. The model's scanner had no end-of-input state for quoted strings. A value like filename="abc was accepted as valid. It also treated an escaped quote as the closing quote, so a filename containing an escaped quote was decoded incorrectly, and the real closing quote was consumed as garbage. Both failures share one root cause: the scanner matched characters instead of states.

Round 2: compile passed, 7 mismatches. The patch added a length check, which fixed the unterminated-string case. But it over-corrected: an empty quoted string (filename="") was rejected, and obs-text bytes inside quotes were rejected. The model fixed a boundary bug by adding a stricter condition, and that condition was wrong in the other direction.

Round 3: 0 mismatches on 5,000 cases. I re-ran with a different seed and 20,000 cases: still 0. Sanitizers stayed clean across all rounds. The parser earned production access for this one narrow job.

What the failures had in common

All five distinct failure classes were boundary states:

• End of input: unterminated quote, dangling escape.
• The escape transition: escaped quote and escaped backslash.
• The empty case: an empty quoted string.
• The edge of the allowed byte range: obs-text.

The model was not bad at the happy path. It was blind to the states where a grammar rule ends. That is a useful prior: when reviewing model-written parsers, review the terminator cases first, not the examples in the prompt.

What the free server changed

Running the gate on MonkeyCode's free server option changed the iteration loop in two practical ways. First, the first compile of a session was noticeably slower than subsequent runs, so I batched all differential cases into one process instead of spawning per case. Second, because rounds were cheap, I stopped polishing prompts and started treating each prompt as a draft for the gate to edit. A failed round cost a few minutes, not a meeting.

The gate script also had a hard timeout, so a hung parser failed the round instead of consuming the session. That is a workflow choice, not a product metric, but it is the difference between a gate and a babysitter.

Limitations and who should not use this

The gate proves equivalence to my oracle on generated inputs. It does not prove RFC conformance, and the generator only explores the grammar I encoded. If the oracle is wrong, the gate is confidently wrong.

Do not use this loop if:

• You have no reference oracle. Differential testing against the model itself only measures self-consistency.
• The grammar is underspecified. The oracle becomes a hidden design document, and the gate will happily lock in your assumptions.
• You can only afford one round. The value of the loop is in the failures; a single pass is just a draft.
• The parser is security-critical. The gate narrows the bug space but does not replace a human review or a coverage-guided fuzzer like libFuzzer.

Lessons learned

1. Write the oracle first. It is the most valuable artifact in the loop, and it forces you to specify the grammar before the model sees the prompt.
2. Prompt for the contract, not the implementation. The interface and the gate define success; the model fills in the middle.
3. Re-run the full corpus after every patch. Round 2 fixed the old failures and introduced new ones; testing only the failing cases would have shipped a regression.
4. Model failures cluster at grammar boundaries. Review terminators, escapes, and empty inputs before anything else.
5. Free infrastructure changes the economics of iteration. When a round costs minutes, you run more rounds and write shorter prompts.

The final parser is about 80 lines and does exactly one job. The gate that reviewed it is about 120 lines and will review the next parser too. That asymmetry — small code, reusable gate — is the part I would copy into any project. If there is interest, I can publish the full gate script and generator as a follow-up.