The Docs Draft Pipeline: What an AI May Write and What You Must Own

The most common documentation failure is not a weak prompt or a lazy writer; it is the absence of a clear boundary between machine-draftable content and human-owned claims. A pipeline that drafts reference sections with free-tier model access and then verifies them with a symbol drift check turns docs into a testable artifact instead of a trust exercise. The model writes the inventory, and the human owns the promises.

Why documentation rots inside a healthy CI pipeline

Documentation bugs share a distinctive property: they are usually discovered by the people who consume the API, not by the pipeline that builds it. A function renamed in the last refactor stays documented under its old name until a user files an issue, and a newly added flag never appears in the docs at all. The root cause is structural, because nothing in the merge pipeline compares the documented surface against the actual code surface.

A prompt cannot know what changed inside a pull request, so the fix has to live in the pipeline around the model. The workflow drafts reference material, validates that every documented symbol still exists, and routes the remaining claims to a human reviewer. That division of labor is the entire design, and each step has a concrete tool.

The ownership boundary: what a model may draft

The first step is to separate documentation into two classes by asking a single question: can this statement be verified against the codebase alone? If the answer is yes, a model may draft it, and if the answer is no, a human must own it. The table below applies that test to the statement types that appear in most API docs.

The reason for the boundary is that normative statements cannot be checked by reading the source code alone. A deprecation promise is a commitment to future behavior, and a security property depends on the deployment environment, so neither can be derived from a function signature. When a model drafts those lines, it is guessing about commitments that only the team can make, and the reader has no way to tell the difference.

The workflow: draft, validate, classify, commit

The pipeline runs on every pull request that touches a public API, and it has four steps that map to four distinct responsibilities.

1. Draft the reference sections. A scheduled job reads the changed modules and produces a candidate diff for the API documentation, covering signatures, flags, and examples. For the drafting step, MonkeyCode's free model access and free server option keep this job off your laptop and out of your token budget. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

2. Validate the symbols. The candidate diff is checked by a drift script that compares documented names against the actual AST of the package, and any mismatch fails the job before a human sees it.

3. Classify the claims. The diff is split by the ownership table, so the human reviewer sees only the normative lines instead of the inventory that a script already verified.

4. Commit with a sign-off. The merged diff includes the generated reference text plus a review line, so the documentation history records who owned each normative claim.

The drift-check artifact

The validation step is small enough to live in a single file, and it uses only the standard library. The script below extracts public symbols from a package with Python's ast module and compares them against the headings of a Markdown API document.

#!/usr/bin/env python3
"""Fail when a public symbol is undocumented or a documented symbol is stale."""
import ast
import re
import sys
from pathlib import Path


def public_symbols(root: Path) -> dict:
    symbols = {}
    for path in root.rglob("*.py"):
        if any(part.startswith(".") for part in path.parts):
            continue
        tree = ast.parse(path.read_text(), filename=str(path))
        for node in tree.body:
            if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
                if node.name.startswith("_"):
                    continue
                symbols[f"{path.stem}.{node.name}"] = node.lineno
    return symbols


def documented_symbols(doc: Path) -> set:
    text = doc.read_text()
    return set(re.findall(r"^#{2,3}\s+`?([\w.]+)`?", text, re.M))


def main() -> int:
    root = Path(sys.argv[1] if len(sys.argv) > 1 else "src")
    doc = Path(sys.argv[2] if len(sys.argv) > 2 else "docs/api.md")
    actual = public_symbols(root)
    documented = documented_symbols(doc)
    missing = sorted(set(actual) - documented)
    stale = sorted(documented - set(actual))
    for name in missing:
        print(f"UNDOCUMENTED {name}")
    for name in stale:
        print(f"STALE {name}")
    return 1 if missing or stale else 0


if __name__ == "__main__":
    sys.exit(main())

Run it locally or inside a CI step with a single command, and the exit code drives the merge decision for the documentation diff.

python doc_drift.py src docs/api.md

- name: Check documentation drift
  run: python doc_drift.py src docs/api.md

The script is deliberately naive, because a simple check that runs on every PR is worth more than a sophisticated one that nobody maintains. It expects one documented symbol per heading, so your API document needs a consistent heading format, and it only inspects top-level module symbols, so nested classes require a small extension.

What the script cannot see

The drift check proves that a documented name exists, but it cannot prove that the description next to that name is true. A model can confidently draft a parameter explanation that matches the signature while being wrong about the actual behavior, and no symbol comparison will catch it. That gap is exactly why the ownership table routes behavioral claims to a human reviewer instead of trusting the draft.

Free-tier generation also has practical limits that shape the design, and you should plan around them from the start. Treat the drafting job as a scheduled batch process rather than an interactive assistant, because a per-PR batch tolerates rate limits and retries far better than a blocking review step. The free server option matters for the same reason, since an unattended job in CI should not depend on a developer's local machine being online.

Who should not use this workflow

Teams without a designated reviewer should not adopt this pipeline, because the human ownership step is the part that prevents false documentation, not the model. Projects whose docs are mostly narrative, such as design histories or migration guides, will get little value from a symbol inventory, and the drift check will only add noise. If your documentation is already generated from a single source like OpenAPI or a type system, the script duplicates work that your generator already performs.

The ownership boundary is the transferable idea here, and the script is just a minimal implementation of one half of it. Run the drift check on your repository for a month and count the false positives, because that number tells you how much trust your documentation actually deserves.