Beyond free-roaming Agents: Architecting a Deterministic 4-node Graph pipeline for near zero false-positive Autonomous VAPT

Beyond free-roaming Agents: Architecting a Deterministic 4-node graph pipeline for near zero false-positive Autonomous VAPT

Every AppSec engineer eventually hits a mathematical scaling wall.

In a high-growth environment, a single application security engineer is often personally responsible for securing dozens of microservices, multiple public-facing API gateways, complex native mobile apps, and continuous CI/CD pipelines. When you are outnumbered 100-to-1 by software engineers shipping code daily, 70% of your offensive testing time is eaten by repetitive friction: subdomain sweeps, manual API parameter fuzzing, writing headers, and inspecting boilerplate responses.

Naturally, when the AI wave hit, many of us tried to offload this friction by feeding targets to LLMs.

But if you’ve ever tried to run a penetration test using a standard conversational LLM chatbot, you know it fails catastrophically in production. The reasons are always the same:

• Ephemeral Context: The context window eventually resets or decays, completely wiping out hours of complex reconnaissance and target memory.
• Phantom Triage: Hallucinated vulnerabilities create massive time-sinks where you end up hunting for vulnerabilities that do not exist.
• Friction & Latency: They require constant, manual "human-in-the-loop" prompting between every single execution step.

Frustrated by these limitations, I set out to build something different.

This is the story of how I evolved my automated security testing from a brittle, free-roaming prompt loop into Okwute: a deterministic, 4-node directed graph pipeline (Mapper → Generator → Executor → Validator) that runs on a headless, self-hosted harness to achieve near zero false-positive autonomous penetration testing.

Phase 1: The .claude Harness (Context on Disk)

My initial attempt at solving LLM amnesia was building an engine, a .claude Harness.

Instead of letting the agent hold execution state in its active context, I decided to decouple the intelligence layer from the ephemeral chat session. The core breakthrough was "loop engineering", creating an execution engine that lives between prompts by anchoring all memory, phase progression, and vulnerability proofs in deterministic filesystem artifacts on a self-hosted workspace.

Instead of isolating learning inside independent target silos, I designed an enterprise-ready workspace topology with a robust Three-Tier Memory System under a central .claude/ root runtime:

.claude/
├── shared/
│   └── org-knowledge.md         # Tier 1: Organization-wide security baselines & false-positive filters
├── products/
│   └── <product-slug>.md        # Tier 2: Shared product-family dynamic memory (e.g., login_api)
├── working/
│   └── <session-id>/
│       └── memory.md            # Tier 3: Isolated session scratchpad & exploration sandbox
└── targets/
    └── payment-gateway/         # Isolated target workspace
        ├── state.json           # Authoritative state-machine ticker & configs
        ├── findings.md          # Cumulative, append-only verified vulnerability proofs
        ├── triage.json          # Human-in-the-loop triage overrides
        └── reports/             # Generated executive & technical vulnerability reports

The Three-Tier Memory Model

This directory layout decouples ephemeral execution logs from enterprise-grade intelligence:

1. Tier 1 (Org Knowledge): Read-only for active agents. It enforces company-wide guardrails and cataloged false-positive exemptions (e.g., "Spring Actuator 401s are intentional").
2. Tier 2 (Product Memory): Keys dynamic vectors across targets of the same product_type. If an agent discovers sequential ID exploitation patterns on a target within a product family, every other target in that family inherits that heuristic, speeding up warm-starts.
3. Tier 3 (Agent Working Memory): Restricts active session scratchpad noise, keeping unverified exploit attempts and exploratory hashes fully isolated until they are ready to be verified.

The system ran as an autonomous state machine. Instead of running an infinite script that would drain API credits, it executed one bounded, deterministic "tick" via a /scan-target command.

During each tick, the harness would:

1. Parse state.json to determine the current testing phase (recon, enumeration, vuln-discovery, chain-development, reporting).
2. Read the local filesystem on disk to reconstruct what it had already scanned.
3. Call the LLM to decide the next logical action.
4. Save the updated results back to disk.
5. Exit.

By writing every single observation, endpoint, and finding immediately to a structured state.json and a Markdown audit trail (findings.md), the context could survive host restarts, process crashes, and API disconnects.

The Concurrency Conflict

To allow multiple instances of this harness to run without stepping on each other's toes, I implemented a lightweight three-tier memory system using SHA-256 context hashing in the file frontmatter:

 `context_hash = SHA-256 (file_content ∖ {context_hash line})`

If a background worker attempted to write an update to a product's shared memory, it would first re-evaluate the hash on disk. If the current hash matched the initial state, the write was committed; if not, a conflict was declared, and the worker safely aborted.

It was an incredible step forward. But as we scaled it, we hit a massive conceptual roadblock: LLMs love to roam free. If you give an LLM agent access to a generic "run terminal command" tool, it will eventually try to skip steps, guess endpoints, or spiral into endless recursive loops trying to fix a single broken curl payload.

We needed a system that enforced absolute discipline.

The Epiphany: James Kettle's "HTTP Terminator"

In August 2026, at DevCon (presented also at Black Hat USA 2026 and DEF CON 34), PortSwigger's Director of Research, James Kettle, published a paradigm-shifting whitepaper titled "Can AI do novel security research? Meet the HTTP Terminator".

In his paper, Kettle tackled a profound question: can an autonomous AI system actually invent new attack techniques, bypass complex security layers, and discover zero-days on live, production systems?

To prove it, he built and open-sourced the HTTP Terminator, an autonomous research engine designed to hunt for HTTP desync vulnerabilities. But his ultimate success was built on a series of harsh, real-world lessons about the cognitive limits of SOTA LLMs. He realized that while LLMs are brilliant at high-level reasoning and pattern recognition, they are notoriously unreliable at state-tracking, structured execution, and exact payload formatting.

Left to their own devices, agents in a freeform loop behave like over-caffeinated interns. They trigger WAF rate-limits, get trapped in recursive debugging loops, hallucinate vulnerabilities, and frequently give up entirely upon seeing defensive headers like Connection: close.

To tame this chaos, Kettle structured the HTTP Terminator around a Four-Phase Discovery Loop:

1. Ideation (with Micro-Inspiration): To prevent models from over-anchoring on existing concepts (context-contamination), Kettle stripped down the input. He parsed 138 technical RFCs into 15,000 tiny, 1-to-3 sentence fragments (micro-inspirations) and prompted the LLM to generate 1-to-5 vectors per fragment, resulting in 30,000 highly unique candidate desync payloads.
2. Evaluation: A simple, non-judgmental execution harness. Instead of pre-configuring what a "successful" vulnerability should look like, the system paired a regular HTTP request with a candidate desync trigger and flagged anomalous deltas in the response status or body length.
3. Weaponization: Moving from a raw desync trigger to verified impact (e.g. Response Queue Poisoning). To bypass LLM safety guardrails ("offensive capability increases"), Kettle utilized Reality Re-framing (renaming local MCP tools to 'Turbo Simulator' so the agent believed it was in a sandbox) and Placebo Capabilities (giving the agent dummy connection-reuse tools to stop it from giving up when connection reuse was blocked). He also designed the "dangling-byte technique", using partial HTTP requests missing a single byte, to systematically defeat the front-end stacked-response connection resets.
4. The Cascade: The heart of true research. The engine took every verified finding and fed it back in to ask: "How can I detect similar behavior elsewhere? Does the origin of that behavior enable other attacks?" This systematic feedback loop led directly to his most significant conceptual breakthrough: Shared-Parser Confusion (the discovery that back-end servers use shared parser code to process both requests and responses, meaning response-only processing directives can be triggered via malicious requests) and a zero-day in Apache Traffic Server (CVE-2026-63078).

Turning the Blueprint into Code: The Genesis of Okwute

Kettle's ultimate takeaway was that "AI vs. Human" is the wrong framing. Instead, modern security engineering is a three-way collaboration: AI vs. Code vs. Human.

He found that starting with an AI-heavy loop is useful for speed, but to achieve consistent, production-grade accuracy, you must gradually move cognitive responsibility to deterministic code. In his exploitation engine, he split templates in half, ensuring deterministic code validated the evidence while the LLM handled planning.

This core design principle was the absolute catalyst for Okwute.

Okwute takes Kettle's four-phase research methodology and compiles it into a rigid, relational, and mathematically-enforced Directed Acyclic Graph (DAG). I eliminated the loose phase drift of legacy tools by locking down each step inside its own database boundary. The Mapper, Generator, Executor, and Validator are the physical implementation of Kettle's Ideation, Evaluation, Weaponization, and Cascade loops, bound forever by strict read/write contracts on SQLite.

Phase 2: Okwute: The Deterministic 4-Node Pipeline

I retired the free-flowing state machine of the .claude harness and completely re-engineered the platform around a rigid 4-node pipeline managed by SQLite.

   ┌──────────────┐      ┌──────────────┐      ┌──────────────┐      ┌──────────────┐
   │    MAPPER    │─────▶│  GENERATOR   │─────▶│   EXECUTOR   │─────▶│  VALIDATOR   │
   │ (Pure Recon) │      │ (Test Cases) │      │ (Burp Suite) │      │ (PoC Proof)  │
   └──────────────┘      └──────────────┘      └──────────────┘      └──────────────┘

The pipeline operates on a strict rule: Each node reads only from its upstream table and writes only to its designated downstream table. No node may skip, jump ahead, or bypass the graph.

Here is how the 4 nodes execute natively in a self-hosted environment:

Node 1: The Mapper (Pure Reconnaissance)

• Command: /run-mapper <target> [<domain-or-url>]
• The Rules: The Mapper has zero judgment responsibilities. It is not allowed to generate payloads or think about exploitation.
• The Job: It runs high-speed subdomain enumeration (subfinder), port scanning (nmap), and passive traffic analysis (httptoolkit MCP).
• Output: It inserts discovered raw endpoints directly into the endpoints table of the target's SQLite database (appsec.db).

Node 2: The Generator (The Threat Modeler)

• Command: /run-generator <target>
• The Rules: The Generator is highly intelligent, but completely sandboxed. It is strictly forbidden from sending a single network packet.
• The Job: It reads unprocessed rows from the endpoints table. Utilizing the LLM's deep contextual reasoning, it evaluates the endpoint parameters, headers, and underlying technology stack. It then plans highly specific test cases (e.g., SQLi, IDOR, CL.0 HTTP Request Smuggling desync paths, unvalidated postMessage handlers).
• Output: It writes candidate payloads and expected behaviors to the test_cases table.

Node 3: The Executor (The Mechanical Gun)

• Command: /run-executor <target>
• The Rules: The Executor is completely mechanical. It has zero intelligence and does not evaluate whether a vulnerability is real or fake.
• The Job: It takes fully-specified HTTP requests directly from the test_cases table and fires them through a headless Burp Suite Professional instance using a custom-built Java proxy (mcp-proxy) that bridges Burp's REST API into the Model Context Protocol (MCP).
• Output: It records the raw response, status codes, and response length directly into execution_results. If a response shows an anomaly (status or length delta compared to the baseline), it flags it and creates an active Repeater tab in Burp Suite for manual traceability.

Node 4: The Validator (The Jury)

• Command: /run-validator <target>
• The Rules: The Validator has the highest evidentiary bar: Zero False Positives. Nothing lands in the findings file without an automated, reproducible Proof of Concept (PoC).
• The Job: It reads anomalous rows from execution_results. It parses the raw HTTP response and compares it logically against the Generator's expected_behavior. It actively attempts to reconstruct a working, step-by-step PoC request-response pair.
• Output: Verified, iron-clad vulnerabilities are written to the verified_findings table on disk and synced to a shared cross-target SQLite memory (appsec_memory.db) to prioritize future test cases on similar tech stacks.

The SQLite Core: How Okwute Keeps State

Because all memory lives in SQLite, there is an immutable audit trail. Data is never deleted; rows are progressed through a state lifecycle (unprocessed $\to$ processed).

Here is a look at how clean and simple the database model is:

The Centralized Memory Evolution: Shifting to Obsidian

The transition from the old .claude harness to Okwute forced us to completely rethink how to manage cross-target security memory.

In the early iterations, I attempted to maintain cross-target knowledge inside flat, prose Markdown files (products/<product-slug>.md). But as multiple agents executed in parallel, this filesystem-centric model created a severe bottleneck: concurrent file writes caused clobbered updates, and the SHA-256 context hashing system would trigger frequent merge conflict halts, entirely defeating fully autonomous runs.

We solved this concurrency crisis by building a Statistical Cross-Target Memory Engine (appsec_memory.db) managed by SQLite. Instead of prose notes, the engine computes real-time mathematical hit-rates for successful exploit vectors:

    `Exploit Hit Rate = confirmed_count / tested_count`

If a specific JWT header spoofing vector yields an 85% success rate on a fintech target, that statistical weight is committed atomically to the centralized database, allowing day-one prioritization when the pipeline spins up on a new target sharing the same product_type.

Bridging Concurrency and Human Readability: The Obsidian Sync

While relational tables solve the multi-agent race conditions, they are notoriously hostile to human operators who need to quickly review, organize, and enrich security findings.

To bridge this gap, I designed a hybrid interface that ports the relational memory directly into a hyper-linked Obsidian Vault using a dedicated Python command-line utility (memory_db.py).

# Synced dynamically via a production orchestrator
python3 .opencode/scripts/memory_db.py sync-vault /workspace/obsidian-security-vault/

This synchronization engine dynamically translates binary SQL tables into a beautifully structured, human-readable Obsidian workspace:

Obsidian_Vault/
├── Index.md                     # Auto-generated, linked directory map of all product types
└── Products/
    ├── fintech_api.md       # Auto-generated vector hit-rates and metrics (Overwritten on sync)
    └── fintech_api - Notes.md # Hand-authored qualitative research and overrides (Preserved forever)

The system automatically splits product-family memory into a dual-file architecture:

1. The Machine Ledger (<product_type>.md): This is a read-only, auto-generated summary updated on every sync. It lists live empirical hit rates, historical vulnerability vectors, and automated false-positive rules. Humans never edit this file.
2. The Human Ledger (<product_type> - Notes.md): This file is created as a blank stub once. It is never overwritten during sync cycles. It is a playground for the human pentester to hand-author qualitative notes, record technology quirks, and outline manual exploitation hypotheses that the AI can read and import as context rules.

This design gives us the best of both worlds: strict relational concurrency for parallel agents, and a beautifully visualized Markdown canvas for the human product owner.

The Orchestration Layer: Multi-Modal Control Planes

While the core SQLite database acts as a decentralized state engine, how the pipeline is actively triggered and managed can be adapted across three distinct operational control planes, depending on the environment:

1. Continuous DevSecOps (Headless CI/CD)

For hands-off, continuous scanning, the entire pipeline is packaged inside a self-hosted Forgejo Actions (or GitHub Actions) CI/CD runner. This headless pipeline runs overnight on a dedicated label-node, managing process state for Burp and the AI harness dynamically.

• The Mechanism: An automated cron or repository dispatch triggers Forgejo Actions to initialize the target's workspace, spin up the headless Burp proxy instance, drive the execution pipeline sequentially, and drop a comprehensive markdown audit summary directly into the Forgejo job dashboard at completion.

2. Terminal-Native Harnesses (The Developer CLI)

For rapid, interactive local testing, security engineers can drive Okwute directly from terminal-native agent environments like Claude Code or OpenCode.

• The Mechanism: By hooking local Model Context Protocol (MCP) servers, engineers can run conversational slash commands (e.g., /run-generator or /run-validator) straight from their local terminal to triage new API endpoints or debug specific payload modifications in real time, combining autonomous execution with on-demand interactive control.

3. Custom-Built Command Dashboard (The Web Control Plane)

To abstract CLI friction entirely, Okwute's pipeline CLI outputs can be fed directly into a custom-engineered Web UI Control Plane.

• The Mechanism: This UI visualizes live telemetry of active target environments, lets security analysts draw trust boundaries on interactive SVG graphs, renders active Burp Repeater tabs side-by-side with LLM-generated payloads, and lets human-in-the-loop operators click-to-approve candidate findings before they sync to the global enterprise database.

Conclusion: Taming the Wave

AI is not going to replace the human-in-the-loop security engineer, but the security engineers who automate their workflows using deterministic graph engineering will completely outscale those who do not.

By moving away from open-ended chat inputs and building structured, database-backed state machines like Okwute, I can let AI do what it does best, reason, classify, and generate patterns, while forcing the underlying infrastructure to remain secure, disciplined, and deterministic.

Stop chatting with your models. Start building compilers for them.

What are your thoughts on agentic security pipelines? Have you explored wrapping Burp Suite or headless proxies into autonomous decision loops? Let’s discuss in the comments!