It all started on June 23rd with a casual post about a VPS Manager benchmark.
Out of curiosity, I decided to ask the author of the benchmark,
Pascal was intrigued. He pointed out a brilliant hypothesis: if a model makes significantly fewer mistakes, the total session cost drops dramatically even if the per-token price is higher. He cited GLM 5.2 as a model that self-corrected multiple bugs during verification to achieve 37/37 tests passing.
Curiosity got the better of me. I spun up my development environment, wrote a custom agent harness, and ran it on Kimi K2.7 using Cloudflare Workers AI.
The V.E.L.O.C.I.T.Y.-OS Series Table of Contents
We are building a bare-metal, self-healing operating system running entirely inside the CPU's L3 cache. Here is the roadmap for this 12-part series:
- Part 1: The Spark — Exposing the "Safe-Room" security leak and building the compiler gate. (You are here)
- Part 2: The NDA Language — Designing a content-addressed triplet representation to cure context bloat.
- Part 3: Ditching the Web Stack — Building a native 30MB IDE with 1,500,000x IPC latency drops.
- Part 4: The Closure JIT — Compiling AST blocks to nested closures and bypassing borrow checker limits.
- Part 5: JIT Math Optimizations — Replacing division operations with precomputed 16-bit lookup tables.
- Part 6: x86-64 Assembler & SCEV-Lite — Compiling scalar loops directly to native code in constant time.
- Part 7: Classic Compiler Passes — Implementing inter-procedural Dead Code Elimination and loop unrolling.
- Part 8: Reclaiming Ring 0 — Exiting UEFI boot services and transitioning the kernel to Ring 0.
- Part 9: Bare-Metal Drivers — Writing a PCI scanner, NVMe block storage controller, and FAT32 parser.
- Part 10: Synaptic Canvas — Rendering a spatial, force-directed GUI based on model token activation vectors.
- Part 11: Swarms & Hot-Patching — Building multi-agent scheduling and zero-downtime RCU driver updates.
- Part 12: Self-Evolution — Handing system control over to a local LLM Terminal that self-optimizes via telemetry.
The Leak: Safe-Room Security
The initial run looked amazing—Kimi successfully completed 19 of the 30 foundation files on my daily free allocation, delivering the cleanest architectural layout of any model tested. But in the meantime, Pascal had run Kimi K2.7 himself and caught a major security blocker on DB credential handling.
This prompted me to dig into the 19 files from my own Foundry run, only to find the exact same mistakes: Kimi had exposed database connection credentials directly in the code.
Pascal pointed out that this wasn't a failure in reasoning—it was a scope failure. Kimi was operating under "safe-room security": it optimized for code correctness against the written spec, assuming it was running in a secure, isolated sandbox rather than a live production environment.
The Solution: Gatekeeper Static Scanning
Pascal suggested that rather than bloating every single system prompt with complex, instruction-taxing security warnings (which models eventually ignore or drift from), I needed a systematic gateway.
That conversation was the spark. I went to work on gatekeeper.rs and built a local security static analysis scanner and sandbox verifier directly into the compilation gate. The rule was simple: before any generated file could be marked as complete and persisted, the Gatekeeper ran systematic regex-based and syntax-tree scans to detect database credentials, hardcoded keys, and common security flaws.
Furthermore, I wired the compiler directly into an isolated JIT sandbox (AssertUnwindSafe) to dry-run the generated bytecode. If the JIT compilation or the dry-run failed, the compiler rejected the output, forced the model to reflect on the diagnostic error, and triggered an automatic self-correction loop.
Here is the architectural flow of how code moves from the LLM model to the secure, bare-metal storage layer:
Here is the core logic from gatekeeper.rs that classifies and verifies LLM-generated code in an isolated environment before committing it to the codebase:
// gatekeeper.rs — Gatekeeper Hybrid LLM Router & Sandbox Verifier
pub enum LlmRoute {
CloudSwarm, // High-complexity planning (GPT-4o/Claude 3.5)
LocalAgent, // Low-complexity execution (Qwen-Coder-0.5B)
}
pub fn classify_query(query: &str) -> LlmRoute {
let q_lc = query.to_lowercase();
if q_lc.contains("architecture") ||
q_lc.contains("blueprint") ||
q_lc.contains("refactor kernel")
{
LlmRoute::CloudSwarm
} else {
LlmRoute::LocalAgent
}
}
// Returns Vec<f32> representing the token activation states (the embedding vector)
// rather than raw bytecode, laying the groundwork for semantic clustering in Part 10.
pub fn route_and_generate(query: &str, site_map: &crate::nda_jit::SiteMap) -> Result<Vec<f32>, &'static str> {
let route = classify_query(query);
match route {
LlmRoute::CloudSwarm => {
// Plan via high-capacity cloud swarm...
generate_bytecode_from_prompt(&format!("/* Cloud Swarm: {query} */"), site_map)
}
LlmRoute::LocalAgent => {
// Direct generation via local model...
generate_bytecode_from_prompt(query, site_map)
}
}
}
This security gate raised the floor for any model running through the pipeline. It was no longer about finding the most "secure" model—it was about building an infrastructure that forced security by construction.
But as the agent continued generating files, I hit another wall: context bloat. The context accumulation of self-correction was costing me valuable seconds and tokens.
In the next post, I'll detail how I tamed the context monster by inventing a new binary format and a multi-agent debate board.
Discussion
How are you all handling LLM "scope failures" in your local agents? Do you prefer prompt engineering or, like me, a hard-coded "Gatekeeper"? Have you noticed your LLM-generated code taking "security shortcuts" like this? I'd love to hear how you're validating AI output in your own pipelines!
Special thanks to , whose peer critique on scope failures pushed me to build this security gate rather than relying on prompt engineering.
Disclaimer: AI was used throughout this project, it is just fitting that it would co-author with me, so special thanks to the Foundry for it's tireless hours toiling away and Gemini for producing the cover image.
