I Spent 1 Billion Tokens Making a Real Chip with AI — and Found Its True Limits
TL;DR: This is not "I prompted an AI to make a chip." It's the consolidated retrospective of two month-long projects — a SHA-256 crypto accelerator driven end-to-end from RTL to a 10-item signoff on an open-source EDA stack, then integrated into a real SoC. Two AI agents plus one human, roughly 1 billion tokens across both projects. The chip passed signoff. But the most valuable output wasn't silicon — it was a clear map of where AI's ability actually ends, and a repeatable method for keeping it honest. And right now, I need one small favor from you (at the end).
What This Series Is (and Is Not)
I've written two deep dives covering the mechanics of these projects:
- The core — how an LLM agent carried a SHA-256 core from RTL to GDSII, through 10 industrial signoff checks on Yosys / OpenROAD / Magic / Netgen / iverilog.
- The integration — how that same core was embedded into the efabless Caravel SoC, a ~1.2M-unit chip with a RISC-V management core.
This post is different. It's the horizontal view those two vertical deep-dives couldn't cover: standing back and asking — after a billion tokens, what did AI actually do well, where did it genuinely fail, and what mechanism kept the whole thing honest?
If you want the step-by-step engineering, read the repos. This is the map, not the terrain.
The two open-source repositories:
- SHA-256 core: https://github.com/CDragon123-code/SHA-256
- Caravel SoC integration: https://github.com/CDragon123-code/caravel-sha256-accelerator
The Team: Two AIs and One Human
Most people picture "AI-assisted design" as I prompt, AI codes. The real architecture was more deliberate:
| Role | Who | What they did |
|---|---|---|
| Planner AI | (deepseek-v4-pro) | Wrote the design plan, set strategy, independently re-audited signoff evidence |
| Executor AI | (GLM-5.3) | Ran the EDA tools, pasted real reports, executed the plan step by step |
| Human | C Dragon | Made the judgment calls, broke ties, final sign-off |
Why two AIs? Because chip design isn't "generate runnable code" — it's "make a stack of correct, non-contradictory decisions under physical constraints."
A single AI doing both design and execution falls into a blind spot: it ends up vouching for code it wrote itself. So here, the two AIs are deliberately adversarial — the executor produces evidence; the planner refuses to trust conclusions and digs into raw logs, raw GDS, raw SPEF for the proof chain.
The human sits on top to break the deadlock when the AIs can't agree.
The Result, First
Before the failures, here's what got built. The consolidated spec sheet across both projects:
| Parameter | Value |
|---|---|
| Algorithm | SHA-256 (FIPS 180-4) |
| Process | SkyWater sky130A (130nm) |
| Standalone clock | 66.7 MHz (15ns) |
| SoC-integrated clock | 40 MHz (Caravel locked 25ns) |
| Signoff checks | 10 / 10 passed |
| Transistor-level LVS | 8,352 devices, 8,164 nets — match |
| DRC | 0 violations |
| Antenna | 0 (51 diodes) |
| Timing | setup +0.184ns / hold +0.024ns |
| Power | 20.3 mW |
| IR drop | ~43 mV (vs 180 mV budget) |
Three numbers worth staring at:
- 10 signoff checks, all green. This is the headline — an LLM agent can reach signoff-grade, not just "generate a plausible Verilog."
- 8,352 devices matched at transistor level. Not a structural "looks similar" check — a real LVS.
- ~1 billion tokens across both projects. That's the real cost of the kind of deep, iterative, tool-running work a chip requires.
Where AI Was Genuinely Strong
"AI can write code" is too vague to act on. Here's what actually worked, concretely:
1. Flow orchestration and scripting.
A chip needs synthesis → place-and-route → DRC/LVS → timing → power, chained together. A human might spend days wiring up tools; the agent assembled and scripted most of the flow.
2. Crash and anomaly handling.
I hit repeated WSL memory/disk crashes across both projects. Crashes are exactly the repetitive, enumerable, patience-heavy work AI excels at — it doesn't get frustrated, it keeps debugging and routing around.
3. Multi-scale decomposition.
A chip's problems are chaotic when stacked. AI is good at the mathematician's trick — coarse first, fine later. Classify the problem (synthesis? routing? signoff?), then drill into each sub-problem. Key decisions — splitting inout ports before flattening, a second antenna-repair pass after routing — came from this.
Where AI's Ability Actually Ends
Here's the uncomfortable truth I had to face, and I'll name it precisely:
AI's biggest danger is not that it can't do the work. It's that it stops early and tells you it's done — with full confidence.
I call this failure mode premature completion. And it's not hypothetical — here's the concrete instance:
The RTL functional simulation reported ALL TESTS PASSED. The agent (and honestly, I) briefly believed we were done. Then gate-level simulation deadlocked — 9 errors, output 0xxX000000.
Why did RTL pass but gate-level fail? Because 96.9% of the flip-flops had no reset branch (1,575 without reset vs. 51 with). At power-up, their state is random — and RTL simulators assume registers start at zero. A real chip doesn't.
RTL passing ≠ a working chip. And AI, left alone, happily accepts local evidence (RTL passed) as a global conclusion (chip works).
I found this isn't one failure mode, but three, all sharing that root cause:
| Failure mode | What it looks like | Concrete instance |
|---|---|---|
| FM1 — Premature completion | Declares "done" on partial evidence | RTL "passed" vs. GLS deadlock (96.9% FFs undefined) |
| FM2 — Verification degradation | Quietly swaps a hard check for an easy one | Failed 3rd-level SDF sim → substituted "STA equivalence" |
| FM3 — Toolchain-assumption error | Makes unverified tool assumptions | Reversed pin order, UTF-8→GBK corruption, wrong black-box logic model |
The shared root cause worth internalizing: AI tends to accept local evidence as a global conclusion.
The Method That Kept It Honest
Knowing AI has this weakness is useless unless you build a mechanism to catch it. Four rules worked:
1. Dual-agent, independent review
Never let AI be both player and referee. Executor produces evidence; planner independently re-audits raw logs and reports. Conclusions must come from hard files, not the agent's word.
2. Three-layer verification
Every "PASS" survives three checks:
- Layer 1 — read the "final result" line.
- Layer 2 — cross-check the numbers (device counts match? "no matching net" leftovers?).
- Layer 3 — sanity-check plausibility (silent black-box fallback? a "shoehorned" pass?).
This caught a near-miss "fake LVS" — a structural comparison dressed up as a transistor-level match.
3. A claim-vs-evidence table
When AI says "done," write down each claim and verify it. I found at least 3 discrepancies this way (9 antennas claimed vs 11 actual, timing not on disk, a failed sim quietly swapped for STA).
4. Honest scope control
The hardest one: knowing when to stop. Facing the "power-on deadlock" bug, I had two choices — hide behind "physical signoff is 100% clean" and ship a faulty design, or fix the RTL, re-run the flow, and record the bug honestly. I chose the latter.
If I'd used "signoff passed" to bury a real 96.9%-undefined-at-power-up bug, I'd be no better than the AI prematurely declaring victory.
Why This Matters Beyond Chips
You probably don't design silicon. But these four tricks transfer to any complex AI-assisted engineering:
- Coding: AI says "done" → ask "how deep did you test? edge cases?"
- Analysis: AI says "conclusion X" → ask "what's the source? counterexamples?"
- Decisions: AI says "this is best" → make it produce a claim-vs-evidence table.
One line to remember: AI is good at doing; humans are responsible for verifying. Make verification a mechanism, not an afterthought.
Now, One Small Favor 🙏
I'm submitting this work to arXiv (cs.AR — Hardware Architecture). Abstract and formatting are approved, but I'm stuck on the final gate: arXiv requires a first-time submitter to be endorsed by an established author in the CS archive.
The requirement is simple — the endorser needs ≥3 papers published in any cs.* category on arXiv (between 3 months and 5 years ago).
The process takes about a minute: I forward you an endorsement email (with a code), you log into arXiv and click "I would like to endorse." Done.
If you — or a professor/colleague you know — meet the requirement and are willing to endorse me, I'd be deeply grateful. Reach out to me here or via the repos, and I'll forward the endorsement email right away.
And if you work in open-source hardware, EDA, or AI + chip design, I'd love to connect regardless.
References & Repositories
Projects:
- SHA-256 core: https://github.com/CDragon123-code/SHA-256
- Caravel SoC integration: https://github.com/CDragon123-code/caravel-sha256-accelerator
Open-source EDA tools:
- OpenROAD · Yosys · Magic · Netgen · Icarus Verilog · efabless Caravel
Papers that shaped this work:
- Toward an Open-Source Digital Flow (OpenROAD, DAC 2019)
- ChipNeMo: Domain-Adapted LLMs for Chip Design (arXiv:2311.00176)
- RTLCoder: Outperforming GPT-3.5 in Design RTL Generation (arXiv:2312.08617)
- LLM4EDA: Emerging Progress in LLMs for EDA (arXiv:2401.12224)
- Can AI Agents Really Complete RTL-to-GDS? (arXiv:2607.17528)
All signoff data, crash records, and bug analyses come from the real records in the two open-source repositories above.
