Spec-Driven Development Isn’t Waterfall — But It Keeps Ending Up There

Spec-driven development isn’t supposed to be waterfall. But without clear workflows and better tooling, it’s easy to end up there.

I recently went deep on spec-driven development. The idea was straightforward: before writing any code, define everything. The full vision, the context, the trade-offs, where the feature fits in the existing architecture, the references. Hand it all to the AI agent with crystal-clear guidance so it can build with minimal supervision.

It looked promising. For about two days.

What I ended up with was thousands of lines of specification documents. Documents that were incredibly hard to review — not because they were poorly structured, but because most of them were generated by the AI itself, correlating the existing architecture, docs, and my guidance into something that looked authoritative. Clear explanations. Perfect formatting. Confident reasoning about every decision.

And that’s exactly where it got scary.

The Confidence Problem Nobody Mentions

Here’s the thing about AI-generated specs that the SDD evangelists aren’t talking about: the AI makes everything look correct.

When a spec is well-formatted, internally consistent, and confidently explained, your brain wants to trust it. It reads like something a senior architect wrote after careful deliberation. But it’s not. It’s a language model pattern-matching against your inputs, producing the most plausible-sounding output it can.

And here’s the trap within the trap: if you push back on something that feels off, the AI doesn’t defend its reasoning. It folds. It assumes you’re right, tries to course correct, and often makes the output worse. Hallucination risk actually goes up when you question it, because now it’s reconciling your objection with a position it never truly held. It’s not reasoning. It’s pleasing.

So you’re stuck. Trust the spec and risk building on wrong assumptions. Question the spec and risk destabilizing it further. Either way, you’ve generated thousands of lines of documentation that are incredibly hard to confidently validate.

After spending far too long trying to get those specs to a place I trusted, it hit me — this felt familiar.

The Intent vs. the Reality

Here’s where I want to be fair: spec-driven development isn’t supposed to be big design upfront.

Marc Brooker, the person building Kiro at AWS, explicitly says “you don’t need to, and probably shouldn’t, develop the entire specification upfront.” Kiro’s own workflow is feature-scoped — requirements, design, and tasks for a single story, not a whole system. GitHub’s Spec-Kit runs in a loop: specify, plan, tasks, repeat per change request. OpenSpec literally states “specs are not frozen contracts; update them when reality changes.”

The vision of SDD is iterative. Living documents. Feature-level scope. Incremental refinement.

But the vision and the practice aren’t the same thing yet.

Birgitta Böckeler, writing on Martin Fowler’s site, tried to untangle what SDD actually means right now and found the definition “still in flux.” She identified three levels — spec-first, spec-anchored, and spec-as-source — and noted that most tools are only spec-first. They help you write a spec before coding, but don’t have clear strategies for maintaining or evolving that spec over time. Even GitHub Spec-Kit’s own community is confused about whether a spec is supposed to live beyond a single change request.

The methodology is ahead of the tooling. SDD can absolutely work — but right now, the tools and workflows don’t do enough to keep you on the iterative path. Without clear guardrails for when to stop specifying and start building, teams default to the thing that feels most natural: writing everything down upfront, as thoroughly as possible, before anyone touches code.

That’s what happened to me. Not because I didn’t know better. Because nothing in the workflow guided me toward “that’s enough, go build and come back.”

The Waterfall Gravity

There’s a reason teams keep falling into this pattern. It has gravitational pull.

When you tell an AI agent to help you write a spec, it wants to be comprehensive. It will map out every component, every edge case, every integration point — because that’s what “thorough” looks like in its training data. And as a developer, you want to feel like you’ve thought of everything before handing off to an autonomous agent. The combination of an AI that defaults to exhaustive and a human who defaults to cautious creates thousands of lines of documentation almost by accident.

This is the same dynamic that made waterfall feel so appealing in the first place. The Agile Manifesto exists because planning everything upfront didn’t work:

Responding to change over following a plan.

SDD proponents will rightly say “that’s not what we’re advocating.” And they’re right. But the tooling needs to actively enforce iterative scope — otherwise that gravity keeps pulling teams toward specifying everything before building anything. The intent is agile. The default behavior, without better rails, is waterfall.

Of course, agile itself got convoluted over the years. Certifications, rituals, dogma that drifted far from the original insight. But the core idea never stopped being true: you learn more from building than from planning.

Thoughtworks — the company whose chief scientist co-authored the Agile Manifesto — just released Technology Radar v34 this week, warning that as AI accelerates code generation, “established practices that ensure discipline become more vital.” They’re not pushing SDD. They’re pushing fundamentals. Iteration. Feedback loops.

What SDD Gets Right

I don’t want to dismiss the methodology. There are real benefits when it works.

Agents need constraints. Without scope boundaries, they expand. Tell an agent to build auth and it’ll add OAuth, SSO, and MFA because that’s what “auth” means in its training data. A spec that says “OAuth is out of scope” genuinely saves time.

Context improves quality. An agent with the full picture makes fewer locally-right-but-globally-wrong decisions. The spec gives it a map, not just turn-by-turn directions.

Team alignment. Multiple people or agents working on the same system need a shared reference point. Specs provide that.

The problem was never “should we spec?” It’s “how do we make sure we spec iteratively instead of falling into the trap of specifying everything at once?”

Thinking in Graphs

Here’s where I’ve landed — and maybe this is just how my brain works, but I think it applies broadly.

I think about projects as graphs, not documents.

At the top, there’s a direction. A north star. What we’re building and why. High-level, intentional, human-defined. It doesn’t need to specify every data model or API contract. It needs to be clear about the destination.

From that direction, you break down into milestones. Each milestone is a meaningful checkpoint — something you can ship, test, or validate. Not a document section. A real deliverable.

Each milestone has its own tasks. And here’s the critical part: the depth of planning for each task happens when you start working on it, not months before. You plan the first milestone in detail. You sketch milestone three at a high level. When you finish milestone one, you know things you didn’t know before — and that knowledge shapes how you plan milestone two.

The direction flows down from the north star. Discovery happens at every node.

Say you’re building a new integration. The north star says: “Users can sync data between System A and System B in real time.” Milestone one might be a basic one-way sync — and when you build it, you discover the API rate limits aren’t what the docs claimed. That changes everything about milestone two. If you’d fully specced bidirectional sync upfront, you’d be rewriting specs instead of shipping software.

This works for humans because it provides structure without drowning you in premature detail. It works for AI agents for the same reason — they need guidance and constraints, but they also need room to discover things during implementation that no spec could have predicted.

Full upfront specification tries to flatten the graph into a document. Every node defined, every edge mapped, before you’ve traversed any of them. That’s not engineering. That’s prophecy.

What This Looks Like in Practice

I’m still figuring this out. There is no perfect workflow yet — that’s kind of the point. But here’s the pattern that’s working:

Thin specs, not thick ones. One page for the current milestone, not twenty pages for the whole system. Define the outcome, the constraints, what’s out of scope. Leave room for what you don’t know yet.

Iterate the spec, not just the code. The spec changes every cycle. Decisions made, assumptions validated or invalidated, things learned by building. A living document, not a contract. This is what SDD’s proponents advocate — we just need clearer workflows and tooling to make this the default path instead of something you have to consciously enforce.

Use agents for exploration, not just execution. Code is cheap now. Build a quick prototype to test an architectural assumption. Throw it away if it’s wrong. A throwaway prototype costs you nothing. Specifying the wrong architecture upfront costs you everything.

Keep the loop tight. In traditional agile, the sprint was two weeks. With agents, the feedback loop can be hours. Specify, build, test, learn, adjust. But only if you keep the scope small enough to actually iterate.

The Gap That Needs Filling

The industry is living through a real-time methodology shift:

1. Vibe coding — prompt and pray. Fast, chaotic, doesn’t scale.
2. Spec-driven development — specify, then execute. Sound in theory, but easy to fall into big design upfront without clear process guardrails.
3. What comes next — the iterative spec workflow that SDD envisions, supported by tooling and processes that actively keep teams on that path. Thin specs. Fast execution. Continuous refinement. Direction without prophecy.

This is the same arc software development has followed before. Waterfall promised control through upfront planning. Agile recognized that the plan always changes. We’re at that inflection point again — the insight of iterative development needs to be baked into the tools and workflows, not just the blog posts and documentation.

A project with a clear direction, meaningful milestones, and task-level depth that’s earned at execution time — not guessed at months before — works for both humans and AI agents. Structure without rigidity. Guidance without false certainty.

The agents are fast. The models are capable. But the bottleneck was never the code.

It was knowing what to build. And you only learn that by building.

Spec-driven development’s vision is right: specs should be living, iterative, and at the center of how we build software with AI. What we need now is better tooling and clearer processes to make that vision the default — so teams stay on the iterative path instead of drifting into the upfront-planning trap that agile was invented to escape. The Agile Manifesto didn’t expire. It just got a new executor — and that executor needs better guardrails.

The post Spec-Driven Development Isn’t Waterfall — But It Keeps Ending Up There appeared first on sudoish.