With AI models and coding tools getting increasingly powerful, it feels developers are stuck in two contradictory narratives:
Developers are finished.
And
AI will not replace developers, but a developer who understands how to use AI will replace one who doesn't.
Between the bold predictions and endless "X technology is dead" Youtube videos, I wanted to see what AI-assisted development looked beyond a greenfield demo.
What would it feel like with a mature codebase with existing behaviour, some questionable design decisions, and most importantly a developer who had to live with the consequences. So I decided to experiment on myself...
Part 1 - Setting Up the experiment
Why a Brownfield Project?
When I explored AI-assisted development, I found TONS of information online. But most of tutorials were focused on greenfield projects.
A greenfield project starts from scratch, giving you the freedom to establish its architecture and structure.
That is useful, but it wasn't what I wanted. The hard part of software development isn't always creating something new. Sometimes it is opening a project two years later and asking:
"Why on earth did I write it this way?"
That was the experiment I wanted. So, I chose one of my first university programming assignments, written when I was still learning object-oriented programming. Having started with C, I was more accustomed to procedural programming, and that experience influenced how I approached the assignment. It is pretty evident from how I originally wrote this assignment:
The application itself was fairly simple. It was a console-based electronic store application where a user could store and query books and electronics. When the program ended, the catalog was written to a file.
Simple enough.
Except the application had validation rules, product-specific behaviour, search functionality, and persistence to account for. And all of it lived in a single 800+ line God class that drove the entire application. The code worked, but its design left plenty of room for improvement. That made it a good candidate for testing whether AI could help refactor an existing codebase without breaking its behaviour.
What was I Actually Trying to Learn?
I went into the experiment with a few questions:
- Does AI-assisted development deliver a meaningful return on investment, beyond simply generating code faster?
- How much of my development workflow should AI actually own?
- Would AI become the central piece of my programming workflow or would it remain what the name suggests: a Co-pilot (Ba dum tss!)
- And, of course: will it replace me!?
For this experiment, I used free tier of GitHub Copilot in auto mode. Mostly because I wanted to measure its usefulness before paying for it. There was another constraint that became increasingly important later.
Tokens
But we'll get to that.
Before I Let AI Loose on the Codebase
Luckily, I had created a README when I submitted the assignment. It documented the application's intended behaviour and gave me something valuable: context. Back then, the README was something I wrote to avoid losing marks. Little did I know I was doing my future self a favour.
Before refactoring anything, I needed a baseline to check whether the application still behaved as expected. I started by looking for seams: places where I could isolate or control parts of the application's behaviour without rewriting the surrounding code.
I learned this concept from Michael Feathers' Working Effectively with Legacy Code. Seams gave me opportunities to isolate behaviour and introduce tests before making larger changes.
Establishing a Behavioural Baseline with Gherkin
I chose Gherkin and Cucumber for my behavioural tests. I'd worked with Gherkin before, although not in Java, so the concepts were familiar even if the setup wasn't. Gherkin lets you describe application behaviour using a simple Given-When-Then structure. Cucumber connects those scenarios to executable tests.
For example, a product search scenario might look like this:
Scenario: Operator runs the program, adds a book, then searches by name
Given a new catalog file
When the operator starts E-Search
When the operator adds products with:
| type | productID | year | name | price | authors | publisher | manufacturer |
| book | 123456 | 2023 | Learning Java | 45.99 | Jane Doe | Education Press | |
And the operator searches for products named "Learning Java"
Then the search result should contain 1 products
The goal was to treat the application as a black box: provide input, observe the output and verify that the result matched the expected behaviour.
Setting up the tests was another task I could have delegated to Copilot. Instead, I watched a tutorial at 2x speed and set up the project myself.
Why?
These tests would become my baseline for evaluating the refactor. I could have asked Copilot to generate them, too, but I wanted to define the expected behaviour independently of the AI doing the refactoring. Otherwise, I risked testing the AI's interpretation of the requirements rather than the application's intended behaviour.
Static Analysis with SonarQube
Behavioural tests weren't enough. They could help me detect changes in tested behaviour, but they wouldn't tell me everything about the resulting code's maintainability or static quality. I also used SonarQube to assess the code. This gave me a second perspective on the refactoring: not just whether the application still worked, but whether the code itself had improved. This was one area where I let AI assist me, and the results were impressive.
Part 2 - The Experiment
Day 1: I had to figure out how to tell AI what I Actually Wanted
This was where things became interesting. I knew what I wanted to improve but I didn't necessarily know the best implementation. That distinction turned out to be extremely important. I started researching how people were structuring AI-assisted development and eventually landed on a spec-driven approach (spec is short for specification).
Instead of telling AI: "Refactor this class.", I would define the desired outcome, the constraints, the behaviour that must remain intact, and the boundaries of the change. Then I would let AI figure out the implementation.
I created a .spec folder at the root of the project and a refactoring section inside it. Each refactoring had a number prefix corresponding to the GitHub issue or piece of work. For example:
./spec/refactoring/1-extract-file-reading.md
Each specification followed a lightweight structure:
Goal
...
Current Implementation
...
Desired Implementation
...
Behaviour to Preserve
...
Acceptance Criteria
...
I kept the format flexible. Not every refactoring required every section and you'll notice that in my rafactoring files as well. Heck, even the file naming was loose, I kept the prefix consistent but the phrase that followed was for humans to quickly understand its purpose.
Template wasn't the core piece. The important part was that I was starting to separate what I wanted from how I wanted it implemented. That was a subtle shift. And initially, I didn't fully understand it.
My first attempts were messy. I was still thinking like a developer who was writing the code myself. I would describe something I wanted changed and then mentally jump ahead to the implementation. But if the point was to have AI perform the implementation, I had to stop doing that. Instead, I needed to become much better at describing the problem. I know this sounds obvious but it wasn't.
The more I worked this way, the more I realized that it was an extension of what I was already doing:
Spending a good amount of time just thinking and defining the problem before implementing it.
If I can't clearly describe what I want changed, how am I expecting an AI model to correctly implement it? This became one of the first major lessons from the experiment.
AI doesn't remove the need for clear and critical thinking. It increases the value of it.
Day 2: I Had to Stop Designing Every Detail
This was one of the strangest parts of the experiment. I had to unlearn the reflex to design the implementation before handing the work to AI. Normally, I'd see a problem and immediately start thinking:
"I'll create this class, move that responsibility there, inject this dependency, change this method..."
Even if Copilot was going to write the code, I was still making all the design decisions. So I started describing refactorings in terms of their intended outcome, boundaries, and behaviour that needed to remain intact. I left the implementation open where I could.
I didn't bother with the traditional "As a user, I want..." format. A concise description and a refactoring specification were enough for the kind of work I was doing. The difficult part was figuring out which details were genuine constraints and which were simply my first idea for a solution.
If I specified every class and method, I was effectively asking Copilot to implement a design I'd already made. If I described the desired outcome and relevant boundaries instead, it had room to propose a different approach. That didn't mean blindly accepting whatever it generated. I still had to evaluate whether its approach made sense.
The challenge wasn't just telling AI what to do. It was knowing what I needed to decide myself, and what I could leave open for AI to solve.
Day 3 - The Workflow Started Taking Shape
As I continued, a repeatable workflow emerged:
This was very different from simply asking AI to write code. The specification became the thing I iterated on. If the AI produced code I didn't like, my first instinct was no longer:
"Let me fix this code."
Instead, it became:
"Did I give it enough information to make the decision I wanted?"
Sometimes the answer was yes: the AI simply made a poor implementation choice. Sometimes the answer was no and the specification was missing an important constraint. That distinction was incredibly useful.
Day 4: AI Was Smart, but It Was Also Gullible
By this point, I'd started thinking about AI differently. Copilot could understand the existing architecture, make changes across multiple files, and implement fairly substantial refactorings. But capability wasn't the same as reliability.
I started thinking of it as a very smart but gullible developer. It could move quickly, generate a lot of code, and propose approaches I wouldn't have considered. But it could also confidently head in the wrong direction if it misunderstood what I wanted.
My role didn't disappear. The work shifted.
Instead of writing every part of the implementation myself, I spent more time understanding the architecture, evaluating design decisions, reviewing changes, and checking that existing behaviour was preserved. I also had to revisit my specifications when they left too much room for misinterpretation. In some ways, I became more involved in the system, not less. I was delegating implementation, not engineering judgment.
Day 5: The Process Finally Clicked
By around Day 5, the process had started to feel natural. I was getting more comfortable breaking refactoring work into smaller changes and giving Copilot enough context to implement them without prescribing every detail.
Something interesting was happening: the codebase was getting smaller, while the documentation was growing.
The giant EStoreSearch class, which had originally contained a lot of responsibilities, was gradually being broken apart. Meanwhile, the specifications documenting each change and the reasoning behind it were piling up.
I didn't want that documentation to become another thing I had to maintain manually. With help from AI, I worked out how to organize the project's specs and documentation into a Docusaurus site. I then set up GitHub Actions to build the site and publish it to Cloudflare Pages whenever changes were made.
The code was becoming simpler, and I was building a clearer record of how and why it was changing.
One of the biggest benefits wasn't just having AI generate code faster. It was having to make my understanding of the system explicit—and leaving behind documentation that I could use long after the experiment.
What didn't go according to plan
The Token Limit Problem
Remember that I was using the free tier of GitHub Copilot. That meant I had to be selective about what I delegated.
In theory, I could ask AI to analyze the entire codebase, explain the architecture, implement a change, review its own work, and explain every decision. In practice, that wasn't always worth the usage or the time. Sometimes, fixing a small mistake myself took ten seconds. Explaining it to Copilot, waiting for another response, and reviewing the change could take several minutes.
AI doesn't need to be involved in every step of development.
I started weighing the cost of delegating a task against the cost of doing it myself. That wasn't a failure of AI-assisted development. It was part of learning where the tool actually provided value.
AI is a tool. It doesn't get bonus points for being involved in everything.
Context Became a Problem
As I became more comfortable with the workflow, I started paying more attention to the context Copilot used. I had old README files, outdated documentation, and remnants of the original architecture sitting alongside specifications describing the intended changes.
Some documents described how the system used to work. Others described how I wanted it to work. Without a clear distinction, they could give the AI conflicting signals.
I began cleaning up outdated documentation and thinking more carefully about which files should be treated as authoritative. A specification describing desired behaviour serves a different purpose from an implementation note, and historical documentation shouldn't automatically be treated as current guidance.
I was learning that documenting a project wasn't just about recording information. It was also about making clear which information to trust, and for what purpose.
Garbage context can produce garbage decisions just as easily as garbage code can.
My Tests Caught What Copilot Missed
Even with specifications and a defined workflow, Copilot didn't always get things right. In one instance, I had explicitly documented the behaviour that needed to be preserved and the boundaries of the refactoring. Yet Copilot still produced a change that failed to account for an expected behaviour. When I questioned it, it acknowledged that it had missed the relevant documentation.
I reran the prompt, and it produced a correct implementation on the next attempt. But the more important part of this experience wasn't that Copilot made a mistake. It was that my independently written end-to-end tests caught the regression before I merged the change.
I didn't have to rely on Copilot to recognize its own mistake. I had an independent way to verify the behaviour That was the value of establishing a behavioural baseline before starting the refactoring: I could evaluate the result against expected behaviour instead of simply trusting that the generated code looked reasonable.
Keeping the Specs in Sync
There was another trade-off I hadn't fully anticipated: the specifications could become stale, too. As the refactoring progressed, I had to keep the code and specifications aligned. If I changed an implementation detail without changing the intended behaviour, the specification might remain valid. But if I wanted to change what the application actually did, I needed to revisit the specification as well.
That added discipline, but it also introduced friction. I couldn't write a specification once and treat it as permanent documentation. It had to evolve alongside the code.
The process also made me more deliberate about behavioural changes. Rather than quietly changing what the application did while refactoring, I had to recognize that I was changing the expected behaviour and update the specification accordingly.
The specification wasn't just a way to guide the implementation. It became another part of the system I had to maintain.
So, Did AI Replace Me?
Partially, yes.
AI took over parts of the work I used to do myself. It handled boilerplate, navigated the existing codebase, and turned well-defined specifications into working implementations. Tasks that would have required me to write every change by hand could now start with a conversation and a specification.
But I still had to decide what was worth changing, what behavior needed to be preserved, and whether the generated code was actually an improvement.
And the more I used AI, the more I realized something: getting code written was only one part of the job.
The harder part was understanding the system well enough to know what to ask for, giving AI the right constraints, and having independent ways to verify its work. Even my specifications became something I had to maintain. I wasn't simply handing work off; I was taking responsibility for a different set of decisions.
I started this experiment wanting to understand how useful AI could be at refactoring an old codebase. I came away with a better understanding of how I wanted to approach software development itself: define the problem, make the intended behavior explicit, change one thing at a time, and verify the result.
AI did take work off my plate. It also made the quality of my thinking more consequential.
It didn't eliminate the need for an engineer. It changed where I needed to be one.
But with that being said, I would still manually program for recreational purposes just because I enjoy doing it. Sure, it may not be the most efficient use of my time because what I would spend programming 4 hours may be done in a couple of minutes or even seconds. But its the act of actively being involved in writing code, compiling, running it and seeing that it worked gives a great feeling which does get lost when AI takes over some of the steps.
I was most efficient when I was performing spec driven refactoring but had most fun when I wrote the tests myself.
