A few years ago, if someone built a polished full-stack application over a weekend, I would immediately assume they were a highly capable developer.
They had probably worked across the frontend, backend, database, authentication, deployment, testing, debugging, and dozens of smaller problems that only become visible when you build a real system.
Today, that assumption is becoming weaker.
Give a capable coding agent access to a repository and it can inspect the codebase, research unfamiliar libraries, implement features, run tests, debug failures, use the terminal, inspect the interface, and continue iterating.
That is incredible progress. I use these tools constantly, and I have no interest in going back.
But I keep returning to one uncomfortable question:
What if AI is increasing our ability to produce faster than we are increasing our ability to think?
This is not another argument about AI replacing developers.
The more immediate concern is the growing separation between producing a working system and actually possessing a reliable mental model of that system.
Building impressive software is becoming normal
Think about a hackathon.
A few years ago, shipping a polished interface, backend, authentication system, database, AI integration, and deployment in 24 or 48 hours was genuinely difficult.
Now imagine 500 participants with access to similarly capable coding agents.
Almost every team can generate a respectable interface. They can create APIs, add retrieval, integrate agents, write tests, produce documentation, and prepare a polished presentation.
The projects will still look different, but implementation becomes less useful as evidence of exceptional ability.
If one team has five more features than another, does that mean they thought five times harder?
Probably not.
This does not make hackathons pointless. It means the thing worth measuring is changing.
The same applies to portfolios. “Look at this application I built” used to tell us a great deal about the developer. Increasingly, we may need to ask different questions:
What did you discover?
What difficult constraint did you overcome?
What did you measure?
Which assumption turned out to be wrong?
What part of the project exists because of your thinking, rather than because software generation has become extremely capable?
Those questions reveal far more than feature count.
We can now build beyond our own understanding
This is the part that concerns me most.
AI makes it possible for developers to create software that is more sophisticated than their own understanding of it.
Suppose an agent receives this request:
Add Redis caching, OAuth, background workers, retries, tracing, and rate limiting.
It implements everything.
The tests pass. The code looks reasonable. The diff is understandable enough, so it gets shipped.
Three months later, production starts behaving strangely. Retries interact badly with a worker. Cached state becomes stale under one specific condition. A request occasionally executes twice during a network failure.
The agent can suggest several explanations and generate several patches.
But now there is a deeper question:
Does the developer understand the system well enough to determine which explanation is actually correct?
There is a difference between producing a system and possessing a model of that system.
Historically, those things were closely connected. You generally had to understand enough of something to build it.
AI is beginning to separate them.
That separation is useful. It allows small teams to attempt projects that once required far more time, money, and expertise.
It can also create a new kind of debt.
Technical debt is stored in the code.
Epistemic debt is stored in the gap between what the system does and what its maintainers understand.
A codebase can look clean. Every test can pass. The architecture diagram can appear perfect.
Yet the team may still be unable to explain its real failure modes.
That debt becomes visible only when the system leaves the conditions under which it was generated and tested.
“Humans will move higher” is not a complete answer
Whenever this problem appears, we move the human one level upward.
AI writes code, so humans will design the architecture.
AI improves at architecture, so humans will focus on product strategy.
AI improves there too, so humans will perform research.
But architecture, strategy, research, and judgment are all forms of reasoning. There is no serious reason to assume that one particular cognitive layer will remain permanently human.
That does not mean every layer is automated today.
It means “move higher” is not a durable philosophy. AI may continue moving upward with us.
Frontier systems are already moving beyond isolated code generation. They can use tools, inspect environments, run experiments, evaluate results, and continue working for longer periods.
So perhaps the useful question is not:
What skill will AI never automate?
Maybe it is:
How do we continue becoming capable humans while increasingly capable machines perform more of our cognitive work?
Understanding a solution is not the same as constructing one
Students will experience this problem before almost anyone else.
Imagine learning recursion.
You do not understand it, so you ask AI.
The first explanation is too abstract. You ask for an analogy. You ask it to visualize the call stack. You request another example.
Five minutes later, something clicks.
That is fantastic. A patient tutor available at any hour can make education dramatically more accessible.
But there is another version of the same interaction.
You encounter a difficult problem and ask AI to solve it before making a serious attempt yourself.
The answer appears. After reading it, the solution feels obvious.
This can create an illusion of competence. Once a clean explanation is in front of us, following it can feel like mastery.
But recognition is easier than reconstruction.
“I understand this solution” is not always the same as “I could have constructed this solution from a blank page.”
A great deal of learning happens before the answer becomes obvious.
You make a prediction.
It fails.
You discover that one of your assumptions was wrong.
You change your mental model.
You remain confused long enough for the structure of the problem to become your own.
Sometimes AI removes useless friction.
Sometimes it removes the friction that was producing learning.
We need to become much better at distinguishing between the two.
The senior pipeline paradox
There is also a longer-term industry problem hiding here.
People often say junior developers can rely heavily on AI because senior engineers will review the result.
But where do future senior engineers come from?
Today’s experienced engineers did not appear fully formed. Their judgment was built through years of debugging, reading unfamiliar code, making poor architectural decisions, responding to production failures, and slowly developing intuition.
A senior engineer recognizes a dangerous pattern because they have seen a similar pattern fail before.
They can challenge a plausible answer because they have spent years being wrong and correcting themselves.
If the next generation delegates most of that formative work to AI, we cannot simply assume that equally capable senior engineers will appear ten years later.
We may automate the apprenticeship while still expecting the experts that apprenticeship used to produce.
The solution is not forcing junior developers to manually write repetitive CRUD applications forever.
It is deliberately creating new ways for them to develop judgment once routine work no longer provides that training automatically.
This is not only an individual habit problem.
It is an educational and organizational design problem.
Maybe we need cognitive gyms
Technology removed huge amounts of physical effort from everyday life.
Cars are better than walking 30 kilometres to work. Machines are better than carrying enormous loads manually.
That is progress.
But our bodies still require physical stress to remain healthy, so we deliberately recreate it.
We go to gyms.
We run on machines that take us nowhere.
We lift heavy objects and place them exactly where they started.
Economically, that activity produces nothing.
Biologically, it maintains capacity.
AI may create the cognitive equivalent.
For most of history, people were forced to exercise certain mental abilities because there was no alternative.
You had to remember.
You had to search.
You had to debug.
You had to sit with uncertainty.
You had to form an explanation before a better one appeared.
Increasingly, those activities become optional.
That does not mean every task should be made artificially difficult. It means intellectual strength may need to become something we maintain deliberately.
Production mode and training mode
The answer is not using less AI for the sake of purity.
If an agent can safely perform six hours of repetitive implementation in twenty minutes, spending six hours doing it manually does not automatically make someone a better developer.
There is no virtue in typing the same boilerplate for the hundredth time.
Use the tools.
Let AI generate repetitive code, search documentation, create tests, inspect logs, and automate boring transformations.
The important question is not whether we use AI.
It is:
Which parts of the thinking loop should we outsource, and which parts should we deliberately continue exercising?
Developers may increasingly need two explicit modes.
Production mode
The objective is to ship.
Use agents, search, generated code, automation, and parallel execution. Remove unnecessary friction and deliver the strongest result possible.
Training mode
The objective is not the product.
The objective is the person building it.
Debug something yourself before asking.
Read the primary documentation.
Attempt the architecture first.
Implement a small subsystem from first principles.
Explain unfamiliar code without assistance.
Predict what a test will reveal before running it.
Training mode is intentionally less efficient.
That is fine.
Efficiency is not always the objective.
Three guardrails for keeping the thinking loop intact
The goal is not to slow development down. It is to make sure the human mental model continues updating alongside the systems being produced.
1. Use falsification-first prompting
Do not always begin with:
Design this system for me.
Begin by constructing your own model:
Here is how I think cache invalidation should work across distributed workers under high write load. Which race conditions, edge cases, and unstated assumptions could break this design?
The difference matters.
In the first interaction, AI supplies the initial model.
In the second, you supply the hypothesis and AI tries to destroy it.
The model becomes an adversary rather than an oracle.
An adversary can make you better.
2. Add a failure-mode gate before merging
Before approving a significant agent-generated change, explain at least two realistic ways it could fail in production.
What happens during a partial network failure?
What happens if the job is executed twice?
What happens if an external service responds slowly but does not completely fail?
What state can become stale?
Which assumption does the implementation rely on?
If you cannot explain the failure modes without asking the same AI that generated the code, you may be carrying more epistemic debt than you realize.
That does not always mean the change should be rejected.
It means the understanding has not yet caught up with the implementation.
3. Trace something directly
In training mode, occasionally bypass the summary.
Read the source code.
Inspect the call stack.
Follow the request through the logs.
Look at the database transaction.
Read the packet trace.
Step through the runtime behavior.
AI explanations are valuable, but they present the system through the model’s abstraction.
Direct tracing forces your own abstraction to form.
You do not need to do this for every bug. Doing it regularly is enough to preserve contact with the lower layers that make higher-level judgment possible.
Do not let AI be both the first thinker and the final judge
These guardrails point toward one simple principle:
Do not let AI become both the first thinker and the final judge.
If AI generates the first solution, the human should meaningfully evaluate it.
If the human creates the first hypothesis, AI can become the critic.
Prediction is another powerful habit.
Before running AI-generated code, predict what it will do.
Before asking why something crashed, write down the most likely explanation.
Before checking a benchmark, estimate the result.
When reality disagrees with you, you have discovered something about your own mental model.
That is where learning happens.
Our projects need a different standard
Projects may also need a harsher test:
If a future model could recreate this entire repository in 30 minutes, what part of the contribution would remain?
A stronger answer might be:
a dataset nobody had,
a reproducible benchmark,
a discovered vulnerability,
or a protocol supported by real evidence.
These are forms of contact with reality.
They are harder to reduce to “the model generated a good repository.”
As implementation becomes cheaper, the strongest projects may become less about adding features and more about producing evidence.
What was observed?
What was tested?
What failed?
What changed because the project existed?
Those questions will matter more than repository size.
Hackathons should measure judgment, not output volume
The same principle could improve hackathons.
AI does not need to be banned. Let every team use it.
But judges could ask different questions:
Where did this problem come from?
How do you know it actually exists?
What did you discover while building?
Which assumption failed?
What part of the work is genuinely new?
Can someone reproduce your result?
Why did you choose this architecture?
Now change one important requirement live. Can you adapt?
Those questions tell us far more about a team than counting integrations or measuring the polish of a generated dashboard.
When implementation is abundant, judgment becomes the signal.
The real danger is not that AI can think
That is the achievement.
The danger is that we quietly decide there is no longer any reason for us to understand what the machine can already do.
There is an enormous difference between:
“AI can do this better than me.”
and
“Therefore I no longer need to understand it.”
The first statement will probably become true for more and more tasks.
The second does not have to follow.
Maybe the important capability of the AI era is not prompting.
Maybe it is something closer to epistemic agency.
Having your own model of a problem.
Your own hypotheses.
Your own curiosity.
The ability to notice when an answer does not make sense.
Then using extremely capable machines to push your thinking further than you could alone.
I do not want a future where developers reject AI simply to prove that they are still capable.
I also do not want one where we become extraordinarily productive while slowly losing the ability to reason without assistance.
The interesting future sits between those extremes.
Use AI aggressively.
Automate what deserves to be automated.
But sometimes think before asking.
Predict before checking.
Attempt before generating.
Understand before approving.
Because the dangerous sentence is not:
“The machine can do this better than me.”
It is:
“Since the machine can do it, there is no reason for me to understand it.”
How are you using AI without letting it replace the parts of programming that made you a better thinker?
