A common mistake in discussions about AI in education is assuming that better homework scores mean better learning. In practice, the two can diverge very quickly.
A large new study of nearly 27,000 students in China, ages 12 to 18, suggests that this gap is already real. About 80 percent of the students reported using AI tools such as DeepSeek and ByteDance’s Doubao, while the remaining 20 percent served as a non-AI comparison group. After six months, the students using AI showed a clear short-term advantage on homework: their scores rose by 18 percent across subjects, and the time needed to finish assignments fell from 64 minutes to 45 minutes on average.
That sounds like a productivity win. But the same study found a much more troubling pattern on monthly exams. The AI users scored 20 percent lower than students who did not use AI. In other words, the group that looked strongest on homework became the group that performed worst when the work had to be done without assistance.
For builders, educators, and product teams, that is the key signal to pay attention to: AI can optimize the completion of tasks without improving the underlying skill. If your workflow only measures output speed or polished submissions, you may be training for dependency instead of competence.
Why this matters for software-minded people
Most developers already understand the difference between a fast demo and a reliable system. A script can produce the right answer once. That does not mean the person who ran it understands the logic, edge cases, or failure modes.
The student data points to the same problem in a learning context:
- Homework becomes easier and faster
- The visible output can even improve
- Retention and independent problem-solving may degrade
- Performance drops when AI is removed from the loop
That last point is especially important. Traditional schooling often assumes that strong homework performance predicts exam performance. This study suggests that assumption may no longer hold when AI is inserted into the middle of the workflow.
The finding was not isolated either. Other research has raised similar concerns. A Brown University professor reported that students did unusually well on a take-home midterm, then performed far worse once the final exam moved in person. The average final score collapsed to 48 percent, after previously staying above 65 percent. Many of the students who had earned perfect scores on the take-home midterm did not even attempt the final.
MIT research also found that students who used AI to help write essays showed lower brain activity than students who wrote without it. Those students also struggled to quote their own writing later, and when they were asked to write again without AI, the reduced brain activity persisted. Additional studies have linked AI use to weaker critical thinking and memory-related issues.
The practical takeaway: do not measure only output
The lesson for anyone building with AI is not "stop using AI." It is to stop treating AI-assisted output as proof of learning, understanding, or long-term capability.
If you are designing a study workflow, classroom process, or internal training system, you need to separate:
- speed of completion
- quality of final output
- ability to reproduce the result without AI
- retention over time
The new student study suggests that these are not the same thing.
How to reduce the dependency problem
You do not need to ban AI to prevent skill atrophy. A better approach is to introduce friction and verification at the right points in the workflow.
1. Make part of the task AI-free
If every assignment can be completed entirely through a model, students will naturally route all effort through it. A more balanced workflow is to require an unaided phase first.
For example:
- brainstorm without AI
- draft key steps or explanations manually
- use AI only after an initial attempt
- submit a short reflection on what the AI changed
This keeps AI as a tool for refinement instead of a replacement for thinking.
2. Add retrieval checks
The study’s exam results matter because they test recall and independent reasoning. If you want to know whether learning is happening, include moments where the AI is removed.
That can be as simple as:
- closed-book quizzes
- oral explanations
- short handwritten summaries
- timed problem-solving without external tools
The goal is not punishment. The goal is to check whether the student can still produce the answer when the assistant is gone.
3. Compare assisted and unassisted performance
If AI is allowed, measure both versions of the task.
A useful pattern is:
- first attempt without AI
- second attempt with AI
- later repeat without AI
That gives you a better signal than homework scores alone. If the AI version improves sharply but the unassisted version stagnates or declines, the tool is helping performance but not capability.
4. Use AI for feedback, not only completion
The strongest use case is often not "write this for me" but "review this with me."
For education, that means AI can help with:
- identifying gaps in reasoning
- suggesting alternative explanations
- generating practice questions
- pointing out missing steps
That preserves the student’s active role. It also makes the model a coach instead of a substitute.
5. Watch for the time-savings trap
The study found that AI users finished homework about 30 percent faster. That is not automatically a good outcome.
In real systems, faster completion can hide weak understanding until a high-stakes moment arrives. If you are a teacher, manager, or product designer, ask whether the speedup is actually reducing learning time or simply removing the struggle that builds memory and skill.
The broader design question
This research fits a pattern that many developers have already seen in tooling: automation can improve throughput while quietly lowering the operator’s awareness of what is happening under the hood.
That is fine for low-stakes repetitive work. It is dangerous in domains where the whole point is to build judgment.
Education is one of those domains. So are onboarding, certification, and any workflow where future performance depends on present understanding.
The actionable response is not to reject AI. It is to design systems that still require humans to think, retrieve, explain, and verify. If you do that, AI can speed up the work without hollowing out the skill behind it.
If you do not, you may get cleaner homework, happier short-term metrics, and much weaker performance when it matters.
