I once collected about 22,000 comments from roughly 140 Korean YouTube videos about AI coding tools and classified them. (Quotes below are translated from Korean.)
I wanted to see what people were asking. What came out was something else.
What the videos teach
Put the titles and tags of those 140 videos in one pile and they say:
How to install. How to get started. How to build an app. Which tool is best.
All of it is "starting." Follow along, a result appears on the screen, the video ends.
What the comments say
The comments sweeping up the likes were telling a different story.
"Verifying AI mistakes takes so much time. Checking every answer for nonsense got so tiring I just do the work myself now." (👍598)
"Coding with AI makes me anxious. If one bug ships, I'm the one responsible. Checking and debugging everything one by one ends up being more work." (👍265)
"I pay every month and it lies about work matters like it's nothing." (👍72)
"Tokens burn too fast… added $50 and it was gone in half a day." (👍30)
It compresses into three complaints: expensive, can't trust it, can't fix it.
The videos teach the start. The people are dying right after the start.
The scariest comment
"Asked it for shampoo recommendations and it recommended one that doesn't exist. Slipped it in between real products — with the weight, the benefits, even a price." (👍49)
That comment is the essence of the problem. When AI is wrong, it doesn't look wrong. The fake sits among the real ones, wearing plausible numbers.
This is why "just write better prompts" is half an answer. Better prompts lower the odds of being wrong. They don't create a way to know when it's wrong. Drop the error rate from 10% to 3% and you still don't know where the 3% is hiding. If that 3% detonates inside payment logic, money leaves the building.
One more finding — where the real questions live
While collecting, I noticed the nature of comments changes with channel size.
multi-million-sub videos real questions/needs = 12% of comments — the rest is reactions and anxiety
10k–300k sub channels real needs = 26% — "I followed along and got stuck RIGHT HERE"
It's the difference between spectators and people actually doing the thing. Comments under the big videos ask "what happens to us in the AI era." Comments under mid-size channels ask "how do I fix this error."
The real questions were in the mid-size channels. And collecting reply threads, not just top-level comments, surfaced 470 more needs that the top level never showed.
So I picked a direction
Content that teaches starting is already everywhere. I decided to work on the stretch right after the start — the can't-trust-it, can't-fix-it stretch.
The earlier posts in this series are the first result of that decision: giving my LLM an exam, planting traps in it, grading by severity instead of pass/fail, why passing still wasn't enough to ship. Every one of them is about turning "I can't trust it" into "I verified it."
People don't die at the start. They die right after it.
But almost everyone is selling the start.
P.S. The classifier that produced the percentages above later sat an exam of its own — the same kind I give my LLMs — and failed three fatal-grade questions. That public correction is its own story, coming later in this series.
Next up: how to steal this exam and port it to your own pipeline, step by step.
