My AI told me to pip install a package that doesn't exist. Turns out someone already weaponized that.

Last week I was working on a FastAPI project and Claude recommended a package called huggingface-cli. Didn't think twice, just pip installed it. Import failed. Nothing worked.

Spent way too long debugging before I actually went and checked PyPI. The package exists, but it's an empty shell. Some security researcher noticed AI keeps recommending this name, so he registered it first as an experiment. Three months. Thirty thousand downloads. Thirty thousand people did exactly what I did.

The scary part is he was running an experiment so the package was empty. What if it wasn't?

After that I got kind of paranoid and went through our entire requirements.txt checking every dependency one by one. Didn't find other fake ones, but the whole process pissed me off. How am I supposed to know next time? Am I going to manually search PyPI every time I add a dependency to make sure it's real? That's insane.

And while I was at it I noticed something else. A couple places where AI called methods that flat out don't exist on the library. prisma.client.softDelete() — Prisma doesn't have softDelete. But the way it wrote it looked completely natural. I missed it in review. Who knows how long it's been sitting there.

And don't even get me started on the tests. Found one that mocked a return value and then asserted the result equaled the mock. What did that test? Nothing. It tested that jest works. Thanks.

const mock = { id: '1', name: 'John' };
jest.spyOn(repo, 'findById').mockResolvedValue(mock);
const result = await service.getUser('1');
expect(result).toEqual(mock); // yeah great job

Coverage looked fine. The test was useless.

I'm having a bit of a trust crisis with AI-generated code right now. I saw a post on r/ClaudeAI the other day saying "Claude isn't dumber, it's just not trying," and honestly that hit a little too close to home. Like how much of what it writes can I actually trust? I searched around for tools that check for this kind of thing — fake packages, fake methods, useless tests — and couldn't really find anything designed for it. Linters don't check if a package exists. Code review can't keep up with the volume.

Feels like something that should be automated but nobody's done it yet. Or maybe someone has and I just can't find it?

Anyone else dealing with this? How do you handle it? If there's a tool I'm missing please tell me before I lose another afternoon to a package that doesn't exist.