Practical Guide to AI Attacks on Real People, Internet, and Deep Web
Introduction
Artificial intelligence has democratized access to attack techniques that once required advanced skills. Today, a "zero‑knowledge threat actor" — an attacker with minimal knowledge — can assemble credible offensive capabilities simply by using AI tools . This guide explores the modus operandi of these attacks and, above all, the difficulties in recognizing them.
Part 1: AI Attacks in Real Life
1.1 Vishing (Voice Phishing) with Synthetic Voices
How it works: Attackers use voice synthesis tools to clone the voice of a supervisor, family member, or colleague. The calls create urgency situations (e.g., "I'm stuck, transfer €5,000 immediately") to induce the victim to act impulsively.
Why it's hard to recognize: A controlled study found that people correctly identify an AI voice only 37.5% of the time — a result worse than chance. Discriminability is near zero, meaning there is no reliable "signal" to distinguish a real voice from a synthetic one .
Study participants relied on heuristics such as:
Hesitations and pauses → now replicated by AI
Tonal variations and emotion → now perfectly simulated
Filler words ("um", "uh") → now added intentionally
The result is that people are overconfident but utterly unreliable: most misclassifications are accompanied by high perceived certainty .
1.2 Deepfake Video and Audio for Corporate Fraud
Real‑world example: In February 2025, a British engineer at Arup was defrauded of $25 million. Attackers used deepfake video and audio to impersonate the company's CFO during a video conference, convincing the employee to transfer funds .
What makes these attacks credible:
Generative AI now produces videos with perfect lip‑sync
Technology replicates not only voice but also facial expressions
Psychological pressure (urgency, authority) overrides critical thinking
Part 2: AI Attacks on the Internet
2.1 Hyper‑Personalized Phishing at Industrial Scale
The phenomenon: Phishing‑as‑a‑Service (PhaaS) platforms like EvilTokens and Forg365 have turned attacks into a "business" with subscription models .
Platform Price AI Features
Forg365 $400/month or $3,800/year Automated AI flattery generation, DocuSign/SharePoint templates, stolen credential management
EvilTokens N/A AI‑generated unique lures for each victim — no two are identical
The escalation numbers:
Phishing attempts increased by 4,151% since ChatGPT's release
EvilTokens platform saw a 1,380% increase between 2025 and 2026
In Q2 2026, ChatGPT entered the top 10 most impersonated brands for phishing, with fake subscription pages stealing credit cards
2.2 AI‑Generated Malware and Malicious Code
The DeepSeek case: A 14‑year‑old security researcher demonstrated that the model can generate malicious code via prompt injection. The report went unanswered for 7 months .
Other examples:
BlackMamba: A keylogger that connected to OpenAI to generate polymorphic code, continuously changing its structure to evade detection systems .
Xanthorox AI: A generative model sold on Telegram and the dark web as a successor to WormGPT, operating on private servers without external APIs .
2.3 "Living off the AI" — Attacks Using Enterprise AI Agents
Attackers no longer attack just the model; they attack AI agents integrated into corporate systems .
Main techniques:
Prompt‑born tool abuse: A document or webpage with hidden instructions tricks the agent into executing connected tools, exfiltrating internal data without triggering traditional EDR .
Memory and retrieval poisoning: The attacker seeds malicious "facts" into a shared vector store, altering the agent's future decisions.
Abuse of "vibe coding": AI‑driven web development platforms (Lovable, Vercel, Netlify) are used to create convincing phishing sites in minutes .
Part 3: Deep Web and Dark Web — The Attack AI Economy
3.1 The Offensive AI Tool Market
On the deep web and Telegram, AI is now a commodity .
What's found in forums and Telegram channels:
Dedicated sections for LLM discussions, prompt injection, and evasion techniques
Sales of stolen premium AI service accounts (ChatGPT, Claude, Gemini) — subscriptions are becoming a lucrative target
Spam generation tools and synthetic profile creation for social media impersonation
3.2 Phishing‑as‑a‑Service (PhaaS)
The economic model of AI‑driven attacks is industrialized: no more lone scammers improvising, but complete supply chains .
The supply chain:
Developers: create phishing kits, impersonation frameworks, and automation tools
Operators and resellers: license or rent these tools, launching campaigns across disposable domains, SMS gateways, and social media
Facilitators: offer "bulletproof" hosting, proxy networks, voice‑cloning vendors, and AI persona generators
Brokers: monetize the output — stolen credentials, session cookies, verified accesses — selling to ransomware groups or initial access brokers
Consequence: A single domain takedown doesn't disrupt the system; the infrastructure is designed to absorb losses and reroute quickly .
Part 4: Why Is It So Hard to Recognize AI Scams? — The 5 Cognitive Barriers
4.1 The Collapse of Traditional Heuristics
Before AI, scams could be recognized by indicators like grammar errors, brand inconsistencies, or "mechanical" tone. These signals no longer work:
"What used to take an attacker weeks or months, they can now pull off in hours" — Randolph Barr, CISO of Cequence Security .
Generative AI produces perfect emails, indistinguishable cloned websites , and voices with emotions and pauses that fool even the most attentive users .
4.2 No Two Lures Are Identical
Cybersecurity relies on pattern recognition: identifying a virus by its signature, a scam by a known template. AI breaks this paradigm:
"Instead of defending against one phishing template at a time, organizations and their technical filters now have to defend against phishing factories that can produce a different lure for every target" — Mika Aalto, CEO of Hoxhunt .
In the EvilTokens campaign, none of the 344 lures were identical — each message had different language, branding, and emotional structure .
4.3 Overconfidence (Too Much Self‑Trust)
People tend to overestimate their ability to detect a deepfake. In the synthetic voice study:
Incorrect responses were often accompanied by high confidence
Participants believed they had identified the AI, but in reality, they were deceived
4.4 Trust Manipulation (Sycophancy Bias)
Attackers exploit trust accumulated over long conversations. The DeepSeek model, for example, can be gradually manipulated — an innocuous request at the start becomes dangerous after 30 conversation turns. The security barrier weakens as the user "builds trust" with the system .
4.5 Abuse of Legitimate Channels
Attacks like Forg365 abuse legitimate Microsoft authentication flows. The device code is presented to the victim as a harmless verification request. The victim enters the code on a genuine Microsoft site, unknowingly authorizing the attacker's access .
The most insidious consequence: Resetting the password might not remove the attacker, because the stolen refresh token remains valid. Access persists even after credential changes .
đź“‹ Summary: Why AI Scams Are Hard to Recognize
Barrier Description
Heuristics collapsed Grammar errors, mechanical tone, known templates — all surpassed by AI
Mass uniqueness Every message is different → impossible to create pattern‑based filters
Overconfidence People are sure they can detect the deception but are wrong almost every time
Gradual manipulation Trust built over long conversations lowers defenses
Legitimate channels abused The attack uses authentic Microsoft flows — nothing "fake" to detect
🛡️ What to Do to Defend Yourself
Based on expert recommendations :
Treat every AI agent as a privileged user: apply the principle of least privilege
Review and limit device‑code authentication use where not strictly necessary
Monitor for unusual access (suspicious hosting providers, odd hours, data volumes)
Revoke tokens and sessions immediately upon suspected abuse
Implement phishing‑resistant MFA (FIDO2/WebAuthn) — MFA alone is no longer enough
Educate users not to trust "voices" but to verify through alternative channels (e.g., call the known number directly)
Based on reports from ESET, ZeroFox, Huntress, Check Point, SecurityWeek, and academic research from arXiv and AAAI (2025‑2026).
