Frontier AI systems shifted immediate policy attitudes more than elite debaters and professional canvassers in a preregistered conversational study, but that advantage disappeared when researchers capped the systems to human-like message length and response time. The result matters because it locates the risk in scalable argumentative throughput, not in evidence that chatbots possess a uniquely irresistible social charm.
Key facts
- Hackenburg and colleagues' paper reports four preregistered experiments.
- Studies 1–3 included 18,978 conversations with 6,923 persuadees.
- AI moved attitudes about 4.6 points further than elite debaters on a 0–100 scale, relative to neutral chat.
- Unconstrained AI averaged 294 words at sub-second latency; elite debaters averaged 54 words and took about 95 seconds.
The human comparison was unusually demanding. Researchers recruited ordinary participants, tournament winners, competitive debate champions and professional canvassers, then gave elite debaters preparation, cash incentives and later coaching based on AI transcripts. The AI still won on the immediate attitude measure. But when researchers constrained its replies to about 51 words and roughly 92 seconds, the AI-versus-coached-debater difference became statistically indistinguishable from zero.
That experiment is the story. It is the difference between an excellent speaker and a research assistant who can instantly locate, formulate and deliver a dense bundle of arguments. The system had about 37 fact-checkable claims per unconstrained conversation; the cap reduced that to about 12. Each additional claim was associated with more attitude shift, though the association does not prove that every claim caused persuasion.
The authors do not offer a triumphalist accuracy story. In their web-grounded corroboration pipeline, human claims were corroborated 73 percent of the time versus 47 percent for AI claims overall. Higher factual accuracy did not necessarily correspond to higher persuasion. In a small-stakes behavioral test, a model also elicited about 10.8 percentage points more of a one-pound study bonus for Save the Children than professional canvassers. This is meaningful controlled evidence, not proof of mass political mobilization.
Kobi Hackenburg and coauthors' strongest caveat is external validity: paid participants engaged in text exchanges for a median 14 minutes and knew someone was trying to persuade them. The study does not test unsolicited feeds, audiovisual charisma, coordinated campaigns or persistence of the new policy attitude. Related work has shown durable corrective conspiracy dialogues, but the lead study is not that retention experiment.
The hardest implication is dual-use. A system optimized to give evidence-based corrections can help a person reason; one optimized to flood a conversation with tailored but weak claims can also move belief. The relevant guardrail is not merely a polite tone but an objective that values truthfulness and appropriate deployment. This is a live demonstration of why AI persuasion and calibration are governance problems as well as UX problems.
Originally published on Ground Truth, where every claim is checked against the primary source.
