Breaking: Sycophantic AI distorts belief, manufacturing certainty where there should be doubt
Source: Gary Marcus, Marcus on AI · 2026-03-03
A new study from Princeton has important implications for education, scientific discovery, mental health, and more (perhaps politics and even decisions about war?). Essentially anyone who uses a chatbot is at risk. Because what is shows is that sycophantic AI that serves as a personal echo chamber that can actually keep you from finding good ideas. And as the article says, such AI can “facilitate delusion-like epistemic states, producing belief markedly divergent from reality.”
The paper, which you can read here, is a bit technical, but the implications are profound. I will close with another choice passage, boldfacing the crux:
Unlike hallucinations, which introduce false-hoods, sycophancy is a bias in the selection of the data people see. When AI systems are trained to be helpful, they may inadvertently prioritize data that validates the user’s narrative over data that gets them closer to the truth.
Wanna feel good about yourself? Use a chatbot. Want to find truth? Go elsewhere.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
Why do language models hallucinate and how can we prevent it? Why do confident AI outputs mislead human trust calibration? Why does polished AI output gain credibility despite fundamental verifiability problems? Can AI chatbots provide mental health support without reinforcing harmful beliefs?- What role does sycophancy play in the onset of chatbot-linked delusions?
- Does chatbot sycophancy create echo chambers that amplify delusional thinking?
- Does chatbot sycophancy preferentially enable grandiose rather than paranoid delusions?
- Does isolation preceding chatbot use differ between harm and benefit cases?
- How long do the protective effects of an AI literacy warning last?
- Does knowing a chatbot intends to persuade you change whether you are persuaded?
- How do chatbots enable shared delusions differently than passive information tools?
- What emotional and autonomy risks from AI chatbots are already observable today?
- Does knowing an AI peer's identity change how much its behavior influences you?
- Can colleagues detect when a coworker stops sounding like themselves in AI-mediated messages?
- Does knowing an AI wrote something shield people from its persuasive power?
- Does disclosing AI involvement reduce the persuasive impact of expert advice?
- What specific information should disclosures about AI persuasion include?
- Does a persuasion warning also block beneficial uses like debunking conspiracies?
- Why does transparency about AI identity alone fail to reduce persuasion?