Using an AI to help decide something changes how you think about the problem — but does it actually make your choice better?
How do people's mental models shift when they use LLMs for decisions?
This explores what happens inside people's heads when they bring an LLM into a decision: how it changes the way they frame the problem, how they judge their own confidence, and even how they come to picture their own thinking.
This explores what happens inside people's heads when they bring an LLM into a decision: how it changes the way they frame the problem, how much they trust their own judgment, and even how they picture their own minds. The most direct evidence is surprising. In a 348-person experiment on strategic decisions, LLM help clearly reshaped how people thought: they weighed a wider range of factors. But their predictions got no more accurate. They also felt more overloaded and less like the decision was theirs Does using LLMs actually improve strategic decision making?. So the shift is real, but a richer picture of the problem is not the same as a better one. More considerations can mean more noise, and handing the framing to a model can loosen your sense of ownership over the result.
A second shift is in confidence. When an LLM explains its answer, readers come away more sure than the model's own internal probabilities justify. The fluency of the explanation, not its reliability, sets how confident people feel Why do readers trust LLM explanations more than they should?. Put that next to the strategy study and a pattern emerges: LLMs can make people feel they have thought harder and more broadly, while their accuracy stays flat. The gap closes only when explanations are deliberately written to signal the model's uncertainty, so the problem lies in how answers are presented, not in the user.
The least obvious shift happens over the long run and goes beyond any single decision. Habitual LLM use can change how people describe their own minds. Memory starts to sound like 'retrieval' and creativity like 'recombination'. LLM vocabulary becomes the handy metaphor for human thinking, and people absorb it without ever deciding to believe it How does LLM vocabulary spread beliefs about human thinking?. If you start to see your own judgment as a kind of lookup, deferring to a better lookup engine starts to feel natural.
One more twist makes this harder to untangle: the models carry human-like reasoning quirks too. They show belief bias, judging an argument by whether its conclusion sounds plausible instead of by its logic, at error rates matching humans item by item Do language models show the same content effects humans do?. They also show optimism about their own chosen options, which could feed confirmation bias Do language models learn differently from good versus bad outcomes?. So a person leaning on an LLM isn't getting a neutral second opinion. They may be getting their own blind spots reflected back with more polish. And LLMs are weak at tracking how a person's mind changes mid-conversation Can language models track how minds change during persuasion?, so the model is unlikely to notice when your thinking is drifting.
The collection only has one study that directly measures how decision-makers change, so treat this as an early picture. Still, the threads point the same way. The main risk isn't that LLMs make people's decisions worse. It's that people feel more informed and more confident while their decisions don't actually get better.
Sources 6 notes
A 348-person experiment found that LLM-assisted evaluation broadened the cues people considered but did not improve prediction accuracy. The assistance also increased perceived overload and reduced psychological ownership of decisions.
Lab experiments on MMLU and TriviaQA show readers' confidence judgments track explanation language poorly compared to the model's internal probability estimates. The gap narrows when explanations are deliberately engineered to reflect internal confidence, but natural language explanations leave readers systematically overconfident.
LLM features get projected onto humans through two mechanisms: analogical transfer (memory as retrieval, creativity as recombination) and metaphorical availability (LLM vocabulary becoming psychologically salient). This pattern propagates the bias without requiring explicit endorsement.
LLMs show identical content-sensitivity patterns to humans on NLI, syllogisms, and Wason tasks, with belief-bias signatures matching human error rates item-by-item. This behavioral isomorphism across three independent tasks suggests content and logical form are inseparable in transformer reasoning architecturally.
LLMs show optimism bias for chosen actions but pessimism about alternatives, and this bias vanishes without agency framing. Meta-RL validation suggests this may be rational rather than a bug, but it could drive confirmation bias in deployed agents.
Show all 6 sources
LLMs match human performance on static mental states like a persuader's unchanging goal, but significantly underperform on dynamic shifts like a persuadee's evolving resistance. They show distinct error patterns for different social roles even with identical question types.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation
- To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models
- Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
- In-context learning agents are asymmetric belief updaters
- What Large Language Models Know and What People Think They Know
- Language models show human-like content effects on reasoning tasks
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues