Does using AI answers actually change how you think, not just what you conclude?

This explores whether exposure to LLM answers changes the process of human thinking, not just its conclusions. The short version: the collection has no study that directly measures critical-thinking skill before and after AI use. It does have several findings that, taken together, suggest LLMs change how people think in ways they may not notice, and that the change isn't always toward better judgment.

The most direct evidence comes from a 348-person experiment on strategic decisions Does using LLMs actually improve strategic decision making?. People who used an LLM considered a wider range of factors, so their mental picture of the problem really did change. Their predictions got no more accurate, though. They also felt more overloaded and less ownership of their decisions. So the AI changed the shape of their thinking without improving its quality, and it loosened their sense that the reasoning was their own. That second effect is easy to miss, and it may matter more for critical thinking than accuracy does.

How you encounter AI output also seems to matter. In a study of AI debates, people arguing directly with an LLM changed their minds only about 7% of the time, while people who simply read the same exchanges were swayed 34–62% of the time Why do LLM audiences shift views more than debaters?. Arguing back acts as a built-in critical filter. Reading passively removes it. Most people meet LLM answers in the passive mode: reading a response, not pushing against it. Related work shows that even ML experts can't reliably tell AI-written abstracts from human ones, and they rated AI-edited text the clearest Can readers tell LLM abstracts from human ones?. Polished, fluent prose lowers the guard that critical reading depends on.

The collection also has a quieter worry: the range of ideas people are exposed to shrinks. LLM essays on debate topics recover only about half of the distinct arguments humans make, and they lean on the same hedged sub-points Do language models flatten the range of public arguments?. Critical thinking needs real alternatives to weigh against each other. If the answers you read keep drawing on a narrower set of arguments, you can become a careful evaluator of a shrinking menu. Answers can also shift with the emotional tone of the question: negative prompts get steered back toward neutral or positive responses Does emotional tone in prompts change what information LLMs provide?. Two people asking the same question in different moods may get different information without knowing it.

For a more philosophical angle, one critique argues that LLMs present knowledge as fixed, finished facts and hide the debates and history that produced them. The same essay points to evidence that people with lower AI literacy are more receptive to that framing Do LLMs obscure the historical processes behind their answers?. Read together, these findings suggest the risk isn't that AI makes people stop thinking. The risk is that it changes the conditions for thinking: fewer competing arguments, more fluent certainty, more passive reading and less ownership. Whether people stay critical may depend less on the AI's quality than on whether they engage with it like a debater instead of an audience member.