What misconceptions hide in how we describe large language models?
Both deflationary framings like "just autocomplete" and anthropomorphic claims about emerging agency capture something real about LLMs, but each may overextend its truth. What distinctions help separate genuine features from overreach?
The paper's central claim is that debates about LLMs "remain structured by persistent folk theories," meaning intuitive, informal explanatory models that "guide attitudes and actions." Two families are named. Deflationary slogans ("just autocomplete," "stochastic parrots," "average of the internet") and anthropomorphic framings ("emergent agents," "protominds," "superhuman reasoners") "each capture genuine features of current systems but mistake those features for the whole." The paper does not ask which camp is right. It treats both as partial models, each accurate about something and wrong about how far it extends.
The mechanism is conflation. The Perspective proposes a "minimal working model" built on four distinctions: pretraining versus the deployed system; the learned distribution versus particular samples; parametric, contextual and external memory; and task competence versus agency. A slogan becomes a misconception when it collapses one of these pairs. The six misconceptions it diagnoses are next-token prediction, regression to the mean, training-data regurgitation, model memory, alignment and understanding. For each, the analysis is said to identify "what the misconception gets right, which distinctions it conflates, and what follows for capability evaluation, system design, and governance." The introduction adds that "folk" describes the mode of explanation, not the speaker's sophistication, so informal and technical accounts can coexist in the same researcher or policymaker.
This is a diagnostic taxonomy of errors in how people model the system, which is a different level from the neighbors. Why do people trust AI outputs they shouldn't? locates failure in the user's cognition when reading outputs. This paper locates it in the explanatory model the user holds of the system, and that model can be wrong in either direction. Are language models and human speakers doing the same thing? names one specific category error. Here that error would be one instance of a wider method: check which distinction a claim about LLMs has flattened. The learned-distribution versus sample distinction also sits close to Should we treat LLM outputs as real empirical data?, which formalizes outputs as draws rather than observations.
The excerpt is only the abstract and one introductory passage, with no discussion section. It states the four distinctions and lists the six misconceptions but gives none of the six analyses. It does not say which distinction each misconception conflates, and it gives no evidence on how widely these folk theories are held. It claims the model is "minimal," not that it is complete or empirically validated. What the passages support is narrower: treat each slogan as a partially true model whose scope needs checking, and ask which of the four distinctions an argument about LLMs has quietly merged. Whether that resolves any of the six misconceptions has to be read in the full paper.
Inquiring lines that read this note 1
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What explains language models' asymmetric difficulty with implicit versus explicit linguistic relations?Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do people trust AI outputs they shouldn't?
When do human cognitive shortcuts fail in AI interaction? Three compounding traps—treating statistical patterns as facts, mistaking fluency for understanding, and avoiding disagreement—may explain systematic overreliance across languages and contexts.
another diagnostic framework for human-AI misreading, located in user cognition rather than in the explanatory model of the system
-
Are language models and human speakers doing the same thing?
Does treating LLM output and human communication as equivalent operations mask fundamental differences in how they work? This distinction shapes how we assess AI capabilities and risks.
one named category error, which this paper's method of finding conflated distinctions would generalize
-
Should we treat LLM outputs as real empirical data?
Can synthetic text generated by language models serve as evidence in the same way observations from the world do? This matters because researchers increasingly rely on AI-generated content without accounting for its fundamentally different epistemic status.
shares the learned-distribution versus particular-sample distinction
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Six misconceptions about large language models: A minimal model and diagnostic taxonomy
- Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentality
- Beyond Hallucinations: The Illusion of Understanding in Large Language Models
- From Human to Machine Psychology: A Conceptual Framework for Understanding Well-Being in Large Language Models
- Are Emergent Abilities in Large Language Models just In-Context Learning?
- The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows
- Word Meanings in Transformer Language Models
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence
Original note title
deflationary and anthropomorphic folk theories of LLMs each mistake a genuine feature for the whole — four distinctions diagnose six misconceptions