SYNTHESIS NOTE
Topics›Human Centered Design›this note

What misconceptions hide in how we describe large language models?

Both deflationary framings like "just autocomplete" and anthropomorphic claims about emerging agency capture something real about LLMs, but each may overextend its truth. What distinctions help separate genuine features from overreach?

Synthesis note · 2026-09-25 · sourced from Human Centered Design

The paper's central claim is that debates about LLMs "remain structured by persistent folk theories," meaning intuitive, informal explanatory models that "guide attitudes and actions." Two families are named. Deflationary slogans ("just autocomplete," "stochastic parrots," "average of the internet") and anthropomorphic framings ("emergent agents," "protominds," "superhuman reasoners") "each capture genuine features of current systems but mistake those features for the whole." The paper does not ask which camp is right. It treats both as partial models, each accurate about something and wrong about how far it extends.

The mechanism is conflation. The Perspective proposes a "minimal working model" built on four distinctions: pretraining versus the deployed system; the learned distribution versus particular samples; parametric, contextual and external memory; and task competence versus agency. A slogan becomes a misconception when it collapses one of these pairs. The six misconceptions it diagnoses are next-token prediction, regression to the mean, training-data regurgitation, model memory, alignment and understanding. For each, the analysis is said to identify "what the misconception gets right, which distinctions it conflates, and what follows for capability evaluation, system design, and governance." The introduction adds that "folk" describes the mode of explanation, not the speaker's sophistication, so informal and technical accounts can coexist in the same researcher or policymaker.

This is a diagnostic taxonomy of errors in how people model the system, which is a different level from the neighbors. Why do people trust AI outputs they shouldn't? locates failure in the user's cognition when reading outputs. This paper locates it in the explanatory model the user holds of the system, and that model can be wrong in either direction. Are language models and human speakers doing the same thing? names one specific category error. Here that error would be one instance of a wider method: check which distinction a claim about LLMs has flattened. The learned-distribution versus sample distinction also sits close to Should we treat LLM outputs as real empirical data?, which formalizes outputs as draws rather than observations.

The excerpt is only the abstract and one introductory passage, with no discussion section. It states the four distinctions and lists the six misconceptions but gives none of the six analyses. It does not say which distinction each misconception conflates, and it gives no evidence on how widely these folk theories are held. It claims the model is "minimal," not that it is complete or empirically validated. What the passages support is narrower: treat each slogan as a partially true model whose scope needs checking, and ask which of the four distinctions an argument about LLMs has quietly merged. Whether that resolves any of the six misconceptions has to be read in the full paper.

Inquiring lines that read this note 1

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

What explains language models' asymmetric difficulty with implicit versus explicit linguistic relations?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
13 direct connections · 91 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

deflationary and anthropomorphic folk theories of LLMs each mistake a genuine feature for the whole — four distinctions diagnose six misconceptions