Do AI models learn language as a set of grammar rules, or as who says what, to whom, and when?
Do LLMs learn abstract grammar or culturally situated discourse patterns instead?
This explores whether language models pick up grammar as an abstract rule system, or whether what they absorb is closer to who says what, to whom, and in which situation, meaning cultural discourse.
The corpus leans toward the second option. One note puts it bluntly: models trained on web text learn 'socially contextualized linguistic action', meaning which speakers make which statements in response to which situations. They model cultural discourse, not language in the abstract, and that explains why they so readily reproduce social positions and personas Do language models learn abstract grammar or cultural speech patterns?. A companion note frames this in Saussurean terms. The model compresses the relational structure of text, with no external referents and no embodied grounding, and that is enough for fluent generation Can language models learn meaning without engaging the world?.
The strongest evidence against the 'abstract grammar' reading is where grammar breaks. Performance declines predictably as sentences get more deeply nested or recursive. Simple sentences are handled well, but complex embedded structures fail consistently. That pattern points to surface heuristics rather than internalized structural rules Does LLM grammatical performance decline with structural complexity?. A similar pattern shows up outside syntax. Causal reasoning beats temporal reasoning because causal connectives ('because', 'therefore') are explicit and frequent in text, while temporal order usually has to be inferred Why do LLMs handle causal reasoning better than temporal reasoning?. In both cases the model is strongest where the pattern is visible and common in the text it was trained on.
There is a wrinkle. The corpus also has a note showing that o1 can build syntactic trees and phonological generalizations through step-by-step reasoning Can language models actually analyze language structure?. That suggests LLMs can reason about grammar explicitly. The note doesn't show that they use abstract grammar internally when they generate text. Being able to describe structure and being driven by structure are different things, and the notes here don't resolve the difference.
If models are learning discourse and not rules, some odd behaviors make sense. The same weights produce a sycophantic chat voice and a falsely objective 'published prose' voice, and each register inherits the failure modes of the text it came from, namely conversational RLHF data and published writing Why do LLMs produce such different writing in chat versus posts?. Frequency in the training text also pulls output toward general words over specific ones, so preferring common phrasings drifts toward abstraction and erases expert-level specificity Does word frequency correlate with semantic abstraction?. Generation itself moves smoothly toward the training distribution and doesn't explore competing positions Does LLM generation explore competing claims while producing text?.
A further tension: if the model has absorbed culture, you might expect it to adapt to context the way a person does. The corpus says the opposite. Alignment training locks it into one communicative identity, which blocks the register-switching and negotiation humans do Can language models adapt communication style to different contexts?. Another note offers a more hopeful reading. Social grounding comes from taking part in language games, so it may grow as LLMs become established participants in human linguistic practice Can LLMs acquire social grounding through linguistic integration?. On that view the cultural-discourse framing describes a starting point that could change.
Sources 10 notes
LLMs trained on web text acquire socially contextualized linguistic action—which speakers make which statements in response to which situations. They model cultural discourse rather than language in the abstract sense, which explains why they reproduce social positions and personas.
Research shows LLMs learn culturally situated discourse patterns by compressing relational structure from text, demonstrating that fluent language generation requires no external referents or embodied grounding.
LLMs show systematic performance decline as syntactic depth and embedding increase. Simple sentences are handled well while complex structures with recursion and embedding fail consistently, suggesting LLMs learned surface heuristics rather than structural grammar rules.
ChatGPT excels at causal relations but struggles with temporal ordering because causal connectives are explicit and frequent in training data, while temporal order is often implicit and must be inferred contextually.
OpenAI's o1 model successfully constructs syntactic trees and phonological generalizations through explicit step-by-step reasoning, revealing that LLM linguistic capability extends far beyond behavioral language tasks to genuine language analysis.
Show all 10 sources
The same model produces sycophantic chat (shaped by RLHF on conversational data) and falsely objective posts (shaped by published prose training). Each register inherits failure modes from its training distribution rather than representing different models or subsystems.
WordNet analysis shows hypernyms (general concepts) occur more frequently than hyponyms (specific ones). Combined with LLMs' frequency bias, this means preferring common paraphrases systematically drifts toward abstraction, erasing expert-level specificity.
Token prediction trains models to continue toward the training distribution, not to explore logically related counterpositions. This smoothness in process produces smooth claims that multiply without generating new perspectives.
System prompts and RLHF training lock models into one communicative identity across all interactions, preventing the contextual register-switching and value trade-offs that characterize human pragmatics. Users cannot reshape model behavior through dialogue negotiation.
Social grounding is acquired through participation in language games rather than possessed innately. As LLMs become established communicative partners in human linguistic practice, they develop elementary social grounding comparable to young children, making the question of LLM understanding time-indexed.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Probing Structured Semantics Understanding and Generation of Language Models via Question Answering
- Conversational Alignment with Artificial Intelligence in Context
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence
- Large Linguistic Models: Investigating LLMs' metalinguistic abilities
- Linguistic Blind Spots of Large Language Models
- From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
- Argument Collapse: LLMs Flatten Long-Form Public Debate
- Semantic Structure in Large Language Model Embeddings