Why do readers interpret the same sentence so differently?
How much of annotation disagreement in NLP reflects genuine interpretive multiplicity rather than error? This explores whether social position and moral framing systematically generate competing but equally valid readings.
The standard assumption underlying NLP benchmark design is that sentences have one correct interpretation. Disagreement between annotators signals annotation failure. The solution is to filter or adjudicate until one answer emerges.
Interpretation Modeling (IM, Cercas Curry et al. 2023) challenges this assumption directly. The study models multiple interpretations of socially embedded sentences, guided by reader attitudes toward the author and reader understanding of implicit moral judgments. Finding: conflicting interpretations are socially plausible. They reflect different social positions and moral framings, not annotation error.
This is not about ambiguous sentences in the traditional sense (lexical or syntactic ambiguity) but about the social and implicit dimensions of meaning in natural communication. A sentence embedded in a social context carries different meanings for readers with different:
- Relationships to the speaker
- Moral frameworks for evaluating the content
- Common ground with the speaker's implied community
The interpretations that result are not all "correct" in a truth-conditional sense, but they are all "valid" in a socially and pragmatically grounded sense — readers with different social positions genuinely understand different things from the same text.
The implication is uncomfortable for NLP: the gold standard that benchmarks aspire to may not exist for a substantial portion of natural language. Treating disagreement as noise produces evaluation systems that measure agreement on easy cases while missing the hard question of how interpretation actually works.
The NLI disagreement literature provides statistical confirmation. "Lost in Inference" (analyzing NLI annotation disagreement across major benchmarks) finds that NLI task performance is not saturated — humans continue to disagree, and that disagreement is not random noise but structured. Human annotation distributions on contested examples carry information that the majority label discards. This is the empirical grounding for IM's theoretical claim: interpretation is irreducibly multiple, and the distribution over interpretations is itself meaningful data.
An additional mechanism: social identity projection. Readers don't just apply their moral frameworks abstractly — they project the likely social identity of the author based on textual cues, then interpret the content through the lens of that projected identity. Two readers who project different author identities from the same text will read the same words as carrying different social stances. This is a grounding claim about interpretation that goes beyond semantic ambiguity.
This connects to Why do speakers deliberately use ambiguous language? — interpretive multiplicity is not a failure of specification but a feature of how socially embedded language operates. Since Do standard NLP benchmarks hide LLM ambiguity failures?, this irreducibility is doubly hidden.
Inquiring lines that read this note 59
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How does rhetorical adaptation affect LLM persuasion and detectability? What makes AI persuasion effective and how can we counter it?- Why does renaming the entity change how compelling the argument feels?
- Do readers with weakly held priors respond more to linguistic features than ideologically committed ones?
- How do readers selectively hold frame-related words in mind?
- How do humans detect which words belong to the same frame together?
- How does frame selection differ from frame application in meaning-making?
- What makes some interpretive postures stick while others fail to form?
- Where does the meaning actually originate in reader-detected resonance across language?
- What makes ambiguity recognition fundamentally important for poetry analysis?
- What signals beyond surface content indicate a passage caused a user's reaction?
- What semantic classifier design avoids lexical variation without genuine conceptual distinctness?
- Why do different readers extract different meanings from identical text?
- Does removing information about who wrote something change how we interpret it?
- Can moral frameworks alone explain why readers understand sentences differently?
- How do readers project author identity from textual cues during interpretation?
- How does semantic framing differ from content injection attacks?
- How much does reader ideology matter compared to the words being used?
- Can readers detect meaning through resonance patterns alone without knowing authorial intent?
- How do politeness strategies depend on semantic ambiguity between literal and intended meaning?
- Can the same predicate generate different projection strength in different contexts?
- What measurement artifacts emerge when annotators interpret the same question differently?
- Do high-disagreement items signal contested values or measurement noise?
- Why do non-attitudes cluster around value-laden questions most relevant to alignment?
- How does constitutional alignment compare to RLHF in removing human annotation costs?
- How does semantic ambiguity differ from structural ambiguity in language?
- Is interpretive multiplicity a bug in language or a feature?
- How does the inability to manage ambiguity undermine literary analysis tasks?
- Why do NLP models fail at recognizing multiple valid interpretations?
- What metrics actually measure disagreement in multi-turn conversations?
- Does endorsement structure outperform content in detecting social controversy?
- Can decreased engagement be distinguished from genuine semantic contradiction?
- Can discourse communities collectively detect disruptions individual readers miss?
- Can persona-based approaches capture genuine disagreement in expert annotations?
- How do human annotators disagree systematically on ambiguous examples?
- Why do posters acknowledge multiple viewpoints without integrating them into coherent judgments?
- Do chain-of-thought prompts help RLVR models predict annotation disagreement?
- Why do NLP benchmarks treat annotation disagreement as noise rather than signal?
- How do organizational roles and peer interpretations shape what an explanation means?
- Why does explanation source matter more than explanation content?
- How does unilateral interpretation differ from mutual communicative uptake?
- Why does lexical difference fail to trigger reader suspicion of artificial origin?
- How can structurally different text produce equivalent real-world effects?
- How do readers interpret AI text differently from human text?
- Can detectors trained for one task reliably perform differently on unexpected text sources?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do speakers deliberately use ambiguous language?
Explores whether ambiguity is a linguistic defect or a strategic tool speakers use for efficiency, politeness, and deniability. Matters because it challenges how we train language systems.
interpretive multiplicity is functionally analogous to ambiguity: not a defect but a feature
-
Do standard NLP benchmarks hide LLM ambiguity failures?
When benchmark creators filter out ambiguous examples before testing, do they accidentally make it impossible to measure whether language models can actually handle ambiguity the way humans do?
this multiplicity is what benchmark design excludes
-
What three layers must discourse systems actually track?
Grosz and Sidner's 1986 framework proposes that discourse requires simultaneously tracking linguistic segments, speaker purposes, and salient objects. Understanding why all three are necessary helps explain where current AI systems structurally fail.
intentional structure is where social framing operates
-
Why do LLM persona prompts produce inconsistent outputs across runs?
Can language models reliably simulate different social perspectives through persona prompting, or does their run-to-run variance indicate they lack stable group-specific knowledge? This matters for whether LLMs can approximate human disagreement in annotation tasks.
the attempt to use LLMs to simulate multiple human perspectives fails because LLMs lack the stable social situatedness that makes interpretation group-specific
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgments
- Can Large Language Models Capture Human Annotator Disagreements?
- We’re Afraid Language Models Aren’t Modeling Ambiguity
- Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence
- Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions
- Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?
- Language models show human-like content effects on reasoning tasks
Original note title
sentence interpretations are irreducibly multiple because social position and moral framing generate competing readings