Can LLMs identify implicit metaphoric mappings that require pragmatic inference?
This explores whether LLMs can recover the unstated 'this maps to that' relationship behind a metaphor — the kind of figurative meaning you can only reach by inferring what the speaker intends, not by decoding the literal words.
This explores whether LLMs can recover the unstated mapping behind a metaphor — the figurative leap that only works if you infer what's meant rather than what's said. The corpus reframes the question in a useful way: metaphor isn't a special category to be detected so much as one instance of a broader pragmatic skill. The Diplomat dataset bundles metaphors, idioms, and puns into a single task — recovering literal meaning from non-literal expression — and argues that what LLMs lack is general semantic decoupling, the ability to separate what a phrase says from what it's doing Can one model handle all types of figurative language?. So the real question isn't 'can the model spot a metaphor' but 'can it infer intended meaning against literal meaning' — which is exactly where pragmatic inference lives.
The discouraging part is what sits underneath that decoupling ability. When semantic content is pulled apart from the reasoning task, LLM performance collapses even when the correct rules are handed to the model in context — they lean on parametric commonsense and token associations rather than manipulating relationships symbolically Do large language models reason symbolically or semantically?. Metaphoric mapping is precisely a structural operation (carry the relations from one domain onto another), so a model that reasons through familiar semantic association rather than structure is likely to handle conventional, high-frequency metaphors well and stumble on novel ones. A related finding sharpens this: models systematically prefer textually frequent phrasings over semantically equivalent rare ones, tracking statistical mass from pretraining rather than meaning Do language models really understand meaning or just surface frequency?. A fresh, implicit mapping has no statistical mass to ride on.
Pragmatic inference also demands holding more than one reading at once — the literal and the intended — and choosing between them by what the speaker must have meant. Here the corpus is blunt: GPT-4 correctly disambiguates only 32% of deliberately ambiguous cases versus 90% for humans, a failure spanning lexical, structural, and scope ambiguity, suggesting models can't hold multiple interpretations simultaneously Can language models recognize when text is deliberately ambiguous?. Metaphor is a managed ambiguity — you're meant to notice the literal reading is wrong and reach past it — so this gap cuts directly at the inferential move the question asks about. Relatedly, entailment work shows models often decide based on whether a conclusion looks familiar from training rather than whether the premise actually supports it Do LLMs predict entailment based on what they memorized?.
The most interesting wrinkle is that explaining a metaphor and using one may be different circuits entirely. 'Potemkin understanding' describes models that explain a concept correctly, fail to apply it, and even recognize the failure — a pattern that implies explanation and execution run on functionally disconnected pathways Can LLMs understand concepts they cannot apply?. So a model might fluently define what 'time is a thief' maps onto while still missing the mapping when it has to act on a fresh one in conversation. That distinction — articulate-about versus operate-with — is the thing worth carrying away here.
A quieter counterweight: LLMs do learn meaning as a fully relational system, compressing cultural and discourse structure from text without any external referent Can language models learn meaning without engaging the world?. That's genuinely how a lot of conventional metaphor works — 'argument is war,' 'up is good' are linguistic conventions, not perceptions — so for the metaphors already sedimented into language, relational compression may be enough. The pragmatic-inference frontier is the novel mapping, built on the fly, that the corpus suggests current models are least equipped to reach.
Sources 7 notes
The Diplomat dataset (4,177 dialogues) reframes metaphors, idioms, and puns as one pragmatic task: recovering literal meaning from non-literal expression. This framing suggests LLMs need better semantic decoupling ability, not more category-specific training data.
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
LLMs show consistent preference for higher-frequency surface forms over semantically equivalent rare paraphrases across math, machine translation, commonsense reasoning, and tool calling. This suggests models track statistical mass from pretraining rather than meaning-recognition as their primary mechanism.
AMBIENT benchmark shows GPT-4 correctly disambiguates only 32% of cases versus 90% for humans. This failure spans lexical, structural, and scope ambiguity—revealing that LLMs cannot hold multiple interpretations simultaneously, a fundamental gap hidden by standard benchmarks.
McKenna et al. (2023) identified attestation bias: LLMs predict entailment based on whether the hypothesis appears in training data, not whether the premise actually supports it. Random premise experiments show models maintain high entailment predictions when hypotheses are attested, proving they respond to memorized propositions rather than premise-hypothesis relationships.
Models can explain concepts accurately, fail to apply them, and recognize the failure—a triple pattern incompatible with human cognition. This indicates functionally disconnected explanation and execution pathways rather than simple knowledge gaps.
Research shows LLMs learn culturally situated discourse patterns by compressing relational structure from text, demonstrating that fluent language generation requires no external referents or embodied grounding.