Should an AI ask you what you mean before it answers, or show several possible answers afterward?
Does disambiguation on the input side differ from the output-side preview approach?
This explores whether resolving ambiguity before the model answers (detecting it and asking, or having the user supply context) is a different problem from showing candidate outputs or interpretations after generation. The corpus has direct evidence on the input side and only indirect evidence on the output side.
This explores whether resolving ambiguity before the model answers is a different problem from showing candidate outputs after it generates. Going by the corpus, they are different problems, and they fail in different places. The corpus has no note on output-side previews as such, so that half of the comparison is my inference from neighboring findings.
Input-side disambiguation has a prerequisite: the model has to notice the ambiguity. On the AMBIENT benchmark, GPT-4 correctly disambiguates only 32% of cases where humans manage 90%. The failures cover word-level, sentence-structure and scope ambiguity, and the pattern suggests models can't hold several readings at once (Can language models recognize when text is deliberately ambiguous?). A model that doesn't notice the ambiguity won't ask a clarifying question. What you get instead is the generic, blended answer described in Why do large language models produce generic responses to vague queries?: vague queries send the model to averaged training-data priors. That note points to remedies on the input side, meaning query verification and getting users to specify their context.
Even when the input side works and the user does clarify, the clarification may not land. Why do language models ignore information in their context? finds that strong training associations can override what's in the context, and that prompting alone can't fix it. There is also a quieter input-side problem: rewording a question changes the answer even when the meaning is unchanged. Higher-frequency phrasings win because the model tracks statistical mass from pretraining, not meaning (Why do semantically identical prompts produce different LLM outputs?). So the input is more than the ambiguity you can see. The wording you choose is a hidden variable in the result.
The output side is different because it doesn't require the model to recognize anything. The closest evidence is the 20-questions test: an LLM doesn't commit to one character or answer. It holds a spread of consistent possibilities and samples one at generation time, so regenerating gives a different but equally consistent reply (Do large language models actually commit to a single character?). Read that way, any single answer is one draw from a spread the model never resolved, and a preview of several outputs would show the user that spread. There's an odd tension here. Sampling exposes alternatives the model can't list when asked directly. Previewing works around the recognition failure, where asking for clarification depends on it.
The remaining question is when to intervene at all. In retrieval, calibrated token-probability uncertainty beats complex multi-call heuristics for deciding when to fetch more information, at a fraction of the cost (Can simple uncertainty estimates beat complex adaptive retrieval?). The note is only about retrieval, but the same self-knowledge signal could plausibly decide when a question deserves a clarifying step or a preview. The corpus doesn't test that. It also doesn't compare the two approaches head to head.
Sources 6 notes
AMBIENT benchmark shows GPT-4 correctly disambiguates only 32% of cases versus 90% for humans. This failure spans lexical, structural, and scope ambiguity—revealing that LLMs cannot hold multiple interpretations simultaneously, a fundamental gap hidden by standard benchmarks.
Unlike social-media context collapse, which flattens multiple audiences, LLM collapse occurs when users provide insufficient contextual scaffolding and models default to blended training-data priors. This distinction suggests remedies should focus on query verification and user-driven context specification rather than platform controls.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Cao et al. and Adam's Law show that semantically identical prompts with different sentence-level frequencies produce systematically different output quality. Higher-frequency phrasings win because models register statistical mass from pre-training, not meaning.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
Show all 6 sources
Calibrated token-probability uncertainty consistently beats multi-call adaptive retrieval on single-hop tasks and matches performance on multi-hop, using a fraction of the LM and retriever calls. The model's self-knowledge proves more reliable than external heuristics for deciding when to retrieve.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
- ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs
- Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
- We’re Afraid Language Models Aren’t Modeling Ambiguity
- Conversational Alignment with Artificial Intelligence in Context
- Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgments
- LLM-Independent Adaptive RAG: Let the Question Speak for Itself
- Aligning Language Models to Explicitly Handle Ambiguity