Giving an AI extra background info — does that make it more likely to ask what you meant, or less?
How does retrieved context change whether models ask clarifying questions?
This explores whether giving a model extra background, such as retrieved documents or remembered facts about the user, makes it more or less likely to stop and ask what you meant. The corpus has no study that tests this directly, but several findings come at it from different sides.
This explores whether giving a model extra background, such as retrieved documents or remembered facts about the user, makes it more or less likely to stop and ask what you meant. The honest answer: none of the papers here measures how retrieval changes how often a model asks questions. Read together, though, they suggest that retrieved context and clarifying questions do the same job, which is filling in what the user left unsaid, and that models are badly set up to tell when that gap is still open.
Start with why models rarely ask at all. Standard RLHF rewards the response that looks most helpful on the very next turn, so a model learns that answering right away beats asking a question Why do language models respond passively instead of asking clarifying questions?. When a vague question gets no extra context, the model doesn't hold back. It falls back on a blend of whatever its training data suggests, which is why answers to vague prompts come out generic Why do large language models produce generic responses to vague queries?. One way to see retrieval is as an automated fix for that missing context. If it works, there is less to ask about. If it doesn't, the model now answers with false confidence instead of with generic hedging.
The interesting part is that not all retrieved context helps. In conversational RAG, what a retrieved memory does matters more than how relevant it scores. Memories that clarify the request improve factual accuracy and the model's attention to the user's constraints, while irrelevant memories actively make answers worse Does retrieved memory quality depend on its functional role?. So retrieval can play the role a clarifying question would have played, or it can quietly push the model in the wrong direction. Another finding makes this worse: when the model's training-era associations are strong, it may ignore what's in its context altogether, and prompting alone often can't override that Why do language models ignore information in their context?. A model that ignores useful retrieved context, or is misled by noisy context, still won't notice that it should ask.
There's also a twist about format. Models answer more accurately when an ambiguity is resolved through an actual question-and-answer exchange than when they get a cleanly rewritten, unambiguous version of the same request Do clarifying dialogues help LLMs answer better than rewritten inputs?. The suggested reason is that dialogue looks more like what models saw in pretraining. That points to a design idea: retrieved context may work better when it's framed as a clarifying exchange than when it's pasted in as a block of facts.
On making models ask well, the training work is promising: reinforcement learning, self-play, and splitting question quality into separate attributes all help Can models learn to ask clarifying questions instead of guessing? Can models learn to ask better clarifying questions through self-improvement? Can models learn to ask genuinely useful clarifying questions?. The most relevant method for retrieval is choosing a question by simulating how each possible answer would reduce uncertainty How can models select the most informative question to ask?. In principle, retrieved context would feed directly into that calculation, so a well-informed model would ask fewer but sharper questions. Whether anyone has tested that is a gap the collection doesn't yet fill.
Sources 9 notes
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
Unlike social-media context collapse, which flattens multiple audiences, LLM collapse occurs when users provide insufficient contextual scaffolding and models default to blended training-data priors. This distinction suggests remedies should focus on query verification and user-driven context specification rather than platform controls.
Retrieved memory type drives response quality more than relevance alone: clarifying memory improves factual accuracy and constraint awareness, while irrelevant memory actively degrades both. Role-aware retrieval and filtering are robustness requirements, not optional optimizations.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
LLMs answer more accurately when given clarifying questions and answers than when given minimally-edited disambiguated versions of the same ambiguous input. The gap exists because clarifying exchanges match the model's pretraining distribution better than artificially constrained rewrites.
Show all 9 sources
Reinforcement learning training increased proactive critical thinking accuracy from 0.15% to 73.98% on deliberately flawed math problems. Notably, inference-time scaling degraded this ability in untrained models but improved it after RL training, suggesting the capability is learnable but fragile without explicit training.
STaR-GATE iteratively finetunes a model on questions that increase response quality, achieving 72% preference over the base model after two iterations. The research shows preference elicitation is trainable through self-play without human question supervision.
The ALFA framework breaks down question quality into theory-grounded attributes (clarity, relevance, specificity) and trains models on 80K attribute-specific preference pairs. Attribute-specific optimization outperforms single-score training, especially in clinical reasoning where asking the right clarifying question directly impacts decision quality.
UoT combines uncertainty-aware scenario simulation with information-gain scoring and reward propagation to identify questions whose possible answers maximally reduce diagnostic uncertainty—providing a principled mechanism for specific, high-value clarification rather than generic prompts.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
- QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning
- STaR-GATE: Teaching Language Models to Ask Clarifying Questions
- Can Large Language Models Reason and Optimize Under Constraints?
- Proactive Conversational Agents in the Post-ChatGPT World
- Learning to Learn from Language Feedback with Social Meta-Learning