Should an AI pause to ask you a question, or just guess and keep going — and what should decide that?
How should systems decide when to ask clarifying questions versus proceeding silently?
This explores how an AI system should decide when to stop and ask the user something and when to go ahead on its best guess, and what the corpus says about why models get this wrong and how to fix it.
This explores the decision point every assistant or agent faces: stop and ask, or keep going on a guess. The most useful finding in the corpus is that ambiguity alone is the wrong trigger. In a Wizard-of-Oz study of people directing a computer-use agent, users wanted the agent to quietly settle small uncertainties, like which menu to click, but to pause before anything consequential, like a payment. What users cared about was the stakes, not how uncertain the agent was When should agents ask before acting on ambiguous commands?. So a good rule is not "ask whenever unsure." It is closer to "ask when a wrong guess would be costly or hard to undo."
Models also don't start out neutral on this choice. They lean toward not asking, and training is part of the reason. Standard RLHF rewards the response that looks most helpful right now, which teaches models to answer immediately instead of finding out what the user actually wants. CollabLLM addresses this by rewarding the long-run value of a conversation, which makes asking a question worth something again Why do language models respond passively instead of asking clarifying questions?. The problem gets worse in tool-using agents, which can chain searches and actions without checking in and drift away from the user's intent along the way. Conversation analysis offers a framework for this: "insert-expansions" are the short side-questions people ask mid-exchange to clarify intent or scope before answering. They prevent misunderstandings instead of repairing them afterward When should AI agents ask users instead of just searching?. A related idea is that staying quiet is a skill of its own. Some systems train an explicit "stay silent" token, or run a hidden check on whether speaking would add anything, because models don't learn timing on their own When should AI systems choose to stay silent?.
Knowing when to ask can be trained, but it is fragile. RL training raised models' ability to spot missing information in deliberately flawed math problems from almost zero to about 74%. The surprise is that giving untrained models more thinking time made them worse at this: they reasoned their way past the gap instead of flagging it Can models learn to ask clarifying questions instead of guessing?. Other routes exist too. Models can improve by fine-tuning on their own questions that led to better answers Can models learn to ask better clarifying questions through self-improvement?. Asking can even emerge without being taught directly, when models learn to treat the conversation itself as a source of information Can models learn to ask clarifying questions without explicit training?.
Once a system decides to ask, the quality of the question matters, and that feeds back into whether asking was worth it. Users respond better to specific questions ("What type of monitor?") than to requests to restate their goal ("What are you trying to do?"). People engage when they can see how their answer will improve the result Which clarifying questions actually improve user satisfaction?. There are principled ways to choose that question. One simulates the possible answers and picks the question that would cut uncertainty the most How can models select the most informative question to ask?. Another breaks question quality into parts like clarity, relevance, and specificity and trains on each one Can models learn to ask genuinely useful clarifying questions?. PleaSQLarify takes a middle route between asking and staying quiet: it ranks the ambiguities in a database query and shows the alternatives visually, so users can resolve them in a few turns without a series of interruptions Can systems surface ambiguity instead of guessing user intent?.
Taken together, a practical policy looks like this. Weigh the cost of a wrong guess against the cost of interrupting. Settle low-stakes uncertainty quietly. When you do ask, ask the one specific question that resolves the most uncertainty. Also keep in mind that the default training pipeline pushes models toward guessing, so leaning toward asking usually has to be built in deliberately. The corpus is thinner on how to measure "stakes" automatically. Most work assumes the system already knows which actions are high-risk.
Sources 11 notes
A Wizard-of-Oz study of 20 participants found users prefer agents to automatically resolve low-stakes UI ambiguity like navigation choices, but to pause and request confirmation before high-stakes actions like payments. Stakes, not ambiguity itself, determined user tolerance for autonomous agent behavior.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Three research programs show LLMs must learn timing as a core skill: DiscussLLM trains silent tokens, Inner Thoughts creates covert reasoning about contribution value, and emotional support contexts require domain-specific initiative models. Humans use continuous internal assessment; AI currently lacks this.
Reinforcement learning training increased proactive critical thinking accuracy from 0.15% to 73.98% on deliberately flawed math problems. Notably, inference-time scaling degraded this ability in untrained models but improved it after RL training, suggesting the capability is learnable but fragile without explicit training.
Show all 11 sources
STaR-GATE iteratively finetunes a model on questions that increase response quality, achieving 72% preference over the base model after two iterations. The research shows preference elicitation is trainable through self-play without human question supervision.
Models trained via SML on complete problems generalize to underspecified tasks by asking for needed information and delaying answers. The training paradigm instills a meta-strategy of using conversation as an information source, addressing the premature-answering failure mode.
Clarifying questions that target concrete information gaps ("What type of monitor?") consistently beat those that ask users to rephrase their needs ("What are you trying to do?"). Users engage most when they can foresee how answering improves results.
UoT combines uncertainty-aware scenario simulation with information-gain scoring and reward propagation to identify questions whose possible answers maximally reduce diagnostic uncertainty—providing a principled mechanism for specific, high-value clarification rather than generic prompts.
The ALFA framework breaks down question quality into theory-grounded attributes (clarity, relevance, specificity) and trains models on 80K attribute-specific preference pairs. Attribute-specific optimization outperforms single-score training, especially in clinical reasoning where asking the right clarifying question directly impacts decision quality.
PleaSQLarify reduced clarification steps to two to eight turns by ranking ambiguities by information gain and letting users explore alternatives visually. Twelve participants used multiple repair strategies, suggesting interfaces should support flexible disambiguation rather than forcing one path.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?
- STaR-GATE: Teaching Language Models to Ask Clarifying Questions
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- Learning to Learn from Language Feedback with Social Meta-Learning
- Proactive Conversational Agents in the Post-ChatGPT World
- DiscussLLM: Teaching Large Language Models When to Speak