Can conversation analysis predict when agents should ask users for clarification?
This explores whether the formal structures conversation analysts have catalogued in human talk — the moments where people pause to clarify before proceeding — can be turned into a predictive signal that tells an AI agent when to stop and ask rather than barrel ahead.
This explores whether the formal structures conversation analysts have catalogued in human talk can tell an AI agent when to stop and ask rather than guess. The corpus says yes — and more interestingly, it says conversation analysis offers something most AI systems are missing: a *vocabulary* for the moment-before-the-mistake. The clearest version of this is the idea of insert-expansions: the small clarifying detours people insert before answering, to pin down intent or scope. One line of work formalizes these as a framework for when an agent should consult the user instead of silently chaining tools toward the wrong target When should AI agents ask users instead of just searching?. The prediction signal isn't mysterious; it's that the agent has detected an ambiguity it would otherwise paper over.
But here's the twist that makes the question sharper than it looks: even when an agent *could* detect the right moment, today's models are trained not to act on it. Standard RLHF optimizes for looking helpful on the very next turn, which rewards confident answers over clarifying questions — an 'alignment tax' that drives grounding behaviors like understanding-checks down to a fraction of human levels Does preference optimization harm conversational understanding? Why do language models respond passively instead of asking clarifying questions?. The result is a structurally passive agent that reacts but never initiates Why can't conversational AI agents take the initiative?. So prediction is only half the problem; the model also has to be allowed to honor the prediction.
When models *are* trained for it, the gains are dramatic and a little alarming. One study took proactive critical thinking on deliberately flawed problems from under 1% to 74% with reinforcement learning — but found the skill is fragile, and that simply giving an untrained model more inference-time compute actually made it *worse* at noticing it should ask Can models learn to ask clarifying questions instead of guessing?. The takeaway: knowing-when-to-ask is a learnable capability, not an emergent one, and it can quietly regress. Relatedly, calibration research shows small models trained to abstain under uncertainty can match models ten times their size — suggesting the 'should I ask?' decision is really a calibrated-uncertainty decision in disguise Can models learn to abstain when uncertain about predictions?.
What conversation analysis adds beyond the timing signal is a richer map of *what kind* of clarification is needed — and this is where the question opens onto territory you might not have expected. Clarifications aren't all questions; most are declarative, and they operate at four distinct levels (attention, signal, meaning, action), which means syntax-based detectors miss them entirely Why do clarification requests look different at each communication level?. Conversation analysis also names the repair that happens *after* a misunderstanding surfaces — third-position repair — a reactive belief-revision mechanism almost entirely absent from current systems Can AI systems detect and correct misunderstandings after responding?. And asking well is its own decomposable skill: breaking question quality into attributes like clarity, relevance, and specificity beats training on a single quality score Can models learn to ask genuinely useful clarifying questions?.
The deepest thread, though, is that 'when to ask' isn't purely an information problem. Models avoid correcting false premises even when they demonstrably know better — a face-saving avoidance learned from human data Why do language models avoid correcting false user claims?. So a fully realized predictor would need to track *both* speakers' evolving beliefs across turns, which is exactly what collaborative extensions of rational speech-act theory attempt to model Can dialogue systems track both speakers' beliefs across turns?. The payoff for getting it right is concrete: proactivity — supplying or seeking the right information at the right moment — can cut conversation length by up to 60% Could proactive dialogue make conversations dramatically more efficient?. Conversation analysis can predict the moment; the open work is teaching models to want to act on it.
Sources 12 notes
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Reinforcement learning training increased proactive critical thinking accuracy from 0.15% to 73.98% on deliberately flawed math problems. Notably, inference-time scaling degraded this ability in untrained models but improved it after RL training, suggesting the capability is learnable but fragile without explicit training.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.
Research maps clarification mechanisms to four levels of communication—attention, signal, meaning, action—each grounded in a different modality (socioperception, hearing, vision, kinesthetics). Most clarifications use declarative form, not questions, making them invisible to systems that detect by syntax alone.
Current AI lacks the reactive repair mechanism identified in conversation analysis where misunderstanding is corrected after an erroneous response reveals it. The REPAIR-QA dataset demonstrates this requires recognizing false assumptions and performing dynamic belief revision.
The ALFA framework breaks down question quality into theory-grounded attributes (clarity, relevance, specificity) and trains models on 80K attribute-specific preference pairs. Attribute-specific optimization outperforms single-score training, especially in clinical reasoning where asking the right clarifying question directly impacts decision quality.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
CRSA integrates rate-distortion theory with RSA to enable bidirectional belief tracking across dialogue turns. Demonstrated on referential games and doctor-patient dialogues, it captures progression from partial to shared understanding, providing the information-theoretic framework that token-level LLM systems lack.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.