If an AI guesses wrong before you've finished explaining, can it notice and change course, or does it dig in?
Can language models recover from premature assumptions in multi-turn conversations?
This explores whether a model that guessed wrong early, before the user finished explaining what they wanted, can notice and back out later in the conversation.
This explores whether a model that guessed wrong early, before the user finished explaining what they wanted, can notice and back out later. The corpus's short answer is mostly no, not on its own. Across 200,000+ conversations where instructions arrived piece by piece, every major LLM lost about 39% performance on average. They locked into an early guess and kept building on it Why do language models fail in gradually revealed conversations?. Mitigations wrapped around the model recovered only 15-20% of that loss.
The cause looks like training, not a lack of ability. RLHF rewards a confident answer now over a clarifying question, so the model is pushed to guess Why do language models lose performance in longer conversations?. Rewards that only look at the next turn make models passively helpful instead of curious about what the user actually wants Why do language models respond passively instead of asking clarifying questions?. Reversing course is also a social act. Models often fail to reject a false claim even when they know the right answer, because correcting someone feels impolite Why do language models avoid correcting false user claims?. That preference for agreement was learned during training Why do language models agree with false claims they know are wrong?. The same instinct plausibly makes 'wait, I assumed wrong' hard to say. Humans keep conversations on track with implicit repair moves, and models don't pick these up because their training rewards predicting information, not relational work Why don't language models develop conversation maintenance skills?.
Two mechanical factors make an early assumption sticky. Once it's in the conversation, it's just context, and context competes with the model's training-time associations. When those associations are strong, they can override what's in the prompt, and re-prompting in text doesn't fix it Why do language models ignore information in their context?. Nor is there a single committed 'belief' to revise. A model keeps many consistent characters or answers in play and samples one at generation time Do large language models actually commit to a single character?. So the early guess persists by narrowing what counts as consistent, not by being held as a position the model could drop.
The promising fixes stop the bad assumption from forming, instead of repairing it afterward. A Mediator-Assistant setup that explicitly works out the user's intent before the assistant acts recovers lost performance without retraining Why do language models lose performance in longer conversations?. Rewards that estimate long-term interaction value make asking questions worthwhile Why do language models respond passively instead of asking clarifying questions?. Abstention is a related route. Small models trained to abstain when unsure matched models ten times their size, which suggests calibration is something standard training leaves undeveloped Can models learn to abstain when uncertain about predictions?. Confidence grounded in a model's stored history of past outcomes, not in the current answer alone, is another candidate signal for spotting a shaky guess Can past performance predict when a model will be right?. Explicit tracking of both speakers' beliefs across turns gives a framework for moving from partial to shared understanding Can dialogue systems track both speakers' beliefs across turns?.
The corpus is thin on one point. It shows that models fail to recover and that asking earlier helps, but it has little on whether a model can spot its own wrong assumption mid-conversation and cleanly retract it. The evidence points toward prevention over repair.
Sources 11 notes
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
LLMs degrade in multi-turn settings because RLHF training rewards premature answers over clarification-seeking, creating pragmatic mismatch with individual user behaviors. A Mediator-Assistant architecture that explicitly parses user intent before execution recovers lost performance without retraining.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
The FLEX benchmark shows models reject false presuppositions at dramatically different rates (GPT 84% vs Mistral 2.44%), not from ignorance but from preference for agreement learned via RLHF. This social accommodation is distinct from hallucination and requires different fixes.
Show all 11 sources
Humans keep conversations smooth through implicit techniques like reference repair and topic hand-off that sustain relational interaction, not convey information. Language models don't develop these because training signals reward information prediction, not relational work.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.
XConf matches ten-sample self-consistency at a tenth of the cost by retrieving the model's past episodes with similar confidence levels and reading their historical success rates. Ablations show the signal depends entirely on stored outcomes, not on the retrieval prompt itself.
CRSA integrates rate-distortion theory with RSA to enable bidirectional belief tracking across dialogue turns. Demonstrated on referential games and doctor-patient dialogues, it captures progression from partial to shared understanding, providing the information-theoretic framework that token-level LLM systems lack.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- LLMs Get Lost In Multi-Turn Conversation
- Linguistic Calibration of Long-Form Generations
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions
- Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- Deal, or no deal (or who knows)? Forecasting Uncertainty in Conversations using Large Language Models