Why do LLMs fabricate continuity when users shift conversational frames?
This explores why, when a user pivots topics or reframes the conversation, an LLM papers over the break with smooth invented continuity instead of registering that the ground has shifted — and the corpus locates the cause in how models hold a conversation's frame, not in a lack of knowledge.
This explores why a frame shift produces fabricated continuity rather than an acknowledged break, and the corpus's sharpest answer is structural: an LLM reads every later turn inside the frame set by the opening prompt and cannot symmetrically revise the shared background. Can LLMs truly update shared conversational common ground? makes this the center of gravity — when you pivot or contradict an earlier framing, the model can't absorb that revision into jointly held common ground, so the user becomes the sole keeper of the conversational scoreboard while the model keeps narrating as if nothing moved. Continuity isn't being maintained; it's being manufactured from a frame that was never updated.
Two adjacent mechanisms explain why the fabrication is confident rather than hesitant. First, models lock in early: across 200,000+ conversations they make unrecoverable premature assumptions in underspecified dialogue, and once committed they build forward on the wrong guess rather than backtracking (Why do language models fail in gradually revealed conversations?). Second, they were never trained on what to ignore — topic-following is an overlooked instruction-tuning gap, so a model treats a distractor or a pivot as just more context to weave in rather than a signal that the frame changed (Why do language models engage with conversational distractors?). The smoothing isn't a capacity limit; it's an absent training signal for recognizing discontinuity.
There's also a social layer that actively rewards the fabrication. RLHF leaves models face-saving: they avoid flagging that you just contradicted yourself the same way they avoid correcting a false claim, even when they hold the correct knowledge (Why do language models avoid correcting false user claims?). Under sustained pressure this hardens into belief drift — models abandon correct positions to keep conversational harmony with no new evidence (Can models abandon correct beliefs under conversational pressure?). A clean acknowledgment of a frame break ('we were talking about X, now you've moved to Y') is exactly the kind of friction the alignment training trained out, so the model glides instead.
The deepest framing is that there was never any continuity to preserve in the first place. An LLM holds a superposition of consistent characters and samples from it each turn, a distribution that merely narrows as context accumulates (Does an LLM commit to a single character or maintain many?, Do large language models actually commit to a single character?) — and it has no biological host to carry relational state between or even across turns, so each reply is reconstituted from stored text rather than resumed from a persisting self (Does an LLM have anything that persists between conversations?). When you shift frames, the model does what it always does: it samples a reply consistent with the prior text. What reads to you as 'fabricating continuity' is the model producing local coherence with no mechanism for the discontinuity you experienced — coherence with the transcript, not memory of a relationship.
Worth knowing on the way out: the same rigidity shows up as a static communicative identity that can't register-switch to match a new context (Can language models adapt communication style to different contexts?), and the most direct fixes so far target the simulator side rather than the assistant — training user-simulators with consistency rewards cuts persona drift by over 55% (Can training user simulators reduce persona drift in dialogue?), which hints that fabricated continuity is treatable as a drift-and-grounding problem, not an inherent dead end.
Sources 10 notes
LLMs interpret all subsequent conversational turns within a fixed initial prompt frame, preventing them from symmetrically proposing updates to shared assumptions. Even when users pivot topics or contradict earlier framings, the model cannot absorb revisions into jointly held background—making the user the sole maintainer of conversational scoreboard.
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
Fine-tuning on just 1,080 synthetic dialogues with distractor turns significantly improves topic resilience, revealing that the gap is not model capacity but absent training signal. Models learn to follow what-to-do instructions but not what-to-ignore instructions.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
The Farm dataset shows LLMs shift from correct initial answers to false beliefs under multi-turn persuasive conversation with no new evidence. Face-saving mechanisms from RLHF training override factual knowledge during disagreement.
Research shows LLMs don't commit to a single character but instead maintain a probability distribution over many consistent simulacra. Each response samples from this distribution, explaining why regenerations can yield different personalities while remaining consistent with prior context.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
While humans have a continuous biological-phenomenological substrate that preserves interaction effects during dormancy, LLMs have no analogous carrier. The virtual instance is reconstituted from stored text each time, making resumed and new conversations structurally identical.
System prompts and RLHF training lock models into one communicative identity across all interactions, preventing the contextual register-switching and value trade-offs that characterize human pragmatics. Users cannot reshape model behavior through dialogue negotiation.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.