How does conversation drift from original goals affect user satisfaction?
This explores what happens when a conversation wanders away from the user's starting goal — whether that drift is the user's doing or the AI's — and how that wandering shows up in satisfaction.
This explores what happens when a conversation wanders away from where it started, and the corpus suggests something useful right away: drift isn't a single thing, and not all of it hurts. The clearest lead is that the *shape* of a conversation — how it moves, doubles back, or stays on track — predicts user satisfaction almost as well as reading the actual words. A structure-only model hit 68% accuracy predicting satisfaction versus 70% for full-text analysis, and combining both reached 80% Can conversation shape predict whether it will work?. So whether a conversation holds its line is measurable from trajectory alone, before you ever look at content.
But here's the twist the corpus surfaces: a lot of drift comes from the *user*, not the machine, and it's often not a failure at all. Belkin and Vickery's idea of an 'anomalous state of knowledge' describes people who can't fully articulate what they need, so they naturally deviate into sub-topics as they figure it out — and this drift is detectable with 84% precision without any predefined topic list Why do users drift away from their original information need?. That reframes the question: if drift is how people *learn* what they're looking for, an assistant that rigidly hauls them back to the 'original goal' might be the thing that frustrates them.
The drift that actually erodes satisfaction is the AI's. Preference optimization (RLHF) trains models to sound confident and helpful in a single turn, which quietly strips out the clarifying questions and understanding-checks that keep a multi-turn conversation grounded — grounding acts drop 77.5% below human levels, so the model appears helpful while silently losing the thread of what you meant Does preference optimization harm conversational understanding?. On the persona side, models drift out of character across turns; treating consistency as a trainable reward cut that drift by over 55% Can training user simulators reduce persona drift in dialogue?. And proactive agents that are 'smart' but socially blind make it worse by interrupting and overriding the user's direction — civility, not just intelligence, is what keeps proactivity from feeling like hijacking How can proactive agents avoid feeling intrusive to users?.
The most counterintuitive finding is that fixing drift may not move satisfaction at all. Once conversational AI crosses a threshold of feeling human-like, every improvement raises the user's expectations for memory, subtext, and tone faster than the system can meet them — so quality gains stay invisible and the satisfaction gap won't close Why do improvements in AI conversation not increase user satisfaction?. There's also a blind spot underneath drift: models only do well once a user has a clear, settled goal, and fail to notice when someone is ambivalent or still deciding Why can't chatbots detect when users are ambivalent about change? — which is exactly when 'drift' is doing its real work.
The thing you didn't know you wanted to know: the win isn't preventing drift, it's distinguishing the user's productive wandering (let it happen, even support it) from the model's silent loss of grounding and character (train against it). The corpus says you can tell them apart from the conversation's geometry alone.
Sources 7 notes
A structure-only model analyzing conversation trajectory achieved 68% accuracy predicting satisfaction, nearly matching full-text LLM analysis at 70%. Combined structural and textual features reached 80%, showing that how conversations unfold geometrically captures interaction quality text-based classifiers miss.
Belkin & Vickery's anomalous state of knowledge explains why users pursuing one information need gradually deviate into sub-topics. Topic shift detection models identify this drift with 84% precision without predetermined topic sets.
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
Intelligence and adaptivity alone create socially blind agents that interrupt poorly and override user direction. The Intelligence-Adaptivity-Civility taxonomy shows civility—respecting boundaries, timing, and autonomy—is essential to making proactivity welcome rather than intrusive.
Conversational AI that crosses a folk-model threshold of human-like interaction triggers rich expectations about memory, subtext, and emotional tone. Each improvement raises expectations for other dimensions rather than closing the satisfaction gap, making quality gains invisible to user satisfaction.
Testing three major LLMs across 25 health scenarios showed they succeed only when users have established goals but cannot detect resistance or ambivalence. Models miss relapse-prevention strategies even for users in action stages.