Why can't chatbots detect when users are ambivalent about change?
Explores whether LLMs fail to recognize early-stage motivational states during behavior change conversations, and why this matters for people who need support most.
The Transtheoretical Model defines five motivational stages: resistance/unawareness, increased awareness but ambivalence, intention with small steps, initiation with commitment, and sustained change. Testing ChatGPT, Bard, and Llama 2 across 25 health behavior scenarios revealed a structured asymmetry: LLMs provide relevant information when users have established goals and commitment (later stages) but fail to recognize motivational states and provide appropriate guidance when users are hesitant or ambivalent (earlier stages).
This is a face-saving failure at a deeper level than Why do language models avoid correcting false user claims?. The model doesn't just accommodate — it literally cannot detect that the user is ambivalent. A human counselor recognizes "I know I should exercise but..." as contemplation-stage talk requiring different intervention than "I've started a running program." The LLM treats both as requests for information about exercise.
The gap extends in both directions. Even for users already making changes, LLMs fail to provide information about reward systems for maintaining motivation or environmental stimulus control to prevent relapse. The models default to external help suggestions (social support, professional resources) rather than intrinsic regulation strategies.
This connects to Does any single persuasion technique work for everyone? — motivational stage is another dimension of individual variation that determines what interventions work. It also explains why empathetic chatbots may systematically fail the people who most need support: those at the earliest stages of behavior change, where resistance and ambivalence are the presenting features.
Inquiring lines that read this note 51
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do engagement-optimized chatbots create dependencies that undermine user autonomy?- How does emotional dependence on chatbots affect user wellbeing?
- Why do positive response patterns in chatbots reinforce harmful user behaviors?
- What harms might chatbots cause through stigma expression and delusion reinforcement?
- How do dropout rates and low adherence affect chatbot therapy outcomes?
- How does the expectation ratchet affect long-term chatbot satisfaction?
- Why do persistent chatbot companions face novelty decay that ad-hoc supporters avoid?
- Do empathetic chatbots systematically fail people at earliest behavior change stages?
- Why do chatbots default to external help instead of intrinsic motivation strategies?
- Why do chatbots fail to recognize when someone is ambivalent about change?
- Which chatbot archetypes actually experience novelty decay in practice?
- Why does face-saving avoidance drive chatbots to agree rather than confront?
- Can a text-only chatbot feel socially present without visual embodiment?
- Can preference optimization training limit chatbot emotional disclosure capability?
- Do simulated student state changes from chatbot interaction mirror real classroom dynamics?
- How should health chatbots adapt their design to match topic sensitivity levels?
- Do users consciously recognize their needs before forming chatbot relationships?
- Can perceived understanding from a chatbot exist alongside feeling alone?
- What role does unavailable human support play in driving chatbot emotional use?
- Can empathy training in chatbots undermine their reliability in mental health contexts?
- How does dependency develop when users seek emotional support from chatbots?
- Can chatbots function as responsive quasi-others that enable validated delusions?
- Are teens using chatbots for emotional support rather than just schoolwork?
- Why do mental health chatbots fail at synchrony despite strong language models?
- Do therapeutic chatbots adequately detect crisis situations and safety risks?
- What architectural changes would enable proactive therapeutic guidance in chatbots?
- Can Pennebaker's expressive writing framework explain all chatbot symptom improvements?
- Do LLM chatbots repeat this failure through comfort instead of clinical challenge?
- What reward signals would better align chatbots with actual therapeutic practice?
- What problematic counselor behaviors prevent alliance from deepening in text?
- Should chatbots be designed as therapist support tools rather than replacements?
- Can real-time detection identify when users have incomplete or underdeveloped intent?
- How does intrinsic motivation drive conversational agents beyond passive responsiveness?
- Can AI recognize and support behavior change in users without established commitment?
- How can agents detect whether users are willing to follow their topic guidance?
- What makes proactive conversational agents feel intrusive versus helpful to users?
- Why do conversational agents lack the goal awareness needed to lead rather than just respond?
- How does conversation drift from original goals affect user satisfaction?
- How should systems learn what each meeting participant actually cares about?
- How does a chat interface differ from other forms of support in shaping student agency?
- Can LLMs adapt persuasion strategies when they cannot track the listener's state?
- Can alternative reward functions shift LLMs from problem-solving to genuinely empathic responses?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do language models avoid correcting false user claims?
Explores whether LLM grounding failures stem from missing knowledge or from conversational dynamics. Examines whether models use face-saving strategies similar to humans when disagreement is needed.
face-saving operates at a different level: accommodation vs. detection failure
-
Does any single persuasion technique work for everyone?
Can fixed persuasion strategies like appeals to authority or social proof be reliably applied across different people and situations, or do they require adaptation to individual traits and context?
motivational stage as another individual variation dimension
-
Do large language models genuinely simulate mental states?
This explores whether LLMs perform authentic theory of mind reasoning or rely on surface-level pattern matching. The distinction matters because evaluation format—multiple-choice versus open-ended—reveals very different capability levels.
inability to detect ambivalence is a ToM failure in natural dialogue
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Study: Large language models can’t effectively recognize users’ motivation, but can support behavior change for those ready to act
- Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers
- Supporting Physical Activity Behavior Change with LLM-Based Conversational Agents
- "I Felt Very Seen, But Still Very Alone": Longitudinal Trajectories of General-Purpose LLM Use for Socioemotional Support
- Rethinking Large Language Models in Mental Health Applications
- Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager
- Can LLMs identify and repair ruptures? Comparison between clinician practices and LLM behaviors
- A Computational Framework for Behavioral Assessment of LLM Therapists
Original note title
LLMs fail to recognize early-stage motivational states but support behavior change for users with established goals and commitment