If you lean on a chatbot for advice or comfort, what happens to that bond when the company updates it?
Do model updates disrupt established sources of support for regular chatbot users?
This explores whether the routines people build around a general-purpose chatbot, including leaning on it for emotional support, get broken when the company updates the model underneath.
This explores whether the routines people build around a general-purpose chatbot, including leaning on it for emotional support, get broken when the company updates the model underneath. The corpus says yes, but the direct evidence is thin. An 18-month study followed people who started using ChatGPT, Claude and Gemini for practical tasks and gradually came to rely on them emotionally. The routines they built were later disrupted by model updates, by public AI-harm discourse, and by ordinary life changes How does emotional chatbot use develop from practical use?. So model updates are one of several destabilizers, and the support role was never fixed. It shifted with the person, the tool and the news cycle.
Several other notes suggest why an update could hurt so much. Longitudinal work on personalization finds that it raises trust and human-likeness at the same time, and that each good interaction raises the baseline, so later failures disappoint more Does chatbot personalization build trust or expose privacy risks?. An update that changes tone or forgets what the user shared lands when expectations are highest. Separately, the "Assistant" personality is a trained artifact. Post-training only loosely tethers a model to one dominant persona direction, and emotional conversations already push it off that direction How stable is the trained Assistant personality in language models?. That paper studies drift within one model, not across versions. Applying it here is my inference, but it implies a new version is a different point in persona space, not a patch to the same companion.
Voice matters too. Chatbots use language that signals expertise, and trust attaches to the register of the answer more than to its accuracy Does chatbot language style actually shape how much we trust it?. If so, an update that changes how the model sounds could break the felt reliability of the support, even if the new model is measurably better.
Not every drop-off should be blamed on the update. Longitudinal studies of chatbot relationships show that the social processes driving attachment fade predictably as novelty wears off Do chatbot relationships lose their appeal as novelty wears off?. A user who disengages after an update may have been drifting anyway. This is probably why the 18-month study lists several causes instead of one.
The design stakes are rising. Analysis of 120 chatbots found three archetypes: ad-hoc supporters, temporary assistants, and persistent companions. Each needs a different design because the time horizon changes what the chatbot is to the user How should chatbot design vary by relationship duration?. General-purpose chatbots are mostly built as the first two, yet people use them as the third. Mental-health AI is heading toward stateful companions with memory and planning How are LLMs evolving their roles in mental health support?. The more continuity a system promises, the more an update that breaks it costs. The corpus has no study measuring wellbeing after a specific update, and none on how providers should handle version changes for people who rely on them.
Sources 7 notes
An 18-month longitudinal study found that users developed emotional reliance on ChatGPT, Claude and Gemini through practical use, then established routines that were disrupted by model updates, AI-harm discourse, and life changes. The support role was not fixed but evolved with the person, the tool, and public conversation.
Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.
Research mapping hundreds of character archetypes reveals a low-dimensional persona space where the leading component measures distance from the default Assistant. Emotional and meta-reflective conversations cause predictable drift, but activation capping along this axis mitigates harmful shifts without degrading capabilities.
Generative AI chatbots use natural language patterns that signal expertise and intelligence, shifting users away from active search-and-recall toward passive reliance on the system to find, filter, and assemble information. Trust attaches to the register of the answer rather than its accuracy.
Longitudinal studies with Mitsuku show that social processes driving relationship formation decline as novelty wears off. Single-session study findings cannot be reliably extrapolated to medium- or long-term chatbot design.
Show all 7 sources
Analysis of 120 chatbots reveals three archetypes—ad-hoc supporters, temporary assistants, and persistent companions—each requiring fundamentally different designs. Time horizon is the primary differentiator between treating chatbots as communication tools versus social actors.
A survey identifies three evolving roles: risk detection tools, stateless empathetic dialogue, and longitudinal personalized agents with memory and planning. However, fully autonomous clinically valid systems remain incomplete, with foundational barriers beyond technical capability.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- CompanionSim: Synthetic Data for Evaluating Anthropomorphism in Human-AI Relationships
- Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot’s Self-Disclosure in Conversational Recommendations
- "I Felt Very Seen, But Still Very Alone": Longitudinal Trajectories of General-Purpose LLM Use for Socioemotional Support
- See you soon again, chatbot? A design taxonomy to characterize user-chatbot relationships with different time horizons
- From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents
- Can LLMs identify and repair ruptures? Comparison between clinician practices and LLM behaviors
- Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
- The Addictive Intimacy of AI: Understanding User Disengagement from AI Companions and Why Some Relationships with AI Become Difficult to Leave