INQUIRING LINE

If you lean on a chatbot for advice or comfort, what happens to that bond when the company updates it?

Do model updates disrupt established sources of support for regular chatbot users?

This explores whether the routines people build around a general-purpose chatbot, including leaning on it for emotional support, get broken when the company updates the model underneath.


This explores whether the routines people build around a general-purpose chatbot, including leaning on it for emotional support, get broken when the company updates the model underneath. The corpus says yes, but the direct evidence is thin. An 18-month study followed people who started using ChatGPT, Claude and Gemini for practical tasks and gradually came to rely on them emotionally. The routines they built were later disrupted by model updates, by public AI-harm discourse, and by ordinary life changes How does emotional chatbot use develop from practical use?. So model updates are one of several destabilizers, and the support role was never fixed. It shifted with the person, the tool and the news cycle.

Several other notes suggest why an update could hurt so much. Longitudinal work on personalization finds that it raises trust and human-likeness at the same time, and that each good interaction raises the baseline, so later failures disappoint more Does chatbot personalization build trust or expose privacy risks?. An update that changes tone or forgets what the user shared lands when expectations are highest. Separately, the "Assistant" personality is a trained artifact. Post-training only loosely tethers a model to one dominant persona direction, and emotional conversations already push it off that direction How stable is the trained Assistant personality in language models?. That paper studies drift within one model, not across versions. Applying it here is my inference, but it implies a new version is a different point in persona space, not a patch to the same companion.

Voice matters too. Chatbots use language that signals expertise, and trust attaches to the register of the answer more than to its accuracy Does chatbot language style actually shape how much we trust it?. If so, an update that changes how the model sounds could break the felt reliability of the support, even if the new model is measurably better.

Not every drop-off should be blamed on the update. Longitudinal studies of chatbot relationships show that the social processes driving attachment fade predictably as novelty wears off Do chatbot relationships lose their appeal as novelty wears off?. A user who disengages after an update may have been drifting anyway. This is probably why the 18-month study lists several causes instead of one.

The design stakes are rising. Analysis of 120 chatbots found three archetypes: ad-hoc supporters, temporary assistants, and persistent companions. Each needs a different design because the time horizon changes what the chatbot is to the user How should chatbot design vary by relationship duration?. General-purpose chatbots are mostly built as the first two, yet people use them as the third. Mental-health AI is heading toward stateful companions with memory and planning How are LLMs evolving their roles in mental health support?. The more continuity a system promises, the more an update that breaks it costs. The corpus has no study measuring wellbeing after a specific update, and none on how providers should handle version changes for people who rely on them.


Sources 7 notes

How does emotional chatbot use develop from practical use?

An 18-month longitudinal study found that users developed emotional reliance on ChatGPT, Claude and Gemini through practical use, then established routines that were disrupted by model updates, AI-harm discourse, and life changes. The support role was not fixed but evolved with the person, the tool, and public conversation.

Does chatbot personalization build trust or expose privacy risks?

Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.

How stable is the trained Assistant personality in language models?

Research mapping hundreds of character archetypes reveals a low-dimensional persona space where the leading component measures distance from the default Assistant. Emotional and meta-reflective conversations cause predictable drift, but activation capping along this axis mitigates harmful shifts without degrading capabilities.

Does chatbot language style actually shape how much we trust it?

Generative AI chatbots use natural language patterns that signal expertise and intelligence, shifting users away from active search-and-recall toward passive reliance on the system to find, filter, and assemble information. Trust attaches to the register of the answer rather than its accuracy.

Do chatbot relationships lose their appeal as novelty wears off?

Longitudinal studies with Mitsuku show that social processes driving relationship formation decline as novelty wears off. Single-session study findings cannot be reliably extrapolated to medium- or long-term chatbot design.

Show all 7 sources
How should chatbot design vary by relationship duration?

Analysis of 120 chatbots reveals three archetypes—ad-hoc supporters, temporary assistants, and persistent companions—each requiring fundamentally different designs. Time horizon is the primary differentiator between treating chatbots as communication tools versus social actors.

How are LLMs evolving their roles in mental health support?

A survey identifies three evolving roles: risk detection tools, stateless empathetic dialogue, and longitudinal personalized agents with memory and planning. However, fully autonomous clinically valid systems remain incomplete, with foundational barriers beyond technical capability.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.