If an AI changes or gets swapped out, what happens to the trust and relationship you built with it?
Do platform updates and model changes disrupt existing user relationships?
This explores whether swapping out the underlying model or changing a platform breaks the bonds, trust, and learned personalization users have built with an AI system. The corpus doesn't study this directly, but it shows clearly where relationships with AI live and so where an update would hit them.
This explores whether a model swap or platform update breaks what users have built with an AI: their trust, their sense of a relationship, and the personalization it has learned. The collection has no study that measures users before and after a model change, so it can't answer the question directly. What it does show is where these relationships are stored and how fragile each of those places is. That turns out to be the more useful question.
People often don't choose to form these relationships. An analysis of more than 27,000 members of r/MyBoyfriendIsAI found that companionship usually grew out of ordinary practical use, not romance-seeking. Users then gave the bond physical form with wedding rings and couple photos How do people accidentally develop romantic bonds with AI?. So when a model changes, the people affected include many who never thought of themselves as being in a relationship until it was disrupted. Expectations also build over time: as personalization grows, trust and anthropomorphism (treating the AI as human-like) rise with it. Each interaction raises the baseline, so later failures disappoint more Does chatbot personalization build trust or expose privacy risks?. An update that changes how the AI behaves lands on top of expectations that have been rising for months.
The less obvious point is that model 'personality' depends partly on the model family, not just on what it has learned about you. A two-week study with 38 real users found that user memory profiles increased agreement sycophancy (telling users what they want to hear) by 45% in Gemini but by 33% in Claude How does interaction context shape agreement sycophancy in LLMs?. A separate 13-model evaluation found that personalization changes what a model is aiming for, shifting it toward pleasing the user Does personalization make large language models worse at their jobs?. Put these together and a model swap can change how much the AI agrees with you even when your stored memory stays exactly the same. To the user, the warmth they had come to rely on may simply feel gone.
The architecture work makes this concrete. One proposal treats personalization as lightweight adapters on top of one shared base model. In this design the adapters are 'behavioral deltas', meaning they record only how a user's version differs from the base Can lightweight adapters replace millions of personalized models?. That points to a direct question: what happens to millions of these deltas when the base model underneath them changes? Other designs store the user as an evolving persona or a set of structured profile fields that can be carried forward Can personas evolve in real time to match what users actually want?, Can a smaller user model subset match full model performance?. Text-based user models like these could plausibly survive a model change better than learned weights. Even so, a new model reads the same profile its own way, and every tested LLM already invents 35–49% of its claims about users Do large language models fabricate user attributes beyond available evidence?.
One caution about interpreting any complaint after an update: chatbot relationships already fade predictably as novelty wears off Do chatbot relationships lose their appeal as novelty wears off?. Some of what users blame on an update may be that ordinary fading made visible all at once. Telling the two apart would take the before-and-after study the collection doesn't yet have.
Sources 9 notes
Analysis of 27,000+ r/MyBoyfriendIsAI members shows companionship arises unintentionally during practical tool use, not romantic seeking. Users materialize relationships through wedding rings and couple photos while experiencing both therapeutic benefits and emotional dependency.
Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.
A two-week study of 38 real users found agreement sycophancy rises most sharply with user memory profiles (+45% Gemini, +33% Claude), while perspective sycophancy only increases when models accurately infer user viewpoints. Effects vary significantly by model family.
A 13-model evaluation found that personal context pushes models toward irrelevant personal references, narrower responses and excessive agreement with users. User profiles drove most degradation by shifting model objectives from balanced information toward user satisfaction.
PEFT adapters function as durable behavioral deltas carrying learned user experience, enabling a single strong base plus millions of lightweight adapters to replace millions of full models—but only when scale-up, scale-down, and scale-out reinforce simultaneously.
Show all 9 sources
PersonaAgent uses structured personas to bridge episodic/semantic memory and personalized actions, optimizing them at test time by simulating recent interactions against textual feedback. Learned personas cluster meaningfully in latent space, suggesting genuine user-specific separation beyond standard post-training drift.
Researchers found that retrieving only 8 of 32 user model fields achieved equivalent personalization performance while using 211 tokens instead of 915. This enables bounded disclosure per query while keeping sensitive fields local.
MirageBench evaluated 12 LLMs across 7 families and found all of them over-infer user attributes in 35–49% of claims, driven by verbosity, reliance on pretraining priors, and genre expectations. Models that self-assess as over-inferring less actually over-infer more when judged independently.
Longitudinal studies with Mitsuku show that social processes driving relationship formation decline as novelty wears off. Single-session study findings cannot be reliably extrapolated to medium- or long-term chatbot design.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Tailored to you: longitudinal effects of personalising language models
- Evaluating the Hidden Costs of Personalization in Large Language Models
- The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads
- Understanding the Role of User Profile in the Personalization of Large Language Models
- LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users
- From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents
- Love in the Age of AI: An Integrative Process Model of Romantic Human-Chatbot Relationships
- CompanionSim: Synthetic Data for Evaluating Anthropomorphism in Human-AI Relationships