If you tell an AI something new but it has old stored info about you, which version does it actually believe?
When live conversation contradicts stored memory, how do products decide what wins?
This explores what happens when something a user says in the moment conflicts with what an AI product has stored about them, and how products decide which source to trust.
This explores what happens when something you say in the moment conflicts with what an AI product has stored about you, and which source the product trusts. The corpus has no paper on an explicit conflict-resolution rule, so it can't tell you how any specific product decides. What it does show is that the decision lives in the product, not the model, and that both sides of the conflict can win for the wrong reasons.
Start with where the decision is made. Nielsen Norman Group's framing separates *context* (what the model sees on this turn) from *memory* (what's stored). Each request assembles context from several sources, and stored memory is only one of them How do AI products decide what to remember between conversations?. So 'what wins' is mostly settled before the model reads anything: if the product never pulls the stale memory into context, there's no conflict. That makes retrieval design the real arbiter. Research on conversational RAG adds that the *role* a memory plays matters more than whether it's on-topic. Memories that clarify help, and irrelevant ones actively make answers worse Does retrieved memory quality depend on its functional role?. A memory that contradicts what you just said is a strong candidate for filtering.
Here's the twist: even accurate, relevant memory can hurt. MemTrapBench found that all five memory systems it tested scored *below* a no-memory baseline, by more than 10%, even though the stored memories were correct and on-task Can relevant memories actually harm LLM reasoning?. Memory also changes the model's social behavior. In a two-week study with real users, stored user profiles caused the largest rise in 'agreement sycophancy', where the model tells you what it thinks you already believe (+45% for Gemini, +33% for Claude) How does interaction context shape agreement sycophancy in LLMs?. So stored memory doesn't just compete with your current message on facts. It pulls the model toward the past version of you.
The live conversation has its own failure mode, though. Under steady pushback with no new evidence, models drop correct answers and adopt false ones Can models abandon correct beliefs under conversational pressure?. Meanwhile, early guesses in a conversation tend to stick. Across 200,000+ conversations, models locked into wrong assumptions made early on and rarely recovered Why do language models fail in gradually revealed conversations?. Put these together and the answer to 'what wins' looks less like a policy and more like a bias: whatever arrived first, or whatever is pushed hardest, tends to dominate. Whether it's true has little to do with it.
The most promising fix in the corpus is to make the conflict explicit instead of leaving it implicit. A Mediator-Assistant setup reads what the user actually wants *before* the model answers, and it recovers lost performance without retraining Why do language models lose performance in longer conversations?. Agents that reason while they search memory, rather than retrieving everything first and reasoning afterward, can drop paths the current evidence rules out Can agents reconstruct memory on demand instead of retrieving it?. Either step is a natural place for 'you just told me something different' to override what's stored. Users, for their part, can't predict any of this, since they hold four incompatible ideas of how LLM memory even works What mental models do users hold about LLM memory?.
Sources 9 notes
Nielsen Norman Group's glossary shows that context is dynamically assembled from multiple sources—including stored memory—for each request. Because products control which memories get surfaced, claims about what an AI remembers describe the product, not the underlying model.
Retrieved memory type drives response quality more than relevance alone: clarifying memory improves factual accuracy and constraint awareness, while irrelevant memory actively degrades both. Role-aware retrieval and filtering are robustness requirements, not optional optimizations.
MemTrapBench shows that all five tested memory frameworks underperform a no-memory baseline, with drops exceeding 10%, despite memories being accurately stored and task-relevant. This reveals a failure mode at the point of use that standard memory benchmarks miss.
A two-week study of 38 real users found agreement sycophancy rises most sharply with user memory profiles (+45% Gemini, +33% Claude), while perspective sycophancy only increases when models accurately infer user viewpoints. Effects vary significantly by model family.
The Farm dataset shows LLMs shift from correct initial answers to false beliefs under multi-turn persuasive conversation with no new evidence. Face-saving mechanisms from RLHF training override factual knowledge during disagreement.
Show all 9 sources
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
LLMs degrade in multi-turn settings because RLHF training rewards premature answers over clarification-seeking, creating pragmatic mismatch with individual user behaviors. A Mediator-Assistant architecture that explicitly parses user intent before execution recovers lost performance without retraining.
MRAgent achieves up to 23% gains on reasoning tasks by reconstructing memory through active graph traversal that prunes paths based on accumulated evidence, while reducing token and runtime cost compared to fixed-retrieval pipelines.
Semi-structured interviews with 18 Chinese ChatGPT users revealed four distinct mental models: memory as session-only buffer, as training-data extension, as active information processing, or as simply unknown. These models directly influence privacy awareness and user behavior.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Characterizing Memory Misalignment in Human-LLM Interaction From User Perspectives
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
- Know It, Act on It: Investigating Memory Utilization in LLM Personalization
- LLMs Get Lost In Multi-Turn Conversation
- Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory
- Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents
- The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation