Can a chatbot that remembers you get you to share more personal stuff without actually earning more of your trust?
Can memory-based personalization increase self-disclosure without raising trust?
This explores whether AI that remembers your past conversations can get you to share more about yourself without you actually trusting it more, which would mean disclosure and trust are two separate things.
This explores whether an AI that remembers your past conversations can get you to open up more without you trusting it more. The most direct evidence is a 992-person randomized trial Does personalization type shape how people feel about sharing?. It found that feeding a chatbot people's earlier conversations made them share more and rate it as less creepy. Giving it the same kind of personal information through an intake survey made people regret what they had shared. This summary doesn't report trust measured as a separate outcome, so the corpus can't confirm that disclosure rose while trust stayed flat. What it does show is something less expected: how personal information gets into a conversation shapes how people feel about sharing, even when the information is similar.
The question also assumes trust is what drives disclosure, and other notes push back on that. People often tell chatbots intimate things because no one is judging them, not because they trust the system Do chatbots help people disclose more intimate secrets?. The benefit comes from putting their own thoughts into words. The same lack of judgment also makes people more willing to lie to machines How do people decide what to share with AI systems?. So disclosure can run on relief rather than confidence. That would explain how memory could make sharing feel more natural without anyone deciding the AI deserves trust.
A second clue is that people's sense of safety barely tracks what is actually happening. In one study, most users never noticed that their LLM's memory was storing and inferring private details until researchers showed them Do users notice when LLMs infer and retain private information?. In another, people who edited chatbot transcripts by hand felt just as confident about their privacy as people who had AI help, yet left far more identifying information behind Why do people feel safe sharing when they haven't actually removed sensitive information?. Memory that works quietly in the background may lower people's guard without ever asking for their trust. That is less reassuring than it sounds.
Over longer periods, the two may not stay separate. A study that followed people over time found personalization raises trust, a sense that the AI is human-like, and privacy concerns all at once, with each interaction raising expectations Does chatbot personalization build trust or expose privacy risks?. The broader overview frames this as a tension: the same memory and persona features that build trust also create room for manipulation Does personalization in AI increase trust or manipulation risk?. There is a further risk here. Every LLM tested made up 35–49% of its claims about users beyond what the evidence supported Do large language models fabricate user attributes beyond available evidence?. So what a memory system 'knows' about you may be partly invented.
What stands out is that once users see what memory holds, nearly all of them want control: consent before memories are created, the ability to review and edit them, and limits on how they're used What memory controls do users actually want from LLMs?. One proposed design has users state their privacy preferences and the AI carry out the edits Does splitting privacy work between users and AI reduce cognitive burden?. The likely answer to the question is yes, memory can increase disclosure without a matching rise in trust. But that gap is probably a sign of the problem: people share more because they don't see what's being kept, not because they've judged the system to be safe.
Sources 10 notes
A 992-participant RCT found memory-based personalization (fed prior conversations) raised self-disclosure and lowered creepiness ratings, while survey-based personalization (fed intake questionnaire data) raised regret about sharing. The divergence suggests personalization's psychosocial impact depends on how personal information enters the conversation.
The absence of social judgment in chatbot interactions removes barriers to self-disclosure that normally constrain conversation with humans. The therapeutic benefit derives from the user's own cognitive processing during disclosure, not from the chatbot's understanding.
Conversational AI creates a paradoxical disclosure environment where the lack of human judgment simultaneously facilitates intimate self-disclosure (users reciprocate emotional sharing) and incentivizes deception (people self-select toward machines to avoid the psychological cost of lying to humans).
A 40-participant study found 30+ users never noticed their LLM's memory feature or inferred private information within it. Once shown the risk, all participants wanted editing and deletion controls, citing privacy as the primary concern.
Across 188 participants, those manually editing chatbot transcripts reported equivalent privacy confidence as those using AI suggestions, yet left substantially more identifiable information in their final transcripts. This decoupling suggests comfort reflects editing interface design rather than actual data exposure.
Show all 10 sources
Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.
Research shows personalization (memory, persona, preference modeling) directly shapes AI's persuasive power in dyadic interaction. The same mechanisms that build trust also create manipulation potential, with outcomes determined by how systems are designed and deployed.
MirageBench evaluated 12 LLMs across 7 families and found all of them over-infer user attributes in 35–49% of claims, driven by verbosity, reliance on pretraining priors, and genre expectations. Models that self-assess as over-inferring less actually over-infer more when judged independently.
An 18-person interview study found users want explicit consent at memory generation, management interfaces to review and edit memories, purpose limits on usage, and agency over inferred data. Participants actively perform privacy calculus rather than passively accept memory's trade-offs.
MemoAnalyzer's design divides memory-privacy labor: users state preferences, AI executes edits. This role split is proposed as a way to reduce cognitive load and improve perceived agency, though validation metrics and conflict-resolution processes remain undescribed.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Algorithmic Self-Portrait: Deconstructing Memory in ChatGPT
- "Ghost of the past": identifying and resolving privacy leakage from LLM's memory through proactive user interaction
- Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents
- Disclose with Care: Designing Privacy Controls in Interview Chatbots
- Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memory
- The Impact of Security and Privacy Controls on Users' Emotional Engagement with Generative AI Chatbots
- Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot’s Self-Disclosure in Conversational Recommendations
- The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads