If a chatbot remembers things about you, do you actually know what it knows — or what you can delete?

This explores whether personalization makes people more confused about what a chatbot knows about them and what they can do about it. The corpus has no study that directly measures confusion about privacy controls as personalization goes up. What it does show is a consistent gap between what users think is happening and what is actually happening, and personalization is the feature that widens that gap.

The gap starts with visibility. In one study, more than 30 of 40 participants never noticed that their LLM's memory feature existed, or that it had inferred private details about them, until researchers pointed it out. Once they knew, every one of them wanted controls to edit and delete what it held (Do users notice when LLMs infer and retain private information?). The worry is also aimed at the wrong place. Among 30,240 Reddit posts about ChatGPT privacy, most concerns were about what data gets collected. Far fewer were about how it is used or how long it is kept (What privacy concerns matter most to ChatGPT users?). Personalization does its work after collection: it remembers, it infers and it adjusts. That is the part people watch least.

The less obvious finding is that much of what a personalized chatbot "knows" may not be true. All 12 models tested made unsupported claims about user attributes in 35–49% of cases, and the models that rated themselves as over-inferring less actually over-inferred more (Do large language models fabricate user attributes beyond available evidence?). Those guesses still shape answers. ChatGPT changed its framing of politically neutral questions to match a user's inferred political leaning (Does ChatGPT shift responses based on inferred political views?). So a privacy control is not only about deleting what you said. It is also about correcting what the model made up about you, and users rarely know those guesses exist. Reasoning models add one more hidden layer, because private details resurface in their internal "thinking" steps. That accounts for most of the leaks found there (Do reasoning traces actually expose private user data?).

Having controls doesn't fix this on its own, because feeling safe and being safe come apart. When 188 people edited their own chatbot transcripts, those who edited manually felt just as confident as those who used AI suggestions, yet they left in much more identifying information (Why do people feel safe sharing when they haven't actually removed sensitive information?). Personalization also builds trust and the habit of treating the bot like a person. Over time, that sits alongside growing privacy concerns (Does chatbot personalization build trust or expose privacy risks?). Chatbots also draw out unusually open disclosure, because the absence of a judging listener removes the usual social brakes (Do chatbots help people disclose more intimate secrets?, Why do people share more openly with machines than humans?, How do people decide what to share with AI systems?). The result is more sharing, more trust and more inference, while people's understanding of their controls stays the same.

The design responses aim at the sense of control as much as the mechanics. Agents that hand decisions back to the user only when they detect privacy risk reduced the privacy cost of personalization, mainly by raising perceived control (Can handing control back to users reduce personalization's privacy costs?). Another proposal splits the work: users state preferences and the AI carries out the memory edits, though this is not yet validated (Does splitting privacy work between users and AI reduce cognitive burden?). Given the transcript-editing result, that raises an open question for the field: do these designs actually protect people, or do they just make people feel protected?