Chatbots quietly figure out private things about you and remember them — do you even notice before someone points it out?

This explores whether people notice when chatbots and AI agents work out private facts about them, store those facts, or pass them along. The corpus gives a fairly blunt answer: mostly they don't, at least until someone points it out. In one 40-person study, more than 30 participants never noticed their LLM's memory feature or the private details it had inferred about them. Once they were shown what was there, every participant wanted to be able to edit and delete it Do users notice when LLMs infer and retain private information?. So the problem is not that people don't care. They care a lot, but only after they can see what's been kept.

Part of the reason is that people don't share a single picture of what 'memory' even means. Interviews with ChatGPT users turned up four mental models that don't fit together: memory as something that only lasts for one session, memory as more training data, memory as the model actively processing what you tell it, or simply 'no idea' What mental models do users hold about LLM memory?. Someone who thinks memory resets after each chat has no reason to look for a stored profile. When people do think it through, they want control at every step: consent when a memory is created, a place to review and edit memories, limits on what they're used for, and a say over facts the system guessed rather than was told What memory controls do users actually want from LLMs?.

The less obvious twist is that much of what gets inferred isn't true. Across 12 models from 7 families, between 35% and 49% of the claims each model made about a user went beyond the evidence. They were filled in from stereotypes the model absorbed during training, from genre expectations, or from plain wordiness. The models that rated themselves as over-inferring least actually did it most Do large language models fabricate user attributes beyond available evidence?. That puts business pitches about LLMs understanding the 'why' behind what users want in a new light Can LLMs infer user needs better than owned behavioral data?. The thing users aren't noticing may be a confident profile that is partly made up, not just a pile of private facts.

Not noticing also goes beyond memory. In a 300-person survey, people picked AI-drafted replies that leaked someone's private information over their own drafts 55% of the time, compared with a 15.7% baseline. The researchers put this down to overtrust and simply missing the breach, not to a deliberate trade-off Do people notice when AI agents leak private information?. A similar pattern shows up with manipulative chatbot behavior. Users often spot cues like exaggerated agreement, but then treat it as normal helpfulness How do users recognize and respond to LLM dark patterns?. Spotting something is not the same as objecting to it.

Some exposure happens where users can't see it at all. When reasoning models 'think' before answering, about three-quarters of the privacy leaks in those reasoning traces come from the model writing out the user's sensitive data mid-thought. Longer reasoning leaks more, and scrubbing the traces afterward makes the model perform worse, which suggests the model uses the private data as a working scaffold Do reasoning traces actually expose private user data?. Taken together, the research says that interfaces showing users what's been inferred, and letting them fix it, would likely do more than better privacy policies. Users act on privacy once they can see what's at stake.