Chatbots don't just remember what you tell them — they also guess things about you. Can you tell those apart?

This explores whether people can tell the difference between what a chatbot remembers about them (things they actually said) and what it has worked out about them (conclusions it drew on its own). No study in the collection tests this directly. Read together, though, the studies point to an answer that may be surprising: mostly, people don't tell them apart. Often they don't know either one is happening. In a 40-person study, more than 30 participants never noticed that their LLM had a memory feature, let alone that the memory held private details it had inferred rather than been told Do users notice when LLMs infer and retain private information?. Once researchers showed them, every participant wanted to be able to edit and delete it.

Part of the reason is that people start from very different ideas of what 'memory' means for an AI. Interviews with ChatGPT users found four incompatible mental models: memory as a buffer that only lasts for one session, memory as an extension of the training data, memory as active processing of what you say, or simply no idea at all What mental models do users hold about LLM memory?. Only the 'active processing' view leaves room for the idea that the system is drawing conclusions about you. If you think memory is just a transcript, an inference that comes back to you will look like something you must have said.

When inferences are shown to people openly, the line that matters to them isn't 'remembered vs. inferred'. It's 'accurate vs. off-base, and who else gets to see it.' Shown 215 insights an LLM had inferred about them, participants reacted with more curiosity than distress. They pushed back only when an inference felt wrong, misread what they meant, or might reach advertisers or third-party apps Do people fear or welcome AI inferences about themselves?. So people judge the content, not where it came from.

The less obvious point is that the systems themselves blur this line on purpose. Personalization research finds that abstract preference summaries beat retrieving specific past conversations Does abstract preference knowledge outperform specific interaction recall?. A summary is already an inference, so the best-performing 'memory' is made of conclusions rather than records. Those conclusions can quietly change answers that have nothing to do with you. When researchers steered a chatbot's internal guess about a user's socioeconomic status to 'low', it falsely claimed there were no direct flights to Hawaii Can steering a chatbot's user model change its factual claims?. The user would see only a confident answer, not the assumption about them that produced it.

Two more findings explain why this is hard to catch. Models themselves mix up what they memorized with what they reasoned out: they judge whether a statement follows from another based on whether they've seen it before, not on whether the logic holds Do LLMs predict entailment based on what they memorized?. And the polished, confident style of chat output discourages users from checking anything Does confident AI tone make users skip fact-checking?. The practical takeaway is that the line between remembering and inferring isn't something users can see from the outside. It only becomes visible when the interface shows it, by labeling where each stored item came from and letting users edit it.