Do people react differently to an AI using things they told it directly versus data it picked up from elsewhere?

This explores whether people respond differently when an AI personalizes from things they told it compared with information that came from somewhere else. The corpus has no study that pits personal disclosures against truly external data such as browsing history or third-party profiles. What it does have points to a sharper idea: people seem to react less to whether the data is theirs and more to how it entered the conversation, and AI systems blur that line by inventing details nobody ever gave them.

The clearest evidence comes from a 992-person experiment Does personalization type shape how people feel about sharing?. In both conditions, all the personal information came from the participants themselves. When the chatbot drew on earlier conversations, people shared more and found it less creepy. When it drew on answers from an intake questionnaire, people regretted sharing more. The information was self-disclosed both times. What differed was whether it came up naturally in the flow of talk or was taken from a form and brought back later. So even with their own data, users seem to notice how it was obtained, and data that arrives through a side channel starts to feel external.

The twist is that some of what feels like 'personalization' isn't based on any data. Across 12 tested models, 35–49% of claims about users went beyond the available evidence Do large language models fabricate user attributes beyond available evidence?. These claims came from stereotypes and assumptions learned in training. The models that rated themselves as over-inferring least actually did it most. To a user, a confident guess can look exactly like something drawn from their history. That makes it very hard to tell 'what I told it' apart from 'what it assumed', and the model can't reliably tell either. A related finding shows that loading personal context can make models drift toward irrelevant personal references and too much agreement Does personalization make large language models worse at their jobs?.

People are also not reliable judges of their own exposure. In one study, users who edited chatbot transcripts by hand felt as safe as users who had AI help, but they left far more identifying information behind Why do people feel safe sharing when they haven't actually removed sensitive information?. Comfort followed the interface design, not what was actually revealed. Over time, personalization raises trust and privacy worry together, and each interaction raises expectations for the next one Does chatbot personalization build trust or expose privacy risks?.

If users can't reliably sort sources themselves, the corpus suggests a different fix: give them control rather than asking them to keep track of where data came from. Agents that hand decisions back to users when privacy risk appears reduced both privacy concern and lost trust, even though they asked for fewer approvals overall Can handing control back to users reduce personalization's privacy costs?. Designs where users state their privacy preferences and the AI carries out the memory edits are proposed to reduce the mental work involved Does splitting privacy work between users and AI reduce cognitive burden?. The open question the corpus leaves behind is whether 'my data vs. external data' is even a distinction users can draw. The evidence suggests what they actually track is whether the personalization feels like part of the conversation or like surveillance.