Do people fear or welcome AI inferences about themselves?

An 18-participant mixed-methods study built the Reflective Layer, a tool that surfaces inferences an LLM could draw from a user's own ChatGPT history, and used it to show participants 215 such inferences drawn from their real conversations. The authors report that reactions ran counter to the privacy-harm framing dominant in prior work: "participants reacted more strongly with curiosity and interest rather than distress and concern." Discomfort was not absent, but it clustered around specific conditions: "discomfort arose mainly when inferences felt misrepresentative of the user or misaligned with expected use." A further axis of acceptability tracked who would see the inference rather than its content alone — participants were markedly less comfortable with advertisers and third-party applications using those inferences than with platform providers.

The study separates two things prior work has blurred: memory (first-hop, the facts a system extracts from what a user explicitly said) and inference (second-hop, "the act of drawing conclusions about a user that are not directly stated in their input"). The Reflective Layer adapts the General User Model (GUM) pipeline, extracting observations and then propositions, each one traceable back to the chat messages that produced it, and scored for sensitivity using continuous Contextual Integrity judgments. Participants' evaluations, the authors argue, separate into three decision points — generation (should the inference have been drawn at all), retention (what can be done once it exists), and transmission (who else gets to use it) — with "the same inference... evaluated favorably at one point and unfavorably at another."

This complicates Does ChatGPT's memory feature feel like a relationship or a machine?, which found that discovering retained memory content mostly backfires into negative expectancy violations; this study, built specifically to surface the more invasive case of unstated inference rather than stored fact, instead finds curiosity predominating, with negative reaction conditional on misrepresentation or scope creep rather than automatic. It also extends Do users notice when LLMs infer and retain private information?, which showed that opacity itself was the problem — once that opacity is removed by a purpose-built probe, reaction is not uniformly protective; it is evaluative, weighing accuracy and fit as much as exposure. The recipient-sensitivity finding (platform versus advertiser versus third-party app) adds a dimension neither neighboring note isolates.

The Reflective Layer is a researcher-built probe adapting the GUM pipeline to exported ChatGPT history, not a shipped product feature, and the sample is 18 self-selected regular ChatGPT users reacting to inferences about themselves specifically — the excerpt does not establish how reactions would hold for infrequent users, for inferences surfaced silently during live use rather than in a reflective study session, or across platforms whose inference pipelines differ from GUM's. What the excerpt does support is that inference acceptability is not simply a content or disclosure-volume problem: it is a context-fit problem, governed by accuracy, purpose-alignment, and recipient, which argues against treating the inference was drawn and the inference was used by X as a single design decision.