Why do people feel safe sharing when they haven't actually removed sensitive information?
Participants reported similar comfort levels whether editing transcripts manually or with AI assistance, despite manual editing leaving far more personal information behind. Understanding this gap reveals how defaults and editing mechanisms shape privacy behavior.
The same 188-participant study of chatbot-based interviews found a split between what participants said about their privacy and what their edited transcripts actually showed: "free-editing and AI-aided users reported similar comfort (per post-task reflections)," yet "their final transcripts diverged" — AI-aided transcripts held far fewer residual PII tokens. Free-editing participants rated their "sense of privacy control and confidence in sharing" about the same as AI-aided participants, even though their "edits often only superficially addressed privacy protection and at times even introduced additional PII by supplying more detailed answers."
The authors read this as a version of the privacy paradox playing out within a single session rather than across general attitudes and behavior: oversharing "may stem less from active risk-taking and more from the power of defaults," where the default is to leave a transcript unchanged, and "many people will simply stick with such a status quo even when given a manual editing option." Because AI-aided editing substitutes a concrete, clickable suggestion for that default, it converts a vague sense of safety into "concrete, actionable edits at the span level" without participants reporting any more worry than the free-editing group did.
This is consistent with the risk side of Does chatbot personalization build trust or expose privacy risks?, where perceived risk and actual exposure move together only loosely as trust builds over time; here the two visibly decouple within one session, with comfort flat across conditions while the PII actually left in the transcript varies widely. The companion note Can AI suggestions help people remove their own personal information? covers the intervention this gap explains the need for.
The study measured self-reported comfort after the task, not real-time privacy judgment during editing, so it can't say whether participants noticed specific PII and chose not to remove it, or simply didn't register it as PII at all — the two call for different fixes. It also can't rule out that free-editing participants were satisficing, stopping once an edit felt "done," rather than lacking privacy concern altogether. The implication the authors draw, cautiously, is that self-reported comfort is not a safe proxy for how much identifiable information a reviewed transcript still contains.
Inquiring lines that read this note 33
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do engagement-optimized chatbots create dependencies that undermine user autonomy? How much do stated privacy concerns predict actual privacy-protective behavior?- Why do people report equal privacy comfort despite removing vastly different amounts of personal information?
- Can editing interfaces create dark patterns where users add sensitive details back after abstractions?
- What privacy risks emerge when AI stores psychological rather than factual data?
- Do transparency tools work better for some user types than others?
- How stable are user privacy attitudes when major AI events occur?
- What privacy safeguards should apply to memories users never explicitly requested?
- What privacy expectations do users hold before learning what systems store?
- Why does giving users full choice fail to prevent privacy leaks?
- How do privacy concerns translate into catching actual information breaches?
- How do major AI ecosystem events shift user privacy sentiment over time?
- How can systems visualize where privacy risks originate in stored memories?
- Does default inaction explain why free-editing participants left more personal information behind?
- Can real-time privacy judgment during editing reveal what participants actually notice as sensitive?
- How much does interface design shape privacy behavior versus underlying privacy concerns?
- Why do perceived privacy risks and actual disclosure decouple when building digital trust?
- Why does reversibility matter more to users than data prevention?
- Why do users trust deletion controls more than preventive privacy controls?
- How do users distinguish between personalization built from their own disclosures versus external data?
- Does giving users memory control actually reduce their privacy concerns?
- What strategies do users already employ to protect sensitive information?
- Can systems bound what gets disclosed per query without user consent?
- Do general populations show the same privacy awareness gaps as college-educated users?
- Do personalization features change how people perceive privacy risks over time?
- Do personalized chatbots amplify confusion about privacy controls?
- How do emotional connections to AI systems affect privacy decision-making?
- Why does information sourced from prior conversations feel less creepy than intake surveys?
- Can memory-based personalization increase self-disclosure without raising trust?
- How does memory personalization increase both trust and privacy anxiety simultaneously?
- How does conversational memory pose different privacy risks than training-time memorization?
Related concepts in this collection 2
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does chatbot personalization build trust or expose privacy risks?
Explores whether personalization features that increase user trust and social connection simultaneously heighten privacy concerns and create rising behavioral expectations over time.
relation: perceived risk and actual disclosure decouple in both, though here within a single session rather than over repeated use
-
Can AI suggestions help people remove their own personal information?
When chatbot interview participants are offered AI-generated suggestions to redact or abstract personally identifiable information from their transcripts, do they remove more sensitive details than they would by editing manually, and does this change affect how engaged or comfortable they feel?
sibling note, same study, the behavior split that explains why the AI-aided mechanism had room to work
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Disclose with Care: Designing Privacy Controls in Interview Chatbots
- Understanding Users' Security and Privacy Concerns and Attitudes Towards Conversational AI Platforms
- The Impact of Security and Privacy Controls on Users' Emotional Engagement with Generative AI Chatbots
- "Ghost of the past": identifying and resolving privacy leakage from LLM's memory through proactive user interaction
- Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
- Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers
- Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot’s Self-Disclosure in Conversational Recommendations
- Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing
Original note title
chatbot interview participants reported similar privacy comfort across editing conditions despite removing far less PII without AI-aided suggestions