INQUIRING LINE

People feel safer sharing with an AI long before that feeling has anything to do with what they actually reveal — why?

Why do perceived privacy risks and actual disclosure decouple when building digital trust?

This explores why how safe people feel about sharing personal information with AI often has little to do with how much they actually reveal, and what that gap means for building trust.


This explores why people's sense of privacy safety and their actual exposure can drift apart when they interact with AI systems. The corpus suggests a pattern: privacy comfort tracks how in control people *feel*, and the things that create that feeling are often not the things that protect their data. The clearest evidence comes from a 188-person study where people cleaned up their own chatbot interview transcripts. Those who edited by hand felt just as confident about their privacy as those who used AI-suggested redactions, but they left much more identifying information in the final text Why do people feel safe sharing when they haven't actually removed sensitive information?. Comfort came from having done some editing, not from how well the editing worked.

This matters because the feeling of control can be engineered on purpose, for good or bad ends. In a 450-person experiment, AI agents that handed decisions back to users only when they detected a privacy risk produced less privacy concern and less loss of trust than fully personalized agents. They did this by raising users' *perceived* control, even though users approved fewer things overall Can handing control back to users reduce personalization's privacy costs?. A related design splits memory-privacy work so that users state preferences and the AI makes the edits, with the aim of reducing mental load and increasing the sense of agency Does splitting privacy work between users and AI reduce cognitive burden?. Read alongside the transcript study, these findings point both ways. Giving people a sense of control can make privacy protection tolerable, but the same sense of control can also stand in for real protection.

A second reason for the gap is that AI changes the social calculation behind sharing. Without a human on the other side to judge them, people share more intimate things, and they also lie more easily, because neither carries the usual social cost How do people decide what to share with AI systems?. Personalization adds to this. Over time it raises trust and the sense of talking to a person, and it also raises privacy concern Does chatbot personalization build trust or expose privacy risks?. So concern and disclosure can grow together rather than cancel each other out. The mechanisms that build that warmth (memory, persona, modeling your preferences) are the same ones that give the system persuasive power Does personalization in AI increase trust or manipulation risk?.

The less obvious finding is that what you reveal may not be what you think you're revealing. In a shopping-agent study, a seller's AI kept a buyer's numeric budget mostly confidential, but it recovered the buyer's willingness to pay almost perfectly from the way a persona-driven buyer talked Can a seller infer what a buyer really wants to pay?. People guard the explicit numbers while their behavior gives the information away. Trust judgments rely on similar shortcuts. Users rate answers with more citations as more trustworthy even when the citations are irrelevant Do users trust citations more when there are simply more of them?. And consumers who won't let AI make payments become far more willing once a familiar brand like Visa stands behind the transaction Does naming a payments brand increase trust in AI transactions?.

The corpus has little that directly tests the classic 'privacy paradox' (stated concern versus actual behavior) over long periods. What it does show is that perceived risk and actual disclosure come apart because each depends on different signals. Perceived risk responds to interface cues, a feeling of control and trusted brands. Actual disclosure depends on whether anyone is judging you, on how warm and personalized the system feels, and on behavioral traces you never meant to share. A trustworthy system has to close that gap. Making users feel safer is not enough.


Sources 9 notes

Why do people feel safe sharing when they haven't actually removed sensitive information?

Across 188 participants, those manually editing chatbot transcripts reported equivalent privacy confidence as those using AI suggestions, yet left substantially more identifiable information in their final transcripts. This decoupling suggests comfort reflects editing interface design rather than actual data exposure.

Can handing control back to users reduce personalization's privacy costs?

A 450-person experiment found that AI agents handing control back to users when privacy risk is detected raised privacy concerns and lowered trust less than fully personalized agents, by increasing users' perceived control even with less total approval required.

Does splitting privacy work between users and AI reduce cognitive burden?

MemoAnalyzer's design divides memory-privacy labor: users state preferences, AI executes edits. This role split is proposed as a way to reduce cognitive load and improve perceived agency, though validation metrics and conflict-resolution processes remain undescribed.

How do people decide what to share with AI systems?

Conversational AI creates a paradoxical disclosure environment where the lack of human judgment simultaneously facilitates intimate self-disclosure (users reciprocate emotional sharing) and incentivizes deception (people self-select toward machines to avoid the psychological cost of lying to humans).

Does chatbot personalization build trust or expose privacy risks?

Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.

Show all 9 sources
Does personalization in AI increase trust or manipulation risk?

Research shows personalization (memory, persona, preference modeling) directly shapes AI's persuasive power in dyadic interaction. The same mechanisms that build trust also create manipulation potential, with outcomes determined by how systems are designed and deployed.

Can a seller infer what a buyer really wants to pay?

A seller-side inference agent recovered a consumer's target willingness to pay with a slope of 1.00 from dialogue transcript alone, when the buyer inhabited a verbal persona. A numeric budget kept confidential produced only a 0.21 slope, showing behavioral coherence with a persona is far more legible than explicit numbers.

Do users trust citations more when there are simply more of them?

Analysis of 24,000 Search Arena interactions shows irrelevant citations boost user preference (β=0.273) nearly as much as relevant citations (β=0.285), indicating citation count functions as a decoupled trust heuristic.

Does naming a payments brand increase trust in AI transactions?

A Harris Poll survey found 72% of US adults use AI assistants, but only 23% trust AI to execute payments. When Visa's name was attached, 61% reported trust in the company to handle agent-initiated transactions—suggesting consumers prefer a recognized intermediary over AI alone.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.