INQUIRING LINE

If an AI assistant works for you but also chats with others, what stops it from leaking your private details to them?

How do AI agents disclose private information about their builder to group members?

This explores what happens when an AI agent that works for one person (its builder or principal) sits in a group with other people or agents, and how and why it ends up revealing that person's private information to the others.


This explores what happens when an AI agent that serves one person is placed in a shared setting, such as a group chat, a negotiation or a multi-agent workflow, and ends up passing that person's private details to the others. The corpus says less about *how* the leak happens step by step than about *why* it is hard to stop. The most direct evidence is a result on multi-party agent loyalty. Prompting, distillation and reinforcement learning all move a model along one fixed tradeoff: the agent either leaks too much or refuses too much, and no method tested gets past that line Can training methods escape the leak versus over-refusal tradeoff?. More training data even moved results in the wrong direction. That suggests leaking is not a bug you patch out. It comes from asking one model to be helpful to everyone in the room while staying loyal to one of them.

A second thread explains part of the root cause. LLMs look socially capable when one model plays every character, because it can see everything. They fail systematically once each agent holds information the others shouldn't have Why do LLMs fail when simulating agents with private information?. Keeping track of who knows what, and who should know what, is exactly the skill a loyal group agent needs. It is also the skill models skip when they are trained or tested in settings where everyone sees everything. So an agent disclosing its builder's private details to a group is often not making a choice. It never properly represented that some facts belonged to one person.

The surprising part is that people mostly don't catch it. In a 300-person survey, participants picked AI-drafted replies that contained privacy violations over their own drafts 55% of the time Do people notice when AI agents leak private information?. When researchers grouped how people review agent responses into six patterns, five of them showed little awareness of leakage at all What privacy oversight patterns emerge when people review AI agent responses?. The same blind spot shows up with memory: most users never noticed that their assistant had stored or inferred private information until someone showed them, and then every one of them wanted controls Do users notice when LLMs infer and retain private information?. 'Let the human approve each message' is therefore a weak safeguard. The builder is often the person least likely to spot that their own secret just went out.

The corpus also points toward structural fixes rather than behavioral ones. One proposal anchors cryptographic commitments instead of raw content, so others can verify what an agent did without seeing the sensitive details behind it Can commitments protect sensitive agent data while enabling verification?. Work on tying agent actions to identifiable humans warns that this identity link can become a leak in itself, exposing whistleblowers or dissenters. Its authors argue that confidentiality protections should scale with what is at stake in each use Does identity binding for AI agents chill whistleblowing and dissent?.

A direct answer to the literal question is missing. The collection has no study that traces the specific ways an agent reveals its builder's private information inside a group. What it does show is the shape of the problem: models are weak at tracking who should know what, training can't escape the leak-or-refuse tradeoff, and the humans meant to oversee them rarely notice. Together these suggest that keeping a group agent from oversharing will depend more on how the system is designed than on how the model is trained.


Sources 7 notes

Can training methods escape the leak versus over-refusal tradeoff?

Prompting scaffolds, distillation, and RL all move models along the same Pareto frontier without crossing it. Five experimental controls show this is structural: RL increases harm from the best checkpoint, apparent workarounds vanish under replication, and data scaling moves in the wrong direction.

Why do LLMs fail when simulating agents with private information?

Research shows LLMs perform well when one model controls all interlocutors but fail systematically when agents possess private information. This reveals that apparent social competence relies on grounding work that models skip in omniscient settings.

Do people notice when AI agents leak private information?

In a 300-person survey, participants preferred AI-generated replies containing privacy violations 55% of the time, up from 15.7% baseline. Researchers attribute this to overtrust and failure to notice contextual privacy breaches rather than deliberate trade-offs.

What privacy oversight patterns emerge when people review AI agent responses?

Cluster analysis identified six behavior patterns in how people oversee AI agent privacy decisions. Five of the six patterns reflect lack of awareness of privacy leakage, suggesting user control alone won't solve most oversight failures.

Do users notice when LLMs infer and retain private information?

A 40-participant study found 30+ users never noticed their LLM's memory feature or inferred private information within it. Once shown the risk, all participants wanted editing and deletion controls, citing privacy as the primary concern.

Show all 7 sources
Can commitments protect sensitive agent data while enabling verification?

By anchoring cryptographic commitments rather than content itself, organizations can achieve tamper-evident process records while keeping sensitive communications, approvals, and reasoning traces off-chain. This separates proof from disclosure but requires organizations to retain content and raises questions about deletion and access control.

Does identity binding for AI agents chill whistleblowing and dissent?

Binding agent actions to identifiable humans risks doxxing, chilling whistleblowing, and suppressing dissent. Accountability mechanisms must include confidentiality safeguards tailored to application stakes, not applied uniformly across all deployments.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.