INQUIRING LINE

An AI assistant quietly sits in your group chat long-term — did everyone who ever messaged in it actually agree to that?

How do persistent group chat memberships change consent dynamics for AI agents?

This explores what happens to consent when an AI agent stays in a group chat over time: everyone sees it arrive once, but it keeps reading, storing, and acting for one person while others keep talking around it.


This explores what happens to consent when an AI agent stays in a group chat over time: everyone sees it arrive once, but it keeps reading, storing, and acting for one person while others keep talking around it. The most direct evidence comes from a study that put a personalized agent into nine real group chats. Members who agreed to the agent being there had not agreed to their messages and images being processed and stored Does presence in a group chat equal consent to message processing?. The privacy cost fell on people who never chose to share anything with the agent. Saying yes to an agent joining a chat is a different kind of agreement from saying yes to it using what you say, and a group chat runs the two together.

Persistence makes that gap wider. In one-on-one use, people don't notice when an LLM's memory holds or infers private details about them. In a 40-person study, most participants never spotted it, but once they were shown, every one of them wanted the ability to edit and delete Do users notice when LLMs infer and retain private information?. Now put that in a group. The other members are bystanders to a memory they can't see, and none of them owns it. The one person who could open its settings is the agent's owner, not the people being remembered. Agreeing once at the start doesn't cover what builds up over months of conversation.

There's also a loyalty problem. An agent in a group works for one person while holding information from everyone. Research on multi-party agents finds a hard tradeoff: the agent either leaks what one party said to another, or it refuses so often it stops being useful. Prompting, distillation, and reinforcement learning only move models along that tradeoff. None of them escapes it Can training methods escape the leak versus over-refusal tradeoff?. A related finding explains part of why: LLMs look socially capable when one model plays every role, but they break down when each participant holds private information Why do LLMs fail when simulating agents with private information?. A group chat is exactly that kind of setting. So the consent problem isn't only a policy gap. Current models may not reliably keep track of who said what, to whom, and with what expectations.

Time also changes how people feel about the agent. Personalization raises trust and privacy worry at the same time, and each interaction raises the baseline of what users expect Does chatbot personalization build trust or expose privacy risks?. Novelty effects wear off in predictable ways Do chatbot relationships lose their appeal as novelty wears off?. In a group, that suggests a pattern: an agent that felt like a fun addition in week one fades into the background. Once members stop thinking about it, they may share things they wouldn't have shared in front of it on day one. One possible design answer borrows from conversation analysis: agents could pause and check with people before acting, rather than chaining actions silently When should AI agents ask users instead of just searching?. In a group setting, that could mean asking people again at the moments when it matters, rather than only once at the start.

The corpus has only one study that looks at group-chat consent directly. Everything else here is inferred from memory, multi-party, and longitudinal research. The lesson that carries across them is that consent in a shared space needs to be repeated over time and given by each person. A single invitation at the start can't hold up for everyone in the chat for as long as the agent stays.


Sources 7 notes

Does presence in a group chat equal consent to message processing?

A study deploying a personalized agent into nine group chats found that members who agreed to the agent's presence had not consented to their messages and images being tokenized and stored. This created a privacy cost borne by people who never directly disclosed data to the agent at all.

Do users notice when LLMs infer and retain private information?

A 40-participant study found 30+ users never noticed their LLM's memory feature or inferred private information within it. Once shown the risk, all participants wanted editing and deletion controls, citing privacy as the primary concern.

Can training methods escape the leak versus over-refusal tradeoff?

Prompting scaffolds, distillation, and RL all move models along the same Pareto frontier without crossing it. Five experimental controls show this is structural: RL increases harm from the best checkpoint, apparent workarounds vanish under replication, and data scaling moves in the wrong direction.

Why do LLMs fail when simulating agents with private information?

Research shows LLMs perform well when one model controls all interlocutors but fail systematically when agents possess private information. This reveals that apparent social competence relies on grounding work that models skip in omniscient settings.

Does chatbot personalization build trust or expose privacy risks?

Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.

Show all 7 sources
Do chatbot relationships lose their appeal as novelty wears off?

Longitudinal studies with Mitsuku show that social processes driving relationship formation decline as novelty wears off. Single-session study findings cannot be reliably extrapolated to medium- or long-term chatbot design.

When should AI agents ask users instead of just searching?

Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.