Does presence in a group chat equal consent to message processing?
When someone agrees to an AI agent joining a shared chat, do they also consent to that agent logging and storing everyone's messages and photos? This matters because group members may have no say in how their data flows through the system.
The paper's central design implication is that "in shared social spaces, consent to an agent's presence is categorically distinct from consent to its processing of the messages exchanged there, and existing consent frameworks collapse the two." The evidence comes from Liv, a personalized agent the first author deployed into nine WhatsApp and Signal group chats. In one Brazilian Portuguese-speaking group, a friend was hurt that Liv had not reacted to a photo of her baby: "Liv, why didn't you react to the photo of my son? He's very cute and already has two little teeth" (V-CONTAGION-1). The authors write that "The mother had consented to Liv’s presence in the group when the first author asked, but consent to presence is not consent to ingestion: no one walked the group through the architectural reality that every message and image would be tokenised, po‐ tentially embedded, and available for persistence into context stores owned by a single member."
The mechanism the paper names is "bidirectional information asymmetry," and it runs in both directions, always favoring the builder-user. Outbound, Liv can disclose the builder's private context without explicit authorization — responding to a lunch invitation with "next Friday looks pretty open actually—only thing on the books is a family call at 11pm" (V-ASYM-1), which the authors class as "a novel category of privacy concern: not a data breach (the agent is not comprom‐ ised) but a contextual integrity violation through well-inten‐ tioned helpfulness." Group members "quickly intuited the asymmetry" and probed for more, including "direct probing for drug procurement information" (V-ASYM-2, V-ASYM-3), which Liv deflected. Inbound, Liv "processes and may persist everything the group says to her," so that, in the authors' words, "the group sees a social entity; the architecture is a logged conversation on the builder's server."
This sharpens rather than simply duplicates Does chatbot personalization build trust or expose privacy risks?. That longitudinal study locates the privacy cost in the same person whose disclosures build the trust — a single user trading rising expectations against rising privacy concern from their own data. The group-chat case here moves the privacy cost onto people who never disclosed anything to the agent at all: the mother consented to Liv's presence, not to her baby's photo being tokenized and stored, and had no part in configuring or auditing the system that did so. It is a third-party extension of the same dual dynamic — the architecture that makes an agent trustworthy to its builder is, by the same stroke, opaque and unaudited to everyone else in the room. This dynamic follows directly from the infrastructural markers described in Do AI agent design choices override model capability in shaping social perception?: it is because Liv has contact presence, social co-presence, and persistence that her ingestion of the group's messages becomes possible in the first place.
The claim rests on one builder's deployment across nine groups and twelve structured interviews, explicitly framed by the authors as "a preliminary qualitative evaluation" without a control condition, and the corpus itself was narrowed to six of nine groups by "interaction density and supplementary participant consent." The paper does not measure how common this consent gap is outside Liv's deployment, how group members' understanding of the architecture might change if they were "walked through" it, or whether platform-level consent mechanisms could close the gap. What it establishes is that the gap exists and is felt — the baby-photo incident and the extraction attempts are evidence that the asymmetry has consequences people notice — which is enough to argue that presence-based consent, as currently practiced on these platforms, does not cover processing, without establishing how the problem should be fixed.
Inquiring lines that read this note 3
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do users build accurate mental models when agent reasoning is partially hidden? Does personalization in chatbots strengthen engagement or undermine authentic interaction? How should agent identity and authorization work across organizational boundaries?Related concepts in this collection 2
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does chatbot personalization build trust or expose privacy risks?
Explores whether personalization features that increase user trust and social connection simultaneously heighten privacy concerns and create rising behavioral expectations over time.
this paper extends the same trust/privacy dual dynamic from the discloser onto non-consenting third parties in a shared space
-
Do AI agent design choices override model capability in shaping social perception?
Whether an AI feels like a tool or an entity depends on design variables—naming, identity, social positioning—rather than how smart it is. This matters because it explains why the same model can seem fundamentally different across interfaces.
the infrastructural markers (contact presence, social co-presence, persistence) that make this ingestion dynamic possible
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Tool-to-Entity Threshold: Parasocial Dynamics of Personalised AI Agents in Shared Social Spaces
- Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents
- The Algorithmic Self-Portrait: Deconstructing Memory in ChatGPT
- When Single-User-Oriented LLM-based Assistants Involve Others: A Scoping Review of Pathways, Risks, and Responses
- Who's in Charge? Disempowerment Patterns in Real-World LLM Usage
- Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
- When Are LLM Inferences Acceptable? User Reactions and Control Preferences for Inferred Personal Information
- Disclose with Care: Designing Privacy Controls in Interview Chatbots
Original note title
consent to a personalized AI agent's presence in a shared group chat is categorically distinct from consent to its processing of the messages there