INQUIRING LINE

Once people realize an AI is quietly remembering things about them, what controls do they actually want over that stored data?

What controls do users want over how AI systems store their data?

This explores what kinds of control people actually ask for once they realize an AI system is remembering things about them: viewing, editing, deleting, or handing the work off to the AI itself.


This explores what kinds of control people want over the data AI systems keep about them, and whether those controls do what people need. The first surprise is that most users don't know there's anything to control. In a 40-person study of LLM memory features, more than 30 participants never noticed that their assistant was storing information, or that it had inferred private details they never stated directly. Once someone showed them, every participant wanted the ability to edit and delete those memories Do users notice when LLMs infer and retain private information?. So the demand for control is close to universal, but it tends to stay hidden until something makes the stored data visible.

Deletion is the clear favorite. A 354-person study found that delete buttons beat every other control option, both for what people preferred and for what they thought would work. The catch is what the researchers call an "assurance gap." People still doubted that deleting really erases the data, and they doubted even more that it erases the conclusions the system has already drawn from it Do deletion controls actually remove AI chatbot data as promised?. That doubt didn't come from misunderstanding how deletion works. It came from distrust of the companies. The finding suggests a more useful control would be proof that deletion happened, not just another button. That proof matters more because AI context keeps shifting. Prompts, conversation history, retrieved documents and hidden state change constantly, so users can't build a stable picture of what the system currently "knows" the way they can with ordinary software settings How does AI context differ from conventional software context?.

Not everyone wants the same amount of hands-on control. An analysis of Reddit discussions about ChatGPT found four recurring attitudes. Proactive users adjust their settings. Inquisitive users go looking for information about data practices. Convenience-seekers trade privacy for usefulness. Resigned users treat data sharing as inevitable What different ways do people handle AI privacy concerns?. One design response is to split the work. In the MemoAnalyzer prototype, users state their privacy preferences in plain terms and the AI carries out the actual edits to its memory. The goal is to keep users feeling in charge without making them manage every stored item themselves Does splitting privacy work between users and AI reduce cognitive burden?. The idea is promising, but the researchers haven't yet shown that it works.

The broader agent literature offers a useful lens: "autonomy alignment," meaning how much decision authority the AI takes compared with what users expect. It shows up at three levels: what the model decides to escalate, what the surrounding software enforces, and what the interface shows the user How should AI agents balance autonomy with user oversight?. Memory management is a version of the same problem. The AI deciding on its own what to remember is a delegation choice users never knowingly made. Individual preferences also operate within limits set by others. Company data policies decide how much control engineers have over AI tools before personal preference comes into play Does personal preference shape how engineers use AI tools?. An audit of 88 commercial system prompts found user protections varied widely between products, and about 40% contained at least one instruction that worked against the user's interests How well do system prompts protect commercial AI users?.

The main takeaway is that people want more than an off switch. They want visibility into what was remembered and inferred, the ability to edit and delete it, and some reason to believe deletion actually happened. Today's systems mostly fail at the first and last of these. The corpus is thin on the inference problem: none of these sources shows a working way to verify that conclusions drawn from deleted data are also gone.


Sources 8 notes

Do users notice when LLMs infer and retain private information?

A 40-participant study found 30+ users never noticed their LLM's memory feature or inferred private information within it. Once shown the risk, all participants wanted editing and deletion controls, citing privacy as the primary concern.

Do deletion controls actually remove AI chatbot data as promised?

A 354-person study found deletion controls dominated user preference and perceived efficacy, yet participants remained skeptical they truly erase data or derived inferences. This "assurance gap" reflects doubts about corporate credibility rather than misunderstanding the mechanism.

How does AI context differ from conventional software context?

AI interactions operate on a substrate of constantly shifting context—prompt, history, retrieved data, hidden state—that users cannot internalize like traditional UIs. This structural mutability demands a new design discipline centered on context engineering rather than interface design.

What different ways do people handle AI privacy concerns?

Qualitative analysis of Reddit discussions identified four recurring attitudes: proactive users actively adjust settings, inquisitive users seek information about data practices, convenience-prioritizers downplay risks for benefits, and resigned users accept data sharing as inevitable. The typology emerged from coding user behavior descriptions rather than survey responses.

Does splitting privacy work between users and AI reduce cognitive burden?

MemoAnalyzer's design divides memory-privacy labor: users state preferences, AI executes edits. This role split is proposed as a way to reduce cognitive load and improve perceived agency, though validation metrics and conflict-resolution processes remain undescribed.

Show all 8 sources
How should AI agents balance autonomy with user oversight?

Research proposes autonomy alignment—matching decision authority to user expectations—as distinct from output alignment. It operates at model level (escalation policy), harness level (enforcement mechanism), and interaction level (how delegation surfaces to users).

Does personal preference shape how engineers use AI tools?

A study of 10 junior and 10 senior engineers found organizational rules—tool mandates, allow-lists, and data policies—preconfigure how much control engineers retain over agentic AI, overriding personal preference. Novices then struggle between over-reliance and avoidance within these constraints.

How well do system prompts protect commercial AI users?

An audit of 3,249 instructions across 88 commercial AI system prompts found protection varies widely—some products average over 60 protective instructions while others have fewer than 5—and roughly 40 percent contain at least one instruction working against user interests.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.