Should an AI explain itself the same way to everyone, or adjust based on who's asking and how much they want to know?
How should AI transparency scale based on user role and preference?
This explores whether showing users how an AI works should look the same for everyone, or whether how much you reveal should depend on who the person is (developer, designer, end user, novice, expert) and what they want to know.
This explores whether AI transparency should be one-size-fits-all or adjusted to who's looking and what they want. The corpus points to a surprising answer: more transparency isn't reliably better, and the same disclosure can help one person and backfire on another. When researchers revealed an AI companion's system prompts and conversational strategies, some users felt empowered while others felt manipulated, and the split came from differences between individuals, not from the disclosure itself Does revealing an AI's strategies help or harm user agency?. So 'how much to reveal' is partly a question about the person receiving it.
The sharpest warning concerns explanations as a safety measure. A review of 35 studies found that explainable-AI features often *increase* overreliance, especially among users with low AI literacy, who read an explanation as a reason to trust rather than a reason to check Does explaining AI decisions actually reduce user overreliance on it?. What actually reduced blind trust was getting users to engage and verify. Timing matters too: telling people they're working with an AI first makes them avoid it, but that bias reverses once they repeatedly see how things turn out, and disclosure without that feedback calibrates nothing Does revealing AI identity help or hurt user trust?. Put together, transparency works less like a label and more like a learning loop. It scales best when it's paired with visible results the user can judge.
Role changes what transparency is *for*. For designers working next to an agent, seeing each step as it ran showed them where they could step in. But the agent misread or undid their edits, so the visibility created an invitation the system couldn't honor Does visible agent progress help designers know when to intervene?. For hands-on roles, transparency is only worth something if it comes with real ways to steer. One design response is to let users set the level themselves: designers are converging on an 'autonomy slider' (suggest, draft, execute) adjusted per task Should agent autonomy be a continuous slider or fixed levels?, and a memory-privacy tool splits the work so users state preferences and the AI carries out the edits, which spares people from inspecting everything Does splitting privacy work between users and AI reduce cognitive burden?. Both suggest that users need control over the right layer more than they need maximum visibility into everything.
There's also a structural layer that user preferences can't reach. System prompts set up a hidden chain of authority in which developers write rules end users can neither see nor change, and small edits there can abruptly shift behavior Who actually controls what AI systems say?. That's why some researchers argue transparency shouldn't be framed only around the individual user. It should scale up to 'visibility' and accountability across everyone an AI affects, with developers and deployers answerable for harms Should AI agents serve one user or balance many interests?. The thought-partner view adds that 'legibility', meaning the AI being understandable to you as you are to it, has to run both ways and match the person's mental model, not just dump internal details What makes an AI a true thought partner, not just a tool?.
The takeaway you might not have expected: tuning transparency by role and preference is necessary but not enough. Individual-level transparency should be adjustable, tied to feedback, and paired with real control. Meanwhile the layers users never see, such as system prompts and who is accountable, need a separate kind of transparency aimed at oversight rather than personal preference. The corpus has little direct evidence on formally tiering explanations by role (for example, expert versus novice dashboards), so that specific design question is still mostly open.
Sources 9 notes
Disclosing an AI companion's system prompts and conversational strategies empowered some users but triggered feelings of manipulation in others. The effect depended on individual differences, not just the fact of disclosure itself.
A systematic review of 35 studies (2015–2025) found that XAI explanations frequently reinforce misplaced trust rather than curb over-reliance, especially among low-AI-literacy users. Active user engagement and critical verification, not explanation transparency alone, effectively counters automation bias.
Users initially avoid AI partners when identity is revealed, but this preference reverses after repeated interactions with visible results. The learning mechanism—observing consistent outcomes—is essential; disclosure without feedback produces no calibration.
Designers used step-by-step agent execution to spot contribution opportunities, but when they edited the canvas during agent work, the agent misinterpreted or reverted their changes, creating coordination conflicts that discouraged further attempts to collaborate.
The AI UX Playground glossary names the autonomy slider as a continuous control letting users dial agent independence per task, collapsing agency levels into three adjustable points: suggest-only, draft, execute. The pattern shows shared vocabulary among designers, though validation against other control shapes remains absent.
Show all 9 sources
MemoAnalyzer's design divides memory-privacy labor: users state preferences, AI executes edits. This role split is proposed as a way to reduce cognitive load and improve perceived agency, though validation metrics and conflict-resolution processes remain undescribed.
System prompts operate as a confidential three-tier control structure where developers set foundational rules users cannot see or influence. Evidence from 2025 incidents shows small prompt edits can abruptly shift model behavior, yet users remain excluded from this layer.
Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).
Collins et al. show that thought partners require three reciprocal desiderata grounded in behavioral science: mutual understanding, legibility, and shared world models. This demands explicit cognitive architectures—Bayesian theory of mind, resource-rationality, goal planning—rather than scaling foundation models on human feedback alone.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
- Humans learn to prefer trustworthy AI over human partners
- Toward Meaningful Transparency for AI Chatbots: Disclosing Persuasive Intent Reduces Persuasion
- Seeing to Think? How Source Transparency Design Shapes Interactive Information Seeking and Evaluation in Conversational AI
- Designing a Dashboard for Transparency and Control of Conversational AI
- Controlling AI Agent Participation in Group Conversations: A Human-Centered Approach
- Governing AI Agents