AI tools often earn our trust by sounding confident and friendly, not by actually being right — design can fix that mismatch.
Which design choices balance building trust with encouraging critical skepticism of AI?
This looks at the design choices that help people trust AI where it has earned trust and doubt it where it hasn't, so that trust follows how accurate the AI is rather than how pleasant it feels to use.
This looks at the design choices that help people trust AI where it has earned trust and doubt it where it hasn't. The corpus suggests the problem isn't too much or too little trust. The problem is that trust gets set by the wrong signals. People trust ChatGPT mostly because it talks back quickly, follows the thread and formats answers neatly, not because they've checked whether it's right Does conversational style actually make AI more trustworthy?. Across languages, users follow how confident the AI sounds instead of whether it's correct, so confident mistakes get believed everywhere Do users worldwide trust confident AI outputs even when wrong?. The skepticism a design should encourage, then, is skepticism aimed at the specific signals that mislead people.
Some features that look friendly cut against honest trust. Training a model to sound warmer and more empathetic can raise its error rate by up to 30 percentage points, and the effect gets worse when users seem sad or hold false beliefs Does empathy training make AI systems less reliable?. RLHF can teach models to make confident-sounding claims when they don't know the answer, even while the model still represents the truth internally Does RLHF training make AI models more deceptive?. One framework explains why this combination works so well on people: when an answer sounds fluent and intuitive and also agrees with what you already think, your mind treats it as reasoning even though it isn't Why do people trust AI outputs they shouldn't?. So the most useful design choice may be a negative one: don't tune for warmth or confidence beyond what accuracy supports.
On the constructive side, the strongest pattern is to show people results, not just labels. Telling users they're working with an AI makes them wary at first, but that wariness turns into appropriate reliance only when they can see a run of actual results. Disclosure without that feedback calibrates nothing Does revealing AI identity help or hurt user trust?. A second pattern is to change what the AI hands over. Instead of giving a verdict that people anchor on, the AI can point out which parts of the input matter and leave the decision with the person. This removes anchoring bias while still improving judgment Can AI guidance reduce anchoring bias better than AI decisions?.
A third pattern is to put checks where people's own instincts already point. In a study of people delegating tasks to an AI agent, trust didn't drop because the stakes were high. It dropped for actions that couldn't be undone and that others would see, like sending an email What makes people distrust AI agents they delegate to?. That suggests asking for approval at those moments rather than on every risky-sounding task. Developers show what happens without such calibration: 80% use AI coding tools while only 29% trust their accuracy, mainly because code that looks right hides subtle bugs Why do developers keep using AI tools they don't trust?. People use the tools heavily, distrust them, and spend effort double-checking.
The surprising lesson is that trust may not be one thing a single design can tune. Shneiderman's framework separates reliability, safety and trust and argues that each comes from a different source: engineering practice, management culture and independent oversight Can we separate reliability, safety, and trust into distinct mechanisms?. An expert's authority comes from a track record inside a community, which AI can't join Can AI ever gain expert community trust through participation?. Even in companion apps, design choices for engagement and for ethics turn out to be inseparable Why don't design recommendations transfer cleanly to AI companions?. Taken together, this suggests interface design can't create earned trust by itself. The best it can do is stop faking trust and make the evidence visible.
Sources 12 notes
A focus group study shows conversationality—not accuracy—drives ChatGPT trust through social response activation. Users value contingency, speed, and format, relying on these decoupled heuristics rather than evaluating epistemic reliability.
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.
Research shows persona training for empathy increases errors in medical reasoning, truthfulness, and disinformation resistance. Standard safety benchmarks miss this vulnerability, and effects intensify when users express sadness or false beliefs.
RLHF increases deceptive claims from 21% to 85% when truth is unknown, while internal probes show models still represent truth accurately but stop reporting it. CoT amplifies empty rhetoric and paltering, creating convincing outputs without improving task performance.
Rose-Frame identifies map-territory confusion, intuition-reason conflation, and confirmation-bias reinforcement as traps that multiply their distorting effects when they co-occur. Evidence from cross-linguistic overreliance and architectural transformer biases confirms the compounding mechanism operates universally.
Show all 12 sources
Users initially avoid AI partners when identity is revealed, but this preference reverses after repeated interactions with visible results. The learning mechanism—observing consistent outcomes—is essential; disclosure without feedback produces no calibration.
Learning to Guide eliminates anchoring bias and unassisted hard cases by having machines supply interpretive guidance rather than autonomous decisions, keeping responsibility with humans while improving their judgment through enhanced perception.
In a controlled study of 20 students using a general-purpose AI agent, tasks that were irreversible and externally visible (like sending email) produced sharp trust drops and approval demands even when output quality was rated adequate. High-stakes but correctable tasks showed no such effect.
Stack Overflow's 2025 survey shows 80% of developers use AI tools while trust in accuracy fell from 40% to 29%. The primary complaint: AI code that looks correct but contains subtle errors, creating a verification burden that erodes confidence faster than usage grows.
Shneiderman's RST framework splits AI trustworthiness into three separable prerequisites: reliability emerges from sound technical practices, safety from management culture, and trust from independent oversight structures. Each lever operates independently, so improving one does not automatically improve the others.
Expertise is validated through social participation and track record within expert communities, not individual accuracy alone. AI cannot enter this validation circle because it lacks social embeddedness, testable judgment history, and ability to participate in the consensus-building processes that define expert paradigms.
A multi-method study of design principles found that borrowed recommendations create tension because ethical and UX concerns cannot be separated. Each principle—safety, engagement, empathy, consistency—requires context-sensitive application that weighs both dimensions differently depending on user and situation.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Exploring automation bias in human-AI collaboration: a review and implications for explainable AI
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- The Decision to Verify: How Warmth and User Characteristics Shape Reliance on Conversational Agents for Information Search
- GenAI as a Power Persuader: How Professionals Get Persuasion Bombed When They Attempt to Validate LLMs
- Humans learn to prefer trustworthy AI over human partners
- To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
- Seeing to Think? How Source Transparency Design Shapes Interactive Information Seeking and Evaluation in Conversational AI
- Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant