SYNTHESIS NOTE
Topics›Human Centered Design›this note

How can product design build human trust in AI systems?

Salesforce groups AI safety controls into five pattern categories meant to keep humans in control. The question explores whether these specific interface patterns actually address worker concerns about AI oversight.

Synthesis note · 2026-10-21 · sourced from Human Centered Design

Salesforce's Responsible AI & Technology team, working with its design, product, and engineering teams, built "standardized human-at-the-helm patterns" described as "standard guardrails implemented across Salesforce AI products designed to improve safety, accuracy, and trust while empowering the human user." The article sorts these guardrails into five named categories: Mindful Friction ("pauses in the user experience to ensure intentional human engagement at critical junctures"), Awareness of AI ("transparency and awareness of the presence of AI-generated content"), Bias & Toxicity Safeguards, Explainability & Accuracy, and Hallucination Reduction ("a set of policies and prompt instructions to limit the scope of what an AI can generate"). The piece opens by citing "a new survey" in which 63% of global workers say human oversight would build their trust in AI, framing the whole pattern set as a response to that reported preference.

Each category is instantiated with specific product patterns rather than left abstract. Citations expose the source of Einstein-generated content. Transparency of AI Content combines a policy requirement — the AI Acceptable Use Policy mandates disclosure when end users interact with an automated system — with a UI element, the Einstein "sparkles" icon, plus real-time check marks for completed AI processes. Model Containment is a prompt-level rule set that blocks "toxic mirroring" and bars reference to protected attributes (gender identity, age, race, sexual orientation, and others) in generated text. Feedback runs through four mediums — edits, hover-overs, explicit feedback, and thumbs up/down — and email generation adds a mandatory confirmation step before sending. Valence of Buttons keeps "send"/"submit" the same color and font as "edit"/"regenerate," reducing the odds a user hits send without reading. Unchecked Demographics defaults demographic attributes to unchecked in generated marketing segments. Planned additions include Trust Safety Detectors (toxicity warnings surfaced in-context inside Prompt Builder, with explanatory popovers) and Confirmation Steps that extend mindful friction to more Copilot junctures "where AI mistakes could happen."

Against How should AI systems be designed across different interaction phases?, this is a narrower, vendor-specific pattern library: Amershi's set organizes its recommendations by when in an interaction they apply, while Salesforce's five categories organize a single company's shipped UI patterns by which trust function they serve. The result instantiates the oversight leg of Can we separate reliability, safety, and trust into distinct mechanisms? at the interface level — "human-at-the-helm" patterns are independent oversight built into product controls rather than organizational process. The mandatory confirmation step before sending a generated email, and the planned Confirmation Steps for Copilot, also target exactly the kind of irreversible, externally visible action that What makes people distrust AI agents they delegate to? found triggers trust withdrawal and regret.

The article is Salesforce's own account of its design system, not an independent evaluation: it offers no data on whether any of the five categories measurably increases user trust, reduces automation bias, or prevents the harms they target, and the 63%-oversight figure is attributed only to "a new survey" with no named source or methodology. The patterns should be read as one vendor's labeled catalog of what it built and why it says it built it, not as evidence that the catalog works as intended.

Inquiring lines that read this note 3

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How should control and autonomy be allocated in human-AI workplace collaboration? How should organizations govern autonomy and access for agentic AI systems?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 115 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

Salesforce organizes AI trust design into five human-at-the-helm pattern categories