INQUIRING LINE

What do office workers actually need from an AI agent before they'll trust it enough to use it at all?

How do workplace users prioritize safety and control in AI agent design?

This explores what people using AI agents at work actually want from them when it comes to staying safe and in control, and how designers are turning those wants into concrete interface choices.


This explores what workplace users want from AI agents when it comes to safety and control, and how designers are turning that into real interface decisions. The most direct evidence comes from a multi-method study of business users. They named human control, reliability, context-awareness and safety as practical requirements. To them these decide whether a tool gets adopted at all, not whether it's merely nice to use What UX principles do workplace users want in AI agents?. The interesting part is what 'control' turns out to mean once you look at how designers put it into practice.

It mostly doesn't mean watching the agent constantly. Gunasekaran argues that what makes an agent feel trustworthy is *perceived* control: the user can see a clear way to step in, even if they rarely use it Does perceived control matter more than actual control for AI trust?. Control also moves earlier in the process. Instead of a single goal box, agent interfaces should collect intent up front: limits, risk tolerance, forbidden actions, and the conditions under which the agent needs permission. A vague instruction gets carried through every step the agent takes without anyone reviewing it Should agentic AI interfaces require users to specify risk tolerance upfront?. Designers are settling on a shared vocabulary for this. One example is the 'autonomy slider', a per-task setting that runs from suggest-only to draft to execute. Nobody has yet tested it against other ways of shaping control Should agent autonomy be a continuous slider or fixed levels?. Salesforce's design system goes further and builds 'mindful friction' into the moments that can't be undone, because that is where users tend to lose trust How can product design build human trust in AI systems?.

The less obvious finding is that safety at work can't be fully designed into the agent itself. One framework applied to more than 2,000 real job tasks found risks that come from how agents, goals, work environments and human relationships interact. These risks appear even when every single part works as intended Can workplace AI risks emerge from interactions alone?. Shneiderman's framework reaches a similar conclusion from the organizational side. Reliability comes from engineering practice, safety from management culture, and trust from independent oversight. Improving one doesn't automatically improve the others Can we separate reliability, safety, and trust into distinct mechanisms?. So when a workplace user asks for 'safety', part of what they need lives outside the product.

Two more findings complicate the picture. First, control may start flowing both ways: agentic systems can hand tasks and responsibilities back to humans, so the user is no longer the only one delegating Can AI systems delegate tasks back to humans?. Second, these demands for control rest on today's limited capability. Leading agents complete only about 30% of tasks in a simulated workplace and struggle most with social interaction and unfamiliar professional software Why do AI agents fail at workplace social interaction?. Part of why users want visible ways to intervene is simply that agents still fail often.

The surprise is that workplace tools may be the good case. A review of nine deployed agents found general productivity assistants like ChatGPT and Claude scored highest on interaction principles. Companion and advisory agents scored lowest, especially on preventing dependency and making the agent's intent visible Do productivity agents actually protect users better than emotional ones?. Business settings seem to push design toward control in a way that emotionally charged settings haven't yet.


Sources 10 notes

What UX principles do workplace users want in AI agents?

A multi-method study identified eight UX principles for workplace AI agents, with business users weighting human control, reliability, context-awareness, and safety as practical necessities for user acceptance and effective collaboration.

Does perceived control matter more than actual control for AI trust?

Gunasekaran argues users need the felt ability to intervene in agentic AI systems to feel safe, not hands-on constant control. The design implication is that UX must make intervention capability visibly available and perceptible, even when users rarely exercise it.

Should agentic AI interfaces require users to specify risk tolerance upfront?

Gunasekaran argues that autonomous AI requires interfaces capturing direction, limitations, risk tolerance, forbidden actions, and permission conditions before execution—not just goal fields. Underspecified intent propagates through every agent action without user review, making upstream specification critical to preventing irreversible harm.

Should agent autonomy be a continuous slider or fixed levels?

The AI UX Playground glossary names the autonomy slider as a continuous control letting users dial agent independence per task, collapsing agency levels into three adjustable points: suggest-only, draft, execute. The pattern shows shared vocabulary among designers, though validation against other control shapes remains absent.

How can product design build human trust in AI systems?

Salesforce's design system sorts AI safeguards into five categories: mindful friction, AI-content awareness, bias safeguards, explainability, and hallucination reduction. Each is implemented as concrete UI patterns rather than abstract principles, targeting the irreversible actions where trust withdrawal typically occurs.

Show all 10 sources
Can workplace AI risks emerge from interactions alone?

A framework modeling agents, goals, environments and human relationships showed that interaction-driven risks can arise even when every component works as intended. Applied to 2,078 O*NET tasks, it identified 8,356 scenarios where goal and environment mediation, alongside agent-human relationships, create risk pathways.

Can we separate reliability, safety, and trust into distinct mechanisms?

Shneiderman's RST framework splits AI trustworthiness into three separable prerequisites: reliability emerges from sound technical practices, safety from management culture, and trust from independent oversight structures. Each lever operates independently, so improving one does not automatically improve the others.

Can AI systems delegate tasks back to humans?

Holldack, Banh and Strobel argue that agentic information systems capable of independent decision-making can transfer rights and responsibilities to humans, reversing the traditional one-directional delegation model. This requires moving beyond agency-primacy frameworks to delegation theory that accounts for bidirectional task assignment.

Why do AI agents fail at workplace social interaction?

TheAgentCompany benchmark shows leading agents achieve 30% task completion in a simulated workplace. Social interaction, professional UI navigation, and domain-specific knowledge are the three primary failure modes, with multi-turn task performance consistently dropping to 35% across enterprise settings.

Do productivity agents actually protect users better than emotional ones?

A heuristic evaluation of nine deployed agents found that general-purpose productivity systems (ChatGPT, Claude) scored substantially higher on 14 HCI principles than emotionally sensitive or high-stakes agents (Character.ai, Consensus, Pine AI). Risk-critical principles like dependency prevention and intent visibility were most violated in companion and advisory contexts.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.