Where do user values break down in agent supervision?
When people use AI agents, their values tend to align with delivered outputs but conflict during oversight. What explains this gap, and what does it reveal about delegation design?
The paper starts from the observation that agent evaluations "typically measure task completion rather than the values users prioritize." It applies Value Sensitive Design, with LLM assistance, to 73,093 first-person Reddit posts about using OpenClaw, an open-source agent that runs on the user's own computer and is driven through chat apps. Each post is coded for a human value, an agent aspect, value fulfillment, and user outcome. The result is 21 values in six groups. Two findings carry the claim. Relative to each aspect's corpus share, values cluster "not at the agent's outputs but at the operating conditions users set around a run." And values were usually met where users described what the agent delivered (five of six groups) but mostly unmet where they described supervising it (all six groups). The authors name this pattern value-sensitive delegation.
The mechanism the paper gives is structural. A "run" is the stretch from the user's instruction to the outcome the user reads afterward, and an agent executes it under the user's credentials "with few checkpoints in between," where a conversational assistant returns a patch for inspection. Because VSD treats a value as interactional, taking shape between a technology's properties, the people affected, and the context of use, a delegated agent stretches that relationship across three moments: the user configures, the task runs autonomously, and the user checks the result. The discussion reports that in four of six value groups, posts concentrated on aspects the user configures rather than aspects the agent exercises without the user. The named groups include Bounded Reach, Reviewability, and Equitable Access, and cost, access, and oversight are the conditions the paper flags.
Against the library, this adds a user-value axis to Does a single benchmark score actually predict agent readiness?. That note decomposes capability into task success, privacy, retention, and similar axes. This paper suggests that even when the output axis looks fine, the axes users care about sit elsewhere. It also gives a user-side counterpart to What makes delegation work beyond just splitting tasks?, which frames delegation as a contract with responsibility, trust, and accountability. Here the contract's weak clause, by users' own account, is oversight. The Bounded Reach and Reviewability groups sit near the concern of Does AI risk increase with the autonomy we give it?, though this paper reports what users say about their values, not measured harm.
The excerpt leaves a lot open. It does not give the share of posts in each group, the effect sizes behind "usually" and "mostly," the reliability of the LLM-assisted coding, or which group is the exception to five of six. Its sample is self-selected users writing about one agent, and the design implications in its Section 6.3 are not in the excerpt. What can be said at this strength is narrower than a claim about agents in general. Among people who chose to write about OpenClaw, the reported dissatisfaction concentrates in the supervision phase and the conditions around a run rather than in what the agent hands back. Completion-based evaluation would not register that.
Inquiring lines that read this note 16
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Why do AI agents struggle to generalize and improve reliably? How can AI systems be proactive while respecting user intent? Can humans maintain meaningful oversight as AI systems become more autonomous?- What role does human oversight play in delivering cheap AI services?
- When developers delegate to other agents, what oversight work actually transfers?
- Does delegating to an AI employee differ from delegating to a human subordinate?
- Who decides what work a coworker agent should do daily?
- How does increased agent autonomy affect a human overseer's ability to monitor?
- What makes supervision the point where users fail to achieve their values?
- Can supervision itself be delegated to an orchestrator agent instead of a human?
- When do human values go unmet during AI agent supervision and why?
- How can AI objective functions incorporate multiple conflicting human values?
- What trust calibration failures occur at the moment users must authorize agent actions?
- What makes users withdraw trust from AI agents they initially delegated to?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does a single benchmark score actually predict agent readiness?
Single-axis benchmarks rank models by one capability—like task success—but ignore privacy, duration, operating mode, and ecosystem fit. Can one number really capture what matters for deployment?
extends the multi-axis argument by adding user values, not only capability properties, as what completion-based scores miss
-
What makes delegation work beyond just splitting tasks?
Delegation is more than task decomposition. What dimensions of a task—like verifiability, reversibility, and subjectivity—determine whether an agent can safely and effectively handle it?
the design-side account of the delegation contract, where this paper reports users' unmet values
-
Does AI risk increase with the autonomy we give it?
Explores whether the risks posed by AI agents scale monotonically with the level of autonomy they're granted, and what the tradeoffs are between human control and agent independence.
adjacent concern with bounds on autonomy, but this paper reports user-stated values rather than measured risk
-
Why do AI agents fail at workplace social interaction?
Explores why current AI agents struggle most with communicating and coordinating with colleagues in realistic workplace settings, despite strong reasoning capabilities in other domains.
a completion measure, the kind of evaluation this paper says overlooks the conditions users set around delegation
-
Does granting agents more autonomy undermine human oversight?
Explores whether the design of autonomous AI systems—by giving agents greater independence—actually weakens the human overseer's ability to catch problems. Matters because oversight is a key safeguard against AI failures.
extends: a position paper offers a mechanism for unmet supervision values, arguing greater agent autonomy leaves users less positioned to oversee and that use erodes the skills oversight needs
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClaw
- Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents
- Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Design Principles for Human-Agent Interaction
- Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
- The Work Behind Delegation: A Framework for Supervising AI Coding Agents
- Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
Original note title
human values in everyday agent use are usually met where the agent delivers and mostly unmet where users supervise it — value-sensitive delegation