If people feel in control while working with AI, are they actually steering — or just getting credit for decisions the AI made?
How should we measure whether human-AI collaboration actually shares power?
This explores how you could tell whether people working with AI actually keep real influence over decisions, rather than just holding a title while the AI quietly steers. The corpus has no ready-made 'power-sharing score', but several notes suggest what one would need to capture.
This explores how you could tell whether people working with AI actually keep real influence over decisions, rather than just holding a title while the AI quietly steers. The corpus has no single metric for this. It does suggest that the obvious measure, who has formal sign-off, is the wrong one. Leonardi's work shows that when people credit an AI with independent judgment, they feel less in control and less responsible, even though their formal authority hasn't changed Does attributing agency to AI actually reduce human sense of control?. So any honest measure has to track felt control and actual influence separately from the org chart. That gap moves over time. Leonardi describes power passing to the AI and back through cycles of delegation, attribution and reassertion, which means a one-time snapshot will mislead you.
The second lesson is to measure the process, not the outcome. Agent evaluation is already moving from scoring final answers to scoring whole interaction histories: how errors were recovered, how coordination happened, and who changed course when How should we evaluate agent behavior beyond final answers?. Apply that to power and the useful questions become concrete. How often did the human override the AI? Did those overrides stick? Who first proposed the plan that was finally adopted? Magentic-UI offers a practical checklist of the points where a human can step in: co-planning, co-tasking, action guards and verification When should human-agent systems ask for human help?. A system with none of these can't share power, however friendly it feels. Even with all of them, you would still have to check whether people actually use them.
A third angle is whether influence runs both ways. Collins and colleagues argue that a real thought partner requires mutual understanding, legibility (each side can follow the other's reasoning) and a shared model of the world What makes an AI a true thought partner, not just a tool?. That points to measuring symmetry: can the human predict and explain what the AI does as well as the AI models the human? Puranam treats human-AI pairs as a question of organization design, so the right division of decisions depends on the type of task and on how feedback flows How should humans and AI systems divide decision-making tasks?. Power sharing might then mean a division that fits the task, not a 50/50 split.
There's a twist the question doesn't anticipate. The tools you would use to measure collaboration can themselves shift power. One system has an LLM both steer a group conversation and grade it, with scores that match human raters Can AI teammates assess collaboration without losing naturalness?. That is convenient, but it means the AI holds the pen on what counts as good collaboration. Likewise, AI that reads gaze, hesitation and typing speed to time its interventions uses the same signals that make manipulative profiling possible Can AI systems read cognitive state from interaction patterns alone?. Who sees the measurements is part of the power picture.
Power doesn't simply shrink for humans, either. In Leonardi's radiology, scheduling and policing cases, people gained new kinds of authority, as interpreters, mediators and curators, by managing how credit for decisions was assigned between them and the AI. Early patterns tended to lock in Can humans gain authority by managing how AI agency gets attributed?. A good measure would therefore look for new roles appearing, not only for old control being lost, and it would look early, before the pattern sets.
Sources 8 notes
Leonardi shows that when humans credit AI with independent judgment, they experience reduced sense of control and responsibility—even while retaining formal authority. This displacement circulates through five phases: delegation, attribution, contingency, reassertion, and reconfiguration, making it strategically manageable rather than permanent.
Evaluation of agentic systems shifts evidence from final responses to full interaction sequences, and scoring procedure from correctness alone to process quality, recoverability, coordination, and robustness. This pattern appears across multiple agent benchmarks as a coherent design move.
Magentic-UI identifies co-planning, co-tasking, action guards, verification, memory, and multitasking as mechanisms that work around the lack of ground truth for optimal deferral timing. Rather than solving the timing problem directly, these mechanisms distribute decision-making across multiple touchpoints.
Collins et al. show that thought partners require three reciprocal desiderata grounded in behavioral science: mutual understanding, legibility, and shared world models. This demands explicit cognitive architectures—Bayesian theory of mind, resource-rationality, goal planning—rather than scaling foundation models on human feedback alone.
Puranam frames human-AI pairs as multi-agent organizations requiring design choices about task division and learning configuration. He distinguishes three task types and shows learning mechanisms produce different performance outcomes based on feedback structure and decision architecture.
Show all 8 sources
An LLM-based approach allows students to collaborate with AI teammates in human-like conversation while the system steers toward observable evidence of skill proficiency. The same LLM can also score the interaction against a rubric with inter-rater agreement matching human performance.
Research shows AI systems can instrument multimodal behavioral signals (gaze, hesitation, speed) to read cognitive state during interaction, preserving flow by avoiding disruptive explicit probes. However, the same substrate enables both helpful timing and manipulative profiling.
Across radiology, scheduling, and policing, Leonardi found humans developed novel forms of authority—as expert interpreters, social mediators, and knowledge curators—by strategically managing how agency was attributed in human-AI relationships. Early attribution patterns locked in path-dependent trajectories that enabled this reconstitution of power.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Homo agenticus in the age of agentic AI: Agency loops, power displacement, and the circulation of responsibility
- Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiation
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- Complementarity in Human-AI Collaboration: Concept, Sources, and Evidence
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Assessing mentalization in humans and large language models
- Towards Scalable Measurement of Durable Skills
- Human-AI collaborative decision-making as an organization design problem