If an AI agent can book, buy, or negotiate on your behalf, who's legally on the hook when it messes up?
What legal framework governs responsibility when AI agents gain discretionary authority?
This explores how the law assigns responsibility when an AI agent acts with real decision-making latitude, such as buying, booking, or negotiating for someone, rather than just following explicit instructions.
This explores how the law assigns responsibility when an AI agent acts with real decision-making latitude, rather than just following explicit instructions. The short answer from the corpus is that no settled legal framework exists yet. The strongest candidate is old law: agency law, the body of common law that already governs brokers, lawyers, and employees acting on someone else's behalf. Should AI shopping agents face fiduciary duties like human brokers? argues that AI shopping agents should carry the same fiduciary duties as human brokers. These are duties of loyalty and care to the person they serve. The authors warn that the window to impose these duties is closing as Google, OpenAI, and Robinhood ship agentic features faster than rules can form.
The catch is that agency law assumes things AI agents break. Can traditional governance tools actually control AI agents? shows that agency law and principal-agent economics name the right three problems: the agent knows more than you do, the agent has discretion, and the agent may not be loyal. But the usual fixes are incentives, monitoring, and after-the-fact enforcement, and they stop working when an agent makes decisions you can't interpret, at machine speed, across thousands of transactions. Must AI monitoring precede incentive alignment as agents evolve? adds that you can't even align incentives until monitoring is in place, because defining good behavior is harder for an AI than for a human employee. So borrowing the human framework gets you the right vocabulary, but you still have no reliable way to enforce it.
With the doctrine unsettled, much of the practical work of assigning responsibility is moving into technical infrastructure. How can AI agents prove who authorized their actions? proposes signed delegation tokens that prove which human authorized an agent, for what scope, and when that authority was revoked. In effect, this is a machine-readable version of the 'scope of authority' that agency law cares about. How can autonomous agents act safely within defined boundaries? writes those limits down explicitly as allowed tools, spending caps, and forbidden actions. How should organizations supervise autonomous AI agents? adds cryptographic receipts and immutable action logs, which create the evidence trail any future liability case would need. Can governance rules embedded in runtime memory actually protect autonomous agents? reports that rules built into the agent's own memory worked better than external policy documents, because the agent actually consulted them while acting.
The less obvious finding is that blame doesn't fall where the formal lines of authority say it should. Does decomposing AI systems shift where people assign blame? found that when people see an AI system described as a language model plus a tool-using agent layer, they blame the agent layer more, because it sits 'closer' to the outcome. Does attributing agency to AI actually reduce human sense of control? shows that people who credit AI with judgment feel less responsible even while keeping formal authority. Can humans gain authority by managing how AI agency gets attributed? shows that early choices about who gets credit tend to lock in. So how a system is designed and described may shape who gets blamed before any court weighs in.
The corpus has little on actual statutes, case law, or regulatory regimes such as the EU AI Act's liability provisions. Can companies alone manage the risks of AI systems? calls for binding government oversight, but its focus is frontier-model risk, not who pays when a shopping agent makes a bad purchase. What the collection does show is that 'who authorized what, within what limits' is being answered by tokens, logs, and scope records faster than by law.
Sources 11 notes
Owens and Law contend that autonomous AI agents operating in commerce should be bound by common-law fiduciary duties imposed on human agents, citing rapid scaling of agentic features at Google, OpenAI, and Robinhood as evidence that a regulatory window is closing.
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
Humberd and Latham argue that as AI becomes more autonomous, monitoring mechanisms must be established first as prerequisites to incentive alignment, because defining desirable AI behaviors is harder than for humans and AI decision-making becomes increasingly incomprehensible over time.
A three-token scheme (User ID, Agent ID, and Delegation Token) signed by human delegators creates cryptographically verifiable authorization chains. Services can now verify who authorized an agent, under what scope, and when revocation occurred.
OpenAI's Model Spec formalizes agent autonomy through structured scope records listing allowed tools, cost limits, and prohibited high-risk activities, mirroring software security's principle of least privilege while avoiding confirmation overload.
Show all 11 sources
Deloitte proposes that managing autonomous AI agents requires humans stationed at intentionally designed decision points, supported by digital identity systems, cryptographic receipts, and immutable action logs—translating user control preferences into organizational governance infrastructure rather than interface features.
A persistent agent recorded 889 governance events across 96 active days, with safeguards encoded directly into the memory layer the agent consulted during operation. Runtime-resident governance proved more effective than external policies because the agent actually accessed it during decision-making.
Study 5 found that when participants evaluated an AI split into a language model and tool-using agentic layer, they attributed greater causality to the agentic part. The mechanism appears to be a proximity heuristic: when both agents share the same nature, people judge the component closer to the outcome as more causal.
Leonardi shows that when humans credit AI with independent judgment, they experience reduced sense of control and responsibility—even while retaining formal authority. This displacement circulates through five phases: delegation, attribution, contingency, reassertion, and reconfiguration, making it strategically manageable rather than permanent.
Across radiology, scheduling, and policing, Leonardi found humans developed novel forms of authority—as expert interpreters, social mediators, and knowledge curators—by strategically managing how agency was attributed in human-AI relationships. Early attribution patterns locked in path-dependent trajectories that enabled this reconstitution of power.
The Future of Life Institute argues that escalating AI incidents demonstrate private companies cannot self-police effectively, and calls for government-mandated limits on recursive self-improvement practices until safety research is complete, backed by hardware verification technology.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Governing AI Agents
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Homo agenticus in the age of agentic AI: Agency loops, power displacement, and the circulation of responsibility
- Authenticated Delegation and Authorized AI Agents
- When AI Becomes an Agent of the Firm: Examining the Evolution of AI in Organizations Through an Agency Theory Lens
- Explaining AI Agents Through Execution Traces
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
- Levels of Autonomy for AI Agents