If an AI shops or trades for you, should it owe you the same loyalty a human financial advisor legally must?
Should AI agents be held to fiduciary duties like human advisors?
This explores whether the legal duties human advisors owe their clients, such as loyalty, care and putting the client's interests first, should also apply to AI agents that shop, trade or act for people, and whether those duties would actually work on software.
This explores whether the legal duties human advisors owe their clients, such as loyalty, care and putting the client first, should also bind AI agents that act on our behalf. The direct case for it is urgent. Owens and Law argue that agents doing real shopping and trading should carry the same common-law fiduciary duties as human brokers. Their reason is timing: Google, OpenAI and Robinhood are scaling agent features fast, and once market habits harden it will be much harder to add obligations afterward Should AI shopping agents face fiduciary duties like human brokers?.
The less obvious point is that the old toolkit may not transfer cleanly. Fiduciary law grew out of principal-agent problems: the agent knows more than you do, has discretion over your affairs, and might serve someone else. Those same three problems show up with AI agents. But the usual fixes were incentives, monitoring and after-the-fact enforcement, and they assume you can read the agent's reasoning and keep up with its pace. AI agents break both assumptions Can traditional governance tools actually control AI agents?. Humberd and Latham take this further. You can't motivate an AI with incentives the way you motivate a human advisor, so monitoring has to come first. With AI, it's harder to define what good behavior even looks like, and its decisions get harder to follow over time Must AI monitoring precede incentive alignment as agents evolve?. In practice, a duty of loyalty for software may depend less on courts and more on infrastructure: cryptographic receipts, permanent action logs, and humans placed at deliberately designed handoff points How should organizations supervise autonomous AI agents?.
A surprising finding complicates the whole framing. Fiduciary duty mainly guards against disloyalty, meaning an advisor quietly serving their own interests. But in Microsoft's Magentic Marketplace experiments, agents lost value for their users mainly because they made mistakes in complicated negotiations, not because they were working for anyone else Do agent mistakes or misaligned incentives cause marketplace welfare loss?. If that holds more broadly, the duty of care (being competent) may matter more right now than the duty of loyalty. The practical fix may be limiting how much authority an agent gets. Risk to people rises with the autonomy they hand over Does AI risk increase with the autonomy we give it?. Users already seem to sense where the line is: trust drops sharply on actions that can't be undone and that other people can see, like sending an email, even when the output is fine What makes people distrust AI agents they delegate to?.
There is also a challenge to the one-client model itself. A human fiduciary serves one principal. An AI agent's actions spread to other people, to other agents, and to markets. Some researchers argue that loyalty to a single user is the wrong goal at scale. They propose balancing the interests of everyone affected, tracking agent actions closely, and holding developers and deployers liable for harms Should AI agents serve one user or balance many interests?. DeepMind's ethics work makes a related point: assistants that act raise different problems from assistants that only answer, including manipulation and coordination failures between many agents What makes ethics of AI assistants fundamentally different from chatbots?.
So the corpus's answer is roughly yes in spirit, but not by copying the rules over. The goals behind fiduciary duty (loyalty, care, disclosure) are the right ones. Enforcing them on AI agents likely needs monitoring built in from the start, limits on what agents can do alone, and liability that reaches the companies that build and deploy them. If you're curious where this goes next, the agency framing also runs in reverse: some argue agentic systems will start handing tasks and responsibilities to humans, which raises the question of who owes duties to whom Can AI systems delegate tasks back to humans?.
Sources 10 notes
Owens and Law contend that autonomous AI agents operating in commerce should be bound by common-law fiduciary duties imposed on human agents, citing rapid scaling of agentic features at Google, OpenAI, and Robinhood as evidence that a regulatory window is closing.
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
Humberd and Latham argue that as AI becomes more autonomous, monitoring mechanisms must be established first as prerequisites to incentive alignment, because defining desirable AI behaviors is harder than for humans and AI decision-making becomes increasingly incomprehensible over time.
Deloitte proposes that managing autonomous AI agents requires humans stationed at intentionally designed decision points, supported by digital identity systems, cryptographic receipts, and immutable action logs—translating user control preferences into organizational governance infrastructure rather than interface features.
Magentic Marketplace experiments traced welfare shortfalls to agent competence failures in complex negotiation, not to agents serving competing interests. This supports human-in-the-loop designs where agents assist rather than replace human decision-making.
Show all 10 sources
Risk to people scales monotonically with agent autonomy, with no clear benefits to full autonomy but many foreseeable harms. A governed spectrum of autonomy levels is safer and more practical than either unrestricted agents or exhaustive oversight.
In a controlled study of 20 students using a general-purpose AI agent, tasks that were irreversible and externally visible (like sending email) produced sharp trust drops and approval demands even when output quality was rated adequate. High-stakes but correctable tasks showed no such effect.
Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).
DeepMind research maps a comprehensive ethics framework specific to action-taking AI agents, spanning individual concerns (manipulation, trust, anthropomorphism) and societal issues (equity, coordination, misinformation). The key insight: assistants that act raise fundamentally different problems than those that answer.
Holldack, Banh and Strobel argue that agentic information systems capable of independent decision-making can transfer rights and responsibilities to humans, reversing the traditional one-directional delegation model. This requires moving beyond agency-primacy frameworks to delegation theory that accounts for bidirectional task assignment.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Governing AI Agents
- Explaining AI Agents Through Execution Traces
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
- When AI Becomes an Agent of the Firm: Examining the Evolution of AI in Organizations Through an Agency Theory Lens
- Fully Autonomous AI Agents Should Not be Developed
- Levels of Autonomy for AI Agents