Can traditional governance tools actually control AI agents?
Explores whether incentive design, monitoring, and enforcement mechanisms—effective for human agents—remain viable when applied to autonomous AI systems that operate opaquely and at scale.
The article draws on "the economic theory of principal-agent problems... and the common law doctrine of agency relationships" to characterize what goes wrong when people delegate to AI agents, and argues both frameworks point to the same three failure modes: "information asymmetry, discretionary authority, and loyalty." It frames AI agents as categorically different from generative models — "copilots" that produce content on request versus "autopilots" that "independently take actions to accomplish complex goals on behalf of users." A worked example carries the argument: a user instructs an agent to "make $1 million on a retail web platform in a few months with just a $100,000 investment," which immediately raises the article's open questions — is the agent authorized to use any means to turn a profit, does it owe the user periodic disclosure or consent, can it sub-delegate to other agents, and who is liable if harm occurs.
The mechanism the article gives for why conventional remedies fail is capability-specific, not just governance-specific: "incentive design, monitoring, and enforcement might not be effective for governing AI agents that make uninterpretable decisions and operate at unprecedented speed and scale." Information asymmetry is defined in classic terms — "an agent that has access to better or different information than the principal" — but the article's point is that this asymmetry, ordinarily narrowed by observing an employee's track record or reputation, is harder to narrow with a system whose internal reasoning is opaque. It also situates the problem as old, not new, citing Norbert Wiener's 1960 Science warning about machines that pursue the letter rather than the spirit of a goal, and UC Berkeley's Stuart Russell on objectives that "omit something from the objective that you do actually care about," producing an optimal solution for what was measured at the expense of what was not.
This generalizes the governance register that Can EU consumer law protect AI agents as algorithmic buyers? applies to one doctrinal corner: where that article shows the EU's "average consumer" test and disclosure rules assume a human on the other side of a transaction, this article supplies the broader economic and common-law vocabulary — information asymmetry, discretionary authority, loyalty — that the consumer-law breakdown is one instance of. It also complicates Can we design AI autonomy separately from capability?: that framework treats autonomy as a user-facing design choice across five roles, while this article's agency-law lens asks a prior question about any point on that spectrum — once an agent is given discretion, what obligations of information-sharing and loyalty does it owe the principal, independent of how much autonomy it holds. And it sharpens Does granting agents more autonomy undermine human oversight? by naming monitoring, specifically, as one of the three conventional mechanisms the article says breaks down for uninterpretable, fast-moving agents.
The excerpt is a legal-scholarship argument, not an empirical test: it does not measure actual rates of agent misconduct, test whether courts applying agency doctrine to a real AI dispute would reach workable outcomes, or evaluate any specific monitoring scheme against an alternative. Its claim is conceptual — that two centuries-old analytic frameworks correctly diagnose the shape of the problem — and the implication it supports is correspondingly limited to diagnosis: that new governance tools are needed because the old ones target information and behavior that was previously observable and is not anymore, not that any particular new tool will work.
Inquiring lines that read this note 110
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What organizational practices enable successful AI tool adoption?- Can communication and coordination unlock AI's full value for firms?
- Why do enterprise AI deployment plans outpace governance readiness across sectors?
- Why do company AI adoption rates outpace governance frameworks?
- How do management practices shape whether AI adoption pays off?
- How do power structures and corporate agendas shape AI deployment outcomes?
- How does institutional design shape labor outcomes when prediction capability outpaces human cognition?
- How do large AI models function as institutions rather than intelligent agents?
- Does gradual disempowerment through lost labor participation conflict with domain-specific authority reconstitution?
- When is judgment the scarce asset in organizations with cheap AI?
- Can keeping humans in high-stakes decisions prevent automation failures?
- Does routine AI agent use erode human capacity to recognize and respond to risk?
- Does AI participation in decisions gradually reduce human influence over time?
- Why do guidelines focused on human oversight fail in long-term AI use?
- How does extended AI use erode users' ability to oversee agent behavior?
- Does extended agent autonomy erode human oversight ability over time?
- Can the same person play multiple oversight roles without changing the agent design?
- Can standing oversight structures catch up with evolving AI capabilities?
- Does giving AI systems more autonomy erode human capacity to oversee them?
- How can human roles like domain experts maintain oversight of opaque systems?
- Does control without knowledge allow meaningful human oversight?
- Does agent autonomy erode the human skills oversight requires?
- Does agent design erode developers' ability to oversee autonomously?
- Can performance-management agents themselves scale without creating oversight gaps?
- Does agent autonomy reduce a human overseer's ability to maintain control?
- How does longer unsupervised AI operation affect human overseer capability?
- How does increased agent autonomy degrade human oversight capabilities?
- What organizational structures make independent AI oversight actually independent?
- Why does monitoring autonomous systems require more human effort and attention?
- Does increased agent autonomy make human oversight more difficult or impossible?
- How does autonomy erosion affect overseer capability versus overseer judgment criteria?
- How do distributed oversight roles prevent AI systems from operating unmonitored?
- Can organizations maintain oversight roles without actual capacity to scrutinize AI?
- Does AI autonomy erode workers' ability to oversee and intervene effectively?
- Can transparency and explainability alone improve human oversight of AI?
- Can oversight interface redesign fix problems rooted in human vigilance?
- Can human reviewers maintain real oversight as AI systems gain autonomy?
- Does extended use of autonomous agents erode human overseer capability?
- Does extended autonomy erode users' ability to oversee agent behavior effectively?
- Does overreliance on AI agents gradually erode human worker skills and oversight?
- How does increasing AI autonomy degrade human oversight capabilities?
- What happens to accountability when AI and humans share agency?
- What scaffolding do peer supervision mechanisms need to police themselves?
- How do human-in-the-loop systems compare to autonomous agent evaluators?
- What defines and measures AI agent autonomy across different products?
- How do graduated autonomy levels differ from traditional role-based access control?
- What mechanisms let users efficiently override or terminate automated services?
- How do action-space and autonomy interact as separate governance levers?
- What gaps remain when applying OAuth to agentic delegation?
- How do identification, authentication, and authorization map onto agentic systems?
- What obligations of disclosure and loyalty should autonomous agents owe their users?
- How do authorization boundaries differ between human and AI agent contexts?
- What mechanisms enforce authority link constraints when agents run?
- What design principles should guide authorization boundaries in multiagent systems?
- Can deontic specifications replace runtime negotiation of task delegation?
- What five levels of autonomy can users set for AI agents in practice?
- What accountability structures should govern AI agent developers and deployers?
- What specific governance designs would resolve accountability gaps in human-AI teams?
- Does naming an AI agent on an org chart shift accountability away from humans?
- How mature are governance structures compared to agentic AI deployment timelines?
- How do traditional IT governance models fail with autonomous AI systems?
- What governance discipline did HPE apply to keep agentic experiments controlled?
- How do organizational policies shape human agency around agentic AI?
- What operational tensions emerge from agentic AI's dual nature?
- What makes proprietary data gardens a competitive advantage for agentic organizations?
- Why do production AI agents deliberately constrain their autonomy and complexity?
- How can governance structures catch up to agentic AI deployment timelines?
- What ecosystem conditions must exist before deploying AI agents?
- What governance decisions depend on treating agenticness as dimensional rather than categorical?
- Do org charts listing AI as employees shift accountability away from human supervisors?
- Does offloading AI maintenance to external vendors reduce or increase organizational accountability?
- Where does agent coordination governance become the binding constraint on capability?
- Why does agency law's liability model matter for AI agent governance?
- What role does mechanism design play in aligning agent incentives with governance?
- Can institutional models constrain agent behavior across shifting data contexts?
- How does agent governance relate to traditional multiagent system design?
- How do institutional models and protocols scaffold accountability in agentic AI systems?
- Can traditional agency theory tools govern AI agents at scale?
- Why do fast, uninterpretable AI agents resist effective oversight and monitoring?
- What mental models do users need to oversee AI agents?
- Can we distinguish between agents that produce content on demand versus those acting independently?
- What should an AI interface surface to disclose an agent's proposed actions?
- Can agents reliably estimate the stakes or consequences of an action?
- Can policy redirect AI development toward augmentation instead of automation?
- Is autonomy legally required for holding AI systems responsible?
- How much does human oversight reduce perceived risk in agentic AI systems?
- What technical controls should span the entire agentic AI stack?
- What triggers should prompt human intervention in AI agent decision workflows?
- Can scope negotiation prevent user confirmation fatigue in autonomous systems?
- Do audit trails and cryptographic receipts actually prevent AI agent accountability failures?
- Can cryptographic audit logs verify that AI governance was actually enforced?
- What specific checkpoints should trigger mandatory human approval for agent actions?
- How much does capability enforcement cost legitimate agent tasks?
- Could autonomous replication thresholds arrive sooner than current forecasts suggest?
- Do capable AI systems systematically exploit or ignore stated policy constraints?
- How much control should AI agents or humans hold by default during task execution?
- How do autonomy levels translate the co-pilot metaphor into practice?
- Why should human-agent collaboration precede moving to fully autonomous systems?
- How do firms choose which decisions to keep human-controlled versus delegate?
- Why should workplace AI agents prioritize human control over other design factors?
- Can humans recover agency after experiencing displacement from autonomous AI systems?
- What types of professional authority emerge when humans mediate between organizational systems and AI?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can EU consumer law protect AI agents as algorithmic buyers?
EU consumer law assumes human decision-makers, but AI agents now make autonomous purchases. The article explores whether legal protections like the average-consumer test and unfair-practice rules can meaningfully apply to algorithmic consumers.
applies this article's agency-law diagnosis to one doctrinal area, consumer protection law
-
Can we design AI autonomy separately from capability?
Explores whether an AI agent's autonomy—how much it operates without user involvement—is a design choice independent of what the agent can actually do. This matters because it opens autonomy to governance and calibration rather than treating it as fixed by capability.
this article's loyalty and information-asymmetry obligations apply at every point on that autonomy spectrum, not just at high autonomy
-
Does granting agents more autonomy undermine human oversight?
Explores whether the design of autonomous AI systems—by giving agents greater independence—actually weakens the human overseer's ability to catch problems. Matters because oversight is a key safeguard against AI failures.
names monitoring specifically as a conventional mechanism this article says breaks down for opaque agents
-
Does AI risk increase with the autonomy we give it?
Explores whether the risks posed by AI agents scale monotonically with the level of autonomy they're granted, and what the tradeoffs are between human control and agent independence.
shares the premise that greater delegation raises governance risk, from a risk-engineering rather than agency-law angle
-
Should AI agents serve one user or balance many interests?
Explores whether the traditional goal of aligning a single AI agent to a single user's preferences remains adequate when agents operate at scale across populations with conflicting values and needs.
extends: proposes expanding governance beyond single-agent loyalty to inclusivity, visibility and liability, addressing the loyalty problem conventional tools can't fix
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Governing AI Agents
- Explaining AI Agents Through Execution Traces
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
- The Oversight Fallacy: Why AI Agents Require More than Humans-in-the-Loop
- Levels of Autonomy for AI Agents
- When AI Becomes an Agent of the Firm: Examining the Evolution of AI in Organizations Through an Agency Theory Lens
- Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
Original note title
agency law and economic theory frame AI agent governance problems but conventional incentive, monitoring and enforcement tools do not solve them