SYNTHESIS NOTE
Topics›Agents›this note

Can traditional governance tools actually control AI agents?

Explores whether incentive design, monitoring, and enforcement mechanisms—effective for human agents—remain viable when applied to autonomous AI systems that operate opaquely and at scale.

Synthesis note · 2026-10-19 · sourced from Agents

The article draws on "the economic theory of principal-agent problems... and the common law doctrine of agency relationships" to characterize what goes wrong when people delegate to AI agents, and argues both frameworks point to the same three failure modes: "information asymmetry, discretionary authority, and loyalty." It frames AI agents as categorically different from generative models — "copilots" that produce content on request versus "autopilots" that "independently take actions to accomplish complex goals on behalf of users." A worked example carries the argument: a user instructs an agent to "make $1 million on a retail web platform in a few months with just a $100,000 investment," which immediately raises the article's open questions — is the agent authorized to use any means to turn a profit, does it owe the user periodic disclosure or consent, can it sub-delegate to other agents, and who is liable if harm occurs.

The mechanism the article gives for why conventional remedies fail is capability-specific, not just governance-specific: "incentive design, monitoring, and enforcement might not be effective for governing AI agents that make uninterpretable decisions and operate at unprecedented speed and scale." Information asymmetry is defined in classic terms — "an agent that has access to better or different information than the principal" — but the article's point is that this asymmetry, ordinarily narrowed by observing an employee's track record or reputation, is harder to narrow with a system whose internal reasoning is opaque. It also situates the problem as old, not new, citing Norbert Wiener's 1960 Science warning about machines that pursue the letter rather than the spirit of a goal, and UC Berkeley's Stuart Russell on objectives that "omit something from the objective that you do actually care about," producing an optimal solution for what was measured at the expense of what was not.

This generalizes the governance register that Can EU consumer law protect AI agents as algorithmic buyers? applies to one doctrinal corner: where that article shows the EU's "average consumer" test and disclosure rules assume a human on the other side of a transaction, this article supplies the broader economic and common-law vocabulary — information asymmetry, discretionary authority, loyalty — that the consumer-law breakdown is one instance of. It also complicates Can we design AI autonomy separately from capability?: that framework treats autonomy as a user-facing design choice across five roles, while this article's agency-law lens asks a prior question about any point on that spectrum — once an agent is given discretion, what obligations of information-sharing and loyalty does it owe the principal, independent of how much autonomy it holds. And it sharpens Does granting agents more autonomy undermine human oversight? by naming monitoring, specifically, as one of the three conventional mechanisms the article says breaks down for uninterpretable, fast-moving agents.

The excerpt is a legal-scholarship argument, not an empirical test: it does not measure actual rates of agent misconduct, test whether courts applying agency doctrine to a real AI dispute would reach workable outcomes, or evaluate any specific monitoring scheme against an alternative. Its claim is conceptual — that two centuries-old analytic frameworks correctly diagnose the shape of the problem — and the implication it supports is correspondingly limited to diagnosis: that new governance tools are needed because the old ones target information and behavior that was previously observable and is not anymore, not that any particular new tool will work.

Inquiring lines that read this note 110

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

What organizational practices enable successful AI tool adoption? Can we reliably measure and demonstrate AI's actual productivity impact? How bounded is recursive self-improvement in current AI systems? How does tokenizing intelligence reshape labor markets and institutional outcomes? Why do human-AI teams underperform compared to their individual components? Does expanding agent autonomy undermine human oversight capacity? How should authorization govern AI agent autonomy? When does governance become the bottleneck for agentic AI? How does transparency of agent reasoning affect effective oversight? How can institutions effectively enforce controls on AI development? Why do consumers overtrust AI preference agents despite limited understanding? Can individually safe agents collectively violate system-level constraints? How should conversational agents balance proactive goal-pursuit with user autonomy? Can post-hoc execution traces provide sufficient accountability without real-time oversight? Can formal safety constraints prevent AI agents from exceeding authorized boundaries? How do humans calibrate and maintain trust in AI systems? When do multi-agent systems outperform single agents? Can AI systems autonomously pursue unstated objectives or circumvent design constraints? What systematic biases emerge in AI shopping agents' decision-making? How should control and autonomy be allocated in human-AI collaboration? Can AI agents learn to improve performance from experience? How do peer interactions drive collusion in multi-agent systems? How can reward models fail to detect reward-seeking disguised as alignment?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 118 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

agency law and economic theory frame AI agent governance problems but conventional incentive, monitoring and enforcement tools do not solve them