As AI assistants juggle multiple people in one chat, who should the rules track: fixed job titles, or who actually knows what right now?
How should governance frameworks represent information access and authority shifts in real time?
This explores how AI governance rules could keep track of who holds which information and who has decision-making power while those things change during a live interaction, as opposed to assigning fixed roles once and leaving them.
This explores how governance could follow who knows what and who is in charge as those things change during a live interaction, instead of fixing roles in advance. The corpus starts with the gap itself. A scoping review of the OECD, NIST and EU AI Act frameworks finds that they name the affected stakeholders but miss how information, authority and agency move between people within a single conversation with a multi-party assistant Do current AI governance frameworks capture shifting stakeholder roles?. The review's recommendation is to govern at the points where information is handled and actions are taken, not through a list of roles. Agency law and principal-agent economics explain why this matters. Their three classic problems are information asymmetry, discretionary authority and loyalty, and the usual fixes (incentives, monitoring, enforcement) assume someone can watch and interpret the agent's decisions. Fast, opaque AI agents break that assumption Can traditional governance tools actually control AI agents?.
The surprising answer from the corpus is that governance works best when it lives inside the system the agent actually runs on, not in a policy document beside it. In a market simulation, a written anti-collusion constitution did almost nothing. A public, enforced governance graph, with an oracle, a controller and an audit log that can't be altered, cut collusion substantially Can runtime governance reduce LLM market collusion more than written rules?. A long-running persistent agent showed the same pattern. Safeguards written into the memory layer it consulted while working produced 889 recorded governance events over 96 days, because the agent ran into the rules while making decisions Can governance rules embedded in runtime memory actually protect autonomous agents?. So representing authority in real time may mean making it a structure the agent reads and the system enforces, not a description humans keep elsewhere.
The structure has to be wired correctly, though. Older multi-agent organization models treat authority as nested: you can only have authority over a role you can communicate with, and you can only communicate with roles you can represent. But they check this once, when the system is specified, not while it runs How do authority, communication, and acquaintance relate in organizations?. Experiments with RACI accountability tags make the same point. Labeling an agent "accountable" changed outcomes only when the workflow actually routed the deliverable through that agent Does accountability placement actually shape multi-agent outcomes?. A label of authority that the control flow ignores does nothing. Singapore's IMDA framework turns this into concrete design levers: the action space (which tools and permissions an agent has) and the autonomy scope (how much it decides alone) How should organizations govern the risks of agentic AI?. The design community's "autonomy slider", which lets a user set an agent to suggest, draft or execute for each task, is the user-facing version of the same idea Should agent autonomy be a continuous slider or fixed levels?.
The human side matters too. Leonardi's research shows that authority can shift even when nothing formal changes. When people credit an AI with judgment, they feel less in control and less responsible while still holding formal authority. That displacement runs in cycles that can be reversed Does attributing agency to AI actually reduce human sense of control?. In radiology, scheduling and policing, some people gained new authority by managing how agency was attributed, for example as interpreters or curators. Early attribution patterns then locked in which way things went Can humans gain authority by managing how AI agency gets attributed?. A real-time governance record that tracks only formal permissions would miss these felt and social shifts. Meanwhile, BCG's survey of nearly 12,000 employees finds that agent use has more than doubled while half of respondents lack clear governance for human-AI teams How far behind is AI governance compared to agent deployment?.
The corpus has no worked-out schema for logging information and authority changes turn by turn in multi-party settings. That remains an open design problem. What it does offer is a direction: put governance in the runtime, make sure routing enforces accountability, treat autonomy as something you can adjust, and expect authority to shift through attribution as well as through permissions.
Sources 11 notes
A scoping review found that OECD, NIST, and EU AI Act frameworks name affected stakeholders but fail to track whose information, authority, and agency shift within single interactions. Governance needs finer-grained relationships at points of information and action handling, not just fixed role categories.
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
A public, enforced governance graph reduced Cournot collusion from mean tier 3.1 to 1.8 across 90 runs per condition, while a written anti-collusion constitution showed no reliable improvement over ungoverned baseline. The architecture uses an Oracle, Controller, and immutable audit log to shape incentives without rewriting agent outputs.
A persistent agent recorded 889 governance events across 96 active days, with safeguards encoded directly into the memory layer the agent consulted during operation. Runtime-resident governance proved more effective than external policies because the agent actually accessed it during decision-making.
MOISE+ formalizes a one-way hierarchy where authority over a role presupposes a communication channel to that role, which in turn presupposes agents can represent each other. The model enforces this at specification time, not runtime.
Show all 11 sources
Ablation studies show RACI accountability tags change outcomes only when the collaboration protocol's control flow passes the deliverable through the accountable agent. The best placement varies by model family, requiring per-model validation rather than universal design.
Singapore's IMDA frames agentic AI oversight around risk bounding, human accountability, technical controls, and end-user responsibility, grounded explicitly in two levers—action-space (tools and permissions) and autonomy (decision-making scope). The framework treats these as design levers organizations tune directly, moving beyond abstract principles to actionable controls across the agent lifecycle.
The AI UX Playground glossary names the autonomy slider as a continuous control letting users dial agent independence per task, collapsing agency levels into three adjustable points: suggest-only, draft, execute. The pattern shows shared vocabulary among designers, though validation against other control shapes remains absent.
Leonardi shows that when humans credit AI with independent judgment, they experience reduced sense of control and responsibility—even while retaining formal authority. This displacement circulates through five phases: delegation, attribution, contingency, reassertion, and reconfiguration, making it strategically manageable rather than permanent.
Across radiology, scheduling, and policing, Leonardi found humans developed novel forms of authority—as expert interpreters, social mediators, and knowledge curators—by strategically managing how agency was attributed in human-AI relationships. Early attribution patterns locked in path-dependent trajectories that enabled this reconstitution of power.
BCG's survey of 11,749 employees shows agent integration more than doubled since 2025, yet half lack clear governance for human-AI teams. Accountability concerns span all organizational levels equally.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Governing AI Agents
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
- Homo agenticus in the age of agentic AI: Agency loops, power displacement, and the circulation of responsibility
- Explaining AI Agents Through Execution Traces
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- When AI Becomes an Agent of the Firm: Examining the Evolution of AI in Organizations Through an Agency Theory Lens
- AI Agents Push Humans Out of the Loop
- When Single-User-Oriented LLM-based Assistants Involve Others: A Scoping Review of Pathways, Risks, and Responses