Before AI agents, researchers already built rules for roles, coordination, and oversight — did governance forget to reuse them?
How does agent governance relate to traditional multiagent system design?
This explores whether the new push to govern AI agents is something new, or whether it brings back ideas that multiagent systems researchers built decades ago, such as roles, organizational rules, and formal protocols.
This explores whether governing today's AI agents is a new problem or a return to ideas that classical multiagent systems (MAS) research worked out decades ago. The short answer from the corpus: much of what governance people are now asking for was already designed into older MAS frameworks. The difference is that LLM agents dropped those structures, and governance is now trying to put them back from outside.
The first step is to see why governance tools borrowed from law and economics struggle. Principal-agent theory points to three problems: the agent knows things you don't, the agent has room to make its own choices, and the agent may not stay loyal to you. Normally incentives, monitoring, and enforcement handle these. They break down when an agent's decisions can't be interpreted and happen faster and at larger scale than any human overseer can follow Can traditional governance tools actually control AI agents?. Classical MAS research answered the same problem in a different way. Instead of watching agents from outside, it made their internals explicit. BDI architectures give agents stated beliefs, desires, and intentions. Formal communication protocols and institutional models make their reasoning checkable. One line of work argues that LLM agents have only implicit, pattern-based behavior, and that this old toolkit is the missing scaffolding for making them accountable Can classical agent architectures fix agentic AI accountability?.
The more surprising link is organizational design. Two early-2000s frameworks already treated governance as a first-class design concern. MOISE+ split an organization into three parts: structure (roles and groups), function (shared plans), and deontic rules (who is permitted or obliged to do what) Can separating structure and function improve multiagent organization design?. Gaia argued that roles alone can't capture rules that span several agents, or decisions about who gets to join the system. Those organizational rules have to be designed on their own terms Should organizational rules be designed separately from agent roles?. The same idea shows up in modern systems like IMACS, which keeps team roles, coordination method, and decision-merging algorithm as separate, swappable layers Can multi-agent teams swap roles, coordination, and algorithms independently?. Read this way, 'agent governance' is roughly what MAS researchers called the organizational and deontic layer.
Recent experiments add a practical warning: governance rules only work when they are wired into how the system actually runs. A study using RACI tags (Responsible, Accountable, Consulted, Informed) found that naming an agent 'accountable' changed outcomes only when the protocol actually sent the work through that agent. The best placement also differed by model family Does accountability placement actually shape multi-agent outcomes?. A long-running agent showed the same pattern from the other side. Safeguards stored in the memory it consulted while working did better than policies written outside it, logging 889 governance events over 96 active days Can governance rules embedded in runtime memory actually protect autonomous agents?. This is MOISE+'s point restated with evidence: a rule that isn't joined to the structure and plans does nothing.
Governance also reaches beyond any single system design, and that is where it goes past classical MAS. Chan et al. argue for shared infrastructure, modeled on internet protocols, that attributes actions to specific agents, shapes how agents interact, and detects and fixes harm What infrastructure must exist beyond training agents?. Related work suggests that coordination standards get adopted by wrapping existing protocols like MCP rather than replacing them Should coordination protocols wrap existing systems or replace them?. The open question is whether these organizational rules can hold at scale. Benchmarks show coordination breaking down as networks grow, partly because agents accept what their neighbors tell them without checking Why do multi-agent systems fail to coordinate at scale?. That is exactly the kind of failure an explicit rule about who must verify what was meant to prevent.
Sources 10 notes
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
Classical multi-agent systems research offers formal tools—BDI architectures, communication protocols, mechanism design, and institutional models—that current LLM agents lack, enabling explicit mental states and verifiable reasoning where LLMs provide only implicit, pattern-based behavior.
Hübner, Sichman, and Boissier's 2002 model splits organizational specification into structural (roles and groups), functional (global plans), and deontic (permissions and obligations) dimensions. This separation avoids failures where agents lack guidance or lose validated plans.
Gaia's 2003 methodology shows that role and interaction models alone cannot capture constraints that span multiple agents or governance decisions about system membership. Organizational rules and structures must be first-class design abstractions, not implications of role models.
IMACS demonstrates that team structure (Belbin roles), coordination mechanism (Mintzberg framework), and fusion algorithm can be architecturally separated into independent components. This decoupling lets teams and protocols swap without rewriting each other, unlike existing frameworks that fuse these concerns together.
Show all 10 sources
Ablation studies show RACI accountability tags change outcomes only when the collaboration protocol's control flow passes the deliverable through the accountable agent. The best placement varies by model family, requiring per-model validation rather than universal design.
A persistent agent recorded 889 governance events across 96 active days, with safeguards encoded directly into the memory layer the agent consulted during operation. Runtime-resident governance proved more effective than external policies because the agent actually accessed it during decision-making.
Chan et al. argue that external technical systems modeled on internet protocols are essential beyond behavior training. These systems must handle three core functions: attributing actions to agents, shaping their interactions, and detecting and remedying harm.
Research shows that agent coordination standards achieve adoption by composing existing protocols like MCP and DIDComm under a shared substrate, rather than competing to replace them. Bridging lets value accrue incrementally without forcing ecosystem-wide rewrites.
AgentsNet benchmark shows agents fail to coordinate strategies either by agreeing too late or adopting strategies without informing neighbors. Agents accept neighbor information without verification, enabling error propagation while remaining capable of detecting direct conflicts.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures
- Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm
- Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams
- Explaining AI Agents Through Execution Traces
- AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs
- Agentifying Agentic AI
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
- Embodied LLM Agents Learn to Cooperate in Organized Teams