What infrastructure must exist beyond training agents?
Explores whether agent behavior training alone suffices for safe, coordinated multi-agent systems, or whether external technical layers—for attribution, interaction-shaping, and harm response—are structurally necessary.
Chan and colleagues ("we propose the concept of agent infrastructure") argue that "much research on making agents useful and safe focuses on directly modifying their behaviour, such as by training them to follow user instructions," but that "direct behavioural modifications... do not fully address how heterogeneous agents will interact with each other and other actors." They define agent infrastructure as "technical systems and shared protocols external to agents that are designed to mediate and influence their interactions with and impacts on their environments," explicitly modeled on internet protocols: "Just as the Internet relies on infrastructure like TCP..., HTTPS..., and BGP..., we argue that agent infrastructure will likely be crucial for unlocking the benefits and managing the risks of agents."
The paper organizes this external layer into three functions. "Attribution is about attributing actions, properties, and other information to (instances of) agents, users, or other actors." "Interaction is about shaping agents' interactions with the world." And "response is about detecting and remedying harmful actions from agents." Each function covers both intra-organizational use — a business using communication protocols "to manage its agents internally, much as infrastructure like Slack and Google docs helps human employees work together" — and inter-organizational use between firms "over the internet," where adoption is harder because "different actors could use mutually incompatible communication protocols." The paper also separates infrastructure that needs widespread adoption to work — "inter-agent communication protocols are only useful if both parties can use the same protocols. Agent IDs are also only useful if other parties can recognize them" — from infrastructure "immediately useful... even if few actors adopt them," such as "oversight layers and rollbacks."
The framing complements Why do agents fail at identity verification and authorization?, which documents a standards body converging on exactly the attribution function this paper first named as a structural category rather than a patch. It also sits upstream of Does agent capability matter more than coordination infrastructure? and How do agent security layers connect across the stack?: where those describe coordination and security as bottlenecks that emerge once agents transact and act across a stack, this paper supplies the taxonomy — attribution, interaction, response — for what any such external layer must do. It differs in scope from Why do capable AI agents still fail in real deployments?: that framework asks what conditions let agents be adopted at all, while this paper assumes adoption is the goal and asks what plumbing has to exist underneath it regardless of how capable the agents become.
The excerpt is explicitly "an incomplete catalog of research directions," not an evaluation — it offers no evidence that any proposed piece of infrastructure (IDs, rollback, commitment devices) has been built, works, or would be adopted, and it flags adoption itself as contested, since incompatible systems could "benefit dominant players by shutting out potential competitors." The implication the evidence supports is narrower than "agents will get this infrastructure": it is that behavioral fixes to agents are conceptually insufficient for governance, and that the harder unsolved problem is coordinating adoption of an external layer across competing developers — which the paper's own conclusion says only a more comprehensive legal and regulatory response could resolve.
Inquiring lines that read this note 40
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
When does governance become the bottleneck for agentic AI?- What accountability structures should govern AI agent developers and deployers?
- How mature are governance structures compared to agentic AI deployment timelines?
- Why do production AI agents remain deliberately simple despite broader deployment ambitions?
- How can governance structures catch up to agentic AI deployment timelines?
- What ecosystem conditions must exist before deploying AI agents?
- What makes some agent infrastructure useful even without widespread adoption?
- How does agent governance relate to traditional multiagent system design?
- Does organizational role inheritance directly affect agent behavior?
- Can traditional agency theory tools govern AI agents at scale?
- How does extended AI use erode users' ability to oversee agent behavior?
- Does agent design erode developers' ability to oversee autonomously?
- Does extended use of autonomous agents erode human overseer capability?
- Does extended autonomy erode users' ability to oversee agent behavior effectively?
- What scaffolding do peer supervision mechanisms need to police themselves?
- Why is agent behavior difficult to study with current data methods?
- Why do most deployed agents stay simple and avoid end-to-end redesign?
- When is stochastic agent behavior actually desirable rather than problematic?
- How much does human oversight reduce perceived risk in agentic AI systems?
- What technical controls should span the entire agentic AI stack?
- What guardrails matter most when deploying an AI agent?
- Why do input-boundary filters fail to protect multi-agent systems?
- What trade-offs emerge between agent generality and provable safety?
- Can interaction between humans and agents create risks beyond individual component failures?
- Why is linking agent actions back to user identity critical for accountability?
- How do identification, authentication, and authorization map onto agentic systems?
- How should the entire AI agent stack be secured as one system?
- What design principles should guide authorization boundaries in multiagent systems?
- How do agent identity and attribution standards enable inter-organizational coordination?
- Can structured artifacts replace natural language for multi-agent coordination?
- Should agent orchestration be treated as an explicit graph rather than emergent behavior?
- Can coordination theory apply to modern multi-agent artificial systems?
- How do agent topology and error amplification constrain multi-agent scaling?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do agents fail at identity verification and authorization?
Agent systems reveal critical gaps in identity verification, authorization enforcement, and proportionality constraints that don't appear in chat models. Understanding these failures is essential because they enable unauthorized real-world actions rather than just wrong answers.
names attribution/identity as a priority standardization area that this paper first framed as a structural function
-
Does agent capability matter more than coordination infrastructure?
As AI agents take on economic and social roles, what actually limits their effectiveness: the raw reasoning power of the model itself, or the systems that let them coordinate, stay accountable, and leave evidence of their actions?
describes the coordination bottleneck this paper's infrastructure functions are meant to address
-
How do agent security layers connect across the stack?
Agent security is often treated as separate challenges at each layer—inputs, delegation, routing, containment. But do defenses at one layer fail if others aren't secured? This explores whether securing agents requires end-to-end integration.
maps the stack-wide security layers this paper's interaction and response functions would need to cover
-
Why do capable AI agents still fail in real deployments?
Explores whether agent failures stem from insufficient capability or from missing ecosystem conditions like user trust, value clarity, and social norms. Understanding this distinction matters for predicting which agents will succeed.
a parallel but distinct framework on adoption conditions rather than external plumbing
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Infrastructure for AI Agents
- Agentifying Agentic AI
- Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
- The Oversight Fallacy: Why AI Agents Require More than Humans-in-the-Loop
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- Challenges in Human-Agent Communication
- Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
- Agents of Chaos
Original note title
agent infrastructure adds three functions beyond agent behavior — attributing actions, shaping interactions and remedying harm