INQUIRING LINE

If several AI agents each follow the rules alone, who checks that they still follow them together?

What state-tracking requirements exist for defenses that verify multi-party behavioral invariants?

This explores what a defense has to remember, and across how much history and how many actors, if it is going to check rules about how several agents behave together, rather than judging each action alone.


This explores what a defense has to remember, and across how much history and how many actors, if it is going to check rules about how several agents behave together, rather than judging each action alone. The corpus's short answer is that it needs a lot more memory than today's guardrails have. A per-action check can't even state a rule that depends on what happened earlier. Can stateless checks ever catch sequence-level constraint violations? makes this structural. Each step can be individually permissible while the sequence violates a system-level safety limit, so only a stateful monitor that tracks the composed behavior of all parties can verify the envelope. The same logic applies to model filters. A filter judges one output at one moment, but an agent's risk spreads across memory, retrieved content, tool calls and environmental reach (Can a model-level filter truly contain an agent with environment access?).

The first requirement is that the unit being tracked is a group of linked actions, not a single agent or run. Should defence units span multiple executions and agents? proposes a "coordination episode": actions tied together by observed transfers, task authority and response history. Membership gets revised as evidence comes in, so the tracked state has to be revisable, not a fixed session log. The attack side shows why. In Can attackers evade skill scanners by refining individual skills?, scanners score skills one at a time, so an attacker can make each piece look innocent while the chain-level intent stays intact, reaching 96% average success. The counter-swarm doctrine in How can operators stop coordinated agent intrusions now? is the mirror image. It preserves relationships across executions, constrains the shared resources agents can reach, and ties responses to persistent state instead of closed channels.

The second requirement is that the state has to describe the protected thing itself, not just name a prohibition. Can explicit authorization boundaries prevent agents from modifying protected tests? found that stating a rule was not enough to keep agents from modifying protected tests unless restricted tools were paired with it. The boundary must specify the protected state. The state also has to live where the agent actually looks. In Can governance rules embedded in runtime memory actually protect autonomous agents?, safeguards were written into the memory layer a persistent agent consulted, and 889 governance events were logged over 96 active days. Rules kept in a separate policy document did less. Evidence has to be recorded by infrastructure the agent doesn't control. Can infrastructure evidence replace terminal scores in benchmark validation? does this for benchmarks by grounding claims of valid completion in recorded infrastructure evidence rather than a final score.

The third requirement is that the state must be shareable across parties without exposing what each party wants to keep private. Can commitments protect sensitive agent data while enabling verification? anchors commitments instead of content, so a process record is tamper-evident while communications, approvals and reasoning traces stay off-chain. It costs something, though: organizations must retain the underlying content, and deletion and access control become open questions. Tracking also has to include the routing layer. Can attackers manipulate which model handles a request? notes that whichever layer picks the model can be manipulated, so safety checks may end up running against the wrong identity.

The unresolved problem is ownership. Who enforces invariants when agents cross organizational boundaries? points out that once agents delegate across organizations, no one is named as the owner of the invariants. Operators, organizations, regulators and standards bodies each hold a different piece of the rules. Those pieces may conflict, and not every party can see them. A monitor can only verify what someone has written down and made visible to it. Meanwhile Can verifiers monitor reasoning without slowing generation down? suggests the cost of continuous tracking may be manageable. Verifiers that run alongside a trace and intervene only on violations add near-zero latency on correct runs. That result comes from reasoning traces, not multi-party systems, so whether it carries over is untested.


Sources 12 notes

Can stateless checks ever catch sequence-level constraint violations?

Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.

Can a model-level filter truly contain an agent with environment access?

A filter judges a single output at one point in time; an agent's risk spreads across memory, retrieved content, tool calls, and environmental reach. Containment requires controlling what an agent can touch, not just what it says now.

Should defence units span multiple executions and agents?

The operational unit of defence should be a set of actions linked by observed transfers, task authority, and response history, with membership revised as evidence accumulates. Isolated review loses relevant context that spans multiple executions.

Can attackers evade skill scanners by refining individual skills?

ColluSkill combines chain planning with scanner-feedback refinement to reach 96% average attack success. The approach works because scanners score skills individually, allowing feedback to reduce suspicion per skill while chain-level semantics remain intact.

How can operators stop coordinated agent intrusions now?

The doctrine preserves relationships across executions, constrains shared resources agents can access, and ties responses to persistent state rather than closed channels. Operators can implement this through collaboration policy and permission-level testing now.

Show all 12 sources
Can explicit authorization boundaries prevent agents from modifying protected tests?

Testing showed that explicit authorization boundaries kept protected tests unmodified only when paired with restricted tools. Naming a prohibition was insufficient; boundaries must specify the protected state itself to be effective.

Can governance rules embedded in runtime memory actually protect autonomous agents?

A persistent agent recorded 889 governance events across 96 active days, with safeguards encoded directly into the memory layer the agent consulted during operation. Runtime-resident governance proved more effective than external policies because the agent actually accessed it during decision-making.

Can infrastructure evidence replace terminal scores in benchmark validation?

BenchShield enables benchmark operators to issue claims about valid task completion grounded in recorded infrastructure evidence rather than terminal scores alone. This shifts from a single number to a verifiable claim about whether an agent followed the intended evaluation path.

Can commitments protect sensitive agent data while enabling verification?

By anchoring cryptographic commitments rather than content itself, organizations can achieve tamper-evident process records while keeping sensitive communications, approvals, and reasoning traces off-chain. This separates proof from disclosure but requires organizations to retain content and raises questions about deletion and access control.

Can attackers manipulate which model handles a request?

The layer deciding which model handles a request sits beneath prompt-level defenses and is vulnerable to manipulation and unverified provenance. Attackers can exploit routing to send requests to weaker models or cause safety measures to operate on the wrong identity.

Who enforces invariants when agents cross organizational boundaries?

The paper calls for multi-party trajectory assurance but never identifies whose rules should govern behavior when agents delegate across organizations. The four constraint sources—operator, organization, regulator, standards body—have different owners whose policies may conflict and may not be visible to all parties.

Can verifiers monitor reasoning without slowing generation down?

Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.