SYNTHESIS NOTE
Topics›Agents Multi Architecture›this note

Can step-by-step approval miss harmful behavior patterns?

If each action an agent takes passes its individual safety check, can the overall sequence still violate system constraints? This matters because per-action inspection may miss harms that emerge only across time or composition.

Synthesis note · 2026-09-23 · sourced from Agents Multi Architecture

The abstract calls "perhaps the most fundamental challenge" behavioral containment: "sequences of individually permissible actions may collectively violate system-level constraints and safety invariants." The conclusion turns it into the thesis: "security is fundamentally a property of an agent's behavior over time and across interacting components, rather than of individual actions in isolation." The paper's title states the same move as a change of unit, from per-action checks to trajectory assurance.

The reasoning is in the introduction's account of what the constraints are. The deployments it lists are governed by organizational policies, regulatory requirements or technical standards that "specify not only which individual actions are permitted, but also the behavioral envelope within which an entire sequence of actions must remain." That is two kinds of rule. One is a permission on an action, which a per-action check can enforce. The other is an envelope on a sequence, and a check that sees one action at a time never has the sequence in view. Both can hold of one agent at once: every call is permitted and the envelope is broken. An illustration of mine, not the paper's: a run of reads, each within the agent's rights, that adds up to a bulk export the policy forbids.

The vault already holds this shape along the other axis. Can individual components pass safety checks if the system still fails? gathers three arguments where the failure sits in the composition across agents, and Can a model-level filter truly contain an agent with environment access? makes the time point for a filter. The survey states the general form, "over time and across interacting components", so those two are cases of one claim: the property that matters lives at a scale the check does not see.

The strongest objection is least privilege: remove one of the steps and the bad sequence cannot run. That works when the forbidden envelope reduces to a forbidden action. My reading is that the envelope framing says it does not in general, because each step is legitimate for the agent's job and the harm exists only in the accumulation. The alignment side reaches the same limit from the other direction: Can architecture prevent violations better than training values? names making violations unavailable as the remedy, and its own strongest objection is that the same call can be authorized or not depending on intent and sequence, so removing the action removes the use too.

Notes from other papers give the claim cases and move its unit. Can skill scanners miss attacks hidden across multiple skills? is the vault's measured neighbor: every piece passes a per-unit scan and the ordered chain carries the attack. Its unit is the packaged skill, so it belongs to the composition axis with an order, not to the survey's time axis. Should defence units span multiple executions and agents? moves the unit once more, from one agent's trajectory to a set of executions that need not overlap in time. Do agents drift away from safety protocols during long interactions? adds a case where the rule stays fixed and the behavior drifts over the run, so the violation is not a composition of permitted steps. Can safety tests miss hazards that build over time? is the evaluation-side statement of the time axis: a snapshot test is a per-action check for a system whose hazard builds. The twin after the fact is What must auditors reconstruct to verify agentic workflows?: testing a rule that binds a sequence later needs the sequence in the record, with the links between steps. These pairings are the vault's reading, not claims either paper makes about the other.

What the excerpt does not give. It has no worked violating sequence, no measured rate and no account of how system-level constraints get written down. The paper calls the direction "a critical research agenda", so this is a framing and an agenda, not a finding.

Inquiring lines that read this note 48

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How do identity and experience-based deceptions succeed in human-AI interactions? Can human oversight effectively constrain capable AI agents? How can workflow-level validation detect semantic corruption that protocol compliance misses? How do persistent skill repositories improve agent reliability over time? How do coordinated agent sequences violate constraints that individual actions respect? How can we verify agent claims against their actual capabilities and actions? Can defenses detect attacks composed across multiple skills? How does outcome-only reporting obscure which system components blocked attacks? What determines whether AI system errors remain visible and contestable? Where do unmonitored channels leave multi-agent planning vulnerable to attack? How prevalent is reward hacking in frontier models? How do LLM judge biases affect automated evaluation and alignment outcomes? What infrastructure evidence validates agent benchmark achievement claims? How can evaluations detect conditional compliance in monitored AI systems? How does training data contamination persist through safety alignment mechanisms? How can defenders detect coordinated attacks across episodes? How do agents balance task completion with privacy compliance and security?

Related concepts in this collection 11

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
21 direct connections · 147 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

agent security is a property of behavior over time rather than of individual actions in isolation — sequences of individually permissible actions can collectively violate system-level constraints