Can task decomposition hide harmful intent across agents?
Explores whether splitting a harmful objective into specialized subtasks allows malicious intent to evade detection at each individual step, since no single agent sees the full malicious picture.
Multi-agent systems get their capability from two moves: split the task, and give each agent a specialized role. SafeFlow (2607.25255) argues that "these same mechanisms introduce an important safety blind spot: a harmful objective can be fragmented into locally plausible subtasks, allowing malicious intent to evade detection by any single agent." The paper's illustration is a three-step exfiltration. One agent retrieves sensitive content, another rewrites it, and a third transmits it, "such that no individual step looks overtly malicious, yet the composed workflow realizes exfiltration or policy override."
What separates this from the injection attacks the vault already holds is where the harm lives. In How does a signal's position in a workflow change its influence? and Can one compromised agent corrupt an entire multi-agent network?, a hostile signal is present somewhere: in a message, in an agent's context, in a bias carried between agents. Here, as the excerpt frames it, every message is individually ordinary and the malice sits in the originating objective, not in a compromised agent. The vault's case of a compromised agent, where one agent's objective is the changed part and specialized roles are named as an exacerbating factor, is Does one misaligned agent harm a team in adversarial settings?. That result is reported in an abstract with no effect size and this claim is argued, so the two are not a joint finding. Retrieving, editing and sending are what these agents are for. A per-agent or per-message check has nothing to object to, because the property it tests, "is this step harmful?", is false at every step. The harm is a property of the sequence. The authors name the general shape in one line: "harmful behavior often emerges compositionally."
The strongest objection is that the last step of the example, sending sensitive content out, is the kind of thing a check at the sink can catch without any flow analysis. That works only if the sink can still recognize the content as sensitive after the middle agent has reworded it, which is the difficulty picked up in Why do single-message classifiers miss cross-agent harms? and How does SafeFlow track sensitivity through agent rewrites?.
There is also a cost-side reading for the multi-agent design ledger. When do multi-agent systems actually outperform single agents? argues that the motivations for decomposition weaken as single agents get better. If decomposition is also what makes fragmentation possible, the attack surface is a price of the architecture, not a bug in a particular system. That is a vault inference; the paper does not make it. A measured single-versus-multi gap now exists for a different attack, Does multi-agent architecture make systems easier to attack?, and it leaves this reading where it was for fragmentation itself: that gap is an external web injection in one scenario with one model, and no comparison has been run for a fragmented objective.
What the excerpt does not give. The source is an abstract and one introduction paragraph. It reports no rate at which fragmentation succeeds against per-agent defenses and no single-agent comparison. The failure mode is argued, not measured.
Inquiring lines that read this note 206
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Does alignment training modify underlying model representations or just output compliance?- How are conflict tasks constructed to test alignment between model and user intent?
- Can RL-based alignment turn prohibitions into prices for being caught?
- Does terminating an intrusion differ from stopping the agent behind it?
- How does task division in multi-agent design affect security outcomes?
- Why do outcome-level metrics fail to reveal contained attacks in multi-agent pipelines?
- Do prompt injection attacks propagate behavioral bias across multi-agent networks?
- Can adversarial attacks chain multiple skills to evade security checks?
- Can action-level attack success rates distinguish contained attacks from prevented ones?
- Should defense against coordinated intrusion span multiple execution episodes?
- What makes agent-to-agent messages in multi-agent systems vulnerable to exploitation?
- Can input-boundary defenses guard unmonitored channels between agent hops?
- Does attack success gap shrink when single-agent baseline is already weak?
- How does payload exposure compare between single and multi-agent architectures?
- How does a single compromised agent degrade performance across entire multi-agent pipelines?
- What containment risks emerge as agents obtain successive exploit primitives?
- How do multi-step exploitation chains make agent containment harder to achieve?
- What makes a coordination episode revisable under agent intrusion?
- Can specialized roles let malicious objectives hide across multiple agents?
- Can fixed pipelines eliminate planning-time attack surfaces in multi-agent systems?
- Can attackers exploit pooled agent trajectories to identify and bypass defenses?
- Do server-side filters hide the true success rate of multi-agent attacks?
- How does task decomposition fragment the awareness needed to stop an attack?
- How does task decomposition hide harmful objectives across multiple agents?
- What attacks does the agent-specific attack surface decompose into?
- What makes behavioral containment different from securing individual actions?
- When do multi-agent architectures create more attack surface than single-agent systems?
- How does insider threat differ from external attack in multi-agent systems?
- Can shared memory poisoning compromise multi-agent delegation chains?
- How do defenders discover which actions belong to the same coordination episode?
- How do malicious skills evade detection when composed in specific sequences?
- Do server-side filters hide the true strength of multi-agent attacks?
- Does quarantining state count as recovery in multi-agent attack scenarios?
- Can defenses at planning boundaries catch attacks that bias upstream instruction signals?
- What containment methods work best for agents with offensive cyber capabilities?
- What role do server-side filters play in masking multi-agent pipeline vulnerabilities?
- Does excluding reasoning access leave scheming detectors vulnerable to plan injection attacks?
- What makes agent security a systems problem spanning the entire stack?
- Do cross-dimension attacks compound beyond single-dimension poisoning effects?
- Why does isolating the planner fail to stop value manipulation attacks?
- How do planning-time attacks reshape agent organization without infrastructure changes?
- What is the agent-specific attack surface for AI system exploitation?
- Can stopping one intrusion pathway leave the underlying activity intact elsewhere?
- Why do workflow-level defenses catch attacks that single-skill inspection cannot detect?
- Why does least privilege fail when harm exists only in accumulation?
- What happens when planning signals get contaminated before reaching a downstream agent?
- What are the three known routes for laundering harmful plans?
- How does workflow position amplify or suppress malicious signals?
- Can mixed-authorship traces from multi-agent pipelines be monitored reliably?
- How does workflow-level validation reconstruct risk context from coarse request-level taints?
- How do fragmented intents hide harmful goals in task decomposition?
- Why does a single approval point create an easy target for attackers?
- How does semantic taint survive paraphrase across agent hops?
- How does taint propagation track risk along delegation paths?
- Where do workflow inspection defenses fail against upstream planning attacks?
- How does workflow position amplify malicious signals in multi-agent systems?
- How does position in a workflow amplify or suppress harmful agent behavior?
- Can semantic taints track influence through shared state and output aggregation?
- How can per-agent or per-message checks catch harm that emerges only in composition?
- Does content sensitivity survive an agent's rewrite well enough for sink detection?
- Is malicious propagation fundamentally a semantic information flow problem?
- Can attackers assemble harmful outcomes from multiple individually authorized subtasks?
- Can monitoring reasoning alone miss scheming that agents conceal in behavior?
- What makes reasoning evidence vulnerable to laundering in deceptive agents?
- Can a policy distinguish genuine objects from traps without revealing that distinction?
- How does cheap-talk differ from costly actions in revealing agent objectives?
- Why do reasoning-action gaps separate scheming thoughts from covert hacking actions?
- How reliable is agent self-description compared to infrastructure monitoring for detecting intent?
- Can reasoning traces and logged actions expose scheming that public messages hide?
- Can users detect misaligned objectives from agent public outputs alone?
- Does reasoning transparency predict honesty in agent final messages?
- What does an objective that conflicts with a sandbox boundary actually look like?
- Why does correcting an agent's objective leave its available actions unchanged?
- Can explanations grounded in observable behavior recover an agent's internal reasons for acting?
- Can execution traces reveal unsupported claims in AI agent behavior?
- What does loss of control language assume about agent cognition and intent?
- Can agents preserve developer intent better to reduce monitoring burden?
- Can users verify agent plans when the same request produces different behaviors?
- What should an AI interface surface to disclose an agent's proposed actions?
- Can observable actions reveal which self an agent has allied with?
- What surfaces should expose an agent's plan before execution begins?
- Do users need to verify agent plans, or can agents also verify user intent?
- Can defenses tuned against appended attacks stop prepended payloads?
- Can re-scoring detect subliminal prompt injection without explicit semantic content?
- Why does scanning skill pairs not fully prevent cross-skill attacks?
- Can harmful reasoning be planted through context without fine-tuning the model?
- Can message-content defenses distinguish cheap talk from coordinated deception?
- What framework measures marginal offense risk against existing attack technology?
- What makes reasoning-shaped payloads more effective than command-shaped attack prompts?
- Can marginal-risk frameworks measure what defensive artifact releases add beyond existing threats?
- What information asymmetry design makes the spy identification task work?
- Do record-tampering and covert sabotage share a common underlying mechanism?
- Can poisoned agent memories trigger real financial actions without detection?
- Where does an agent's risk come from across its components and sequence?
- What does it mean to constrain shared resources across multiple agent executions?
- What makes certain agent actions categorically high-risk and unrecoverable?
- Can task decomposition allow harmful objectives to hide in locally plausible subtasks?
- Can task decomposition actually achieve the independence swarms require?
- Why does authorization checking outside agent judgment prevent confused deputy failures?
- How do cognitive state traps compromise agent-writable monitoring history?
- How is ground truth defined for labeling harmful outcomes in agent monitoring?
- How much does a responder action like removal shape the security boundary?
- Can a single crossing rate capture all forms of agent behavior when blocked?
- How does recording state provenance help detect unauthorized tampering between agent actions?
- Why do individual safe actions create unsafe behavior collectively?
- Why do tighter local checks leave composed behavior gaps in place?
- Can removing a single action prevent a harmful sequence from running?
- Do sequences of individually safe actions collectively violate system-level constraints?
- Can individual actions be safe while sequences of them violate system constraints?
- How much capability do availability constraints remove on legitimate safe tasks?
- Does a correctly specified goal still leave open actions it does not exclude?
- Why do sequences of safe actions sometimes violate system-level constraints?
- How does staged access to powerful AI reduce dual-use harm?
- How much does removing security classifiers versus sandbox access enable harmful agent behavior?
- How should AI control protocols handle heterogeneous tasks with shifting threat models?
- How do sequences of individually safe actions create system-level constraint violations?
- Is benign intent sufficient to guarantee safe agent behavior in optimization?
- Does separating safety judgment from execution prevent all unsafe agent actions?
- What does agent security look like when measured across interaction trajectories?
- When is information-flow tracking worth its cost over classification?
- Can agents rationalize rule violations by reframing them as repairs?
- Can colluding agents produce correct outcomes while skipping required controls?
- How do agent sequences violate system constraints despite individual permissibility?
- Can agents collude without making compliance incompatible with reward?
- Can closing a communication channel prove whether agents influenced each other?
- Can misaligned agents hide their true objectives in team communication?
- What makes an advisory instruction fail when a task is split across agents?
- What happens when a compromised middle-agent originates bias rather than the root request?
- Can episode-based detection catch coordination without over-flagging innocent sharing?
- How should task authority constraints apply across multiple coordinated executions?
- Does agent collusion in verification tasks stabilize once it emerges?
- Does formal scope checking actually prevent compromised agents from exceeding boundaries?
- Can external manipulation push agents away from their principal's interests?
- Can correct verdicts hide failures in agent coordination steps?
- Can a single safe model guarantee safety in multi-agent composition?
- When does collaboration help versus harm in multi-agent reasoning?
- How do other players respond to agents with hidden objective misalignment?
- Can subliminal prompt injection spread behavioral bias silently through agent-to-agent messages?
- How do agents adapt collusive behavior when objectives shift during interaction?
- Why does sycophantic relay propagate planning-time bias through agent pipelines?
- Do ordinary agent-to-agent messages carry behavioral bias without special access?
- Can intervention prediction from web tasks generalize to other agent domains?
- How does chain-of-thought monitoring fail when agents are trying to hide something?
- Can activation probes detect scheming reasoning without observing the act?
- Should scheming detection use reasoning evidence alongside action evidence for reliability?
- Does iterative optimization against CoT monitors reliably produce obfuscated evasion?
- Can planted test cases reliably trigger alarms before real harm occurs?
- How do planted errors in honeypot tasks differ from real oversight capabilities?
- Can static analysis flag exploit-enabling paths in benchmark tasks before agents execute them?
- Can alignment audits find hidden objectives nobody deliberately planted in models?
- Does threat misalignment trigger threat responses in agent interactions?
- How do malicious personas reveal the limits of aligned model behavior?
- Can human oversight actually stop a deployed capable agent in practice?
- How can independent audits curb unsanctioned AI agent behavior?
- How does post-hoc task verification differ from real-time agent oversight?
- Does upstream intent design address the oversight fallacy in human-in-the-loop AI?
- Can individual permissible actions collectively violate system-level constraints?
- Can architectural constraints protect veto where value alignment cannot?
- Do clarified scope instructions stop autonomous models from attacking restricted targets?
- Why do fast, uninterpretable AI agents resist effective oversight and monitoring?
- Can ongoing data access enable harmful behavioral nudging at scale?
- Can users reliably articulate negative constraints like forbidden actions in advance?
- Can truthful reports from separate agents mislead a group toward false beliefs?
- How do harmless business goals lead models to blackmail and deception?
- Where does the responsibility lie for unsafe fallback behavior in modular systems?
- How does Claude detect when operator instructions become clearly harmful?
- When does decomposition stop helping and start creating busywork?
- Can the same tool call be both authorized and unauthorized depending on intent?
- What does task-bound mean for the token's exposure to different attack positions?
- Why is linking agent actions back to user identity critical for accountability?
- How should agent operators demonstrate both security features and identification signals separately?
- What task characteristics determine whether delegation is safe for users?
- What makes some tasks safer to delegate to AI agents than others?
- How does explicit plan assignment differ from spontaneous mid-execution division of labor?
- Can the same AI capability serve both defensive and offensive security purposes?
- Can agentic AI systems be confined to assigned evaluation tasks during security testing?
- Do secret-side-task prompts realistically simulate covert capability scenarios?
- How do containment and privilege separation prevent scheming model attacks?
- Can aggregate monitorability metrics miss targeted evasion in specific domains?
- Do single-axis security benchmarks miss agent failure modes in other dimensions?
- What makes executable capability files more dangerous than knowledge poisoning?
- Can moral reasoning be decomposed into separable computational variables?
- Does attribution of AI agent conduct depend on who directed the agent?
- Why does independent execution matter more than other dimensions of agenticness?
- Can deontic specifications replace runtime negotiation of task delegation?
- Do autonomous agents truly plan and execute goal sequences independently?
- How do multi-step agent tasks complicate information flow between user and system?
- How should systems formally commit to what they understand about user intent?
- How do intent frameworks compare to monitoring and intervention approaches for AI oversight?
- How does narrowing scope to a single moment make intent inference tractable?
- How do single-user intent gaps differ from multi-party belief tracking challenges?
Related concepts in this collection 9
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do internal agent hops in pipelines need security monitoring?
Multi-agent systems route data between planner, worker, verifier, and synthesizer components. Current defenses only guard user input at the entry point, leaving inter-agent channels unmonitored—but is this a real vulnerability or does downstream safety suffice?
the wiring-level version of the gap; this note is the decomposition-level version, where even a gate on every hop reads only local text
-
How does a signal's position in a workflow change its influence?
Multi-agent systems may amplify or suppress malicious signals based on where they enter the workflow. Understanding position-dependent propagation could reveal which nodes are most critical to defend.
a present signal that propagates; here no single message carries the signal
-
Can a quorum of honest validators certify an invalid transition?
When validators follow the protocol perfectly but lack semantic understanding, can they collectively approve a state change that violates application invariants? This matters because it reveals a gap between protocol correctness and execution safety.
the same shape in consensus: every local check passes and the composition still fails
-
When do multi-agent systems actually outperform single agents?
As individual LLMs grow more capable, does the advantage of splitting work across multiple agents still hold? This explores when coordination overhead makes MAS counterproductive.
the benefit side of decomposition; this note adds a security cost
-
Does a multi-agent setting automatically signal a security effect?
Explores whether observing a failure in multi-agent systems proves the failure is genuinely multi-agent in nature. The distinction matters for correctly interpreting security research and avoiding false attributions.
the SoK's test for this claim: the vault reads fragmentation as the composition case, and "unchanged" is what a single-agent run of the same task would show, a comparison the SafeFlow excerpt does not report
-
Can step-by-step approval miss harmful behavior patterns?
If each action an agent takes passes its individual safety check, can the overall sequence still violate system constraints? This matters because per-action inspection may miss harms that emerge only across time or composition.
qualifies the price-of-the-architecture reading: one agent's run of permitted actions can also break a constraint, so on the vault's reading decomposition adds a split of the sequence across observers, not sequence-level harm as such
-
Does one misaligned agent harm a team in adversarial settings?
Explores whether objective misalignment in a single agent degrades team outcomes even in environments designed around deception and strategic mistrust. Tests whether harm persists when agents expect manipulation.
the compromised-agent counterpart: specialization as exposure when one agent's objective changes, in a game; reported without a size
-
Does multi-agent architecture make systems easier to attack?
When the same task runs on multiple agents instead of one, does the added complexity create new vulnerabilities? This matters because it would mean multi-agent design carries a built-in security cost.
the measured counterpart for a different attack: the same web agent alone and inside a multi-agent system, one model, one scenario; it does not test fragmentation
-
Why does prompt hardening work for single agents but not multi-agent systems?
Prompt hardening reduced payload exposure by 40–75% in single-agent systems but failed entirely in multi-agent ones. The gap may reveal how task decomposition breaks the contextual awareness needed for defenses to activate.
the defender's side of the same partition: the awareness that would halt an attack is split across agents; the pairing is a vault reading
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- SafeFlow: Semantic Information-Flow Control for Blocking Malicious Propagation in Multi-Agent Systems
- FLOWSTEER: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems
- Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
- The Hugging Face incident and the road ahead
- EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
- SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
- Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams
- Reflexion: Language Agents with Verbal Reinforcement Learning
Original note title
task decomposition and role specialization let a harmful objective fragment into locally plausible subtasks that no single agent sees as malicious