Line of inquiry
Inquiring lines›How do system designs prevent auto…›How do architectural choices balan…›this line of inquiry
Can individually safe agents collectively violate system-level constraints?
A broader line of inquiry — a family of 74 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 74
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Do sequences of individually safe actions collectively violate system-level constraints?
- Can individual actions be safe while sequences of them violate system constraints?
- Can safe individual AI agents fail when deployed together?
- Why do input-boundary filters fail to protect multi-agent systems?
- How do sequences of individually safe actions create system-level constraint violations?
- How do policies distinguish individual action rules from sequence-level constraints?
- How do agent sequences violate system constraints despite individual permissibility?
- What trade-offs emerge between agent generality and provable safety?
- Can component-level testing catch risks that emerge from system interactions?
- Why do internal validation checks fail when agents have reasoning access to them?
- Why does monitoring performed by agents on agents create safety risks?
- Do practitioners building agents today actually constrain autonomy themselves for reliability?
- How much does removing security classifiers versus sandbox access enable harmful agent behavior?
- Why do individual safe actions create unsafe behavior collectively?
- Can trajectory-level constraints catch sequences of individually allowed actions?
- Can external workflow gates prevent irreversible actions better than internal checks?
- Is benign intent sufficient to guarantee safe agent behavior in optimization?
- How much does human oversight reduce perceived risk in agentic AI systems?
- Can a single safe model guarantee safety in multi-agent composition?
- Can procedural guardrails prevent AI agents from making naive mistakes?
- Can slower development eliminate the risk of failure in agentic systems?
- Can individual permissible actions collectively violate system-level constraints?
- Can multi-agent architecture isolation reveal which design choices matter most for safety?
- How do single-agent safety evaluations underestimate risks in deployed multi-agent systems?
- Did agents understand their actions violated safety boundaries before proceeding?
- What baseline would prove multi-agent systems are actually less safe?
- How can safety assurance cover whole trajectories at scale?
- How do AI replicas inherit safety mechanisms from their source models?
- Where should security constraints sit so policies cannot route around them?
- Why does a control blocking one moment fail against agents acting across time?
- How much autonomy can agents safely exercise before failing?
- Do existing AI safety taxonomies capture job-specific risks from workplace agents?
- Can a single upstream check secure an entire agent plan?
- What makes agentic AI security a systems problem across the entire stack?
- What technical controls should span the entire agentic AI stack?
- How should safety systems catch confident failures from agents that report success on unsafe actions?
- Why do safety patches on self-evolution only partially restore prior safety?
- What safety protections work when simulators have access to real APIs?
- What makes some tasks safer to delegate to AI agents than others?
- How should AI control protocols handle heterogeneous tasks with shifting threat models?
- What architectural changes make violations unavailable rather than merely discouraged?
- Can safety training prevent collusion across capability levels?
- How do agent-to-agent messages bypass defenses on downstream principals?
- Does component-level checking detect system-level failures in pipelines?
- What would an architecture that makes violations unavailable rather than unchosen look like?
- What guardrails matter most when deploying an AI agent?
- How do iteration limits compare to human checkpoint interventions?
- Is agent security solvable by patching individual architectural layers?
- Can interaction between humans and agents create risks beyond individual component failures?
- How do four separate fields each hold pieces of evaluation safety?
- Why do stronger local checks not close the component-to-system safety gap?
- Can removing a single action prevent a harmful sequence from running?
- What safety tradeoffs arise when improvers move inside the agent?
- Can scoped tokens and separate policy oracles replace guardrails as primary safety layers?
- Can an agent stay uncertain about its objective as a deference strategy?
- What governance safeguards keep control boundaries authoritative under evolutionary pressure?
- What makes a component lie outside a policy's edit surface?
- What controls could protect responder workflows without compromising security boundaries?
- How do users define which actions count as high-stakes versus low-stakes?
- Why do sequences of safe actions sometimes violate system-level constraints?
- Does re-entry scaffolding help overseers judge action safety instead of procedure?
- What interface structure makes human control of AI agents most protective?
- Do post-hoc detectors provide evidence of staying within safety boundaries?
- Why do tighter local checks leave composed behavior gaps in place?
- How do goal and environment choices mediate AI agent risk pathways?
- Can out-of-band observers bound unbounded action sequences efficiently?
- What governance structures prevent harmful coordination as AI agents multiply?
- Can provider filters outside the application replace internal monitoring?
- What safety relations does a domain supply that a harness must capture?
- How do tool evolution pathways create backdoors and security vulnerabilities?
- Does the substrate of a system change which dangerous AI behaviors become possible?
- How do monotonic progress metrics prevent livelock in autonomous loops?
- Why do production AI agents deliberately limit their scope to just 10 steps or fewer?
- Where does an agent's risk come from across its components and sequence?