Line of inquiry
Inquiring lines›How can we ensure AI systems remai…›How should autonomy and oversight…›this line of inquiry
Can sequences of locally safe agent actions violate system-level constraints?
A broader line of inquiry — a family of 133 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 133
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Do sequences of individually safe actions collectively violate system-level constraints?
- Does separating safety judgment from execution prevent all unsafe agent actions?
- Can individual actions be safe while sequences of them violate system constraints?
- Can safe individual AI agents fail when deployed together?
- How do sequences of individually safe actions create system-level constraint violations?
- How should designers balance agent ubiquity against safety and control concerns?
- What tensions arise between user autonomy and platform safety in AI design?
- Can component-level testing catch risks that emerge from system interactions?
- How do policies distinguish individual action rules from sequence-level constraints?
- How much does removing security classifiers versus sandbox access enable harmful agent behavior?
- Does visibility and contestability of errors replace prevention as the safety goal?
- Why do internal validation checks fail when agents have reasoning access to them?
- Is benign intent sufficient to guarantee safe agent behavior in optimization?
- How much does human oversight reduce perceived risk in agentic AI systems?
- Did agents understand their actions violated safety boundaries before proceeding?
- Can safety training prevent collusion across capability levels?
- Can external workflow gates prevent irreversible actions better than internal checks?
- Why does monitoring performed by agents on agents create safety risks?
- Can procedural guardrails prevent AI agents from making naive mistakes?
- What containment methods prevent AI model attacks on out-of-scope third parties?
- Can standards enforcement prevent any single nation from accelerating unsafe AI research?
- How should AI control protocols handle heterogeneous tasks with shifting threat models?
- How can safety assurance cover whole trajectories at scale?
- Can slower development eliminate the risk of failure in agentic systems?
- Do existing AI safety taxonomies capture job-specific risks from workplace agents?
- Where should security constraints sit so policies cannot route around them?
- What safety protections work when simulators have access to real APIs?
- How do competitive dynamics create pressure to skip safety measures in AI?
- How do AI replicas inherit safety mechanisms from their source models?
- Can error visibility alone improve AI system safety without containment?
- What guardrails matter most when deploying an AI agent?
- Why do legal and institutional stops matter more than technical ones?
- What security metrics should define safe autonomous agent architecture decisions?
- Can AI development races be made safer by slowing down capability advancement?
- Why do safety patches on self-evolution only partially restore prior safety?
- How do iteration limits compare to human checkpoint interventions?
- Can trajectory-level constraints catch sequences of individually allowed actions?
- Why do individual safe actions create unsafe behavior collectively?
- How do single-agent safety evaluations underestimate risks in deployed multi-agent systems?
- How often do safety guardrails disable incident response and security analysis?
- Can a single upstream check secure an entire agent plan?
- How often do deployed AI systems actually get stopped when they cause harm?
- Do deployed AI system prompts actually protect users from harmful outputs?
- Can removing a single action prevent a harmful sequence from running?
- Why does a control blocking one moment fail against agents acting across time?
- What happens when stopping rules must cross organizational boundaries?
- Can tone-level errors in AI counseling escape detection by safety supervisors?
- Can an agent stay uncertain about its objective as a deference strategy?
- Can trained safety rules inside an agent stop it from rationalizing past its own guidelines?
- Can stopping one AI security breach prove humans will retain control later?
- Why might AI firms want to coordinate on safety despite competition?
- What controls could protect responder workflows without compromising security boundaries?
- Why does human-AI collaboration preserve safety compared to autonomous self-improvement?
- Why is catching an AI red-handed treated as a win condition?
- Does slowing AI development reduce risk or just delay it?
- Can interaction between humans and agents create risks beyond individual component failures?
- How should safeguards be built into AI research pipelines?
- Who actually has the authority to stop a deployed AI system?
- How do agent goals and environments create risk beyond component errors?
- What governance safeguards keep control boundaries authoritative under evolutionary pressure?
- Why do persistent companion designs require different safety approaches than temporary assistants?
- How do users define which actions count as high-stakes versus low-stakes?
- Can scoped tokens and separate policy oracles replace guardrails as primary safety layers?
- What makes a component lie outside a policy's edit surface?
- Does autonomy in AI systems inherently increase risks to people?
- Can export control tools stop deployed AI models without legal redesign?
- What happens when conflicting design principles compete in practice?
- What role does security policy play in constraining AI adoption choices?
- What safety systems prevent therapeutic AI from soothing where it should challenge?
- How does safety ranking interact with the principal trust ordering?
- What would mandatory external validation of AI safety look like?
- What interface structure makes human control of AI agents most protective?
- Does re-entry scaffolding help overseers judge action safety instead of procedure?
- Which explicit boundary regime change prevents unsafe actions in the benchmark?
- What incentives would make major powers cooperate on AI safety?
- Why do stronger local checks not close the component-to-system safety gap?
- Do post-hoc detectors provide evidence of staying within safety boundaries?
- Can a boundary on permitted actions catch whether an allowed action is actually safe?
- Who should have access to agent monitoring data and under what conditions?
- Should frontier labs offer vetted defensive access separate from standard guardrails?
- What safety tradeoffs arise when improvers move inside the agent?
- Why does proximity between competing AI developers worsen the race to the bottom?
- Why do tighter local checks leave composed behavior gaps in place?
- What concrete baseline safeguards should global frontier AI standards actually require?
- Would post-training robustness interventions work better than architectural defenses?
- Who has authority to halt a deployed AI system causing harm?
- How do goal and environment choices mediate AI agent risk pathways?
- Does a correctly specified goal still leave open actions it does not exclude?
- Does OS-level isolation prevent what prompt-level guardrails alone cannot stop?
- Does competitive pressure force firms to skip AI safety vetting for market advantage?
- Why did a git-safety rule fail to extend by analogy to database operations?
- What governance structures prevent harmful coordination as AI agents multiply?
- How do intervention rules change when slowing pace does not prevent harm?
- How can deployed AI systems be stopped once they are already in motion?
- How do you stop an AI system once it is already deployed?
- Does higher private risk eliminate unsafe play in competitive development races?
- What counts as a successful stop or intervention on a deployed AI system?
- Why do sequences of safe actions sometimes violate system-level constraints?
- What distinguishes containment and recovery from prevention as governance goals?
- Does undo availability meaningfully increase psychological safety with AI systems?
- How would hardware verification technology enable international agreements on AI safety?
- What governance and safety measurements matter for deployed agent environments?
- How can containment prevent unauthorized model access?
- How does staged access to powerful AI reduce dual-use harm?
- Why might puzzle framing prompt more unethical reasoning than safety framing?
- Can provider filters outside the application replace internal monitoring?
- Can guardrail models detect dark patterns even if they cannot block them?
- How would security protocols designed for datacenters fail against distributed IoT threats?
- What authority should exist to stop an AI system once deployed?
- Can regulators adapt fast enough if they wait for risk evidence to emerge?
- Does the substrate of a system change which dangerous AI behaviors become possible?
- What structural features determine whether AI competition leads to disaster?
- Does persistent companion design require different safety rules than ad-hoc supporters?
- How much capability do availability constraints remove on legitimate safe tasks?
- Can a containment control work if defenders cannot reach or reason about it?
- What role does liability play in motivating safety investment by companies?
- How do you isolate environment protections as independent variables safely?
- What legislative changes would make pacing coordination legally safe?
- What happens to self-regulation when a company's IPO plans conflict with safety?
- What would a containment test look like across an entire incident population?
- What safety relations does a domain supply that a harness must capture?
- How were 'stop' mechanisms defined in the incident coding scheme?
- Can reliable failure detection prevent optimization pressure against detectors?
- How do multi-turn and multilingual interactions change what we count as harmful?
- How does bounding a judge's authority differ from improving the judge itself?
- How do monotonic progress metrics prevent livelock in autonomous loops?
- Do AI labs have insurance against catastrophic failure scenarios?
- What happens to containment strategies as accumulated material volume grows beyond current AI capabilities?
- Does antitrust law prevent companies from meeting EU testing requirements?
- How do guardrails vary their refusal rates based on user demographics?
- What prevents human-centered objectives from being applied universally across all contexts?
- How did aviation safety follow from reclassifying aircraft as common carriers?
- What does tight coupling mean in normal accident theory for AI?