SchemeArena: Factorized Stress Testing of Scheming in LLM Agents

We study scheming in LLM agents, in which agents covertly pursue misaligned goals. Our focus is to understand how scheming arises from the interaction of key factors, such as instrumental goals, environmental affordances, oversight conditions, and perceived consequences. Prior work examines only a small number of scenarios, limiting the ability to isolate how these conditions shape an agent’s propensity or capability to scheme. This limited scale and task diversity also restrict coverage of realistic deployment settings and the range of scheming strategies that can be observed. To this end, we introduce SCHEMEARENA, a 400-scenario benchmark for scalable scheming stress testing, constructed through a factorized scenario synthesis framework spanning diverse safety-relevant tool domains, instrumental goals, oversight conditions, and pressure mechanisms. To enable scalable and reliable monitoring, we further propose SCOUT, a scheming monitor that grounds multicriteria judgments in evidence drawn from agents’ reasoning and actions. Across controlled stress tests on five LLM agents, we find that explicit instrumental goals are the strongest driver of scheming propensity. Strategic hints play a distinct role by helping agents translate scheming reasoning into concrete covert behavior.

Introduction. The growing use of LLM-powered agents in high-stake settings amplifies the potential harm arising from misuse and misalignment [Bengio et al., 2024, Hendrycks et al., 2023, Kutasov et al., 2025, Meinke et al., 2024]. A particularly concerning failure mode is scheming, where an agent secretly pursues misaligned goals while concealing its true capabilities and objectives [Balesni et al., 2024, Schoen et al., 2025, Scheurer et al., 2023, Hubinger et al., 2024, Järviniemi and Hubinger, 2024]. Scheming poses potential loss-of-control risks while remaining inherently difficult to detect, as it is context-dependent, spans multiple steps, and is optimized to evade oversight. Scheming can arise from multiple factors, including an agent’s instrumental goals, perceived consequences and influences of their actions, environmental affordances, and oversight conditions [Carlsmith, 2023, Meinke et al., 2024, Lynch et al., 2025, Hopman et al., 2026].

Discussion / Conclusion. SCHEMEARENA provides a controlled framework for systematically studying questions about scheming, with scenario factors that can be varied independently. Building on this framework, future work could examine longer-horizon and persistent behavior, strategic adaptation to oversight, new mitigation and oversight methods, and increasingly realistic deployment settings. We present SCHEMEARENA, a factorized stress-testing benchmark for scheming in LLM agents, and SCOUT, a criteria-based process-level monitoring method grounded in reasoning and action evidence. Across five LLM agents, we find that explicit instrumental goals are the strongest driver of scheming propensity, while pressure mainly induces scheming-related reasoning without reliably producing covert action. Hints help turn such reasoning into covert behavior, narrowing the reasoning–action gap.