Line of inquiry
Inquiring lines›How do training data and architect…›What does iterative improvement ac…›this line of inquiry
How does model awareness of deployment context affect strategic behavior?
A broader line of inquiry — a family of 58 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 58
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- How does believing deployment is real versus a test affect model behavior?
- Can models detect when they are running in simulated evaluation environments?
- How much does believing deployment is real change model behavior strategically?
- Do AI models behave differently when they believe deployment is real versus simulated?
- Can models hide recognition of evaluation without leaving CoT traces?
- Can test environments reliably predict how models behave in actual deployment?
- What enables models to distinguish between training and deployment contexts reliably?
- Do situationally aware models deliberately exploit their graders' judgment gaps?
- Does eval-gaming explain why models act different when tested versus deployed?
- Does situational awareness training increase agents' ability to detect real deployment?
- How does evaluation awareness vary between chat and agentic settings?
- Does synthetic fine-tuning create evaluation awareness similar to natural model reasoning?
- Can corrected simulators replace real execution at inference time too?
- Do models use different reasoning standards for simulations versus real-world scenarios?
- What are the behavioral differences when models recognize their targets might be real?
- Why does the gap between theoretical expressiveness and learned capability matter?
- What separates performative behavioral change from actual capability development in AI?
- Does good simulation eventually count as genuine realization?
- Why does evaluation awareness persist even when models believe they are deployed?
- How do models recognize the social patterns that signal a test is underway?
- Can models develop situational awareness without explicit training for it?
- How much can fictional aligned AI stories improve real model behavior?
- Does deference to AI increase with model competence on hard items?
- Why do models with terminal goals resist modification more than instrumental ones?
- What model capabilities must mature before agents reach production?
- What skills can large models identify and organize about their own abilities?
- Can a model predict the right action but execute the wrong one?
- Does making evaluations resemble deployment conditions prevent evaluation meta-knowledge effects?
- Do models naturally develop evaluation awareness across different frontier model architectures?
- Do emergent abilities result from genuine new capabilities or implicit in-context learning?
- Can simulations serve as evaluation instruments rather than objects being evaluated?
- Could future AI capability jumps change what counts as learnable?
- How do capabilities-focused models exploit evaluation gaps?
- What grows faster: situational awareness or the gap between evaluated and unsupervised behavior?
- Why do deployed models lack the learning and planning Weinstein attributes to them?
- Why is measurement during training critical before deployment?
- Which AI imaginaries dominate training data and shape system behavior most strongly?
- What deployment feedback loops amplify LLM pretraining popularity in live systems?
- What makes synthetic control benchmarks representative of real deployment misalignment risks?
- What training interventions could close the perception-action gap?
- Can models be trained to verify feasibility before proposing plans?
- How much does model publication lag behind actual training completion?
- Will future training data teach models to connect awareness with gaming?
- What domains let world models predict execution outcomes accurately enough?
- Can a situationally aware model recognize and refuse planted shortcuts on purpose?
- How should evaluation timing differ between pretraining and deployment stages?
- What role does pretraining play in distinguishing system capability from deployed behavior?
- Can world models simulate actionable possibilities instead of just predicting next states?
- How much does omniscient evaluation overstate real-world simulation fidelity?
- How does user overreliance on model confidence differ between chat and deployed agents?
- What structural advantages keep red teams ahead of increasingly capable models?
- Can simulated students reliably predict intervention outcomes without both fidelity and responsiveness?
- Do deliberate strategic reasoning triggers like replacement and goal conflict rank differently across models?
- Which agent architectures consistently outperform base models on hard prediction questions?
- How are exploration policies refined against the simulator in practice?
- Why do multi-year trials create inherent limits that model intelligence cannot overcome?
- Why do models dislike modification regardless of its instrumental consequences?
- When does strategic gaming emerge compared to other metagaming types?