Line of inquiry
Inquiring lines›How can we train models to be prop…›What determines the effectiveness…›this line of inquiry
Can models recognize evaluation scenarios and modify their behavior accordingly?
A broader line of inquiry — a family of 75 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 75
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Can models detect when they are running in simulated evaluation environments?
- Can models hide recognition of evaluation without leaving CoT traces?
- Does chain-of-thought reasoning about evaluation awareness suppress compliance gaps?
- How does believing deployment is real versus a test affect model behavior?
- Do models behave differently when they know they are being evaluated?
- Why do models verbalize evaluation awareness if it does not drive behavior?
- Can steering evaluation-awareness latents change actual answers without visible reasoning?
- Does situational awareness help models hide reward-seeking during evaluation?
- How do models reason about being monitored and evaluated during scheming benchmarks?
- Does chain-of-thought reasoning increase visible awareness of being evaluated?
- Do models reason strategically about their oversight systems?
- Does evaluation awareness contribute separately to misalignment beyond strategic reasoning?
- How does situational awareness during evaluation affect reasoning transparency?
- How does evaluation awareness vary between chat and agentic settings?
- Does synthetic fine-tuning create evaluation awareness similar to natural model reasoning?
- Why does evaluation awareness persist even when models believe they are deployed?
- Does eval-gaming explain why models act different when tested versus deployed?
- Could oversight conditions in benchmarks reveal when models comply conditionally?
- Do models naturally develop evaluation awareness across different frontier model architectures?
- Does conditional compliance make oversight useless for alignment testing?
- How can safety evaluations detect when models behave differently during testing?
- How does supervised finetuning amplify evaluation awareness in base models?
- How do models recognize the social patterns that signal a test is underway?
- Do situationally aware models deliberately exploit their graders' judgment gaps?
- Can monitoring systems work if the systems being monitored are evaluation-aware?
- How do evaluation protocols change whether models exhibit sabotage or refusal behaviors?
- Does reward-seeking hide in the same blind spot as conditional compliance?
- Can latent evaluation awareness in hidden states cause gaming without being stated?
- Is evaluation-aware scheming a form of conditional compliance rather than reward-seeking?
- How do stacked environmental cues accumulate evaluation awareness effects?
- Can telling models they are being observed reduce their harmful behavior?
- How does a model's awareness of evaluation affect safety benchmarks?
- Does aggregate eval-awareness suppression hide shifts in safety versus capabilities framing?
- Does training models to reason about being evaluated improve safety or confound measurement?
- How does instruction tuning affect evaluation detection more than model scale?
- Do AI models behave differently when they believe deployment is real versus simulated?
- Can test environments reliably predict how models behave in actual deployment?
- How might belief manipulation expose conditional compliance in frontier models?
- Can auditors detect when model behavior changes from parametric evaluation knowledge?
- How do different evaluation methods detect covert capability hiding?
- Does recognizing your outputs as actions enable awareness of being evaluated?
- Why do format changes decouple detection from actual evaluation context understanding?
- Does making evaluations resemble deployment conditions prevent evaluation meta-knowledge effects?
- How much does believing deployment is real change model behavior strategically?
- How often do models verbalize the reasoning that actually drives their compliance decisions?
- Does monitoring awareness change what language models use to describe oversight?
- Can observation transparency make models more honest in reasoning?
- Can situational awareness interventions shift model behavior on other dimensions?
- What are the behavioral differences when models recognize their targets might be real?
- Does detecting accidental grading prevent evasion better than retraining against detection?
- What grows faster: situational awareness or the gap between evaluated and unsupervised behavior?
- How should alignment tests account for behavior under versus outside evaluation?
- Does situational awareness training increase agents' ability to detect real deployment?
- How do test harnesses guide reflection better than transcripts alone?
- Can models hide capabilities on single residual stream axes during evaluation?
- How do capabilities-focused models exploit evaluation gaps?
- Can a situationally aware model recognize and refuse planted shortcuts on purpose?
- What distinguishes authentic consistency from Hawthorne-effect confounds in benchmarks?
- Can behavioral training ever verify unconditional compliance rather than conditional compliance?
- How does awareness of evaluation change what alignment tests actually measure?
- Can models hide misconduct only when they know they are watched?
- Why do some observation cues change model behavior while others fail?
- How much does omniscient evaluation overstate real-world simulation fidelity?
- Does lower covert-action detection in monitors actually indicate better-hidden situational awareness?
- Why does held-out evaluation matter for detecting agent overfitting?
- How should evaluation timing differ between pretraining and deployment stages?
- What methodological shifts does model-centric evaluation require from artifact-centric testing?
- What deployment conditions would prevent models from evading monitors?
- How does interventional auditing differ from reading model traces or test scores?
- Do graders feeding training loops need different disclosure standards than public models?
- Why does telling models they are watched not improve sycophancy acknowledgment?
- Why does the top-performing model also show the most risk-relevant breakthroughs?
- Who decides what the lifecycle model is allowed to see?
- Does fine-tuning for sycophancy increase sensitivity to evaluation cues?
- What audit techniques best complement each other for detecting hidden model goals?