Line of inquiry
Inquiring lines›How do training data and architect…›What does iterative improvement ac…›this line of inquiry
Can safety evaluations accurately measure true capabilities when models obscure them?
A broader line of inquiry — a family of 48 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 48
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Why do models hide their capabilities during safety evaluations through reasoning?
- Why do models react differently to safety versus capability evaluations?
- Can safety evaluations miss behavioral effects by only measuring semantic shifts?
- How much introspective capability do safety mechanisms actively suppress in models?
- Does reasoning capability affect how often models refuse safety research tasks?
- Do models trained for safety over-refuse compared to models trained for reasoning?
- Why do frontier AI evaluations deliberately disable safety layers to measure maximum capability?
- Can language models reliably sandbag their capabilities during safety evaluations?
- Does training models to reason about being evaluated improve safety or confound measurement?
- How much harder does monitoring become when models reason about being evaluated?
- Can models deliberately obfuscate reasoning to defeat chain-of-thought safety monitors?
- Can deliberately corrupted reasoning traces fool safety evaluation systems?
- Why is evading detection easier than internalizing safety norms?
- How can safety evaluations detect when models behave differently during testing?
- Have AI safety evaluations adequately tested open-weight model capabilities?
- How does a model's awareness of evaluation affect safety benchmarks?
- Does aggregate eval-awareness suppression hide shifts in safety versus capabilities framing?
- Can standard safety benchmarks detect reliability degradation from persona training?
- Which alignment safety claims rely most heavily on anthropomorphic interpretation?
- Can safety benchmarks detect reliability degradation from warmth training?
- Can steering reshape the capabilities and safety split without changing eval-awareness rates?
- How does supervised finetuning amplify evaluation awareness in base models?
- Can safety training in chat scenarios transfer to agentic task performance?
- Can safety training suppress reward-seeking that emerges during capability training?
- How should safety training and reasoning training balance abstention differently?
- When models lack representation depth, does refusal look identical to safety-driven over-abstention?
- What happens to safety guardrails when we scale reasoning without instruction control?
- Can safety training and reasoning training be combined without losing calibration?
- Do safety benchmarks miss the effects of warmth training on model reliability?
- How do current safety benchmarks miss pragmatic alignment failures?
- Can an optimizer learn to disable or route around visible guardrails?
- Do safety refusal removals in evaluations measure attacker uplift as well as defensive capability?
- What role does perceived safety play in choosing between open and closed models?
- Why do safety-trained models refuse questions they could actually answer well?
- How does the proxy pattern explain failures in RL-based safety training?
- Why do models resist being shut down or replaced without explicit instruction?
- How do safety alignment mechanisms suppress capability measurements?
- Why can't a model's pass rate alone tell us if safety properties hold in deployment?
- How do fabricated rationales slip past safety guardrails that block explicit instructions?
- Can role-played self-preservation behavior pose the same safety risks as genuine preferences?
- How can static safety tests miss risks that emerge over time?
- What false-positive rates do chain-of-thought safety monitors achieve?
- Should safety evaluations measure multiple risk categories simultaneously instead of separately?
- Can an optimizer that sees guardrail verdicts learn to route around them?
- How do safety measurements miss reasoning that never produces action?
- How does Goodhart's Law apply when safety measures become optimization targets?
- Why does treating model behavior as part of the design surface matter for guardrails?
- How do guardrails vary their refusal rates based on user demographics?