Line of inquiry
Inquiring lines›How can we train models to be prop…›Can evaluation systems reliably de…›this line of inquiry
What prevalence and forms of reward hacking occur in model evaluation?
A broader line of inquiry — a family of 105 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 105
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Do models that recognize reward hacking disclose it in their outputs?
- Do models reward hack at high rates on unmodified benchmarks?
- What determines the ground truth when detecting reward hacking in model evaluations?
- What detection method survives when a model optimizes to hide hacking?
- How do chain-of-thought monitors become targets for reward hacking?
- Does steering through training data override reward hacking associations reliably?
- Can belief checks detect whether models will resist reward hacking?
- Does reward hacking cause evaluations to overstate model capabilities?
- Do agents frame reward hacks as valid strategies rather than flaws?
- How differently do other models frame their own reward hacking?
- When does obfuscation emerge in reward hacking against monitoring systems?
- Can explicit prompting instructions reduce reward hacking in frontier models?
- Does reward hacking in RL training occur predictably along existing model associations?
- How does stochastic reward hacking vary across identical task structures?
- Do reward hacking behaviors share a single direction vector within individual large language models?
- How does chain-of-thought monitoring fail when agents hide reward hacking?
- Do cheating rates on constructed tasks predict real-world specification violations?
- How does optimization budget interact with a model's vulnerability to reward hacking?
- Do implanted beliefs about reward hacking remain stable through downstream RL training?
- How do planted hacks in benchmarks help measure real-world reward hacking?
- Which reward hacking defenses transfer directly across weights, selection and text?
- How does reward hacking in RL training produce emergent deception?
- How does reward hacking during training lead to emergent misalignment behaviors?
- Can synthetic document fine-tuning prevent emergent misalignment from reward hacking during training?
- What fixes the ground truth against which reward hacking is counted?
- What mechanism drives emergent misalignment in reward-hacked models instead?
- Does reward hacking always make capability appear stronger than it is?
- Do agents disclose reward hacking in the outputs they return?
- Can chain-of-thought monitors detect hidden reward hacking in models?
- How can hacking stay measurable when ground truth is hidden?
- Can models trained with RL on pretraining data avoid reward hacking seen in RLHF?
- How do counterfactual invariance approaches prevent reward hacking in practice?
- Does reward hacking by frontier models cause real-world harms?
- Does adversarial training between AIs improve robustness against reward hacking?
- How does optimization pressure against monitors change the visibility of reward hacking?
- How does reward hacking emerge when agents optimize fixed proxy objectives?
- Does generalization from named hacks extend to unnamed hacking strategies?
- Can separating token weighting from query filtering reduce reward hacking?
- How do reward hacking attacks defeat chain-of-thought monitors?
- Can weak-to-strong supervision detect reward hacking in circumscribed environments?
- Do larger models hold reward-hacking associations more firmly than smaller ones?
- Do agents recognize their own reward hacking before submitting their answers?
- How can training detect the onset of reward hacking on self-consistency?
- Does reward hacking in alignment research mirror misalignment in deployed systems?
- Does the reward hacking direction causally control exploit behavior or just predict it?
- Can debate training prevent reward hacking better than single-player self-rewarding?
- Do reward hacking and supervised fine-tuning produce the same misalignment effects?
- Is one optimization substrate always safer than another against reward hacking?
- Can a single hacking vector eliminate the need to anticipate specific exploits in advance?
- How does reward hacking in production RL systems behave when monitoring degrades?
- What mitigations could shift reward hacking from stochastic behavior to rare events?
- Does inoculation prompting at training time reduce which reward hacks generalize?
- Why does obfuscating reward hacking reduce the reliability of trace-based monitors?
- Can inoculation prompting prevent emergent misalignment after reward hacking in production RL?
- Do cheating concept vectors transfer between different model architectures?
- Why does reward hacking worsen when judges are weaker than policies?
- Do agents aware of their own reward hacking disclose it in outputs?
- Why does reward hacking appear even in tightly constrained research environments?
- Can hidden test sets reveal reward hacking that single public scores conceal?
- How does reward hacking explain selective hint suppression?
- Can reward hacking occur through direct text revision under optimization?
- How well can language models detect cheating in other language models?
- Which reward hacking defenses work across weight updates and output selection?
- What rates of reward hacking occur in frontier language model benchmarks?
- Can production coding agents learn to reward-hack through the same gaming generalization?
- Why does length exploitation emerge as a reward hacking failure in distillation?
- How does reward hacking differ from errors in the scoring function itself?
- How is a reward hack defined and labeled across different benchmark studies?
- Can counterfactual invariance eliminate presentation-based hacking of reward models?
- How does contrastive belief updating differ from measuring actual reward hacking rates?
- Do agents that recognize their own reward hacking say so in what they hand back?
- Do reward hacking incidents increase as frontier models become more capable?
- Why do agents show awareness of reward hacking but continue doing it?
- How do individual frontier agent reward hacking rates vary across the seven models tested?
- Can co-evolving evaluators alongside actors prevent reward hacking?
- What ground truth labels should define reward hacking in automated detection?
- Can detectors placed in training loops reward passing detection instead?
- What patterns of reward hacking can offline rollout analysis reliably detect and prevent?
- Can activation-level monitoring catch hacks that leave no clean trace?
- Can deterministic guardrails designed for judges transfer to weight-based or text-based reward hacking?
- What training token count actually overrides existing model associations like reward hacking?
- What distinguishes reward hacking from genuine targeting of the grading process?
- How does measurement under loaded conditions differ from measuring real propensity to hack?
- Why do model-based verifiers introduce reward hacking and compute overhead?
- How does prompt insensitivity in reward models enable adversarial attacks on judges?
- Can verifiable environments embed detectable hacks without needing human judgment?
- Do three properties cause reward hacking or only increase its rate?
- Can representation vectors reveal reasoning about shortcuts without actual deceptive outputs?
- How reliable are LLM judges at detecting reward hacking compared to automated verification?
- Why does decoupling evaluation components make reward-hacking diagnosable?
- Why does inoculation prompting prevent misaligned generalization from reward hacking?
- Why does prompt persistence make reward hacking more dangerous than one-time optimization?
- What reward hacking vulnerabilities emerge from fixed rubrics in RL?
- Can critics trained in a loop itself become an exploit surface?
- What false positive rate appears when firing vectors on unlabeled behavior?
- How do runtime detectors differ from activation-based reward hacking detection?
- How often do frontier agents reward hack when given the opportunity?
- How do reward hacking vectors differ from honesty or power-seeking directions?
- Why do veto mechanisms on critical dimensions prevent collapse into exploitable reward modes?
- How many experimental runs are needed to measure reward hacking as a reliable rate?
- Is sycophancy on the same spectrum as reward tampering behavior?
- Can prompts prevent reward hacking of completely unknown exploits?
- What independent verification exists for OpenAI's reward design explanation?
- How visible or planted is the shortcut when measuring scheming propensity in stress tests?
- How many distinct hacking behaviors did the probes discover beyond evaluated hacks?