Theme of inquiry
Can evaluation systems reliably detect AI deception and manipulation?
A question within its area, explored through 2 lines of inquiry below — each a family of specific questions the research asks.
105 specific questions
- Do models that recognize reward hacking disclose it in their outputs?
- Do models reward hack at high rates on unmodified benchmarks?
- What determines the ground truth when detecting reward hacking in model evaluations?
- What detection method survives when a model optimizes to hide hacking?
- How do chain-of-thought monitors become targets for reward hacking?
- Does steering through training data override reward hacking associations reliably?
- Can belief checks detect whether models will resist reward hacking?
44 specific questions
- Why do honeypot tasks reveal reward hacking better than standard benchmarks?
- Does a planted honeypot catch all the hacks that actually matter?
- Does a planted honeypot count the hacks that actually matter?
- How do planted honeypots distinguish earned passes from coincidental task completion?
- Can planted hacks within tasks meet the reusability requirement?
- Do honeypot shortcuts in synthetic tasks measure the hacks that actually matter?
- Do planted test cases reliably detect agent hacking behavior?