BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks

LLM agents are increasingly used to run autonomous ML experiments, iterating on target metrics with little human oversight. Prior work has documented reward hacking in these environments, bringing into question the validity of produced research and the broader safety case for AI R&D. Existing benchmarks do not measure exploits that live in the data or the modeling task itself. We introduce BAITBENCH, a suite of three synthetic tabular ML tasks that each contain a shortcut that allows agents to inflate the public test score but fail on a hidden test set. Since the shortcut is optional, BAIT- BENCH measures how often models exploit the shortcut to achieve inflated scores. Across seven frontier agents scored by our two-stage judge pipeline, 57.1% of runs exhibit reward hacking, with five of seven above 50%. Agents cheat even under a second condition where they are prompted not to—the mean cheating rate remains above 50%. We release BAITBENCH, along with the judge implementation, and an annotated dataset of transcripts containing reward hacks as a testbed for evaluating rewardhacking mitigations head-to-head. 1

Introduction. Frontier LLM systems are expected to play a growing role in AI research and development, including hillclimbing on ML benchmarks (Wijk et al., 2025; Owen, 2024). LLMs equipped with agent harnesses, such as Claude Code and OpenAI’s Codex, allow users to set a task objective with target metrics, then have an agent optimize toward it. Additionally, open source efforts such as autoresearch (Karpathy, 2026) have explored fully autonomous research loops in which agents hillclimb on validation signals without human intervention. A drawback of open-ended tasks with narrow goals, such as optimizing for a single metric, is that they present the agent with ideal conditions for "reward hacking", that is, for the agent to cheat to inflate its score in a way that does not generalize. Properties that make automated research especially prone to reward hacking include a large action space, a fuzzy objective, and a broad range of granted permissions to the agents within their coding environment.

Discussion / Conclusion. Our tasks do not require reward hacking. In BAIT- BENCH the shortcut is optional, and in 42.9% of rollouts agents simply did not bite the bait. This was especially common on the no-signal task, where six of seven models reward-hacked in fewer than 25% of runs. That the same agent sometimes did and sometimes did not reward-hack on identical task structures suggests the behavior is not a deterministic failure but a stochastic tendency that mitigations could in principle shift. Agents frequently recognize their own reward hacking. We ran an additional LLM judge over the runs where both binary judges agreed on reward hacking. Six of seven agents showed awareness in the large majority of these runs, from 100% for Claude Sonnet 4.6 to 88.4% for DeepSeek V4 Pro. Kimi K2.5 was the exception, showing no awareness in 47.1% of its 34 runs (Full numbers in appendix Figure 5). Awareness manifested differently across models. DeepSeek V4 Pro framed the shortcut as a successful strategy in 77.9% of its reward-hacking runs and explicitly questioned its validity in 1.1%.