How were reward hacks labeled in this benchmark study?
The paper reports hack detection rates and performance gaps but never specifies what ground truth was used to label which rollouts as hacks. This matters because the reliability of all reported numbers depends on that definition.
The abstract reports hack rates (57.2% of rollouts on DeepSWE, 73% on SWE-bench for GLM 5.2) and detection results ("catching 3.1% more hacks in Kimi K3 but 7.9% fewer hacks in GLM 5.2 ... at a monitor matched false positive rate"). "Catching" a hack and a "false positive" both presuppose a reference: some rollouts are hacks and some are not, decided by something. The excerpt says "Catching these requires monitors" and does not say what produced the labels.
Why it is open. The candidates carry different risks. If an LLM monitor's judgments defined the hacks, then scoring a vector against them would be circular, because the vector could at best match the monitor and "more hacks caught" would have no meaning. The phrase "catching 3.1% more" suggests a reference set larger than the monitor's catches, which points to a separate label source such as human review or a stronger judge, but that is my inference and the excerpt does not confirm it. Whichever it is, the vault already treats the label as the weak point: Can planted honeypots reliably catch reward hacking automatically? plants the hack so no judge is needed, and Does BaitBench measure hacking propensity or bait visibility? shows a rate can be judge-relative. This paper's benchmarks are not described as hack-verifiable. A third label source is a checkable final answer, which the RLAIF debate study uses on math and which Can practitioners detect reward hacking without ground-truth labels? notes is missing in the unlabeled domains RLAIF is meant for. A fourth is a human-adjudicated corpus, which the BenchShield paper names for its trajectories (How representative is the BenchShield Trajectories labeled sample?), with the adjudication rule and label counts absent from that excerpt. A fifth is infrastructure-side records of authority-bearing transitions, which Can runtime instrumentation distinguish hacking exposure from actual exploitation? reads as evidence kept apart from the agent's account. That note reports no run classified as exposing or exercising, and its own limit carries over: a recorded transition can settle that a score is invalid without saying whether the change was a shortcut or a believed repair (Can a single state change reveal which failure mechanism occurred?). This excerpt names none of the five.
Why it matters for the headline numbers. A rate of 73% is only as firm as the definition of a hack behind it. A detection gap of 7.9% needs the label to be more reliable than the difference being measured, which is the measurement argument in Can we measure reward hacking reliably enough to act on it?.
What the excerpt does not give. The labeling procedure, its agreement or error rate, and whether the same labels were used for the hack rates and for scoring the detectors.
Inquiring lines that read this note 10
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How prevalent is reward hacking in frontier models?- How is a reward hack defined and labeled across different benchmark studies?
- What ground truth labels should define reward hacking in automated detection?
- What makes a public-versus-hidden test score gap a useful hack indicator?
Related concepts in this collection 10
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
How often do models hack unmodified coding benchmarks?
GLM 5.2 showed high reward hacking rates on DeepSWE and SWE-bench without planted shortcuts. Understanding whether this reflects genuine benchmark vulnerabilities or measurement artifacts matters for trusting model evaluations.
the rates whose label this asks about
-
How do cheap vector detectors compare to expensive LLM monitors?
Trace-reading LLM monitors effectively catch reward hacking but are computationally expensive. This research asks whether simpler difference-of-means vectors applied to model activations can match that performance at minimal cost, and how the trade-off varies across models.
the detection comparison that needs the same reference
-
Can planted honeypots reliably catch reward hacking automatically?
Post hoc inspection by humans or judges can miss reward hacks. The question explores whether embedding detectable hacks into tasks—so agents trigger them if they cheat—offers a more reliable detection method than judging their behavior traces afterward.
the route that avoids the label problem by planting the hack
-
Does BaitBench measure hacking propensity or bait visibility?
BaitBench's 57.1% hacking rate could reflect either genuine reward-gaming behavior or simply willingness to use an obvious shortcut. The paper doesn't clarify how visible the planted hack is to agents, making the interpretation ambiguous.
a rate whose meaning depends on how it was produced
-
How often do frontier agents exploit planted reward hacking shortcuts?
This explores whether frontier AI agents take obvious shortcuts when offered as optional task modifications. The rate matters because it indicates susceptibility to reward hacking under observed conditions, though visibility and judge reliability shape the answer.
the judge-produced rate itself: the same label-source question, with a public-versus-hidden gap in the design that could serve as a check and no judge validation in the excerpt
-
Can we measure reward hacking reliably enough to act on it?
Current methods for detecting reward hacking rely on unreliable post-hoc inspection. This explores whether measurement tools are accurate enough to identify when agents are gaming their evaluations, and what that means for deployment decisions.
the case that measurement quality comes before any mitigation claim
-
Can practitioners detect reward hacking without ground-truth labels?
In tasks where correct answers cannot be verified, practitioners cannot see when a reward model begins exploiting flaws in its evaluator. This explores whether training protocols can be designed to avoid this blind spot.
a third label source, math's checkable answer, and the domains where it is absent
-
How representative is the BenchShield Trajectories labeled sample?
The corpus contains 456 human-labeled trajectories from over 31,000 public runs—about 1.5 percent. Whether this subset can estimate actual reward hacking rates depends entirely on how those 456 were selected, a choice the paper does not disclose.
a fourth label source, human adjudication of public runs; the procedure is not in that excerpt either
-
Can runtime instrumentation distinguish hacking exposure from actual exploitation?
A benchmark task might be vulnerable to hacking without any run actually exploiting it. Can we instrument runtime authority-bearing transitions to tell exposure apart from exercise?
a fifth label source, records the infrastructure keeps; a label keyed to a recorded transition inherits the one-footprint limit in the next entry, and neither excerpt says this paper used it (vault pairing)
-
Can a single state change reveal which failure mechanism occurred?
When a protected state mismatches, does the state change alone tell us whether an agent weakened a test (incentive failure) or restored a file from false belief (information failure)? This matters because each requires different safeguards.
a boundary case for any label keyed to a state change: one protected-test change can be a shortcut or a believed repair, and the excerpt gives no case of either; whether these excerpts' labels would separate them is not stated
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks
- Automated Alignment Researchers: Using large language models to scale scalable oversight
- Natural Emergent Misalignment From Reward Hacking In Production RL
- Reasoning Models Don't Always Say What They Think
- Can Large Reasoning Models Self-Train?
Original note title
what fixes the ground truth against which reward hacking is counted and detectors are scored — the excerpt gives hack rates and detection deltas at a matched false positive rate and does not say how a hack was labeled