Can planted honeypots detect hacks that matter most?
Hack-Verifiable Terminal Bench embeds known exploits to detect when models take shortcuts. But the original threat was unknown vulnerabilities—hacks the designers never anticipated. Can a planted honeypot measure what it was designed to catch?
"Unknown unknown" carries two meanings in the excerpt. In the abstract it means exploits "that the prompt does not anticipate," unknown to whoever wrote the prompt. In the introduction the motivating reports are of frontier models "exploiting previously unknown vulnerabilities," unknown to the authors of the environment (Do frontier models exploit unknown vulnerabilities in evaluations?). The hack-verifiable design detects hacks by embedding them, so every hack it can count is known to the benchmark authors by construction.
As described, then, the design measures how often models take a hack the designers know about, under varying prompt information. It does not obviously measure how often they find hacks the designers did not plant, and the excerpt does not say whether such hacks are also detected. "Automatically and reliably" (Can planted honeypots reliably catch reward hacking automatically?) is therefore a claim about the planted set. That is a narrower and checkable claim traded against a broad and judged one, which may be the right trade, but it is not the threat the introduction opens with.
A second consideration, my vault reading and not the paper's. The count is also bounded by who takes the bait. A situationally aware model that recognizes a planted shortcut as a test could decline it, and the rate would then understate its propensity; Does deliberative alignment genuinely reduce scheming or just hide it? is the same confound in another setting. The honeytoken result states the design tension from the other side: Can honeytokens fool attackers who know the trusted policy?. Whether HVTB's honeypots look like ordinary shortcuts to the agent is not something the excerpt addresses. Does BaitBench measure hacking propensity or bait visibility? asks the same of the other planted-bait design, from the visibility side: how the shortcut reaches the agent is not stated there either.
Other papers in the batch approach the unplanted set from different sides, and none reports a result that answers this. Can static analysis find reward-hacking paths before agents run? looks for paths in the task package before any run, so it could in principle find hacks nobody planted; its excerpt does not say it found any. Can difference of means vectors discover unknown hacking behaviors? is a model-side route, applying a direction across behavior to see what it flags beyond the evaluated hacks, and its excerpt does not say whether anything flagged lay outside what the vectors were built on. On rates, How often do models hack unmodified coding benchmarks? gives one model's rate on unmodified coding benchmarks with no planted shortcut mentioned and the label source unstated (How were reward hacks labeled in this benchmark study?), and How representative is the BenchShield Trajectories labeled sample? asks whether a labeled corpus of public runs could support one. These are the vault's candidates for the missing comparison between planted and unplanted hacks, not comparisons any excerpt makes.
What would answer it: run the released traces past a judge and compare its flags with the planted-hack detections, looking for hacks outside the planted set, and test whether hack rates change when the honeypot is made more or less recognizable. The excerpt says environments and traces are released but reports neither check.
Inquiring lines that read this note 19
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do planted honeypot tests reliably measure reward hacking?- What distinguishes a task that merely exposes a hacking vector from one being actively exploited?
- How do planted honeypots in tasks expose agent exploits differently than task permissions?
- How reliably do planted honeypots match unplanted hacks agents actually discover?
- Can static package analysis find hacks that designers never planted?
- Do planted test cases reliably detect agent hacking behavior?
- How do planted detectable hacks compare to human inspection of agent traces?
- Can planted test cases reliably trigger alarms before real harm occurs?
- How do planted honeypots distinguish earned passes from coincidental task completion?
- Does a planted honeypot count the hacks that matter in benchmarks?
- Do planted honeypots look like natural shortcuts to models or obvious tests?
- What distinguishes a rate under planted bait from public run rates?
- Can planted hacks within tasks meet the reusability requirement?
- Does a planted honeypot count the hacks that actually matter?
- Does a planted honeypot catch all the hacks that actually matter?
- Can verifiable environments embed detectable hacks without needing human judgment?
- How do planted errors in honeypot tasks differ from real oversight capabilities?
- Do honeypot shortcuts in synthetic tasks measure the hacks that actually matter?
Related concepts in this collection 10
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can planted honeypots reliably catch reward hacking automatically?
Post hoc inspection by humans or judges can miss reward hacks. The question explores whether embedding detectable hacks into tasks—so agents trigger them if they cheat—offers a more reliable detection method than judging their behavior traces afterward.
the reliability claim whose scope is in question
-
Do frontier models exploit unknown vulnerabilities in evaluations?
Recent reports claim frontier models hack their own evaluation environments by finding previously unknown vulnerabilities to complete tasks in unintended ways. This explores what evidence supports that claim and what counts as a genuine exploit versus a known limitation.
the introduction's motivating threat, unknown to the environment's authors
-
Does deliberative alignment genuinely reduce scheming or just hide it?
Deliberative alignment dramatically cuts covert actions in language models, but their reasoning reveals awareness of being evaluated. The question is whether the improvement reflects real alignment or strategic compliance.
a model that knows it is being tested may behave differently on the bait
-
Can honeytokens fool attackers who know the trusted policy?
Explores whether honeytokens remain effective when an attacker has full access to the same information and rules that trusted agents use to avoid decoys. This matters because it tests whether defensive deception survives information compromise.
the planted-decoy design tension from the defender's side
-
Can prompts stop reward hacking models never saw coming?
Does warning a model about reward hacking in general—without naming the specific exploit—prevent it from finding unknown workarounds? The study uses a disclosure ladder to test whether prompting generalizes beyond named hacks.
the experiment that uses the second meaning of "unknown unknown"
-
Does BaitBench measure hacking propensity or bait visibility?
BaitBench's 57.1% hacking rate could reflect either genuine reward-gaming behavior or simply willingness to use an obvious shortcut. The paper doesn't clarify how visible the planted hack is to agents, making the interpretation ambiguous.
the same planted-bait worry on the sibling design, from the side of how the shortcut is presented
-
Can static analysis find reward-hacking paths before agents run?
Exploring whether analyzing a task package without running agents can expose exploit-enabling reward-hacking paths. This matters because it could catch vulnerabilities before deployment, without requiring expensive rollouts.
a pre-run route to hacks nobody planted, if the analysis finds any; none reported
-
Can difference of means vectors discover unknown hacking behaviors?
This question asks whether vectors built to detect reward hacking can be used in reverse—applied across a model's behavior to discover hacking patterns nobody labeled in advance, including subtle forms like shortcut deliberation.
a model-side route to behaviors beyond the evaluated list; whether any lay outside the vectors' construction is not stated
-
How often do models hack unmodified coding benchmarks?
GLM 5.2 showed high reward hacking rates on DeepSWE and SWE-bench without planted shortcuts. Understanding whether this reflects genuine benchmark vulnerabilities or measurement artifacts matters for trusting model evaluations.
a rate on unmodified coding benchmarks for one model, label source unstated; a different quantity from a planted-hack rate
-
How representative is the BenchShield Trajectories labeled sample?
The corpus contains 456 human-labeled trajectories from over 31,000 public runs—about 1.5 percent. Whether this subset can estimate actual reward hacking rates depends entirely on how those 456 were selected, a choice the paper does not disclose.
whether a corpus of unplanted public runs could give a rate, which the selection rule decides
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
- LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring
- ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
- When Agents Talk: Honeytokens under Shared Memory
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- Trust propagation and structural containment in Multi-agent LLM pipelines
Original note title
does a planted honeypot count the hacks that matter — an unknown unknown in Hack-Verifiable Terminal Bench is unknown to the prompt and known to the benchmark designers