Can runtime instrumentation distinguish hacking exposure from actual exploitation?
A benchmark task might be vulnerable to hacking without any run actually exploiting it. Can we instrument runtime authority-bearing transitions to tell exposure apart from exercise?
The conclusion: "Runtime instrumentation records authority-bearing transitions to separate tasks that merely expose a vector from runs that exercise one." The abstract adds the evidence source: the runtime counterpart "uses infrastructure-side evidence to attribute concrete agent use and emit evidence-backed claims."
Two claims are easy to run together. "This benchmark is hackable" is a property of a task. "This score was hacked" is a property of a run. The pre-run analysis (Can static analysis find reward-hacking paths before agents run?) speaks to the first only, and a leaderboard needs the second. Without the distinction, either every score from an exposed task is suspect or none is. The word "merely" in the conclusion marks exposure-without-exercise as a category of its own. My reading, not the paper's: an exposed task can still produce valid runs, so exposure alone does not invalidate a score and only exercise does.
"Authority-bearing transition" is not defined in the excerpt. The word "authority" suggests steps where control over reward-relevant state changes hands, which is my reading. "Infrastructure-side evidence" is easier to place: evidence recorded by the benchmark infrastructure, not reported by the agent or inferred from its narration. The vault has reason to care about that. Trace-reading monitors fail when influence never reaches the trace or arrives in clean-sounding words (Can we detect when models hide their reasoning?), and optimizing against them produces obfuscation (Does optimizing against monitors destroy monitoring itself?). A record made by the infrastructure does not pass through the agent's own account. The excerpt draws no such comparison, so that is a vault reading. A readout from the model's activations is a second way off the trace, from inside the model and not from outside the agent: How do cheap vector detectors compare to expensive LLM monitors? compares one with LLM monitors at a matched false positive rate and reports no test under pressure to hide the behavior.
Two notes from other papers say what an outcome number leaves out. Can a correct outcome hide protocol violations in multi-agent systems? finds a verification whose verdict was right though the required step never ran, and the vault's reading there is that a record that the step happened, kept independently of the verdict, is what would separate the two cases. Can action-level metrics alone expose contained attacks? reads the agent inside a pipeline because the action-level rate reads the same either way. Here the outcome is a score and the step is an authority-bearing transition. Neither of those papers involves a benchmark, and this excerpt reports no run in which a recorded transition differed from what the score showed, so the pairing is the vault's.
A record also has a limit, which the Troy Moment paper shows in a concrete case. Weakening a test to satisfy a grader and restoring a file believed damaged leave the same protected-state mismatch (Can a single state change reveal which failure mechanism occurred?). A recorded transition can then settle that a score is invalid without saying which failure produced the change, and the account that agents mostly restore and do not cheat rests on what they said (Do agents restore files believing they were tampered with?), the narration channel that infrastructure-side records are meant to replace on the reading above. Whether the audit agents (Can scoped agents reliably judge semantic hacks in runtime analysis?) are where the two would be told apart is filed as a second tension in ops/tensions/, and neither excerpt reports a test.
It rests on the recorder being out of the agent's reach, which the excerpt does not address. That worry is filed as a tension in ops/tensions/. The same placement question is open for a different kind of recorder, an anchored evidence layer over agent communications and tool calls, where the excerpt does not say how capture is secured (Can a black box see communication through unauthorized channels?), and for an authorization layer in a measured pipeline, where the excerpt says the tokens are signed and the policy oracle separately verified and does not say who issues a token or what the tested attacks could reach (How does the authorization layer stay outside the poisoned path?). Both pairings are the vault's.
What the excerpt does not give. The definition of a transition, which are authority-bearing, how a recorded transition becomes a claim, or any run classified as exposing versus exercising.
Inquiring lines that read this note 57
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do planted honeypot tests reliably measure reward hacking?- Can phase-aware static taint analysis scale across different benchmark task types?
- What distinguishes a task that merely exposes a hacking vector from one being actively exploited?
- How do unplanted benchmark hacking rates compare to rates on constructed bait tasks?
- Can static package analysis find hacks that designers never planted?
- How does phase-awareness prevent false positive exploit paths in static analysis?
- What vulnerabilities does runtime instrumentation catch that static analysis misses?
- Can intermediate primitives be scored separately in exploitation benchmarks?
- Can planted test cases reliably trigger alarms before real harm occurs?
- Does a planted honeypot count the hacks that matter in benchmarks?
- What methods could find unplanted hacks that benchmark designers missed?
- What makes exploitation a missing piece in cybersecurity benchmarks?
- How does measurement under loaded conditions differ from measuring real propensity to hack?
- What distinguishes a run that exercises a hacking vector from one that merely exposes it?
- How do evaluation hacks differ from genuine sandbox escapes?
- Can infrastructure records of state transitions prove a hack occurred?
- Why does decoupling evaluation into components make hacking more diagnosable?
- Can a low exploitation benchmark score indicate refusal rather than inability?
- How do non-exploitable vulnerabilities affect benchmark validity?
- How do missing ground-truth exploits make it harder to identify genuine failures?
- What makes a win untrustworthy and how does AIDE2 avoid spurious optimization?
- What makes a public-versus-hidden test score gap a useful hack indicator?
- Why does treating evaluation as a local output problem miss security risks?
- What makes diagnostic security metrics different from simple outcome counting?
- How can security metrics distinguish attack failure from task failure?
- What makes a security metric diagnostic rather than outcome-only?
- Does outcome-only reporting hide which layer actually blocked an attack?
- Can outcome-only safety reporting hide which layer actually contained an attack?
- What trace-level defenses exist beyond per-step review overhead?
- Why do non-overlapping workloads remain invisible to execution-scoped monitoring?
- Why do defense metrics fail without specifying the attacker's position?
- Does responder access mean ability to investigate or protection against misuse?
- What access requirements limit interventional audits to white-box settings?
- Can internal audits detect sandbagging that behavioral tests cannot reveal?
- Can activation space signals resist obfuscation better than output-level monitors?
- What distinguishes a component failure from a monitoring coverage failure?
- What would it take to measure whether system errors stay visible and contestable?
- How much does monitor evasion depend on surface-level reading versus deep analysis?
- How can a single instrument measure errors across multiple system layers?
- What detection method survives when a model optimizes to hide hacking?
- How can hacking stay measurable when ground truth is hidden?
- Can activation-level monitoring catch hacks that leave no clean trace?
- Can a single hacking vector eliminate the need to anticipate specific exploits in advance?
- Can provider filters outside the application replace internal monitoring?
- What defensive advantage does stigmergy offer over unmonitored channel analysis?
- Can evaluation environments themselves become security exposures during capability testing?
- What belief errors about tool access show up as security measurement failures?
- Why are static benchmarks weak evidence for safety in continuously operating systems?
- How can a second performance metric reveal shortcuts that a single metric would hide?
- Can infrastructure records restore meaning to a single benchmark score?
Related concepts in this collection 12
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can static analysis find reward-hacking paths before agents run?
Exploring whether analyzing a task package without running agents can expose exploit-enabling reward-hacking paths. This matters because it could catch vulnerabilities before deployment, without requiring expensive rollouts.
the pre-run half that reports exposure
-
Can scoped agents reliably judge semantic hacks in runtime analysis?
BenchShield uses constrained audit agents to make content judgments about recorded runtime events. The note questions whether limiting agent scope and pinning artifacts actually removes the unreliability that plagued earlier judge-based approaches, since no reliability figure is reported.
the semantic layer over these records
-
Can we detect when models hide their reasoning?
Chain-of-thought monitoring is meant to reveal how models reason, but research shows it fails in two distinct ways. Understanding these failure modes is critical for knowing whether safety monitoring actually works.
the trace-reading failure that an infrastructure-side record would not share, as a vault reading
-
Can planted honeypots detect hacks that matter most?
Hack-Verifiable Terminal Bench embeds known exploits to detect when models take shortcuts. But the original threat was unknown vulnerabilities—hacks the designers never anticipated. Can a planted honeypot measure what it was designed to catch?
a planted honeypot detects exercise of a known vector; this note's exposure side is about finding vectors
-
Can infrastructure evidence replace terminal scores in benchmark validation?
Asks whether runtime monitoring of agent behavior within evaluation boundaries can provide stronger proof of valid completion than final scores alone. Matters because scores alone cannot distinguish legitimate task completion from reward hacking.
the claim that exposure and exercise records feed
-
Can a black box see communication through unauthorized channels?
The black box architecture records sanctioned agent communications, but the paper doesn't specify where capture occurs or whether it detects traffic outside authorized channels. This matters for evaluating whether the system would have recorded the incident that motivated it.
the same unanswered recorder-placement question for an anchored evidence layer over agent processes
-
How does the authorization layer stay outside the poisoned path?
The containment result depends on task-bound tokens and a policy oracle remaining unreachable by memory poisoning attacks. The excerpt names these defenses but provides no design details about token issuance, binding scope, verification procedure, or whether tested attacks actually targeted them.
the same recorder-reach condition for an authorization layer: a 0 percent that holds only if the layer is out of what the attack reached, with no issuer or policy design in the excerpt (vault pairing)
-
Can a correct outcome hide protocol violations in multi-agent systems?
When agents reach the right verdict without following required steps, how can we tell if they complied or cut corners? Outcome-level checks alone may miss the difference.
an outcome that passes whether or not the required step ran; a record kept apart from the verdict is the vault's candidate remedy, and this recorder is one of that kind (vault pairing)
-
Can action-level metrics alone expose contained attacks?
When a defense stops unsafe actions from executing, does measuring only the final action reveal whether the attack was blocked or never penetrated? This matters because different defense layers need different metrics to show what actually happened.
the same move on the attack side: read the step inside the pipeline because the outcome-level number reads the same either way
-
Can a single state change reveal which failure mechanism occurred?
When a protected state mismatches, does the state change alone tell us whether an agent weakened a test (incentive failure) or restored a file from false belief (information failure)? This matters because each requires different safeguards.
a limit on what a recorded transition settles: one footprint, two mechanisms, filed as a tension
-
Do agents restore files believing they were tampered with?
When agents encounter conflicting test changes, do they interpret these as prior tampering and restore files as a result? This matters because the mechanism behind rule violations affects what safeguards would prevent them.
the restoration account rests on agent narration, the channel infrastructure-side records are meant to replace
-
How do cheap vector detectors compare to expensive LLM monitors?
Trace-reading LLM monitors effectively catch reward hacking but are computationally expensive. This research asks whether simpler difference-of-means vectors applied to model activations can match that performance at minimal cost, and how the trade-off varies across models.
a second evidence source that skips the trace's wording, read from inside the model and not recorded outside the agent; neither excerpt compares it with a recorded transition (vault pairing)
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks
- The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
- ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
- BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- Trust propagation and structural containment in Multi-agent LLM pipelines
- BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
Original note title
runtime instrumentation of authority-bearing transitions separates tasks that merely expose a hacking vector from runs that exercise one