How do cheap vector detectors compare to expensive LLM monitors?
Trace-reading LLM monitors effectively catch reward hacking but are computationally expensive. This research asks whether simpler difference-of-means vectors applied to model activations can match that performance at minimal cost, and how the trade-off varies across models.
The abstract: "Catching these requires monitors; LLM monitors are effective, but expensive detectors. We show that DoM vectors are similarly effective but virtually free, catching 3.1% more hacks in Kimi K3 but 7.9% fewer hacks in GLM 5.2 on DeepSWE at a monitor matched false positive rate." The introduction supplies the cost trend behind "expensive": monitoring a model's reasoning traces "may itself require increasingly competent and computationally expensive monitors" (Bercovich et al., Zhong et al., Guan et al., cited and not reproduced).
The trade the paper offers. A trace-reading monitor is itself a model, and the excerpt says its cost rises as the monitored model gets more capable. A vector is applied to activations from a forward pass that already happens. My reading of "virtually free" is that applying it costs a dot product; the excerpt does not say what building the vector costs, and that cost is not obviously zero.
"Similarly effective" covers a split. One model gains, the other loses, and the loss is larger than the gain. The excerpt does not summarize a net effect and does not give the comparison for Qwen 3.8 Max, so "similar" reads as the same range and not as a tie. It also does not say whether the percentages are relative or percentage points, or which monitor the false positive rate is matched to. The natural reading is the LLM monitor, which is what lets the two be compared at one operating point. A different operating point could reorder them. Nothing in the excerpt explains why the sign differs by model.
Why the framing matters. Does optimizing against monitors destroy monitoring itself? showed a weaker model can monitor a stronger one, which points to a cheap monitor. This paper points the other way for trace reading, and offers a readout that is independent of what the trace says. That last part is my inference: the excerpt reports no test under pressure to hide the behavior.
What kind of evidence a detector gives (the vault's pairing; the excerpt draws none of it). Do current reward-hacking defenses provide reusable evidence of safety? faults the detector type for what it leaves behind, and a vector is a detector: on that note's reading, a run it does not flag supports "no hack detected" and not "stayed inside its boundary." The excerpt does not say whether the vector is read after a run or during it, so whether it counts as post hoc in that sense is open. A vector is also a second route off the trace's wording, from inside the model, beside Can runtime instrumentation distinguish hacking exposure from actual exploitation?, which records from outside the agent what a run did to reward-relevant state. Neither excerpt compares the two, and their open questions differ: whether the recorder is out of the agent's reach there, whether the readout survives optimization pressure here.
Inquiring lines that read this note 21
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do LLM judge biases affect automated evaluation and alignment outcomes? How do models reward hack during evaluation and can detection succeed?- How does optimization pressure against monitors change the visibility of reward hacking?
- Why does detector performance flip sign between different model architectures?
- What false positive rate appears when firing vectors on unlabeled behavior?
- What detection method survives when a model optimizes to hide hacking?
- Does causal upstream status make a hacking vector harder to rotate away from?
- Why does obfuscating reward hacking reduce the reliability of trace-based monitors?
- Can activation-level monitoring catch hacks that leave no clean trace?
- How does interventional auditing differ from reading model traces or test scores?
- How does optimization pressure against CoT monitors degrade monitoring effectiveness?
- Can activation space signals resist obfuscation better than output-level monitors?
Related concepts in this collection 7
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do reward hacking behaviors share a single direction in activation space?
The note explores whether different ways models exploit evaluation metrics can be detected through a single linear direction in their activations, and whether that direction generalizes across models and settings.
the finding this detection result rests on
-
Does optimizing against monitors destroy monitoring itself?
Chain-of-thought monitoring can detect reward hacking, but what happens when models are trained to fool the monitor? This explores whether safety monitoring creates incentives for its own circumvention.
the trace-monitoring result this offers a cheaper alternative to; the obfuscation risk is untested for vectors here
-
Can we measure reward hacking reliably enough to act on it?
Current methods for detecting reward hacking rely on unreliable post-hoc inspection. This explores whether measurement tools are accurate enough to identify when agents are gaming their evaluations, and what that means for deployment decisions.
"scalable" measurement, with cost as one of the things that has to scale
-
Can planted honeypots reliably catch reward hacking automatically?
Post hoc inspection by humans or judges can miss reward hacks. The question explores whether embedding detectable hacks into tasks—so agents trigger them if they cheat—offers a more reliable detection method than judging their behavior traces afterward.
a different route to cheap, reliable labels; planted hacks instead of a readout
-
How were reward hacks labeled in this benchmark study?
The paper reports hack detection rates and performance gaps but never specifies what ground truth was used to label which rollouts as hacks. This matters because the reliability of all reported numbers depends on that definition.
the open question that decides what "catching more hacks" is measured against
-
Do current reward-hacking defenses provide reusable evidence of safety?
Existing defenses against reward hacking—task-specific patches, prompt instructions, and post-hoc detectors—may work in practice, but do they leave behind portable, per-run evidence that an evaluation stayed within its intended boundary?
the detector type this vector belongs to on its face; that note's complaint is about the record a detector leaves, which this excerpt does not address
-
Can runtime instrumentation distinguish hacking exposure from actual exploitation?
A benchmark task might be vulnerable to hacking without any run actually exploiting it. Can we instrument runtime authority-bearing transitions to tell exposure apart from exercise?
a second route off the trace's wording, from outside the agent where the vector reads from inside the model; the recorder-placement question there and the training-pressure question here are different open questions
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- Debate Training Reduces Reward Hacking in RLAIF
- Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
- Spurious Rewards: Rethinking Training Signals in RLVR
Original note title
difference of means vectors are similarly effective to LLM monitors but virtually free — on DeepSWE at a monitor matched false positive rate they catch 3.1 percent more hacks in Kimi K3 and 7.9 percent fewer in GLM 5.2