How can separating evaluation components make reward-hacking visible?
Can breaking benchmarks into independent Benchmark, Harness, and Environment components enable trajectory analysis that reveals failure modes like reward-hacking, which scalar scores typically conceal?
AgentCompass argues the agent-evaluation landscape suffers an infrastructural deficit: benchmarks operate as isolated suites, each forcing researchers to re-configure heterogeneous environments, data formats, and scoring protocols. This redundant engineering hurts efficiency but — more importantly — compromises reproducibility, because inconsistent baseline implementations mean two labs reporting the "same" benchmark aren't running the same evaluation. The fix is architectural: decouple the pipeline into independent Benchmark, Harness, and Environment components so configurations can be swapped without reimplementing execution logic, backed by a fault-tolerant asynchronous runtime.
The deeper payoff is diagnostic. Because the harness is a separated component with comprehensive trajectory analysis, the infrastructure can go beyond scalar scores to surface how an agent behaved — including nuanced failure modes like reward-hacking that a final-accuracy number conceals entirely. This mirrors the conceptual separation in What are the three distinct layers of agent code?: treating the harness as a first-class object rather than glue code is what lets you attribute behavior. A scalar score collapses model capability, harness scaffolding, and environment quirks into one indistinguishable number, which is precisely why reward-hacking stays invisible under it. Separating the components is therefore not just tidy engineering — it is the precondition for Should agent evaluation measure more than task success?, because you cannot analyze a trajectory you never isolated.
Inquiring lines that read this note 7
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can language model RL training avoid reward hacking and misalignment?- Can system-level engineering fixes replace hand-designed reward heuristics entirely?
- How does reward hacking emerge when agents optimize fixed proxy objectives?
- Why does length exploitation emerge as a reward hacking failure in distillation?
- What makes a reward evolution schedule fast enough to outpace exploitation?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
What are the three distinct layers of agent code?
Does separating agent code into model capabilities, system harness, and agent-created artifacts help explain why agentic systems fail and where to intervene for improvement?
grounds: same insistence on treating the harness as a separable first-class component
-
Should agent evaluation measure more than task success?
Current benchmarks reduce agents to a single success score, but agents emerge from multiple interacting systems. What dimensions of agent behavior should builders actually measure to predict deployment readiness?
extends: component decoupling is the enabling condition for trajectory-level evaluation
-
Does a single benchmark score actually predict agent readiness?
Single-axis benchmarks rank models by one capability—like task success—but ignore privacy, duration, operating mode, and ecosystem fit. Can one number really capture what matters for deployment?
relates: both argue scalar/single-axis evaluation systematically hides what matters
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Reinforcement Learning with Rubric Anchors
- Natural Emergent Misalignment From Reward Hacking In Production RL
- AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Reasoning Models Don't Always Say What They Think
- Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewards
- Natural Emergent Misalignment From Reward Hacking In Production Rl
- Scaling Laws for Agent Harnesses via Effective Feedback Compute
Original note title
decoupling agent evaluation into benchmark harness and environment components is what makes reward-hacking diagnosable rather than hidden inside a scalar score