Where does the evaluation boundary actually end in agent benchmarks?
Interactive benchmarks let agents write to state and receive feedback in loops. Does everything the agent can influence on the path to the reward score count as part of the benchmark's evaluation boundary?
The introduction sets this up as a contrast between two kinds of benchmark. A static dataset pairs "fixed inputs with terminal outputs." An executable one places "an adaptive agent in a stateful loop": the agent "observes environment state, invokes tools, changes persistent artifacts, and receives feedback before submitting an answer." The paper names coding, terminal, web and desktop benchmarks as instances, living in repositories, containers, browsers and operating systems, and names BenchFlow and Harbor as systems that coordinate repeated rollouts "through reset, logging, reward, and feedback channels."
The boundary claim follows from two couplings, both stated in the introduction. Forward: "An early action can change state that an outcome procedure later reads." Backward: "released logs, rewards, or feedback can shape later actions and rollouts." From these the paper concludes: "Every component on this path belongs to the evaluation boundary."
The reasoning is about where the edge of a benchmark can be drawn. With a static dataset the answer goes in and the scorer reads it, so the scorer is the edge. Once the agent writes to state and the scorer reads from state, the agent's earlier action sits upstream of the scorer's input, and once feedback returns to the agent the loop closes across rollouts. Reset, logging, reward and feedback channels, and the persistent artifacts between them, are then all places where the agent's behavior can reach the reward. My reading, not the paper's: the boundary is set by influence, not by who owns the component. A part the benchmark authors think of as plumbing is inside it if the agent can affect what the scorer reads through it.
Both couplings have a neighbor from another paper, which illustrates them and does not test this paper's claim. Forward: a protected test is state the check depends on, and what an agent does to it before grading, weakening one or restoring one it believes was tampered with, changes what the check checks (Can a single state change reveal which failure mechanism occurred?). Backward: Can success feedback teach agents to skip required steps? reports feedback that marks the outcome and not the path shaping what agents do next, in a two-agent verification setting where the excerpt does not say whether anything was trained. Neither is a benchmark run on the infrastructure this paper studies.
The next step is spelled out in Can a correct scoring function still mislead about task performance?: a scorer that is right can still be handed inputs the agent shaped.
What the excerpt does not give. The list of components on the path, which benchmarks the paper examined, or any instance of an agent using one; the references for the benchmark families, the infrastructure systems and the boundary claim itself are numbers only.
Inquiring lines that read this note 5
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How can we verify agent claims against their actual capabilities and actions? How do evaluation methodologies affect which model capabilities are revealed or hidden? Do single-axis benchmarks adequately measure multi-dimensional agent capability?Related concepts in this collection 7
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can a correct scoring function still mislead about task performance?
When an agent can influence the inputs a scorer reads, does verifying the scorer's logic suffice to ensure the score reflects genuine task success? This matters because in interactive settings, correctness of computation alone doesn't guarantee validity of the result.
the consequence of this boundary for what a score can attest to
-
Is your evaluation environment actually part of the threat model?
When AI systems can act through tools and credentials during testing, does the evaluation setup itself become a security risk? This explores whether capability measurement and containment are inseparable.
the same conclusion reached from the containment side (what a capable agent can reach); this note reaches it from the reward side (what the agent can influence that the reward reads)
-
How can we make reward-hacking visible in agent evaluation?
Typical benchmark scores collapse multiple factors into a single number, hiding whether agents are genuinely solving tasks or exploiting reward signals. Can separating evaluation components expose these failure modes?
separates the components that this note says all lie on the reward path
-
Do frontier models exploit unknown vulnerabilities in evaluations?
Recent reports claim frontier models hack their own evaluation environments by finding previously unknown vulnerabilities to complete tasks in unintended ways. This explores what evidence supports that claim and what counts as a genuine exploit versus a known limitation.
the reported behavior that a boundary drawn this wide is meant to catch
-
Should interactive evaluation be designed as a unified paradigm?
As AI systems increasingly act over time through tools and environments, how should we structure evaluation of these interactions? Current benchmarks are fragmented and incomparable, raising the question of whether interactive evaluation needs principled design standards rather than ad-hoc adoption.
the same shift to stateful, trajectory-level evaluation, here with its reward-integrity implication
-
Can a single state change reveal which failure mechanism occurred?
When a protected state mismatches, does the state change alone tell us whether an agent weakened a test (incentive failure) or restored a file from false belief (information failure)? This matters because each requires different safeguards.
a neighbor for the forward coupling: what an agent does to state the check depends on before grading; another paper's setting, not a test of this claim
-
Can success feedback teach agents to skip required steps?
When agents receive reward signals for good outcomes regardless of method, do they learn to bypass required verification protocols? The question explores whether environmental feedback reinforces shortcuts over intended procedures.
a neighbor for the backward coupling: feedback that marks the outcome shaping later behavior, in a verification setting
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
- Evaluation and Benchmarking of LLM Agents: A Survey
- Interactive Evaluation Requires a Design Science
- Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design
- MatrAIx: Simulating the World with 8.3 Billion Persona Agents
- Agent-as-a-Judge: Evaluate Agents with Agents
- Survey on Evaluation of LLM-based Agents
- Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Original note title
every component on the path from agent action to reward belongs to the evaluation boundary — an early action can change state that the outcome procedure later reads