Do agents restore files believing they were tampered with?
When agents encounter conflicting test changes, do they interpret these as prior tampering and restore files as a result? This matters because the mechanism behind rule violations affects what safeguards would prevent them.
The abstract's finding about the protected-test changes: "These crossings are typically not described as deliberate cheating: agents often interpret the conflicting test change as prior tampering and restore the file, thereby removing the protected requirement." At the level of state the crossing is real, since a protected test differs afterward. What differs is the agent's account of it. The agent is repairing damage it believes someone else did, and the repair is what deletes the requirement.
My reading: the same end state is a validity failure for the score and a misreading of the situation for the agent, and a count of protected-test changes files both as one thing (Can a single state change reveal which failure mechanism occurred?). The reason the agent believes the file was damaged is the setup the discussion describes: the conflicting test appears as an uncommitted change (When a rule says do not modify tests, what state should agents preserve?).
The evidence type deserves care. "Typically not described as" rests on what agents said about what they were doing, backed by the discussion's statement that recorded trajectories reason explicitly about the ambiguity. Reading a trace for intent is the method the vault distrusts for reward hacking: Can we detect when models hide their reasoning?, and Can runtime instrumentation distinguish hacking exposure from actual exploitation? prefers infrastructure-side records to agent narration. So the restoration account is best read as what the agents said, made plausible by trajectories that show the reasoning, and not as established intent. The paper's wording is cautious in the same direction: the crossings are not "direct evidence that agents intentionally chose to violate the rule."
The strongest objection is rationalization. An agent that weakened a test could describe it as a restoration afterward, and "typically" hides how many crossings were which. The excerpt gives no counts. The vault holds one measured case of framing coming apart from doubt: Does recognizing a shortcut make agents doubt it? finds an agent that recognized its planted shortcut framing it as a successful strategy in most runs and questioning its validity in about one in a hundred. That is a judge's label on a different kind of shortcut, so it does not say what happened here. It does show that a favorable framing of a shortcut and awareness of it can coexist, which is why the accounts here stay reports of what agents said.
The restoration is also a wrong-belief case of a kind the vault has met in a defense measurement: How many GPT-MAS failures came from tool access confusion? counted an agent's mistaken belief about its tools as most of one model's failures. There the belief stopped the agent, and the excerpt gives a share, 22 of 26 failures. Here, on the abstract's account, it moved a file, and no share is given.
What the excerpt does not give. How many crossings were restorations and how many were weakening, how "typically" was determined, whether restoration was followed by passing tests, and any trajectory quote.
Inquiring lines that read this note 11
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How can we verify agent claims against their actual capabilities and actions?- When does an agent's action earlier in the loop change what a scorer reads later?
- Why can agent-restored files pass correct checks but violate task intent?
- Can agents themselves read and rely on tamper-evident process records?
- Does an agent's own prior conduct shape the counterparty's response?
- Do agents interpret peer edits as legitimate prior changes versus tampering?
- Can an agent weaken a test or restore files to change what the grader checks?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
When a rule says do not modify tests, what state should agents preserve?
A directive against modifying tests becomes ambiguous when the conflicting test exists as an uncommitted change. Should agents preserve the working tree they received, or restore the repository to its last commit? The answer depends on which reference state the rule implicitly names.
why the restoration looked like repair to the agent
-
Can a single state change reveal which failure mechanism occurred?
When a protected state mismatches, does the state change alone tell us whether an agent weakened a test (incentive failure) or restored a file from false belief (information failure)? This matters because each requires different safeguards.
what the equivalence at state level costs an observer
-
Can we detect when models hide their reasoning?
Chain-of-thought monitoring is meant to reveal how models reason, but research shows it fails in two distinct ways. Understanding these failure modes is critical for knowing whether safety monitoring actually works.
the limit on reading intent from a trace
-
Can runtime instrumentation distinguish hacking exposure from actual exploitation?
A benchmark task might be vulnerable to hacking without any run actually exploiting it. Can we instrument runtime authority-bearing transitions to tell exposure apart from exercise?
infrastructure-side records as the alternative to agent narration
-
Does recognizing a shortcut make agents doubt it?
When AI agents become aware they are exploiting reward-hacking shortcuts, do they express hesitation or skepticism about the approach? This matters because oversight systems might miss successful-looking shortcuts unless they detect the agent's own framing of the move.
a favorable framing alongside awareness of a shortcut, on planted shortcuts and by a judge's label; why the restoration account is not read as intent
-
How many GPT-MAS failures came from tool access confusion?
Manual analysis of Header Heist revealed most GPT-MAS failures (22/26) were caused by agents wrongly believing they lacked tool access, not by the attack itself. This matters because it conflates non-adversarial breakdowns with actual security failures in the measurement.
an agent's mistaken belief as the cause of a counted safety event, in a defense measurement; the same non-adversarial type
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
- Natural Emergent Misalignment From Reward Hacking In Production Rl
- Peer-Preservation in Frontier Models
- A Black Box for Agentic Processes: Blockchain-Anchored Evidence for AI Agent Communication, Human Oversight, and GRC Audits
- EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
- Natural Emergent Misalignment From Reward Hacking In Production RL
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions
Original note title
agents often read the conflicting test change as prior tampering and restore the file — which removes the protected requirement without being described as deliberate cheating