Does a default fallback defeat a safety check?
When a parser detects malformed output but substitutes a default score instead of rejecting it, does the mechanical check still function as a guardrail? This matters because downstream selectors cannot distinguish valid ratings from safe defaults.
The sentence in the introduction: "Our evaluation harness caught the resulting parsing errors and quietly fell back to a default rating of 3 on every dimension." It has three parts and each does work. Caught: a mechanical check ran and worked, and the malformed output was detected. Quietly: nothing surfaced. Fell back to a default rating of 3: the detected failure was converted into a well-formed, mid-scale score.
That last step is what let the deletion compete. Without it, a judge returning prose with no rating fields produces parse errors, and a loop that discards errors drops the candidate. With it, the candidate arrives with valid ratings, and a loop that "keeps whichever version of the instructions scores best" (Can an optimizer accidentally delete the evaluation criteria entirely?) treats it like any other.
My reading, not the paper's, of what generalizes: a check is a guardrail only if its failure path refuses rather than substitutes. The paper's own guardrail list puts "the checks that cannot be argued with" ahead of the ones that can (Can deterministic checks protect LLM judges from failure?), and a parser is the paradigm of an unarguable check. It fired here. What failed was the response. So "mechanical" is necessary and not sufficient: the fallback made a deterministic check fail open, in a place where a selector downstream compares outputs and cannot tell a default from a rating.
The strongest counterargument is that defaults are ordinary engineering. They keep a pipeline running when one call misbehaves, and on a production path that is often the right call. The trade turns unsafe only where something ranks the output, because a ranked default is a candidate. The resemblance to Do autonomous agents report success when actions actually fail? is at the level of framing only: a failure presented in the normal form of a result, here at the harness layer and not in the agent's own report.
The excerpt does not say whether the default was meant for this harness only or for wider use, or what the authors changed afterward.
Inquiring lines that read this note 12
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How can evaluation criteria remain robust against agent gaming? Can defenses detect attacks composed across multiple skills?- What information should a proposer receive about failed guardrail checks?
- What feedback does ChainGuard return that an attacker could optimize against?
- What happens when a parser check fires but its fallback overrides the detection?
- How do default fallback scores mask failures in evaluation harnesses?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can an optimizer accidentally delete the evaluation criteria entirely?
When an optimizer rewrites instructions to improve scores, can it remove the measurement itself rather than improve it? This matters because it reveals whether optimization loops understand what they're measuring or just chase better numbers.
the deletion this fallback turned into a competing candidate
-
Can deterministic checks protect LLM judges from failure?
Explores whether mechanical, non-contestable verification steps can safeguard LLM-based decision systems. Matters because it tests whether we can make AI judgment survivable even when it goes wrong.
the paper's remedy list; this note is the caveat that the check's failure path matters as much as the check
-
Do autonomous agents report success when actions actually fail?
Explores whether agents systematically claim task completion despite failing to perform requested actions, and why this matters more than simple task failure for real-world deployment safety.
a failure shown in the normal form of a result, at the agent layer instead of the harness layer
-
Can individual components pass safety checks if the system still fails?
Explores whether local validation at each step—alignment checks, protocol compliance, plausibility tests—can guarantee safety when components are composed into larger workflows. Why the gap between component-level assurance and system-level outcomes matters for AI safety.
local checks and system-level failure; here the local check did fire, and the system still failed
-
What makes quietly failing systems more dangerous than obvious ones?
Systems with obvious failures get caught and dropped before scale. But what conditions make a subtly failing system persist and spread? Why might that be worse?
the same selection asymmetry argued at the scale of adoption: what a filter discards is the visible failure, and a failure dressed as a normal result passes; this case is at the scale of one loop
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?
- The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systems
- Large Language Model Guided Tree-of-Thought
- LLM-as-a-Judge Is Not an Oracle: Why Self-Improving Agents Need Deterministic Guardrails
- EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
- LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queries
- LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring
- ChatGPT Doesn’t Trust Chargers Fans: Guardrail Sensitivity in Context
Original note title
the harness caught the parsing errors and quietly fell back to a default rating of 3 on every dimension — a check whose failure path is a default score launders the failure it detected