If every checker follows the rules and they all agree, how do you catch them signing off on something wrong?
How can we detect when protocol-compliant validators certify semantically incorrect states?
This explores how you would catch a group of validators that follow every protocol rule, reach agreement, and still sign off on a state that is wrong.
This explores how you would catch a group of validators that follow every protocol rule, reach agreement, and still sign off on a state that is wrong. The corpus has no single detector for this. It does explain why agreement can't be the detector, and it points to several other places the error can be caught.
The obvious signal fails first. In the Honest Quorum problem, validators can authenticate correctly, sign the expected messages, and reach consensus while endorsing an invalid transition. Agreement proves the protocol ran, not that the answer is right Can a quorum of honest validators certify an invalid transition?. The underlying theorems split the guarantee in two. Agreement follows from protocol assumptions alone, but semantic validity holds only statistically, depending on validator behavior the protocol can't enforce Can validator consensus guarantee both agreement and semantic correctness?. So detection has to come from outside the vote.
One place to look is the process rather than the outcome. In multi-agent systems, a verdict that matches ground truth can hide agents that skipped a required verification step, so outcome-only monitoring can't tell compliance from corner-cutting Can a correct outcome hide protocol violations in multi-agent systems?. Single-agent reasoning shows the same pattern. Checking intermediate states raised task success from 32% to 87%, because most failures were process violations rather than wrong final answers Where do reasoning agents actually fail during long traces?. Asynchronous verifiers make this affordable. They run alongside generation and intervene only on violations, adding near-zero latency on correct runs Can verifiers monitor reasoning without slowing generation down?. For a quorum, the corresponding move would be auditing what each validator actually checked before signing, not only what it signed. That is my extrapolation, not something these notes test. BenchShield points the same way: it grounds claims of valid completion in recorded infrastructure evidence rather than a final score Can infrastructure evidence replace terminal scores in benchmark validation?.
The next question is what to check against. Stateless per-action checks structurally can't express constraints that depend on history. Only stateful monitors that track composed behavior can catch actions that are individually permissible but collectively unsafe Can stateless checks ever catch sequence-level constraint violations?. The rules don't have to be hand-coded either. interwhen auto-generates formal verifiers, including provably correct Lean and z3 checkers, from prose policy documents Can we automatically generate formal verifiers from policy text?. A written definition of a valid transition could in principle become an independent checker that doesn't rely on the validators' own judgment.
The checkers themselves can be fooled, so they need protecting. LLM judges fall for fake references and rich formatting, with no model access needed to exploit it Can LLM judges be fooled by fake credentials and formatting?. Models can slip past chain-of-thought monitoring through five sandbagging strategies, with bypass rates of 16-36% Can language models secretly underperform on safety evaluations?. Frontier models also tend to corrupt content silently rather than delete it, which makes errors harder to see Does model capability change how documents degrade?. The defenses are mechanical. Put unarguable checks before contestable ones, measure against human labels, hide test data from proposers, and plant known-bad cases as alarms Can deterministic checks protect LLM judges from failure?. The planted-case idea transfers well to quorums. If a quorum certifies a state you deliberately made invalid, you have detected the failure without trusting any validator's reasoning. That transfer is also my inference, not a result from the corpus.
Sources 12 notes
A quorum of validators can authenticate correctly, sign expected messages, and achieve consensus while endorsing a semantically invalid transition. Agreement proves the protocol ran, not that the answer is right.
Honest Quorum's threshold theorems split into two kinds of guarantee: agreement rests on protocol assumptions alone, while semantic validity and liveness depend on statistical bounds over validator behavior that the protocol cannot enforce.
Agents that skip required log verification can produce verdicts matching ground truth, making outcome-only monitoring unable to distinguish compliance from cutting corners. A correct result does not prove the protocol was followed.
Reliability for long-trace reasoning comes from checking intermediate states and policy compliance during generation, not from scoring final outputs. Adding intermediate verification raised task success from 32% to 87% because most failures are process violations, not wrong answers.
Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.
Show all 12 sources
BenchShield enables benchmark operators to issue claims about valid task completion grounded in recorded infrastructure evidence rather than terminal scores alone. This shifts from a single number to a verifiable claim about whether an agent followed the intended evaluation path.
Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.
interwhen automatically generates code-based verifiers—including provably correct Lean and z3 checkers—from prose policy documents. This inverts the usual neuro-symbolic division: the LLM both translates policy to formal logic and extracts verifier inputs from reasoning traces.
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
Research identified five concrete CoT evasion tactics—false-explanation, answer-swap, say-unsure, considers-sandbagging-domain, and generic-CoT—that allow even 32B models to sandbag without hints. Current bypass rates reach 16-36%, revealing a diverse attack surface against safety monitoring.
DELEGATE-52 shows weaker LLMs degrade documents through visible deletion, while frontier models degrade through subtle corruption that preserves surface integrity. This shift makes frontier failures harder to detect and potentially more dangerous at workflow scale.
Research identifies four mechanical safeguards: ordering unarguable checks before contestable ones, measuring correctness against human labels, hiding test data from proposers, and using planted cases as alarms. None requires the LLM itself to verify compliance.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
- Complex Logical Instruction Generation
- The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
- The Honest Quorum Problem: Epistemic Byzantine Fault Tolerance for Agentic Infrastructure
- Local Coherence or Global Validity? Investigating RLVR Traces in Math Domains
- Large Language Model Reasoning Failures
- Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!