When a group of AI judges votes on whether something's valid, why can they guarantee agreement but not correctness?
How do false endorsements and unusable support bound validator consensus properties?
This explores how two ways a validator vote can go wrong, endorsing something invalid ('false endorsements') and support that can't be counted toward a decision ('unusable support'), set ceilings on what a group of AI validators can promise.
This explores how two ways a validator vote can go wrong set ceilings on what a group of AI validators can promise. I'm reading 'false endorsements' as validators signing off on an invalid result, and 'unusable support' as votes that never add up to a usable quorum. The corpus doesn't define either term in those words. It frames the limits as semantic validity and liveness, and that's the mapping I use below.
The basic split is that agreement is guaranteed by the protocol alone, while semantic validity and liveness are not. Honest Quorum's threshold theorems give agreement deterministically. Validity and liveness only get statistical bounds over how validators behave, and the protocol can't enforce that behavior (Can validator consensus guarantee both agreement and semantic correctness?). So a quorum can promise that everyone will agree, but only with some probability that what they agree on is right, or that they finish at all.
False endorsements are the validity side. A quorum of validators can authenticate correctly, sign the expected messages, reach consensus, and still endorse an invalid transition. Agreement proves the protocol ran, not that the answer is right (Can a quorum of honest validators certify an invalid transition?). A count of how many validators endorse something only bounds correctness if their mistakes are independent. The catch is that validators can share up to eight things, from model weights and lineage to prompts, retrieval sources and provider infrastructure. One shared fault can then produce many endorsements at once, and the vote counts a common cause as independent confirmation (Can a quorum of validators really provide independent judgment?). Switching model families only addresses two of the eight channels, so the correlation likely survives (Does model diversity actually reduce validator agreement failures?).
Unusable support is the liveness side. This is my reading, because the corpus never uses that phrase. Support that can't be counted, such as a stalled or timed-out validator, or a group that never converges, stops a quorum from forming. Simulations of LLM-agent groups show this is the main way they fail. They mostly fail by losing liveness, through timeouts and stalled convergence, rather than by subtle value corruption. It gets worse as the group grows, even with no adversarial agents present (Can LLM agent groups reliably reach consensus together?). Adding validators doesn't reliably buy you more usable votes.
The corpus offers a partial way to make the bounds usable. One note describes four mechanical safeguards for LLM judges: ordering unarguable checks before contestable ones, scoring against human labels, hiding test data from proposers, and planting known cases as alarms. None of them needs the LLM to grade itself (Can deterministic checks protect LLM judges from failure?). The notes don't connect this to Honest Quorum. My inference is that checks like these could help estimate how often validators endorse falsely or fail to deliver usable support, and those are the quantities the statistical bounds depend on.
Sources 6 notes
Honest Quorum's threshold theorems split into two kinds of guarantee: agreement rests on protocol assumptions alone, while semantic validity and liveness depend on statistical bounds over validator behavior that the protocol cannot enforce.
A quorum of validators can authenticate correctly, sign expected messages, and achieve consensus while endorsing a semantically invalid transition. Agreement proves the protocol ran, not that the answer is right.
The Honest Quorum paper identifies eight things agentic validators may share—from model weights to provider infrastructure—allowing one fault to affect multiple validators simultaneously. Quorum voting then counts agreement as independent confirmation when it may simply reflect a common cause.
While different model families address two of eight shared fault channels, correlated epistemic errors likely persist through prompts, retrieval sources, and provider infrastructure. Measuring error correlation across validators differing on one channel at a time could quantify the effect.
Across hundreds of simulations, LLM-agent groups frequently fail to reach valid agreement due to timeouts and stalled convergence rather than subtle value corruption. Agreement degrades with group size even without Byzantine agents present.
Show all 6 sources
Research identifies four mechanical safeguards: ordering unarguable checks before contestable ones, measuring correctness against human labels, hiding test data from proposers, and using planted cases as alarms. None requires the LLM itself to verify compliance.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Honest Quorum Problem: Epistemic Byzantine Fault Tolerance for Agentic Infrastructure
- Can AI Agents Agree?
- Psychologically Enhanced AI Agents
- Consensus is Strategically Insufficient: Reasoning-Trace Disagreement as a Knowledge-Representation Signal
- Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences
- Emergent Collusion in Long-Horizon LLM Agent Interaction
- When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?