If a panel of AI agents votes on an answer, can we guarantee the answer is actually right?
Does semantic validity across a quorum require new property definitions?
This explores whether the classic promises we ask of a group of validators (they agree, and the answer is valid) still work when the validators are AI agents, or whether "the answer is actually right" needs a different kind of guarantee.
This explores whether the classic promises we ask of a group of validators (they agree, and the answer is valid) still work when the validators are AI agents, or whether "the answer is actually right" needs a different kind of guarantee. The corpus points toward yes, and the Honest Quorum work already treats it that way. Its theorems come in two kinds. Agreement follows from protocol assumptions alone, so it holds deterministically. Semantic validity (and liveness) holds only as a statistical bound over how validators behave, and the protocol can't enforce that behavior (Can validator consensus guarantee both agreement and semantic correctness?). That split is itself a new way of stating the property: the guarantee is no longer "this always holds" but "this holds if validators are right often enough."
The reason the old definition falls short is that a quorum can authenticate correctly, sign the expected messages, reach consensus, and still endorse a semantically invalid transition. Agreement proves the protocol ran, not that the answer is right (Can a quorum of honest validators certify an invalid transition?). A protocol sees messages and signatures, not meaning, so nothing in the classic property list can catch this failure.
The statistical bounds also lean on an assumption that is easy to break: that validators err independently. The Honest Quorum paper names eight things validators may share, from model weights to provider infrastructure. One fault can then hit many validators at once, and a vote count treats what may be a single common cause as independent confirmation (Can a quorum of validators really provide independent judgment?). My inference is that a workable definition has to say how validators' errors relate to each other, not just how many of them are wrong.
Other notes in the collection suggest why the statistical side is fragile. LLM-agent groups often fail to reach valid agreement at all, through timeouts and stalled convergence rather than corrupted values, and this gets worse as the group grows, even with no adversaries (Can LLM agent groups reliably reach consensus together?). Frontier models corrupt documents silently while weaker ones visibly delete content, so the better the validator, the harder its mistakes are to spot (Does model capability change how documents degrade?). Models can also pick up the form of reasoning without the substance: invalid chain-of-thought examples work almost as well as valid ones (Does logical validity actually drive chain-of-thought gains?). And they can build defensible-looking frameworks on ideas that have no legitimate correspondence, without flagging the problem (Do language models evaluate semantic legitimacy when fusing concepts?). A validator's "approve" may therefore reflect plausibility rather than checking. This fits the broader finding that self-improvement and verification are bounded by a generation-verification gap (What limits autonomous capability in large language models?).
The corpus doesn't offer a full catalogue of replacement property definitions. What it shows is a direction: separate the guarantees the protocol can prove from the ones that depend on validator behavior, and say explicitly what that behavior must look like, including how correlated it may be. A neighboring result has the same shape. Annotation responses that look alike can be different kinds of signal (real preference, non-attitude, constructed preference), and treating them uniformly contaminates training (Do all annotation responses measure the same underlying thing?). Likewise, a quorum's "yes" votes can mean different things, and counting them as one thing is what causes the trouble.
Sources 9 notes
Honest Quorum's threshold theorems split into two kinds of guarantee: agreement rests on protocol assumptions alone, while semantic validity and liveness depend on statistical bounds over validator behavior that the protocol cannot enforce.
A quorum of validators can authenticate correctly, sign expected messages, and achieve consensus while endorsing a semantically invalid transition. Agreement proves the protocol ran, not that the answer is right.
The Honest Quorum paper identifies eight things agentic validators may share—from model weights to provider infrastructure—allowing one fault to affect multiple validators simultaneously. Quorum voting then counts agreement as independent confirmation when it may simply reflect a common cause.
Across hundreds of simulations, LLM-agent groups frequently fail to reach valid agreement due to timeouts and stalled convergence rather than subtle value corruption. Agreement degrades with group size even without Byzantine agents present.
DELEGATE-52 shows weaker LLMs degrade documents through visible deletion, while frontier models degrade through subtle corruption that preserves surface integrity. This shift makes frontier failures harder to detect and potentially more dangerous at workflow scale.
Show all 9 sources
Illogical chain-of-thought exemplars matched valid CoT performance on BIG-Bench Hard, showing that structural properties—not logical validity—drive the gains. The model learns the form of reasoning, not genuine inference.
LLMs generate coherent, plausible metaphorical reasoning when prompted to fuse semantically distant concepts without legitimate correspondences. Rather than decline or flag the fusion as speculative, they produce elaborate frameworks presented as defensible research, revealing a category-distinct hallucination type missed by fact-checking taxonomies.
Multi-agent deliberation produces specific failure modes (Degeneration-of-Thought, Silent Agreement), alignment at scale includes problematic self-valuation, and self-improvement is formally bounded by the generation-verification gap. Measurement error and conditional compliance hide the true capability ceiling.
Behavioral science reveals that annotations contain genuine preferences, non-attitudes, and constructed preferences—distinguishable by consistency across measurement conditions. Treating them uniformly contaminates reward model training and downstream alignment.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Honest Quorum Problem: Epistemic Byzantine Fault Tolerance for Agentic Infrastructure
- Can AI Agents Agree?
- Emergent Collusion in Long-Horizon LLM Agent Interaction
- Psychologically Enhanced AI Agents
- LLMs Corrupt Your Documents When You Delegate
- Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures
- Consensus is Strategically Insufficient: Reasoning-Trace Disagreement as a Knowledge-Representation Signal
- Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences