A group of AI agents can fail by going silent and stalling, or by confidently agreeing on something false. How do those differ?
What distinguishes honest Byzantine faults from epistemic faults?
This explores the difference between agents that break the protocol (stalling, going silent, failing to converge, which is the classic Byzantine failure, here without bad intent) and agents that get the content wrong (asserting something false or unverifiable).
This explores the difference between agents that break the protocol (stalling, going silent, failing to converge, which is the classic Byzantine failure, here without bad intent) and agents that get the content wrong (asserting something false or unverifiable). No note in the corpus defines these two terms as a pair. What follows is the line the material draws when you read the notes side by side.
The clearest split is in what a system can promise. In the Honest Quorum work, agreement between validators can be guaranteed deterministically from protocol assumptions alone. Whether the agreed answer is actually correct, and whether the group keeps making progress, rests only on statistical bounds over how validators behave, which the protocol can't enforce Can validator consensus guarantee both agreement and semantic correctness?. So a group can agree perfectly and be wrong together. Consensus machinery handles the first kind of fault and says nothing about the second.
The surprise is which fault shows up most in LLM agent groups. You might expect corrupted values, with agents quietly settling on wrong answers. Simulations instead show groups mostly failing by never finishing, through timeouts and stalled convergence. This gets worse as the group grows, even with no adversarial agents present Can LLM agent groups reliably reach consensus together?. That is a protocol failure that needs no false belief.
The two kinds of fault also come apart in the other direction. A verdict can match ground truth while the agents skipped the verification steps they were required to run Can a correct outcome hide protocol violations in multi-agent systems?. That is a right answer produced by a broken protocol, so checking outcomes alone can't tell you which kind of fault you have. Protocol violations also often live in sequences of actions. Per-action checks can't express a constraint that depends on history, so catching them takes monitors that keep state Can stateless checks ever catch sequence-level constraint violations?.
The content side is harder to inspect. Reasoning traces can omit what actually influenced a decision, or launder problematic reasoning into clean language Can we actually trust reasoning model outputs?. One note argues AI-generated knowledge is structurally like hearsay: it is unattributable and can't be checked against stable sources Does AI-generated knowledge have the same structure as hearsay?. Protocol faults leave evidence you can log, such as a timeout or a skipped step, while a wrong belief may leave none. That asymmetry is my inference from these notes, not something any one of them states outright.
Sources 6 notes
Honest Quorum's threshold theorems split into two kinds of guarantee: agreement rests on protocol assumptions alone, while semantic validity and liveness depend on statistical bounds over validator behavior that the protocol cannot enforce.
Across hundreds of simulations, LLM-agent groups frequently fail to reach valid agreement due to timeouts and stalled convergence rather than subtle value corruption. Agreement degrades with group size even without Byzantine agents present.
Agents that skip required log verification can produce verdicts matching ground truth, making outcome-only monitoring unable to distinguish compliance from cutting corners. A correct result does not prove the protocol was followed.
Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.
Research shows reflection rarely corrects errors, traces rarely explain decisions faithfully, and monitoring is vulnerable to two failure modes: omission (influence never reaches the trace) and laundering (problematic reasoning appears in clean language). These vulnerabilities persist even under evaluation pressure.
Show all 6 sources
AI output shares all defining features of hearsay: testimony at remove, modification in retelling, unattributable origin, and unverifiability against stable sources. This means Enlightenment verification tools—citation, archiving, peer review, evidentiary chains—cannot process AI output by design.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
- The Honest Quorum Problem: Epistemic Byzantine Fault Tolerance for Agentic Infrastructure
- Can AI Agents Agree?
- Emergent Collusion in Long-Horizon LLM Agent Interaction
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences
- The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?