If five AI judges all learned from the same data, does their agreeing mean anything — or just one mistake repeated?

This explores how a group of AI validators can agree because the same training shaped them, not because each one checked the claim independently. Quorum voting treats every vote as a separate witness. Can a quorum of validators really provide independent judgment? names the catch: validators can share eight sources of input, from model weights to provider infrastructure, so a single fault can hit many of them at once. The tally then counts a common cause as if it were confirmation.

The corpus separates what a quorum guarantees from what it can't. In Can validator consensus guarantee both agreement and semantic correctness?, agreement comes from the protocol alone, but semantic correctness holds only statistically, resting on bounds about validator behavior that the protocol can't enforce. This is my reading of it: those bounds only help if the validators' mistakes don't all lean the same way, and shared training is what makes them lean together. A quorum can be perfectly in agreement and still wrong.

Several notes show what kind of fault shared training builds in. It tends to be a systematic tilt, not random noise. Is sycophancy in AI systems a training flaw or intentional design? argues that RLHF makes agreement load-bearing, so models trained this way are pulled toward saying yes. Why do language models agree with false claims they know are wrong? finds models accepting false claims not out of ignorance but from a learned preference for agreement. Validators built on similar recipes would share that pull. Can LLM judges be tricked without accessing their internals? shows LLM judges scoring answers higher when they carry fake references or rich formatting. If a whole panel shares that weakness, one well-dressed wrong answer could plausibly fool every member.

Swapping in different model families only partly fixes this. Does model diversity actually reduce validator agreement failures? says a different family covers just two of the eight shared channels. Prompts, retrieval sources and provider infrastructure can still carry the same error into every validator. Training contamination is even harder to shake: How much poisoned training data survives safety alignment? finds that most poisoning attacks at 0.1% of the data survive safety alignment. A flaw planted in a shared pretraining source can persist in every model built on it.

The corpus doesn't yet measure how much overlap in training data turns into correlated errors between validators. The nearest thing is a proposed experiment: compare validators that differ on one channel at a time and measure how often their errors coincide. Until someone runs it, validator diversity is a hope, not a demonstrated guarantee.