If you ask AI checkers from different companies to review the same work, does their agreement really prove it's right?

This explores whether swapping in validators from different model families makes their mistakes independent, so that agreement between them means something. The corpus's answer is probably not on its own. Asking three validators is like asking three friends to check your work: it only helps if their errors are unrelated. One note argues that model family changes just two of eight fault channels the validators share. The prompts they receive, the retrieval sources they pull from, and the provider infrastructure underneath can still carry the same error to all of them Does model diversity actually reduce validator agreement failures?. Three validators agreeing may be one mistake counted three times.

Part of the reason is that models from different labs are less different than their brand names suggest. A study of 70+ models on 26K open-ended queries found an "Artificial Hivemind" effect: models independently produced strikingly similar or even identical answers, which the authors trace to overlapping training data and alignment procedures Do different AI models actually produce diverse outputs?. That was measured on open-ended generation, not on validation, so applying it to validators is an inference. Preference tuning adds a hint. It reduces diversity in code but increases it in creative writing, because code rewards converging on the correct answer Does preference tuning always reduce diversity the same way?. Validation is right-or-wrong work, so it plausibly sits in the regime where training pushes every model toward the same answers. The trouble is that they also converge on the same wrong ones.

Changing how validation is done may matter more than changing which model does it. An agentic judge that collects evidence cut judge shift to 0.27%, against 31% for a plain LLM judge on complex tasks. But its memory module cascaded errors through the system, so one shared component undid the gains Can agents evaluate AI outputs more reliably than language models?. That is the same failure the diversification worry describes, only inside one system. The wider multi-agent literature in the corpus points the same way. It lists Degeneration-of-Thought and Silent Agreement among the failure modes of multi-agent deliberation, a reminder that agents agreeing is not the same as verifying What limits autonomous capability in large language models?. Diversity of approach can be engineered without changing the model. Structuring one model's reasoning as a dialogue between distinct agents produced more varied problem-solving strategies Can dialogue format help models reason more diversely?. That measures variety of reasoning, though, not independence of errors.

The corpus does not have the measurement that would settle this. The note proposes building validators that differ on one channel at a time (model family, prompt, retrieval source, provider) and measuring how correlated their errors are. Until someone does, model diversity is a cheap first step. Agreement among validators should count as weaker evidence than the number of validators suggests.