When a group of people or AIs judges an idea, does talking it over beat checking it against the outside world?

This explores whether groups judging ideas need information from outside their own discussion. The corpus has no note that tests this directly, but neighboring findings point the same way: outside evidence is what keeps a group's verdict about the idea rather than about how the conversation went. Without it, groups drift toward agreement, style, and the audience's existing beliefs.

Discussion alone adds less than you'd expect. Language model groups reproduce the human pattern where talking helps average members more than top performers, but they get there through greater conformity, earlier convergence, and less unique information surfacing than human groups do (Do language model groups mimic human group reasoning patterns?). If a group converges before anyone brings in new material, the discussion is mostly agreement. Knowledge that was never in the room also can't be swapped in later. Diverse ideation teams without genuine senior expertise do worse than a single competent agent (Does cognitive diversity alone improve multi-agent ideation quality?). Diversity gives the group more to react to, but only real domain knowledge lets it separate good ideas from plausible-sounding ones.

Evaluators are also easy to fool when they only see the ideas as presented. Models trained to imitate ChatGPT fooled human raters by copying its confident, fluent style while gaining no real capability (Can imitating ChatGPT fool evaluators into thinking models improved?). In debates, voters' prior political and religious beliefs predicted outcomes better than the language did (Does what readers believe matter more than what debaters say?). Framing a claim as already-accepted background makes it less likely to be scrutinized at all (Why are presuppositions more persuasive than direct assertions?). Even the visible reasoning can mislead: logically invalid chain-of-thought examples work nearly as well as valid ones, so reasoning that looks sound is weak proof (Does logical validity actually drive chain-of-thought gains?). A group judging by how convincing an idea sounds is judging exactly what these findings say is unreliable.

Outside evidence breaks that loop, and there's a clean argument for why. An information-theoretic proof shows that a check reading only the generated text can't reliably tell whether a reflection helped when the truth depends on something outside the text. Checks grounded in the environment can (Can transcript alone tell whether a reflection helps?). The practical results fit. An agent-based judge that collects its own evidence shifted only 0.27% across conditions, against 31% for a plain LLM judge on complex tasks (Can agents evaluate AI outputs more reliably than language models?). For ideas specifically, a pipeline that extracts an idea's claims, retrieves related work, and compares them reached 86.5% reasoning alignment with human reviewers on 182 ICLR submissions, beating holistic LLM judgments (Can structured pipelines make LLM novelty assessment reliable?). Novelty can't be judged from inside the idea. It is a relationship to what already exists, and comparison is how people evaluate anyway (Do comparisons help users evaluate items better than isolated descriptions?).

Evidence has to be chosen and handled carefully, though. Picking evidence by explaining why each piece matters beat similarity re-ranking by 33% while using half as many chunks (Can rationale-driven selection beat similarity re-ranking for evidence?), so more evidence isn't better, but relevant evidence is. The evidence-collecting judge also had a failure: its memory module cascaded errors, so one bad piece of evidence spread through later judgments (Can agents evaluate AI outputs more reliably than language models?). External evidence is what a group's evaluation stands on, and it needs the same scrutiny as the ideas being evaluated.