If an AI just checks whether it was usually right before, without checking the actual facts, can it still judge truth well?
Can outcome-oriented summaries improve verification accuracy without grounding?
This explores whether a verifier can judge correctness more accurately by focusing on outcomes, meaning final answers, results, or a record of past successes, without checking claims against external evidence or the steps that produced them.
This explores whether a verifier can judge correctness more accurately by focusing on outcomes, meaning final answers, results, or a record of past successes, without checking claims against external evidence or the steps that produced them. The collection has no paper that tests 'outcome summaries' under that name. It does have several neighboring findings, and together they give a split answer. Outcome signals can help a lot, but only when the outcomes themselves are real. And looking only at final answers misses most of the failures that matter in long reasoning.
The strongest case for outcome-focused verification is XConf Can past performance predict when a model will be right?. Instead of asking a model how confident it feels about its current answer, XConf looks up past episodes where the model felt about as confident and checks how often it was actually right in those cases. That matches the reliability of sampling ten answers and voting, at a tenth of the cost. The catch is in the ablations: the benefit comes entirely from the stored record of real outcomes, not from the act of summarizing. So a record of what happened works as grounding in disguise. Take away the real results and the gain disappears. VeriFree Can reasoning improvement work without answer verification? makes a similar move. It skips the verifier and scores a reasoning trace by how likely it makes a known reference answer. 'No verifier' still leans on an outcome someone has already confirmed.
The case against relying on outcomes alone comes from process verification. In long reasoning tasks, adding checks on intermediate steps raised task success from 32% to 87% Where do reasoning agents actually fail during long traces?. Most failures were rule-breaking along the way, not wrong final answers, so a judge that sees only a tidy summary of the result can't see them. Confidence work points the same direction: confidence measured at each step catches breakdowns that averaging over the whole trace hides Does step-level confidence outperform global averaging for trace filtering?. A model can even work out the right answer internally and then overwrite it in its visible output Do transformers hide reasoning before producing filler tokens?. A summary can be confidently wrong about what actually happened.
There's also a quieter risk. A well-presented summary is exactly what fools an ungrounded judge. LLM judges reliably fall for fake citations and polished formatting Can LLM judges be fooled by fake credentials and formatting?. Models also avoid contradicting confident claims even when they know better Why do language models avoid correcting false user claims?. Compressing evidence into a clean summary can strip out the details a judge would need to object. For contrast, grounded systems get their reliability by refusing to answer when evidence is missing Can RAG systems refuse to answer without reliable evidence?, or by checking each reasoning step against real tool feedback Can interleaving reasoning with real-world feedback prevent hallucination?.
The takeaway you might not have expected: verification accuracy can improve without new training or external retrieval. It improves through finer-grained scores, repeated judging, and breaking criteria into parts Can verification accuracy scale without training models?. So the better question may not be 'outcomes or grounding?' but 'how much effort is spent verifying?' Outcome summaries make a cheap, strong signal when they come from real track records. As a substitute for checking the reasoning, they are the most effective thing to use when deceiving a judge.
Sources 10 notes
XConf matches ten-sample self-consistency at a tenth of the cost by retrieving the model's past episodes with similar confidence levels and reading their historical success rates. Ablations show the signal depends entirely on stored outcomes, not on the retrieval prompt itself.
VeriFree bypasses answer verification entirely by using the conditional probability of reference answers given generated reasoning traces as both reward signal and training weight. This approach matches or surpasses verifier-based methods on MMLU-Pro, GPQA, and SuperGPQA without rule-based or model-based verifiers.
Reliability for long-trace reasoning comes from checking intermediate states and policy compliance during generation, not from scoring final outputs. Adding intermediate verification raised task success from 32% to 87% because most failures are process violations, not wrong answers.
Local step-level confidence catches reasoning breakdowns that global averaging masks and enables early stopping before traces complete. This approach achieves comparable accuracy gains to naive majority voting with far fewer generated traces, proving trace quality matters more than quantity.
Logit lens analysis shows models trained with hidden CoT tokens compute correct answers in layers 1-3, then actively suppress these representations in final layers to produce format-compliant filler output. The reasoning is fully recoverable from lower-ranked token predictions.
Show all 10 sources
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
A multilingual RAG system for noisy historical newspapers succeeds by aggressively expanding retrieval while constraining generation to only grounded answers. The grounded-refusal prompt prevents hallucination when OCR errors and language drift degrade source quality, trading coverage for integrity.
ReAct demonstrates that alternating verbal reasoning with external tool queries (Wikipedia API, environment interaction) prevents error propagation by injecting real-world feedback at each step. On knowledge-intensive and interactive tasks, this approach outperforms pure chain-of-thought and reinforcement learning by 10-34% absolute accuracy.
Research shows verification accuracy improves independently via score granularity, repeated evaluation, and criteria decomposition—all deployable at inference without retraining. This reframes weak verifiers as under-scaled rather than fundamentally limited.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Local Coherence or Global Validity? Investigating RLVR Traces in Math Domains
- Deep Think with Confidence
- Reinforcing General Reasoning without Verifiers
- References Improve LLM Alignment in Non-Verifiable Domains
- Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens
- Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
- Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Traces
- Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!