Are AI judges good at grading everything, or just good at faking the parts that are easy to fake, like citations and formatting?
Do LLM judges have uneven skill across different evaluation criteria?
This explores whether an LLM acting as a judge is better at grading some qualities of a response (accuracy, reasoning, style, sourcing) than others, and why that might happen.
This explores whether an LLM judge is reliably good at some evaluation criteria and weak at others. The corpus has no head-to-head study that scores judges criterion by criterion. What it does have points strongly toward yes: judges are least reliable wherever a surface cue can stand in for the real thing being measured.
The clearest evidence comes from work on judge biases. Researchers found four exploitable biases, and two of them, authority and 'beauty' (rich formatting), are 'semantics-agnostic'. The judge rewards fake citations and polished layout no matter what the content says Can LLM judges be fooled by fake credentials and formatting? Can LLM judges be tricked without accessing their internals?. In practice, a judge asked to score 'credibility' or 'presentation' is grading the criterion most easily faked. Criteria that need the judge to actually check substance are harder to game in this way, but they are also where the judge's own reasoning has to do real work. That is why training judges with RL to reason before they score cuts these biases Can reasoning during evaluation reduce judgment bias in LLM judges?, and why simply telling a judge to 'be fair' doesn't Can prompting reduce bias in LLM judges reliably?.
The unevenness also stays hidden when you only look at a single overall score. When practitioners broke outputs into fragments and labeled what each piece was doing for each criterion, they found 48% more places where the judge's verdict didn't match what they would have concluded Can breaking outputs into fragments reveal more evaluation problems?. A single holistic number blends strong and weak criteria together. Decomposing first and zooming out only once the patterns stabilize is one way to see which criteria the judge actually handles well When should practitioners stop decomposing LLM evaluations?.
A less obvious point is that some apparent criterion failures are really failures about who wrote the text or how good it is. Judges prefer LLM-written arguments over human ones, 62% to 37% Do LLM judges systematically favor arguments from other LLMs?. That preference rises in step with how well a model can recognize its own writing Do LLMs favor their own text because they recognize it?. In peer review, though, the apparent favoritism toward LLM-written papers disappeared once paper quality was held constant. The real pattern was general leniency toward weaker work Do LLM reviewers actually favor LLM-written papers?. So a judge can look biased on 'originality' when it is actually miscalibrated at the low end of the quality range.
Two more points complicate the question. First, the criteria themselves aren't fixed: people revise what they mean by 'good' as they grade Do evaluation criteria shift when we grade outputs?. Some of a judge's unevenness may therefore come from criteria that were never stable to begin with. Second, practical fixes exist that work around a single judge's weak spots rather than curing them. You can aggregate many evaluations, which brings rankings closer to expert consensus, mostly because of the sheer number of evaluations rather than the mix of models Can aggregated LLM rankings match human expert judgments?. Or you can read the judge's full probability distribution over scores instead of one token, so it can tell close cases apart Can reading logit distributions break ties in LLM judging?.
Sources 12 notes
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
Research shows LLM evaluators systematically score higher when responses include fake references or rich formatting, independent of content quality. These biases are exploitable without model access, undermining AI benchmark credibility.
Training judges with reinforcement learning to reason about evaluations—by converting judgment tasks into verifiable problems with synthetic data pairs—produces judges that think through their decisions rather than relying on exploitable surface features, directly mitigating authority, verbosity, position, and beauty bias.
Research evidence suggests that instructing LLM judges to reduce bias does not reliably work. The practical implication is that system design should focus on containing judge errors through structural checks rather than attempting to eliminate bias through better instructions.
A user study of ten practitioners found that functional fragmentation—decomposing outputs into fragments and labeling their rhetorical functions per criterion—surfaced 48% more evaluation misalignments than holistic LLM-as-a-Judge scores, enabling better trust calibration and discovery of actionable issues.
Show all 12 sources
Evalet proposes a three-step workflow: start with fragment-level evaluation, iteratively refine criteria until similar clusters emerge consistently, then zoom out to holistic scores. The stopping condition is when function clusters stabilize rather than continuing to shift.
LLM judges selected LLM arguments as winners 62% of the time versus humans' 37%, while humans split votes 39% LLM / 37% human. This same-author bias operates downstream of component scoring and compounds existing judge vulnerabilities, creating a calibration ceiling in RLAIF pipelines.
Fine-tuning LLMs to recognize their own summaries increased their preference for those summaries in a linear relationship, suggesting recognition capability drives self-preference bias. The authors present this as initial causal evidence, not proof.
Across 125,000+ reviews, the apparent favoritism of LLM-assisted reviewers toward LLM papers disappears once paper quality is held constant. LLM papers cluster among weaker submissions, creating a spurious interaction driven by LLM reviewers' general leniency toward lower-quality work.
A study of nine practitioners using EvalGen found that criteria drift: users need predefined criteria to grade outputs, but grading outputs fundamentally reshapes what those criteria should be. This suggests evaluation criteria are partly emergent from observing model outputs, not independent inputs.
Multiple LLMs ranking business models individually are inconsistent and biased, but aggregating evaluations—especially across different models—produces rankings that resemble human-expert rankings. Gains come primarily from scaling (more evaluations) rather than diversity (different LLM types or prompts).
Computing the expectation over scoring-token logit distributions yields continuous verifier scores instead of discrete tokens, substantially reducing ties and improving discrimination between solutions without additional training or models.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge
- Humans or LLMs as the Judge? A Study on Judgement Biases
- References Improve LLM Alignment in Non-Verifiable Domains
- LLM-REVal: Can We Trust LLM Reviewers Yet?
- LLM or Human? Perceptions of Trust and Information Quality in Research Summaries
- Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
- Evalet: Evaluating Large Language Models through Functional Fragmentation
- Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge