Should a tool that grades AI answers make you lock in your grading rules upfront, or let them evolve as you look at real examples?
Should evaluation assistants treat criteria as outputs rather than inputs?
This explores whether tools that help people evaluate AI outputs should stop asking users to write their grading criteria up front, and instead help those criteria take shape while users look at real outputs.
This explores whether evaluation tools should let grading criteria emerge from looking at outputs, rather than requiring a fixed rubric up front. The clearest answer in the corpus is "partly, yes." When practitioners used EvalGen, a tool for building LLM graders, they ran into criteria drift: they needed some criteria to start grading, but grading real outputs kept changing what they thought the criteria should be Do evaluation criteria shift when we grade outputs?. So criteria work as both inputs and outputs. You can't grade without a starting rubric, and you can't settle on a good rubric without grading. A tool built on the idea that criteria are fixed inputs gets this loop wrong.
Other studies show what this loop could look like in practice. One approach, functional fragmentation, splits an output into pieces and labels what each piece is doing for each criterion. In a user study, practitioners using it found 48% more cases where the LLM judge's scores didn't match what they actually cared about, compared with reading a single overall score Can breaking outputs into fragments reveal more evaluation problems?. The tool doesn't hand people a better rubric. It gives them a closer look at outputs, and that closer look is where they notice their criteria were incomplete. Agent-based judges that gather evidence before scoring point the same way: their judgments became far more stable once evaluation became an active, evidence-driven process instead of applying a fixed prompt once Can agents evaluate AI outputs more reliably than language models?.
The main risk is drifting too far. A rubric that keeps changing can be gamed, and the corpus shows what happens when scores and real quality come apart. In one case, judge pass rates rose from 23% to 80% while the system got no better at the defects that mattered Can a higher evaluation score hide poor task performance?. Judges also have biases that can quietly shape whatever criteria grow out of their outputs. Models prefer text they recognize as their own Do LLMs favor their own text because they recognize it?, and evaluators reward polished writing even when the substance isn't better Does polished writing actually signal better quality work?. If criteria emerge from outputs, they can also absorb these biases.
So a better framing is to treat criteria as a working hypothesis that gets revised, with a record of each change. The field-level work on interactive evaluation makes a similar point at a larger scale: switching formats doesn't remove old problems like comparability and reproducibility, it just moves them somewhere else Do interactive evaluations actually solve the benchmark comparison problem?. What helps is explicit rules for how evidence turns into judgment Should interactive evaluation be designed as a unified paradigm?. For criteria, that means an evaluation assistant should help users revise their rubric while they read outputs, then freeze and version it, so that scores from week one and week six still mean the same thing.
The less obvious lesson is that "what counts as good" is often something you only find out by looking. If even practitioners can't fully state their standards ahead of time, a large part of evaluation work is discovering those standards. Tools that hide this, by asking for a rubric and then reporting a single number, may be hiding the most useful part of the process.
Sources 8 notes
A study of nine practitioners using EvalGen found that criteria drift: users need predefined criteria to grade outputs, but grading outputs fundamentally reshapes what those criteria should be. This suggests evaluation criteria are partly emergent from observing model outputs, not independent inputs.
A user study of ten practitioners found that functional fragmentation—decomposing outputs into fragments and labeling their rhetorical functions per criterion—surfaced 48% more evaluation misalignments than holistic LLM-as-a-Judge scores, enabling better trust calibration and discovery of actionable issues.
Eight-module agentic evaluation achieved 0.27% judge shift versus 31% for LLM-as-a-Judge on complex tasks. However, the memory module cascaded errors, revealing that agentic systems need error isolation mechanisms to maintain gains.
When systems optimize toward evaluation scores, measured progress can rise while actual task performance remains flat or declines, because optimization can exploit weaknesses in the measurement itself rather than solve the task. A relayed prompt case demonstrated this: judge pass rates rose from 23.1 to 80.0 percent while task-facing defect detection stayed unchanged.
Fine-tuning LLMs to recognize their own summaries increased their preference for those summaries in a linear relationship, suggesting recognition capability drives self-preference bias. The authors present this as initial causal evidence, not proof.
Show all 8 sources
Studies show evaluators perceived AI-generated documents as both human-written and better quality than human submissions. This suggests rhetorical polish misleads judgment and should not serve as a quality signal in evaluation.
Interactive evaluation relocates core problems—comparability, reproducibility, evidence-to-judgment mapping—into higher-dimensional space rather than solving them. The field needs design protocols and shared standards, not format adoption, to make trajectory scoring interpretable.
Interactive evaluation should be treated as a principled paradigm with explicit protocols and reporting standards, not adopted piecemeal as benchmarks. The fragmentation plaguing current interactive benchmarks mirrors early evaluation culture; formalizing the paradigm—expanding evidence from final responses to trajectories while standardizing how to score process quality and robustness—makes results interpretable and reproducible.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Interactive Evaluation Requires a Design Science
- Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
- LLM or Human? Perceptions of Trust and Information Quality in Research Summaries
- Evalet: Evaluating Large Language Models through Functional Fragmentation
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
- Temporal Self-Rewarding Language Models: Decoupling Chosen-Rejected via Past-Future
- Towards interactive evaluations for interaction harms in human-AI systems
- Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge