Do evaluation criteria shift when we grade outputs?

EvalGen, a mixed-initiative interface for building LLM evaluation assertions embedded in the ChainForge prompt-engineering tool, ran a qualitative study with "nine (9) industry practitioners" who were building evaluation pipelines for their own LLM prompts. The study's headline finding, which the authors name "criteria drift": "users need criteria to grade outputs, but grading outputs helps users define criteria." Some criteria, the paper reports, "appears dependent on the specific LLM outputs observed (rather than independent criteria that can be defined a priori)." The stated implication is direct: approaches that "assume the independence of evaluation from observation of model outputs" rest on a premise the study's own participants did not hold to in practice.

EvalGen's design mirrors the problem it surfaces. The tool has an LLM propose natural-language evaluation criteria from the user's prompt, then generates candidate assertions (Python functions or LLM "grader" prompts) for each criterion; the user supplies "good"/"bad" grades on a sample of outputs, and those grades drive automatic selection of assertions by "alignment" — a combination of coverage (catching outputs the user marked bad) and false-failure rate (not flagging outputs the user marked good). Because grading is the step where users actually encounter the range of model outputs, the paper finds that grading itself revises what the criteria should be — criteria function as an output of the process, not only an input to it. The discussion pushes this toward an epistemic question: whether "alignment" is "an actualizable goal" at all, and whether the field's working assumption of a "ground truth' set of labels we merely need to elicit" survives contact with how practitioners actually evaluate.

This complicates Does iterative prompt engineering undermine scientific validity?'s diagnosis that criteria instability in prompt evaluation is a researcher-bias problem fixable with pre-specified, validated protocols: EvalGen's practitioners were not being sloppy — criteria drift showed up even for people deliberately trying to align evaluators with their own preferences, suggesting the instability is structural rather than a discipline failure. It also offers a mechanism for why Do LLM benchmarks actually measure what they claim to measure?: if criteria genuinely shift once evaluators observe outputs, a benchmark's upfront, rarely-revisited construct definitions are exactly where validity gaps would accumulate over time.

The study is qualitative and small — nine practitioners, evaluating one interface's workflow — so the excerpt does not establish how widespread criteria drift is outside prompt-engineering contexts, nor does it quantify how much criteria change or when drift stabilizes. The paper also does not resolve its own epistemic question about whether alignment is achievable; it leaves this as an open question for future evaluation assistants rather than an argued position. What the excerpt does support: evaluation tooling that treats criteria as fixed and elicitable upfront is likely to be systematically wrong about its own targets, not merely imprecise in hitting them.