When an AI aces a test, does that mean it can do the real job, or just the gradable parts?

This explores what gets warped when a real task is turned into something a machine can grade automatically, and where a benchmark score and real-world ability come apart. The corpus points to several separate distortions, and they don't all push in the same direction.

The first distortion is which tasks get measured at all. Benchmarks favor tasks that are precisely specified and easy to grade automatically, so messy, long-horizon work mostly drops out. The result is that automated benchmarks can both overstate and understate what frontier systems do. Do automated benchmarks hide what frontier AI systems can really do? proposes the alternative: run long, untidy real tasks, read the logs qualitatively, and report the cost. That approach also catches emerging capabilities earlier. A second thing that disappears is the user. Outcome-only scoring hides how different people phrase requests and how they judge results, which is why Can simulated users reveal what offline benchmarks miss? builds simulated users, with 8.3 billion persona records, to put that variety back into evaluation.

The second distortion is that the test may not be independent of the model. Qwen2.5-Math-7B can reconstruct 54.6% of MATH-500 from partial prompts, yet it scores 0.0% on the later LiveMathBench. That gap suggests the earlier score reflects memorization more than reasoning (Does RLVR success on math benchmarks reflect genuine reasoning improvement?). The same work also found that on clean benchmarks only correct rewards improve performance. Contamination doesn't mean the underlying change is fake, though. Can genuine reasoning activation coexist with contaminated benchmarks? argues that real reasoning behavior and inflated benchmark gains sit at different measurement levels and can coexist. So a score can be wrong about a real effect, or right for the wrong reason.

The third distortion is that a single number doesn't say what produced it. A benchmark result measures the model plus everything wrapped around it. Can execution harnesses lift model performance without retuning weights? shows this: optimizing the execution system around frozen weights lifted DeepSeek-V4 Flash by 5.4 points on Terminal-Bench 2.1, with no change to the model. The number also doesn't say whether the run was legitimate. Can infrastructure evidence replace terminal scores in benchmark validation? argues for backing scores with recorded infrastructure evidence about whether the agent followed the intended path. And a score stripped of its source and setting can't support comparisons, which is the case for Can benchmark scores be trusted without knowing their origin? keeping provenance attached. A quieter version of the same problem comes from instruction tuning. Models trained on empty or deliberately wrong instructions do about as well as models trained on correct ones, because what transfers is the shape of the output (Does instruction tuning teach task understanding or output format?). Part of a good score can therefore be familiarity with the format rather than understanding of the task.

Making evaluations interactive doesn't fix this. Do interactive evaluations actually solve the benchmark comparison problem? finds that comparability, reproducibility, and the link from evidence to judgment move into a higher-dimensional space instead of going away, and that the field needs shared design protocols more than a new format. The common thread across these notes is that a benchmark score is a compressed claim, and each distortion above is something the compression discarded: the messy tasks, the users, the model's independence from the test, or the conditions of the run.