Once a new AI benchmark is released, how long until most researchers start using it as their yardstick?

This explores how quickly a newly released benchmark becomes the yardstick most researchers report against. The corpus has no adoption-rate data, so it can't say whether that takes months or years. What it does show is which benchmarks the field reaches for, and what happens once it does.

The same few benchmarks keep turning up as fixed targets. The Darwin Gödel Machine reports its self-improvement as a 2.5× jump on SWE-bench and a 2.2× jump on Polyglot (Can AI systems improve themselves through trial and error?). A harness-optimization paper reports its gains on Terminal-Bench 2.1, and it re-runs the same runbook on newer models like GPT-5.6 and DeepSeek-V4 Flash without changes (Can execution harnesses lift model performance without retuning weights?). That's reuse and persistence, which isn't the same as speed. A benchmark that outlasts several model generations works like shared infrastructure, and the version number suggests these yardsticks get revised in place rather than replaced.

The corpus also suggests which benchmarks spread easily: the ones that are simple to score. Automated benchmarks favor precisely specified, auto-gradable tasks, which distorts what they show about frontier systems, and messier long-horizon evaluations catch emerging capabilities earlier (Do automated benchmarks hide what frontier AI systems can really do?). If a benchmark's usefulness to a research group depends on how cheaply it can be run and compared, adoption will favor the cheap-to-score ones. Nothing here measures that directly, so treat it as a reading of the evidence rather than a finding.

Adoption also has consequences that matter more than its speed. One analysis of 960 real occupational workflows argues that the field optimizes what it measures, and that it has measured contests rather than work (Why do agent benchmarks not predict real economic value?). Once a benchmark is widely adopted, scores on it can stop reflecting what they were meant to. Networks can pass every test while their internal structure stays incoherent, and standard benchmarks can't detect the difference (Can AI pass every test while understanding nothing?). Even the tools that grade the results can shift: LLM judges showed 31% judge shift on complex tasks, against 0.27% for an evidence-collecting agent judge (Can agents evaluate AI outputs more reliably than language models?).

As benchmarks multiply, tracing where a number came from becomes its own problem. Benchmark Radar keeps source identities and citations attached to each score, because scores without provenance can't reliably support model comparisons (Can benchmark scores be trusted without knowing their origin?). So the question the corpus can actually answer is what a benchmark does to a field once the field has adopted it. It steers effort toward what's easy to grade, and adoption pulls faster than validation.