Can a benchmark measure generalization without hiding training data?

Chollet's paper proposes guidelines for what a general AI benchmark should look like, then builds "a new benchmark closely following these guidelines, the Abstraction and Reasoning Corpus (ARC), built upon an explicit set of priors designed to be as close as possible to innate human priors." Its stated position: "intelligence testing should control for scope, priors, and experience: every test task should be novel (measuring the ability to understand a new task, rather than skill) and should assume an explicit set of priors shared by all test-takers." It adds that "ARC explicitly assumes the same Core Knowledge priors innately possessed by humans."

The design directly operationalizes the paper's warning that skill tests are gameable: by making every task novel and fixing in advance the priors available to every test-taker, the benchmark removes the two levers the paper names — "unlimited prior knowledge" and "unlimited training data" — that let a system be credited with skill that masks, rather than demonstrates, its own generalization power. The paper reports the resulting asymmetry plainly: "ARC can be fully solved by humans, but cannot be meaningfully approached by current machine learning techniques, including Deep Learning."

This move — manipulating what's given to the test-taker rather than just scoring raw output — parallels How much guidance do AI systems need to conduct research independently?, where ASI-Bench holds a research project fixed and withdraws guidance in stages so results isolate what guidance a model still needs; ARC instead holds priors fixed and varies task novelty, but the logic of controlling the input condition to make the output interpretable is the same. It also bears on Can AI systems invent new concepts rather than reuse trained ones?: that paper's vocabulary gap — the difficulty of inventing new representational primitives rather than searching within a fixed space — is close to what this excerpt calls generalization difficulty, since both locate the hard part of intelligence in handling representations not given in advance. See the sibling note Should we measure intelligence as skill or learning efficiency? for the definition ARC is built to measure.

The excerpt's own limitations section concedes that generalization is "not quantified" for ARC itself, and that ARC's validity — whether performance on it predicts real-world usefulness — "should be investigated via large-sample size statistical studies on humans," which the paper says had not been done when it was written. So ARC's claim to measure "a human-like form of general fluid intelligence" rests on the control-for-priors design logic above; the paper is explicit that it is a design intention, not yet a validated measurement.