Should we judge intelligence by what a system already knows, or by how fast and cheaply it picks up something new?

This explores whether intelligence is better measured by how efficiently a system learns than by what it can already do. The clearest case in the collection is François Chollet's argument Should we measure intelligence as skill or learning efficiency?. He says a high score on a task tells you very little on its own, because enough training data can buy skill without any real learning ability behind it. His alternative is to ask how much new capability a system gets out of a given amount of experience and built-in prior knowledge, on tasks that are actually new to it. On that view, a model that aces a benchmark after seeing millions of similar examples may be less intelligent than one that does a little worse after seeing a handful.

The collection shows why this matters in practice. Models trained to imitate ChatGPT fooled human evaluators by copying its confident, fluent style, yet they gained nothing in factual accuracy or in handling unfamiliar tasks Can imitating ChatGPT fool evaluators into thinking models improved?. That is Chollet's warning in miniature: performance on familiar tasks can be bought, and it is easy to mistake for capability. A different line of work rethinks what efficient intelligence looks like. Memory-Amortized Inference proposes that minds save effort by reusing past reasoning paths stored in memory instead of working everything out from scratch Can cognition work by reusing memory instead of recomputing?. If that's right, learning efficiency is partly a matter of how well past experience is organized for reuse.

The less obvious payoff is that the same distinction keeps showing up when we try to measure people working with AI. Deployment telemetry, meaning the usage data AI products collect, records what people produce with assistance. It cannot show whether they are building any lasting ability of their own; it sees "expertise in use" but not "expertise forming" Can we measure whether AI erodes independent skill?. Anthropic's engineers report big productivity gains, but they also worry that handing routine coding to Claude erodes the hands-on practice they need to catch its mistakes Does AI assistance erode the skills needed to oversee it?. Output goes up while the ability to learn and check may quietly go down, and current metrics can't tell the difference.

Our informal signals of human skill are even weaker. Fluent AI output makes users feel more competent than they are Does processing ease mislead users about their own competence?. Across three studies, people's self-ratings of their AI competence were almost unrelated to their measured performance Can self-ratings replace objective performance scores for AI competence?. Meanwhile, recruiters reward candidates who claim AI skills without much checking whether the claims are true Do AI skills help candidates get more job interviews?. Whether the subject is a model or a person, we keep measuring visible skill at a single moment, which is the thing Chollet says misleads.

So yes, intelligence can be defined as skill-acquisition efficiency, and the corpus makes a strong case that it should be. What it doesn't have is a ready-made way to measure that in everyday settings. Doing so would mean tracking learning over time with controlled exposure to new tasks, which neither benchmarks nor product telemetry currently do.