Can one score really tell you how capable an AI is, or does it hide what it's good and bad at?
Can a single competence score capture multiple separable dimensions of capability?
This explores whether one number can stand in for competence when competence is really several different things at once, and what the corpus says happens when we collapse it (or refuse to).
This explores whether one number can stand in for competence when competence is really several different things at once. The corpus says mostly no: a single score hides differences that matter, and the more useful move is to keep the dimensions apart.
The clearest case is AI agents. Capability breaks into at least five separable axes: task success, privacy compliance, long-horizon retention, mode-shift behavior, and ecosystem readiness. Models ranked first on one axis often rank lower on others, so a single-axis benchmark can be agent-capability-is-a-vector-across-separable-axes-single-axis-benchmarks-system|systematically misleading about deployment readiness. Social intelligence follows the same pattern. The SOTOPIA framework needs seven dimensions running at once, including goal completion, believability, keeping secrets, and financial outcomes. It also shows that something like efficiency can be measured on its own: humans average 16.8 words per turn against GPT-4's 45.5 (Can social intelligence be measured across seven dimensions?). Goal completion alone would miss that gap.
The same result appears where you might expect a single verdict to work. Persona fidelity is a judgment call, but breaking it into three functional dimensions and assembling the evidence consistently beats asking an LLM judge for one overall grade, especially on harder tasks (Can breaking persona fidelity into parts improve how we judge it?). Prompt quality also turns out to be a structured space with six dimensions grounded in communication theory. Improving one dimension cascades into others, so the dimensions are separable to measure but not independent of each other (Can we measure prompt quality independent of model outputs?). That is a caveat to "separable": you can measure the axes apart, but a change in one can move the others.
A single score can also mislead in less obvious ways. Imitation-trained models fool human evaluators by copying ChatGPT's confident, fluent style, while factuality and generalization to new tasks stay unchanged. The overall impression of competence improved and the underlying capability didn't (Can imitating ChatGPT fool evaluators into thinking models improved?). Emergent abilities show a related problem in the metric itself. Sharp jumps in capability mostly vanish when you swap a discontinuous metric for a continuous one, so what looks like a real change is partly an artifact of how the score was built (Are LLM emergent abilities real or measurement artifacts?). Self-reports don't rescue you either. Across three studies, self-rated AI competence correlated only .055 with objective performance, so you can't swap in a cheaper proxy (Can self-ratings replace objective performance scores for AI competence?).
The trade-off shows up in training too. Structured tasks lower output entropy while creative ones raise it, so optimizing them together can hurt open-ended ability. Ordering the training by which tasks help or hurt the others beat joint training by 6.2% (Does training order reshape how models handle different task types?). Rubrics work the same way. Using them as accept/reject gates avoids the reward hacking that happens when you fold them into a single dense reward (Can rubrics and dense rewards work together without hacking?). In both cases, collapsing distinct signals into one number is where things go wrong. Once capability is a vector, you can route on it: versioned capability vectors let agents be matched to tasks without manual wiring (Can semantic capability vectors replace manual agent routing?).
So a single score can summarize competence only if you accept that it discards the differences that decide whether a model is ready for a particular job. The corpus doesn't offer a rule for when one number is good enough. It does show repeatedly that the dimensions come apart in practice.
Sources 10 notes
Capability decomposes into task success, privacy compliance, long-horizon retention, mode-shift behavior, and ecosystem readiness. Models ranked highest on one axis often rank lower on others, making single-score evaluations systematically misleading for real deployment.
SOTOPIA framework operationalizes social intelligence across Goal, Believability, Knowledge, Secret, Relationship, Social Rules, and Financial dimensions. Humans produce 16.8 words per turn versus GPT-4's 45.5, revealing efficiency as a measurable capability in social interaction.
PRISM decomposes persona fidelity into three functional dimensions and consistently beats direct LLM-as-a-judge baselines across three benchmarks, especially on harder tasks. This suggests fidelity is inherently multidimensional and better captured through decomposed evidence than single verdicts.
Research identifies six evaluable dimensions—Communication, Cognition, Instruction, Logic, Hallucination, and Responsibility—with 20 sub-criteria based on Grice, cognitive load theory, and instructional design. Improvements in one dimension cascade to others, revealing prompt quality as a structured space rather than a flat checklist.
Imitation models fool human evaluators by mimicking ChatGPT's confident, fluent style while failing to improve factuality or generalization on novel tasks. The ceiling is set by base model capability, not fine-tuning method—better fundamentals, not shortcuts, drive real improvement.
Show all 10 sources
Sharp, unpredictable capability transitions vanish when using continuous metrics instead of discontinuous ones. The same model outputs show smooth predictable improvement with scale, suggesting emergence is a measurement choice rather than a real behavioral change.
A pooled analysis of three studies found a correlation of only .055 between self-reported and objective measures of AI competence, with confidence intervals including zero. This provides no basis for substituting self-assessment for demonstrated performance.
Omni-Thinker shows structured domains decrease output entropy while creative domains increase it. BWT-guided scheduling—training structured tasks first—yields 6.2% gains over joint training by preventing entropy collapse from damaging open-ended capabilities.
DRO shows that using rubrics to accept or reject rollout groups—rather than converting rubric scores into dense rewards—prevents reward hacking. This separation preserves the categorical strength of rubrics while letting token-level rewards optimize within valid answers.
Versioned Capability Vectors embedded in HNSW indices couple semantic matching with policy and budget constraints, making capability discovery a first-class operation that scales sub-linearly as agent heterogeneity increases.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Reinforcement Learning with Rubric Anchors
- xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems
- LLMs Corrupt Your Documents When You Delegate
- Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference
- The False Promise of Imitating Proprietary LLMs
- Are Emergent Abilities of Large Language Models a Mirage?
- Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Tasks
- Are Emergent Abilities in Large Language Models just In-Context Learning?