AI agents are ranked by test scores, but the things users care about most may not show up on the scoreboard.

This explores the gap between how agents get scored and what people actually care about when they use them. The corpus points to two mismatches. The scoreboard is too coarse, and it looks at the moment an agent delivers, while users' values break down at other moments.

Start with the scoreboard. One note argues that agent capability is a vector across at least five separable axes: task success, privacy compliance, long-horizon retention, mode-shift behavior, and ecosystem readiness. Models that rank highest on one axis often rank lower on others, so a single score can mislead about whether an agent is ready to deploy Does a single benchmark score actually predict agent readiness?. Even task success alone hides a lot. Two agents with identical success rates can differ enormously in efficiency and reliability, and those differences depend on memory, context, and verification behavior that a pass/fail number never records How should we measure agent system performance beyond task success?. The benchmarks also test the wrong kind of task. An analysis of 960 real occupational workflows found agents clear abstract contests but fail long-horizon professional work. The authors blame benchmark design rather than model capability: the field optimized what it measured, and it measured contests, not work Why do agent benchmarks not predict real economic value?.

Then look at what users say they value. A study of 73,093 Reddit posts about the OpenClaw agent found that users' values were met in five of six groups when they described what the agent delivered. During supervision, values went unmet in every group. Supervision here means configuring the agent, watching it run on its own, and reviewing the result Where do user values break down in agent supervision?. A capability axis measures the output. Users are also living through the loop around it. A second failure sits earlier in that loop. When users reveal their goals gradually over several turns, even the best models fully align with all of a user's intents only 20% of the time. They also uncover fewer than 30% of user preferences by asking questions, and the note names passivity and premature assumptions as the systematic causes Why do AI agents miss most of what users actually want?. Benchmarks usually hand the agent a finished task description, so this kind of failure never shows up in the score.

Users also judge agents on things that aren't task performance at all. The Partner Modelling Questionnaire found that people size up a dialogue agent along three factors: perceived competence (49% of the variance), human-likeness (32%), and communicative flexibility (19%) How do users mentally model dialogue agent partners?. Two of those three are about how the agent interacts, not whether it solved the task. Much of what users experience as reliability comes from the system around the model. Reliable agents offload memory, reusable skills, and interaction protocols into a harness layer instead of relying on model scale Where does agent reliability actually come from?. Scoring the bare model can therefore miss what the user is actually touching.

The widest version of the mismatch is that capability is not the thing that decides whether an agent survives in the world. A historical analysis running from GPS to modern AI finds that capable agents stall when five ecosystem conditions are missing: value generation, personalization, trustworthiness, social acceptability, and standardization Why do capable AI agents still fail in real deployments?. The corpus doesn't have a single study that maps user values onto each capability axis. Read together, though, these notes give a consistent picture. Benchmarks reward delivering the answer, while users want to be understood, to stay in control while the agent runs, to trust it, and to have it fit into their work.