How do you know if a human-plus-AI team is actually working well, beyond just checking if the final answer is correct?

This explores how to judge whether a human-AI interaction system is working, beyond checking whether the AI produced a correct output. The corpus doesn't name one best metric. What it does show is that the systems which measure well use several kinds of evidence together. One review of 291 papers on human-GenAI interaction found that 'evaluation strategy' is one of six core design choices, alongside how much control users have and how the model gives feedback How should designers structure human-GenAI interaction systems?. So how you measure success should depend on what the system is for and how people use it. It isn't a fixed scorecard you bolt on afterward.

The clearest lesson is to measure the whole interaction, not just the end result. Research on agent evaluation argues that the evidence should cover the full sequence of steps, and that scoring should look at process quality, whether the system recovers from mistakes, and how robust it is How should we evaluate agent behavior beyond final answers?. The surprising finding: two systems with identical task-success rates can differ enormously in efficiency, reliability, and readiness for real use How should we measure agent system performance beyond task success?. A single success percentage can hide the differences that matter most. Some researchers go further and argue that interactive evaluation needs its own agreed protocols and reporting standards. Without them, results from different benchmarks can't be compared Should interactive evaluation be designed as a unified paradigm?.

On the human side, the metrics that carry weight are about perception and experience. Customer satisfaction (CSAT) moves in response to small design choices: in an A/B test, a single warm opening message raised it from 72.8% to 78.4% Does conversation design ownership measurably improve AI agent performance?. Studies of how people mentally model dialogue agents found that perceived competence explains about half of their impressions, with human-likeness and communicative flexibility explaining the rest How do users mentally model dialogue agent partners?. That gives you three things to measure, not one 'satisfaction' number. User preference and cognitive load matter too. AI-generated task-specific interfaces beat plain chat in over 70% of comparisons Do generated interfaces outperform text-based chat for most tasks?, which fits the argument that chat is often the wrong format for structured, goal-directed work Is chat the wrong default interface for AI products?.

Two less obvious points are worth knowing. First, good metrics can check two audiences at once. Interface changes aimed at helping AI agents raised agent task completion, and human studies found no loss in usability for people Can interface design help agents complete computer tasks better?. Second, signals like gaze, hesitation, and typing speed can measure a user's mental state continuously without interrupting them with surveys. But the same data that lets a system time its help well also enables manipulative profiling Can AI systems read cognitive state from interaction patterns alone?. So the choice of what to measure is also an ethical choice.

The gap in the corpus: there's no head-to-head study ranking these metrics against each other. The best supported answer is to combine three kinds of evidence. Look at the whole trajectory, not just task success. Add measures of how users perceive the system (competence, satisfaction, cognitive load). And use behavioral signals carefully. Then weight them by the system's purpose, the way the six-dimension framework suggests.