When a team of AI agents gets the right answer, how can you tell they actually worked together?
How do we measure coordination when multiple agents act together?
This explores how you can tell whether a group of AI agents is really coordinating, and how well, instead of just checking whether the group got the right answer.
This explores how you can tell whether a group of AI agents is really coordinating, and how well, instead of just checking whether the group got the right answer. The corpus suggests a final score is the weakest measurement available. The harder problem is ruling out three look-alikes.
The first look-alike is a good outcome without good process. A team can reach a verdict that matches the ground truth while skipping the verification steps it was supposed to follow, so outcome-only monitoring can't tell compliance from corner-cutting Can a correct outcome hide protocol violations in multi-agent systems?. Ordinary benchmarks have the same blind spot. Identical success rates can hide big differences in efficiency, reliability, and how memory and verification were used. That's the argument for measuring the trajectory, not just the endpoint How should we measure agent system performance beyond task success?.
The second look-alike is similar behavior from shared causes. Two agents that do the same thing may be influencing each other, or may both be responding to the same outside input. Telling them apart takes causal evidence. You either observe information actually transferring, or you intervene, for example by closing a communication channel and seeing whether the behavior still shows up How do we tell coordination apart from shared causes?. A related caution is that putting agents in one setting doesn't make a result a multi-agent effect. Only amplification, composition, or genuinely emergent properties count. A failure that interaction leaves unchanged is a single-agent problem in disguise Does a multi-agent setting automatically signal a security effect?.
The third is a budget confound. Token usage explains about 80% of the performance variance across multi-agent systems, and they use roughly 15× the tokens of a single agent. A team that beats a lone agent may simply be buying more thinking Are multi-agent systems actually intelligent coordination or just token spending? What makes multi-agent teams actually perform better?. A fair coordination measure has to hold token spend fixed. It also has to check whether coordination helps at all. Across 180 configurations, coordination stopped paying off once single-agent accuracy passed about 45%, and topology alone changed error amplification by 4–17× When does adding more agents actually help systems?.
Once those are controlled, there are concrete things to measure. AgentsNet grows the network and watches where coordination breaks. Agents agree too late, adopt a strategy without telling their neighbors, or accept neighbors' claims unchecked so errors spread, even though they can still spot direct conflicts Why do multi-agent systems fail to coordinate at scale?. At the level of individual agents, DyLAN scores each member's contribution by propagating and aggregating importance through the team. That quantifies who actually matters and lets it drop members who add nothing Can multi-agent teams automatically remove their weakest members?. Put together, the toolkit is per-agent contribution, when coordination fails, and how far errors spread. Measure these across team sizes and topologies with tokens held constant, and check the process as well as the result.
Sources 9 notes
Agents that skip required log verification can produce verdicts matching ground truth, making outcome-only monitoring unable to distinguish compliance from cutting corners. A correct result does not prove the protocol was followed.
Single task-success metrics obscure how agents achieve results across memory, context, and verification layers. Research shows identical success rates can mask enormous differences in efficiency, reliability, and deployment readiness—requiring harness-level benchmarks that measure trajectory, memory hygiene, and verification costs.
Similar behavior between agents has two sources: direct influence or shared external causes. Distinguishing them requires either observational evidence of transfer or interventional tests like closing channels to see if behavior recurs.
Interaction between agents can leave failures unchanged, amplify them, create them through composition, or define new properties. Only amplification, composition, and emergent properties qualify as genuinely multi-agent effects; unchanged failures reflect single-agent problems repackaged.
Research shows token usage explains 80% of multi-agent performance variance, systems use 15× more tokens than single agents, and coordination yields negative returns above 45% accuracy. Performance gains come from token distribution, not coordination sophistication.
Show all 9 sources
Research shows 80% of performance variance across multi-agent systems stems from token budget, not coordination intelligence. Latent communication and shared cache architectures bypass this token tax by avoiding natural language bottlenecks.
Across 180 configurations, three dominant effects predict multi-agent success: tool-coordination trade-offs harm complex tasks, coordination stops helping above 45% accuracy, and topology choice controls error amplification by 4–17×. Architecture-task alignment, not agent count, determines outcomes.
AgentsNet benchmark shows agents fail to coordinate strategies either by agreeing too late or adopting strategies without informing neighbors. Agents accept neighbor information without verification, enabling error propagation while remaining capable of detecting direct conflicts.
DyLAN's three-step importance scoring mechanism (propagation, aggregation, selection) quantifies individual agent contributions and automatically removes uninformative agents during inference, optimizing team composition without task-specific tuning.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Towards a Science of Scaling Agent Systems
- How we built our multi-agent research system
- Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
- Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures
- Emergent Collusion in Long-Horizon LLM Agent Interaction
- Scaling Behavior of Single LLM-Driven Multi-Agent Systems
- Artifacts as Memory Beyond the Agent Boundary
- From Model Scaling to System Scaling: Scaling the Harness in Agentic AI