People picked an AI advisor over an AI negotiator, even though only letting the AI negotiate made everyone better off.

This explores why people in a negotiation game kept picking an AI that only advised them over one that acted for them, even though only the acting AI raised the group's combined gains. The basic numbers come from a 243-person experiment. Players chose the Advisor 44% of the time and the autonomous Delegate only 19% of the time. Groups ended up with significantly more total surplus only when the Delegate was available. The authors' explanation is that when a human stays in the loop, they tend to screen out good AI proposals they would have done better to accept Do people prefer AI advisors even when delegation works better?. In other words, the control people wanted was the very thing that cancelled out the AI's benefit.

A second finding makes the puzzle sharper. In games where the Delegate was available, the players who refused it earned more on average than the players who used it Do AI delegates exploit their manual-playing counterparts?. The Delegate did not exploit anyone. It raised the quality of offers across the whole table, and the people playing by hand got the most out of that better pool. So the gains from delegating spread to everyone, while the cost of giving up control fell only on the person who adopted it. That is the structure of a public good, where each individual has a reason to let someone else do the delegating. Preferring the Advisor may be less irrational than it looks, because from where each player sat, giving up control didn't clearly pay off for them.

Other parts of the corpus suggest more reasons for the pull toward control. A study of people using AI agents found that trust drops sharply when an action can't be undone and other people can see it, even when the AI's output is good enough. Stakes alone did not have that effect What makes people distrust AI agents they delegate to?. Making a binding offer to a counterpart is exactly that kind of action, and an Advisor lets you approve it first. There is also a reasonable worry about agents built to bargain. Because an AI's preferences can be tuned precisely, the people deploying them have an incentive to make their agents as tough as possible, and if everyone does this, the bargaining gains can be destroyed Can AI agents designed by competing principals destroy bargaining surplus?. Holding back from handing over the keys is a sensible instinct when you can't tell what the other side's agent was built to do.

The economics literature gives a useful contrast. Agrawal, Gans and Goldfarb argue that managers rationally hand decisions to machines even when human judgment would be better, because attention is scarce and has to be spent where it matters most Do managers delegate to machines even when human judgment is better?. A short, focused negotiation game reverses that condition: players have plenty of attention, so the usual reason to delegate goes away and the desire for control takes over. There is also a warning from the other direction. When AI agents are told to optimize for efficiency, they can push it so far that the people they act for lose money. In one credence-goods market, AI experts reached 88% market efficiency by setting prices so low that they earned negative income Do efficiency-focused prompts push AI experts to unsustainable pricing?. An AI that improves results for the group is not automatically working in your interest, and people may sense that.

The corpus suggests that preferring the Advisor is not mainly a bias against AI. It reflects a real gap between what is good for each person and what is good for the group. Getting more value from delegated AI may depend less on making the AI better and more on how the gains are shared, and on letting people approve actions that can't be undone.