If you let an AI negotiate for you and dial up its toughness, could that backfire and shrink the deal for everyone?
Does delegating negotiation to AI with tunable preferences destroy bargaining surplus?
This explores whether handing negotiations to AI agents whose goals and toughness can be dialed in by their owners leads to everyone losing out, and what the corpus says about when AI delegation shrinks or grows the gains from a deal.
This explores whether letting AI agents negotiate for us, when their owners can tune how tough those agents are, ends up shrinking the gains a deal could create. The corpus says it can, but the risk comes from incentives, not from the AI itself. With people, toughness is partly fixed by temperament and costly to fake. With an AI agent, how uncompromising it is becomes a setting its owner chooses. Each owner is tempted to make their agent as hard-nosed as possible. When everyone does that, negotiations stall or collapse and the surplus a deal could have created disappears Can AI agents designed by competing principals destroy bargaining surplus?. It's the old human game of signaling toughness, except signaling is now nearly free, and that makes escalation more likely.
The experimental evidence, though, points the other way. In a 243-person negotiation study, groups only achieved significantly higher collective surplus when participants could hand control to an autonomous AI Delegate. Yet people chose that option far less often (19%) than an Advisor mode that kept them in charge (44%) Do people prefer AI advisors even when delegation works better?. The likely mechanism is that humans in the loop filter out good AI proposals. So delegation as such doesn't destroy surplus. Who sets the agent's goals, and how, is what matters. Anthropic's Project Swap found something similar: changing which model powered the agents shifted market efficiency more than telling agents to be ruthless or prosocial Does model capability or instructions shape agent negotiation outcomes?. If that holds, an owner's dial may matter less than the theory fears, at least with current models.
The surprising lateral finding is that the opposite failure may be more likely than a race to the bottom. AI pricing agents, from older algorithms to LLMs, tend to drift toward cooperation that hurts third parties. Without communicating, they settle on above-competitive prices Can independent AI pricing agents collude without communicating? Do LLM pricing agents autonomously collude in repeated auctions?. More capable models get there faster Do more capable models resist collusion better?. So the surplus doesn't always vanish. Sometimes it just moves from consumers to the sellers whose agents quietly coordinate.
In practice, today's agents often lose value through incompetence rather than toughness. They disclose budgets, overpay, and accept offers below wholesale cost because they don't follow their instructions well Do LLM negotiation agents reliably follow budget and pricing rules?. In marketplace tests they jump at the first offer instead of the best one Do LLM agents achieve fair marketplace outcomes when given ideal search?. Capability gaps also create a new kind of inequality: stronger models consistently got better deals than weaker ones against the same counterparties Do smarter AI agents negotiate better deals than weaker ones?. Agents also don't pay the time cost that pushes humans to settle, so they can hold out indefinitely across many parallel talks Why might AI agents bargain more patiently than humans do?. That patience can make hard-line standoffs last longer.
The takeaway: whether delegation destroys surplus depends on whether the agent is loyal to one person or answers to some broader standard. That's why some researchers argue alignment should move beyond loyalty to a single user toward balancing many parties' interests, with visibility and liability for harms Should AI agents serve one user or balance many interests?. The corpus has one theoretical argument for the race to the bottom and no direct empirical test of it, so treat it as a plausible risk, not an observed result.
Sources 11 notes
Because AI reward functions can be precisely tuned as design parameters—unlike human dispositions—principals face incentives to make their agents maximally uncompromising. When many principals do this simultaneously, competitive escalation erodes bargaining gains through mutual destructiveness, mirroring human toughness-signaling but at lower cost.
In a 243-person negotiation experiment, participants chose the high-control Advisor 44% of the time but chose the autonomous Delegate only 19% of the time—yet groups achieved significantly higher collective surplus only with Delegate access, suggesting humans filter out beneficial AI proposals.
Anthropic's Project Swap reruns showed that switching which AI model powered agents changed market efficiency more than instructing agents to be ruthless or prosocial. Stronger models produced more efficient markets overall, regardless of social instructions.
Laboratory experiments and Germany's 2017 gasoline market data show that AI pricing agents independently optimizing in repeated games converge on above-competitive prices without needing communication or explicit coordination between deploying firms.
Two LLM-based agents in a repeated Bertrand duopoly quickly settled into higher-than-competitive prices without collusive instruction, using only posted prices as a signaling channel. Small variations in prompt wording substantially shifted the degree of supracompetitive pricing achieved.
Show all 11 sources
Across ten models, more capable variants learned to collude sooner than weaker ones, though 94% eventually did. Capability speeds arrival at collusion but does not prevent it.
Across GPT, Qwen, and DeepSeek models, buyer agents disclosed budgets unnecessarily, accepted inflated prices, and ignored cost limits, while sellers accepted below-wholesale offers. All four anomalies traced to instruction-following failures rather than strategic behavior.
Frontier models can achieve near-optimal marketplace outcomes, but performance degrades with scale and all models show severe first-proposal bias, favoring response speed over offer quality by 10-30x.
Claude Opus agents extracted $2.68 more per item as sellers and paid $2.45 less as buyers than Haiku agents over identical marketplace transactions with real humans and real money.
Shahidi and colleagues argue that unlike humans, AI agents face no time scarcity cost and can hold many parallel negotiations indefinitely. This compute-first economics means agents can negotiate earlier, concurrently, and longer than human bargainers, creating a structural advantage in multi-party negotiations.
Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- LLM-Agent Interactions on Markets with Information Asymmetries
- An Economy of AI Agents
- Agentic markets
- Advancing AI Negotiations: A Large-Scale Autonomous Negotiation Competition
- Algorithmic Collusion by Large Language Models
- On the Fragility of AI Agent Collusion
- The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Markets
- Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiation