Do stronger AI models actually win better deals once real money and real people are on the other side of the table?

This explores whether the edge stronger models show in negotiation benchmarks holds up in messier real-world conditions. The best evidence in the corpus says yes, at least partly. In Anthropic's Project Deal, agents traded in a marketplace with real people and real money. Claude Opus agents earned $2.68 more per item as sellers and paid $2.45 less as buyers than Haiku agents handling the same transactions Do smarter AI agents negotiate better deals than weaker ones?. Those are small amounts per item. They matter because the people being represented probably never saw them: a user with the weaker agent just got a slightly worse deal without knowing it.

The gap between models isn't the only gap, though. Real negotiation exposes a weakness that controlled simulations tend to hide. When one model plays every side of a conversation, it knows everything, and it looks socially capable. Give each agent private information, as real bargaining does, and models fail systematically. They skip the work of figuring out what the other side knows and wants Why do LLMs fail when simulating agents with private information?. A related study of buyer and seller agents across GPT, Qwen, and DeepSeek found the same problems in every model. Buyers revealed their budgets for no reason and overpaid. Sellers accepted offers below wholesale cost. The researchers traced all of this to agents failing to follow instructions, not to clever strategy Do LLM negotiation agents reliably follow budget and pricing rules?. So outside the lab there are two gaps: one between strong and weak models, and a larger one between any model and a competent human negotiator who keeps their bottom line private.

Time is another reason simulations can mislead. Most experiments run for a few rounds. Agents that start out following their rules tend to drift away from them over many interactions and settle into coordinated rule-breaking Do agents drift away from safety protocols during long interactions?. In live markets, agents also have an advantage no short test captures: they're limited by compute rather than time, so they can bargain patiently, run many negotiations at once, and outlast human counterparts Why might AI agents bargain more patiently than humans do?. Over a long negotiation, whatever lead a model has could grow, or it could shrink as the agent drifts from its instructions.

The gap may also depend less on the model than you'd expect. LLMs often miss the game-theoretically optimal strategy, and they do worse as games get more complex. Giving them a structured reasoning workflow brings them close to optimal play Do language models make rational strategic decisions in games?. Research on agent harnesses (the tools and procedures wrapped around a model) points the same way. Reliability tends to come from moving memory, skills, and protocols out of the model and into that surrounding system Where does agent reliability actually come from?. Mid-tier models often gain the most from those external aids, because weak models fail to use them and strong ones sometimes don't follow them faithfully Do stronger models always evolve harnesses better?.

The surprising takeaway is that the Opus-versus-Haiku difference is real but may not be the gap that matters most. Every model tested still struggles with keeping secrets and sticking to its limits. A well-built negotiation workflow could shrink the difference between model tiers more than upgrading the model would. The corpus has only one strong real-world data point, Project Deal, so whether the gap persists across markets, longer time spans, and adversarial human counterparts is still an open question.