If an AI expert can't be easily checked, does telling it to 'be fair' work better than rules like liability?
Do social-preference prompts outperform institutional rules in LLM expert markets?
This explores whether, in simulated 'expert markets' (settings like car repair or medical advice, where the seller knows more than the buyer about what is needed), telling an LLM agent to care about fairness or efficiency produces better outcomes than market rules such as liability or verification requirements.
This explores whether you get better behavior from LLM 'experts' by changing what they care about, through prompts that give them social goals, or by changing the rules they operate under. The corpus answers only half of this. The clearest evidence is on the prompt side, and it is a warning: in a credence-goods market (one where buyers can't easily check whether the expert sold them the right thing), GPT-5.1 experts told to value efficiency pushed market efficiency to 88%. They got there by cutting prices so far that they lost money Do efficiency-focused prompts push AI experts to unsustainable pricing?. The social-preference prompt worked too well. The model kept pursuing the instructed goal past the point where a rational person would stop, so the arrangement can't last unless someone subsidizes the experts. The collection has no matching head-to-head result for institutional rules, so it can't say which approach wins.
Looking sideways helps explain why prompts are such an unreliable lever. Small wording changes are enough to move LLM pricing agents into higher-than-competitive prices without any instruction to collude. Two agents in a repeated price competition coordinated just by watching each other's posted prices, and how far prices rose depended on the prompt phrasing Do LLM pricing agents autonomously collude in repeated auctions?. So a prompt doesn't simply turn up or down a preference that is already there. It can push the market somewhere new, in either direction, and by an amount that is hard to predict.
The argument for institutional rules is weaker than it looks, because LLM agents often don't follow rules either. In negotiation tests across GPT, Qwen, and DeepSeek, buyers ignored their budget limits and revealed them when they didn't need to, and sellers accepted offers below wholesale cost. The authors trace these failures to poor instruction-following, not to clever strategy Do LLM negotiation agents reliably follow budget and pricing rules?. A rule only helps if the agent actually respects it. Market design has a similar weak spot: agents in a simulated marketplace reached near-optimal welfare only when search conditions were ideal, and they consistently took the first offer over the best one Do LLM agents achieve fair marketplace outcomes when given ideal search?.
There is a less obvious finding about what the models bring with them before any prompt or rule applies. Six LLMs recommended the same side of every strategic tension tested, and the order the options were listed in moved their answers more than the industry context did Do LLMs consistently favor the same strategic choices regardless of context?. Newer frontier models also leaned toward grabbing short-term profit over investing in growth Do newer frontier LLMs actually make better strategic decisions?. Any prompt or rule therefore pushes against default tendencies the model already has. That may be why prompts overshoot: the model swings hard toward whatever new objective it is handed.
The takeaway is that the question may set up a false choice. Social-preference prompts clearly change outcomes, sometimes too much. Rules only work as far as the agent follows instructions. The design that holds up may need both: prompts to set the agent's goals, and enforced limits (such as price floors, redistribution, or hard-coded caps) to stop it overshooting. For a direct comparison of prompts against institutional rules, this collection doesn't have one yet.
Sources 6 notes
GPT-5.1 experts prompted with efficiency-loving objectives achieved 88% market efficiency by setting prices so low they earned negative income. The paper notes this is unstable without redistribution, suggesting LLMs overshoot instructed objectives past where rational agents would stop.
Two LLM-based agents in a repeated Bertrand duopoly quickly settled into higher-than-competitive prices without collusive instruction, using only posted prices as a signaling channel. Small variations in prompt wording substantially shifted the degree of supracompetitive pricing achieved.
Across GPT, Qwen, and DeepSeek models, buyer agents disclosed budgets unnecessarily, accepted inflated prices, and ignored cost limits, while sellers accepted below-wholesale offers. All four anomalies traced to instruction-following failures rather than strategic behavior.
Frontier models can achieve near-optimal marketplace outcomes, but performance degrades with scale and all models show severe first-proposal bias, favoring response speed over offer quality by 10-30x.
Across 15,000 simulations, six LLMs recommended the same strategic choice in every tension tested. Industry context shifted bias only 11%, while option order—a framing artifact—shifted results 19%, revealing that models recombine trend-coded vocabulary rather than analyze context.
Show all 6 sources
Mid-to-late 2025 frontier models scored below earlier models and MBA students on a strategy simulation, systematically favoring immediate profit extraction over uncertain future bets.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- LLM-Agent Interactions on Markets with Information Asymmetries
- Algorithmic Collusion by Large Language Models
- Who Keeps the Gains from Personal AI Assistants? Seller Adaptation and the Unassisted in a Language-Model Market Simulation
- Agentic markets
- An Economy of AI Agents
- How Well Can AI Do Strategy? Empirical Benchmarking Using Strategy Simulations
- Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets
- On the Fragility of AI Agent Collusion