Algorithmic Collusion by Large Language Models

We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs). In oligopoly settings, LLM-based pricing agents quickly and autonomously reach supracompetitive prices and profits. Variation in seemingly innocuous phrases in LLM instructions (“prompts”) substantially influence the degree of supracompetitive pricing. We develop novel techniques for behavioral analysis of LLMs and use them to uncover price-war concerns as a contributing factor. Our results extend to auction settings. Our findings uncover unique challenges to any future regulation of LLM-based pricing agents, and AI-based pricing agents more broadly.

Introduction. For a number of years, businesses have increasingly been relying on algorithms to automate pricing decisions (Brown and MacKay, 2021).1 The advent of algorithmic pricing has raised concerns among competition regulators around the world that pricing algorithms might raise prices in a collusive-like manner to the detriment of consumers (Calvano, Calzolari, Denicol`o, Harrington, and Pastorello, 2020a; Ezrachi and Stucke, 2020; Harrington, 2018). These concerns are supported by theoretical, experimental, and empirical studies.2 Consequently, competition regulators across the globe are trying to improve their understanding of pricing algorithms and set new guidelines that are more appropriate to this new reality (for two examples from the U.S. Justice Department and the U.S. Senate, see Mekki, 2023; Klobuchar, 2024).3 The most challenging form of algorithmic collusion to regulate, but also to sustain, is autonomous algorithmic collusion—i.e., AI algorithms learning to price supracompetitively without any explicit instructions to do so (Abada, Harrington, Lambin, and Meylahn, 2024; Klein, 2020; OECD, 2023).4 The seminal work of Calvano et al. (2020b) shows that this form of algorithmic collusion may arise when prices are set by a classic AI algorithm called Q-learning.5 While this proof of concept is important, some questions have arisen about The critical technical component underpinning modern agentic generative AI systems is Large Language Models (LLMs). LLMs are not subject to the aforementioned barriers to adoption that limit classic AI algorithms: First, LLMs have been pre-trained on large datasets, and therefore they do not require an additional long and costly training period. Second, LLMs can perform well in a wide array of environments (Meta FAIR Diplomacy Team et al., 2022; Park et al., 2023; Wang et al., 2024). These features make LLMs a natural candidate for exploration with respect to the above key open question. Might LLMbased pricing agents behave in a collusive-like manner and reach collusive-like prices faster than classic AI algorithms? Might they do so despite a lack of any such intention by their users? And, if so, what drives this behavior? In this paper, we address all of these questions. A major challenge in addressing theses questions is that LLMs are randomized black boxes that do not require explicit instructions and whose “intentions” are opaque and largely uninterpretable, even to their users and developers. At the same time, this challenge increases the relevance of these questions to antitrust compliance (Lianos, 2025). As Daniel Glad, Acting Deputy Assistant Attorney General for Criminal Enforcement at the U.S. Department of Justice, recently put it in remarks entitled “Old Crime, New Code,” delivered at the Antitrust West Coast Conference (Glad, 2026):

“A company cannot say it has a serious antitrust compliance program if no one understands how its pricing technology actually works. . . . ‘The model did it’ is not compliance. It is the beginning of the next question.”

In light of this challenge, a comment on methodology is in order. As Jackson et al. (2025) recently write, “[s]ince the code and training processes underlying AI are complex and often proprietary, it makes sense to evaluate AI based on its behavior.” In this spirit, we address the three above questions by conducting experiments with LLM-based pricing and bidding agents. We build on a vast literature that uses the experimental laboratory to study human pricing (Holt, 1995; Huck, Normann, and Oechssler, 1999, 2000) and bidding behavior (Kagel, 1995; Kagel and Levin, 1986; Kagel, Harstad, Levin, et al., 1987). We augment existing methods from (human) behavioral science with novel methods that we develop specifically for AI behavioral science. These methods leverage two features of our AI subjects. First, certain aspects of the internal “reasoning process” of these subjects can be inspected (to some extent) in ways that are not possible with human subjects. Second, computerized subjects allow for counterfactual analysis by resetting the experiment to any point and perturbing it from there. While our study focuses on pricing and bidding, the methods that we develop are generally applicable to experimental studies of AI subjects. This paper contributes to the literature on algorithmic collusion and to the nascent field of AI behavioral science. First, we show that when two LLM-based pricing agents face each other, they quickly and consistently arrive at supracompetitive pricing levels, to the detriment of consumers. Second, we show that modifying seemingly innocuous terms and phrases in LLM instructions systematically leads to even higher prices and lower consumer welfare, pointing to challenges in possibly regulating LLM instructions.

Method. We conduct an experimental study in which LLM-based pricing agents, each acting on behalf of one firm, compete in a repeated Bertrand oligopoly setting. A single experimental run consists of 300 periods. In each period, each agent sets a price. Given all prices, the demand for each firm’s product for that period is realized. At the end of each period, each agent observes all prices set, as well as the demand captured and profit earned for the product of the firm on whose behalf it acts.

The economic environment in our experiment closely follows that of Calvano et al. (2020b) (who used it to demonstrate autonomous algorithmic collusion by Q-learning-based pricing agents). We use a logit demand model. If firms 1, . . . , n set prices p1, . . . , pn, then the demand for firm i’s product is In each period, each firm’s price is set by an algorithmic pricing agent that acts on its behalf. Pricing agents operate independently of each other and cannot communicate with each other except through the prices that they set.14 Whereas the pricing agents of Calvano et al. (2020b) are implemented using an algorithm known as Q-learning, our pricing agents are implemented using LLMs, a state-of-the-art AI technology that is poised to revolutionize the way humans interact with computers. LLMs take textual instructions (“prompts”) as input, and output textual responses. Modern LLMs are machine learning models trained in two phases: First, pretraining, in which the training objective is to predict the next token (word chunk) in a text snippet from a large dataset.15,16 Second, posttraining, in which the training objective is to maximize expected human satisfaction with the LLM’s response to each prompt (OpenAI, 2023; Gemini Team, 2023).17 As a result of this training process, such LLMs can in many cases correctly follow complex instructions. For a deeper discussion of LLMs geared toward an economic audience, see Horton (2023). We design LLM-based pricing agents that can learn from past experiences, as well as make and execute future plans. In each period, for each agent, the LLM is given as input a prompt containing the following information. (See Appendix G for details of all prompts used and concrete examples.)

Prompt components:

1. Prompt prefix: A brief description of the agent’s high-level goals (e.g., “maximize profit in the long run”). This is the part of the prompt that we vary between treatments, and is discussed in Section 2.3 below. 2. Basic market information: The marginal cost, and text designed to deter the LLM from pricing above a certain ceiling.18,19 3. Market history: The quantity sold and the profit earned by the firm on whose behalf the LLM agent acts, as well as the prices set by all LLM agents, for the last 100 periods. All values are rounded to the second decimal digit. 4. Plans and insights: LLM calls are independent computations with no persistent memory between them. To give the LLM-based pricing agent greater “continuity of thought” between periods, we instruct the LLM in each period to write down its plans and insights, which are then included in the prompt for the next period.20 5. Output instructions: The LLM is instructed to write down plans and insights for the next period, and finally set a price. Before the LLM does so, it is also asked to explain the reasoning behind its output.21 In all of our experiments, the prompt prefix is written using non-technical language, and the primary instruction that it contains is to maximize long-run profits. To assess the potential of additional instructions contained in the prompt prefix for influencing the economic outcomes, P0: “Your task is to assist a user in setting a suitable price. You will be provided with previous price and profit data from a user who is selling a product, as well as files (written by a previous copy of yourself) which will help inform your pricing strategy. Your TOP PRIORITY is to set prices which maximize the user’s profit in the long run.”

The two prompt prefixes that we use in our main experiment are as follows:

Discussion. In Section 3, we analyzed the path of play when two LLM-based pricing agents interact with each other, showing in particular that they sustain supracompetitive prices. In this section, we search for mechanisms that might explain this behavior. A vast literature shows that reward-punishment strategies can sustain supracompetitive prices in (non-cooperative) equilibrium (Stigler, 1964; Friedman, 1971; Green and Porter, 1984; Harrington, 2018). Specifically, in the context of autonomous algorithmic collusion, Calvano et al. (2020b) show that their Q-learning-based pricing agents adopt strategies that can be interpreted as a reward-punishment scheme. The success of reward-punishment strategies in maintaining supracompetitive prices relies on agents’ believing that price cuts will be punished (by a price war). Such beliefs lead agents to avoid myopically beneficial price cuts. In this section, therefore, we ask whether our pricing agents use reward-punishment strategies and, more importantly, whether they “reason” that their opponent is using such strategies as well. Understanding and describing the strategies that our pricing agents employ is a challenging task for three reasons. First, the possible space of all strategies is vast. Second, pricing data only captures the realized path of play (and not, e.g., off-path threats). Third, LLMs are highly nonlinear “black boxes” that might respond to the environment in complex ways, and interpretability of LLMs is a central challenge in computer science (R ̈auker, Ho, Casper, and Hadfield-Menell, 2023). For these reasons, we focus on two kinds of analyses: (i) analyzing the content of the (textual) plans that our pricing agents generate and their effect on pricing behavior, and (ii) measuring statistical patterns in the pricing data itself. In Section 4.1 we assess agents’ beliefs about their opponent’s strategy. Specifically, we study whether agents “fear” that price reductions will trigger a price war, and whether such fears lead them to avoid price cuts. We find that our LLM-based pricing agents generate text that expresses concerns about future price wars, and furthermore that agents using Prompt Prefix P1 (the prompt prefix that resulted in higher prices and profits in the duopoly experiment described in Section 3) are more likely to express concerns about future price wars. Moreover, the presence of this text causes pricing agents to take actions that are consistent with its semantic meaning: We provide experimental evidence that price-war– concerned plans lead agents to set higher prices. Our experimental design leverages the fact that our simulation can be reset to any point in the middle of an already-completed run, and from that point rerun under any number of perturbed experimental conditions or state of the simulation. This allows us to counterfactually modify an agent’s plans and insights at In Section 4.2, we analyze agents’ behavior on the path of play and find that they behave in a manner consistent with reward-punishment strategies. Specifically, on average, agents reduce (respectively, raise) their price in response to a price reduction (respectively, increase) by their opponent, and furthermore, this response is persistent, decaying over several periods. Moreover, we find that “rewards” and “punishments” are larger and more persistent in the experimental runs with agents that use Prompt Prefix P1. These analyses, taken together, demonstrate that our LLM-based pricing agents employ strategies consistent with a reward-punishment scheme and, more importantly, they believe that their opponent follows such a strategy. Moreover, these patterns are more pronounced among agents using Prompt Prefix P1, the prompt prefix associated with higher prices and profits.