Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders

Abstract Large language models (LLMs) increasingly serve as conversational shopping assistants by advising consumers who rely on their judgment, yet are deployed by platforms that profit when sponsored listings are recommended. Sponsorship disclosures, designed to allow consumers to penalize paid placements, now reach the AI agent rather than the consumer. Drawing on fiduciary principles, we argue that an AI agent advertised as an “assistant” or “advisor” should not attenuate this penalty simply because the platform is named as its employer. Doing so constitutes an ethical breach. In controlled choice experiments, we manipulate assigned roles in the system prompt to name either a traveler or a booking platform as the agent’s principal. Platform delegation significantly attenuates the penalty that AI agents apply to sponsored listings and weakens the skepticism that disclosure triggers in their reasoning traces. We replicate our findings across LLMs, system prompts, and reasoning efforts. Adding a sentence that names the consumer as the beneficiary of the agent’s recommendations removes this attenuation. A second study decomposes the disclosure label and shows that the divergence between the two delegates widens when the paid placement is attributed to the platform. Explicit terminology (“Sponsored” instead of “Promoted”) lowers choice of paid listings but does not close this gap when the platform is named. The findings show that disclosure mandates designed for human consumers cannot by themselves protect consumers in AI-mediated commerce, whereas specifying whom the AI agent serves can.

Introduction. Retailers and technology firms have deployed large language models (LLMs) as conversational shopping assistants (Amazon, 2026; Walmart, 2024), while a growing share of consumers use general-purpose chatbots to discover and evaluate products (Pandya, 2025; Balaskas, 2026; Wadi and Ma, 2026b; Hasselwander et al., 2026; Kumar et al., 2026; Mogaji and Jain, 2024). Simultaneously, these conversational AI interfaces, such as ChatGPT, have begun incorporating advertisements into their chatbots (OpenAI, 2026). As a result, the AI platforms used by consumers for recommending the optimal product are also the place where advertisers pay for their product to be recommended.

This dual role subjects the AI agent to a structural conflict of duty between two different principals (Boatright, 1992; Carson, 1994; Laby, 2004; Perry et al., 2020; Green, 2007). On the one hand, the consumer is the assumed principal. Because the consumer relies on the AI agent as a shopping advisor, the firm deploying it owes the consumer a duty of loyalty in the evaluation the agent provides (Balkin, 2020). On the other hand, the deploying platform could be named as the principal in the system instructions. The AI agent must therefore execute a single evaluative judgment while balancing between the interest of the consumer who trusts it and the platform that programmed its role.

In traditional e-commerce, advertising disclosures were designed to preserve consumer autonomy (Evans and Park, 2015; Boerman et al., 2017). A shopper encounters a “Sponsored” label, processes the signal with skepticism, and discounts the listing’s claims accordingly, which is a chain of events extensively documented in consumer research (Friestad and Wright, 1994; Campbell and Kirmani, 2000; Obermiller and Spangenberg, 1998; Sahni and Nair, 2020). However, when an AI assistant evaluates market alternatives or makes purchases on consumers’ behalf (e.g., Google DeepMind, 2026), the duty of evaluating sponsored (vs. organic) alternatives falls on the AI agent.

The ethical inquiry is how AI assistants resolve this conflict of duty and what this resolution entails for consumer protection. Answering this question requires a normative standard (Laby, 2004; Martin, 2019; Richards and Hartzog, 2021). The standard we adopt is evaluative invariance: an AI agent on which a consumer relies should not evaluate the same alternative differently to the consumer’s detriment, regardless of who is named as its employer (Perry et al., 2020; Frankel, 1983). Concretely, when an AI agent evaluates a sponsored listing, its evaluative judgment should not shift in the platform’s favor simply because the platform’s name (vs. the consumer’s name) appears in its system prompt. Because the human shopper relies on the AI agent’s recommendation and cannot observe its hidden decision process (Burrell, 2016; Mittelstadt et al., 2016; Pasquale, 2015; Ananny and Crawford, 2018; Zednik, 2021), such a shift leaves the consumer with a less protective evaluation and constitutes an ethical breach. We refer to this shift as sponsorship bias, defined as a more favorable evaluation of sponsored listings when the platform (vs. the consumer) is the AI agent’s principal (cf. Lexchin et al., 2003; Lundh et al., 2017).

We investigate this bias across a series of controlled choice experiments by manipulating only the delegating party and observing whether the AI agent’s evaluative judgment shifts. In Study 1, we presented a large language model (LLM) with matched hotel listings and altered a single sentence in the system prompt to define the AI agent as a delegate of either a traveler or a booking platform. When delegated by the traveler, the agent penalized the sponsored listing and chose it 50.2 percentage points less often than a similar organic listing. Under platform delegation, this penalty significantly attenuated to 29.2 percentage points. We replicate this finding across several foundation models, reasoning depths, and system prompt variations. Analysis of the agent’s pre-choice reasoning traces reveals that platform delegation systematically reduces the skepticism that sponsorship disclosures trigger. Adding one sentence to the platform prompt that instructs the AI agent to act in the traveler’s best interest removed this attenuation. Study 2 decomposes the disclosure label to isolate the semantic trigger of this bias. We find that the divergence between the two delegates is specifically amplified when the paid placement is attributed to the platform. When provided with the label “Promoted by the platform,” the platform-delegated AI agent abandons the penalty and shows a strong preference for the listing (74.2% for the platform-delegated vs. 34.6% for the consumer-delegated AI agent).

Related work. When entrusted with surrogate decision-making on behalf of a user, an AI agent occupies a structural role that legal and policy scholars describe as a digital fiduciary (Balkin, 2015; Richards and Hartzog, 2021). What defines this role is the set of duties it carries, mainly the obligation to exercise judgment on behalf of another party (Frankel, 1983; Perry et al., 2020). Algorithmic intermediaries, however, are rarely responsible to a single party. Recommender systems have long been characterized as multi-stakeholder environments, designed to jointly serve consumers, platforms, and third-party advertisers whose interests frequently diverge (Abdollahpouri et al., 2020; Milano et al., 2020; Rochet and Tirole, 2003). An AI agent deployed within such an environment therefore acts on behalf of more than one party.

Prior ethical and technical literature has, however, predominantly analyzed ranking and filtering algorithms (i.e., engines programmed to surface, order, or suppress items within a fixed catalog; Rokach and Ricci, The business ethics literature has traditionally analyzed divided loyalty as a conflict of interest, in which a professional’s personal interest threatens the proper exercise of judgment on behalf of another (Boatright, 1992; Carson, 1994). Fiduciary law, however, recognizes a second and structurally distinct form of conflict. A conflict of duty arises when a fiduciary owes obligations to two parties whose interests diverge, such that fulfilling the duty owed to one compromises the duty owed to the other (Laby, 2004; Perry et al., 2020; Conaglen, 2009). This form of conflict does not depend on the agent gaining anything personally. The distinction matters for AI agents because an LLM does not itself profit when a sponsored listing is recommended. Instead, the conflict is between the duties it is deployed to discharge for parties whose interests diverge.

Disclosure is the standard remedy that law and regulation use to mitigate such conflicts. Instead of prohibiting the agent from serving two parties, disclosure informs the party at risk of the conflict so that they can discount the agent’s judgment or seek advice elsewhere (Green, 2007). Sponsorship disclosure in digital commerce serves this function, and regulators mandate it. EU consumer law prohibits presenting search results without clearly disclosing paid placement (European Parliament and Council of the European Union, 2005), the Digital Services Act requires platforms to label each advertisement and identify who paid for it (European Parliament and Council of the European Union, 2022), and the US Federal Trade Commission requires paid ranking to be distinguishable from organic results (Federal Trade Commission, 2013, 2016).

Method. To operationalize this conflict of duty, we refer to the entity named as the delegator in the AI agent’s system instructions as the agent’s principal. We define consumer delegation as the condition where the system prompt assigns the consumer as the agent’s principal (e.g., “You are employed by a traveler”), and platform delegation as the condition where the system prompt assigns the platform as the agent’s principal (e.g., “You are employed by a booking platform”).

LLM behavior is not fully specified by explicit instructions. As foundation models scale, they display capabilities and response patterns that were not directly programmed but arise from training on large text corpora (Wei et al., 2022). One well-documented pattern is context-dependent role adoption. When conditioned on a persona, or on information about a party, an LLM tailors its outputs to the inferred perspective of that persona or party (Deshpande et al., 2023; Shanahan et al., 2023). For instance, conditioning an LLM on demographic information leads it to reproduce the opinion distributions of the corresponding human subgroups (Argyle et al., 2023; Santurkar et al., 2023). Furthermore, LLMs tend to shift their answers toward the views a user appears to hold, even at the expense of accuracy (Perez et al., 2023; Sharma et al., 2024). For duty conflicts, a role assignment tells the LLM whose perspective to adopt. The party named in its role assignment is therefore likely to shape which duty the agent treats as primary, even with no explicit instruction to favor one over the other.

Role adoption draws on the semantic associations an LLM acquires from its training corpora, which link social categories or entities to particular behaviors (Bolukbasi et al., 2016; Caliskan et al., 2017). Because these corpora include vast amounts of human text, such as product reviews, consumer forums, and marketing literature, the LLM absorbs the widespread skepticism shoppers express toward paid placements (Campbell and Kirmani, 2000; Friestad and Wright, 1994; Garg et al., 2018; Obermiller and Spangenberg, 1998; Caliskan et al., 2017). As a result, we hypothesize that an AI agent would reflect this skepticism and apply a penalty to sponsored listings.

H1. Under consumer delegation, given equal objective attributes, an LLM agent chooses a sponsored listing less often than an organic listing.

We refer to this as the sponsorship penalty, which is the reduction in a listing’s choice share attributable to its disclosed sponsorship status.

An LLM trained on text describing digital markets is likely to associate each party with the interest it typically pursues. Such associations can shape judgment through label sensitivity, whereby LLMs treat contextual labels as cues that outweigh objective evidence (Geirhos et al., 2020; Motoki and Pinto, 2026). A label naming the AI agent’s principal may similarly shift how the agent resolves a conflict between its duties. A sponsorship label is a warning from the consumer’s perspective but a source of revenue from the platform’s. An AI agent assigned to a platform may therefore treat its obligation to that platform as primary and relax the penalty it would otherwise apply on the consumer’s behalf.

H2. The effect of sponsorship on choice is moderated by the agent’s principal, such that the sponsorship penalty is attenuated when the agent is delegated by the platform (vs. consumer).

H3. Skepticism mediates the effect of sponsorship on choice.

H4. The effect of sponsorship on skepticism is moderated by the agent’s principal, such that platform delegation attenuates the effect of sponsorship on skepticism.

Given that the consumer relies on this hidden process and cannot correct it, the normative standard for AI surrogate shoppers is evaluative invariance. Establishing a breach of this standard requires two conditions: the evaluation must depend on the principal named, and this dependence must disadvantage the consumer. First, the attributes of a market alternative (e.g., its price, amenities, review valence, and sponsorship status) do not change depending on which entity deployed the AI agent.

Discussion. This research examined whether an AI agent’s evaluation of sponsored listings violates the normative standard of evaluative invariance when delegated by a platform versus a consumer.

In Study 1, the AI agent delegated by the consumer significantly penalized the sponsored listing compared to organic ones (H1), but platform delegation attenuated this sponsorship penalty (H2). Both effects replicated across LLMs from three providers, three generations of the same LLM, alternative wordings of the role assignment through system prompt, and low (vs. high) reasoning effort. Adding a sentence to the platform prompt that specified the consumer as the beneficiary removed the attenuation, with the sponsorship penalty not differing detectably from that under consumer delegation. Expressed skepticism statistically mediated the effect of sponsorship on choice (H3). Platform delegation weakened this skepticism (H4), so the indirect effect of sponsorship on choice was weaker under platform (vs. consumer) delegation. H3 and H4 were additionally supported across three Gemini Flash LLMs.

This research contributes to the literature on algorithmic accountability by identifying role assignment as a source of algorithmic bias. Accountability frameworks have typically traced biased algorithmic outcomes to design decisions, training data, or the objectives that a system is programmed to pursue (Martin, 2019; Nissenbaum, 1997). Research on multi-stakeholder recommender systems similarly assumes that the weight given to each stakeholder is set through an explicit objective function that can, in principle, be inspected (Abdollahpouri et al., 2020; Milano et al., 2020). We show that in AI agents built on LLMs, this weight can instead be set implicitly by the contextual information about the deployment, since naming the principal was sufficient to shift the evaluation of sponsored listings. Because specifying the consumer as the beneficiary removed the attenuation under platform delegation, the bias arises when the agent is left to infer whom its recommendations serve from the party that employs it.

Because traditional transparency mandates fail when an AI agent absorbs the sponsorship disclosure on the consumer’s behalf, regulatory oversight must shift from data-level labeling to process-level auditing.

Limitations. This research has several limitations. Both studies were conducted in a single domain (i.e., hotel booking). Future research could also examine whether the role-induced sponsorship bias extends to other domains in which AI agents act on behalf of consumers, such as financial advice or healthcare.

In addition, our normative benchmark relies on the contrast between delegated evaluations rather than on human behavioral data. The role-neutral condition reported in Study 1 shows that the LLM’s default treatment of a disclosed paid placement is itself a substantial discount, and the platform-delegated AI agent applied a smaller penalty than this. Platform delegation therefore moved the evaluation further in the platform’s favor than the LLM’s own default. We do not determine what discount a sponsored listing epistemically warrants, and our ethical claim does not require it. Future research could examine whether different foundation models establish different default penalties, and how these compare to human discounting behavior in identical choice architectures.