Does list order alone shift which hotel an LLM recommends?

In the same randomized conjoint audit of twelve LLM travel assistants described in How do LLM hotel assistants weight reputation signals differently than humans?, the position of a hotel card within the five-card candidate set was itself randomized alongside the seven reputation attributes. The audit reports that "list position—a content-free artifact—shifts recommendations causally, worth about $12 per night." By importance share, position's effect (4.1%) "exceeds those of two genuine reputation signals—chain affiliation (2.0%) and review recency (1.7%)," and "in one model the first-position advantage is an order of magnitude larger than the panel average, rivaling substantive signals outright."

The paper's second theoretical move is to treat the LLM assistant's own interface as part of the signal environment it audits: because "no hotel controls" its position in the list presented to the model, and position "carries no quality information," a measurable causal effect of position on selection means position becomes "a de facto reputation signal manufactured by the platform" rather than a neutral artifact of presentation. The design isolates this from retrieval by holding the five-hotel candidate set "fixed and exogenous" and randomizing only attributes and position within it, so the effect measured is the selection stage's response to ordering alone, not a compounding of upstream retrieval ranking and selection.

This is structurally close to Do language models favor sources regardless of item quality?, which finds a different content-free attribute — source-domain label — causally swaying LLM selection independent of the item's actual content; both studies show LLM choice responding to a surface cue nobody but the platform or retrieval system controls. It also complements Do shopping agents respond more strongly to price and rating cues?: that audit shows LLM agents over-respond, relative to humans, to manipulable content cues like price and rating, while this one shows a cue with zero content — ordering alone — still moves the same kind of model.

Because the audit fixes the candidate set and randomizes only within it, the $12-per-night estimate describes the selection stage's sensitivity to order, not any compounding effect from an upstream retrieval or ranking step that a deployed assistant would also apply before the five cards are ever assembled; the paper flags extending the audit to that retrieval stage as its own next step. The panel is twelve fixed-version models, "not a census of deployed assistants." What the finding does support is that a design choice entirely inside the assistant's interface — how retrieved candidates are ordered before the model picks among them — functions as a pricing lever on hotels worth real dollars per night, one no property can observe, contest, or manage, unlike a low organic search ranking it could at least see.