Whose hotel does the AI recommend? An algorithm audit of reputation signals in LLM-assisted hotel selection

Travelers increasingly ask large language model (LLM) assistants which hotel to book, making these systems gatekeepers of property visibility—yet what moves their recommendations is undocumented. We conduct a pre-specified algorithm audit using a randomized choicebased conjoint: across personas, prompt templates, and twelve open-weight and proprietary models, assistants choose among five hotels whose guest rating, review volume and recency, management response, chain affiliation, price, eco-certification, and list position are independently randomized. We estimate the average marginal component effect of each signal on the probability of recommendation. Guest rating and price dominate (a top rating raises selection by 31.6 percentage points; a high price lowers it by 30.0), reproducing human valence-and-price primacy but over-weighting eco-certification and ignoring management response. List position—a content-free artifact—shifts recommendations causally, worth about $12 per night. Stated reasons track revealed weights imperfectly. The findings ground generative engine optimization and the accountability of AI infomediaries in causal evidence.

Keywords: large language models; algorithm audit; electronic word-of-mouth; conjoint analysis; generative engine optimization; hotel selection

Introduction. The most important shop window a hotel has is no longer a page of search results—it is increasingly the context window of a conversational assistant. When a traveler types “which of these hotels should I book?” into a general-purpose large language model (LLM), the assistant does not return a ranked list to be browsed and filtered; it returns a recommendation, often a single property, accompanied by a short justification. Industry tracking indicates that roughly forty percent of United States travelers used generative-AI tools to plan a trip during 2025, an increase of about eleven percentage points year over year, and that the share of travelers who begin trip planning at a conventional search engine has fallen sharply while the share beginning ∗Equal contribution. Corresponding author: Mirzasamadcontact@gmail.com. †Fandaqah, Al Khobar, Saudi Arabia. ‡Hamdard University, Karachi, Pakistan. arXiv:2606.16344v1 [cs.AI] 15 Jun 2026 at a generative-AI platform has more than doubled (Phocuswright, 2025). Forward-looking surveys point the same direction: a majority of travelers now expect to use AI for trip planning within the year, and among prospective users, obtaining recommendations is the single most common intended use. Since major assistants began enabling hotel search and booking directly inside the chat in 2025, the LLM has moved from advising on the periphery of the funnel to occupying its decisive center—the point at which one property, and not its competitors, is surfaced to a ready-to-book traveler. This relocation of the decision turns the assistant into an algorithmic infomediary: a gatekeeper whose selection function determines which eligible properties a traveler ever sees (Metaxa et al., 2021). For an individual hotel, the stakes are concrete and asymmetric. A property that the assistant systematically declines to surface is, for that growing population of travelers, effectively invisible—and, unlike a low organic search ranking, the hotel cannot observe the slot it failed to win, cannot see its competitors’ treatment, and is given no account of why. Suppliers have nonetheless begun to act. A nascent industry practice, generative engine optimization—the deliberate shaping of content to win inclusion and prominence in generative-engine answers, advanced as the successor to search engine optimization—already treats the LLM’s selection behavior as a surface to be engineered (Aggarwal et al., 2024). Yet this practice is being sold and adopted with essentially no causal evidence base: the inputs that actually move an assistant’s recommendation are undocumented, and the field has no estimate of which reputation signals shift the machine’s choice, in which direction, or by how much.

Optimization advice is therefore running well ahead of measurement.

We argue that the question with both theoretical and managerial bite is a supply-side one: among the reputation signals a hotel can actually manage—its average guest rating, the volume and recency of its reviews, whether management is seen to respond, its brand or chain affiliation, its price, and its third-party eco-certification—which causally change the probability that an LLM recommends the property, and with what relative weight? This question inherits decades of human evidence as a benchmark. The electronic word-of-mouth (eWOM) literature has established not only that these cues move human booking behavior but, crucially, their relative ordering: review valence (rating) is consistently the dominant lever, far outweighing review volume, with management response, brand, price, and green certification playing measured secondary roles (Yang et al., 2018; Proserpio and Zervas, 2017; Assaker and O’Connor, 2023).

Whether the gatekeeper that increasingly stands between hotels and travelers reproduces that hard-won ordering, or reweights the cues in idiosyncratic and commercially consequential ways, is unknown.

We answer this question with a randomized, choice-based conjoint algorithm audit (Sandvig et al., 2014; Metaxa et al., 2021; Hainmueller et al., 2014). In each choice set, an LLM travel assistant is asked to recommend one of five synthetic hotel cards whose reputation attributes— rating, review volume, review recency, management response, chain affiliation, price, and Green Key eco-certification, together with the list position of each card—are independently randomized.

Because the attributes are randomized, the average marginal component effect (AMCE) of each signal on the probability of recommendation is causally identified, in interpretable percentagepoint units, and is comparable across models and decoding temperatures (Hainmueller et al., 2014; Swait and Louviere, 1993).

Related work. and hypotheses 2.1 AI-mediated travel discovery Travelers increasingly begin hotel discovery not at a search engine or an online travel agency but in conversation with a general-purpose LLM assistant. Industry tracking indicates that roughly forty percent of United States travelers used generative-AI tools to plan a trip during 2025—an eleven-point year-over-year increase—while the share starting trip planning at a conventional search engine fell from about half to roughly a third over the same period and the share starting at a generative-AI platform more than doubled (Phocuswright, 2025). With assistants now able to surface and, on some platforms, transact bookings directly within the chat, the LLM increasingly occupies the position once held by ranked search results: it decides which of many eligible properties a traveler actually sees recommended.

This shift turns the assistant into an algorithmic infomediary whose selection function has direct commercial consequences for hotels. Suppliers have already begun to respond. The emerging practice of generative engine optimization—optimizing content for inclusion and prominence in generative-engine answers, the successor to search engine optimization—treats the LLM’s selection behavior as an object to be influenced (Aggarwal et al., 2024). Yet the inputs that actually move an assistant’s recommendation are not publicly documented, and an individual hotel cannot observe why it was or was not surfaced. The managerially urgent question is therefore which reputation signals that a property can manage—its guest rating, the volume and freshness of its reviews, whether management visibly responds, its brand affiliation, its price, and its sustainability credentials—causally change the probability that an LLM recommends it, and with what relative weight. Existing evidence does not answer this. The eWOM literature quantifies how these cues move human booking behavior and market outcomes (Section 2.2), and a nascent strand examines LLM travel preferences (Section 2.5), but no study has audited the supply-side reputation signals an LLM rewards when acting as the gatekeeper. We close this gap with a randomized, choice-based conjoint audit that identifies the average marginal component effect (AMCE) of each signal on the LLM’s recommendation (Hainmueller et al., 2014).

Method. 3.1 Design overview and rationale We treat the language model as an algorithmic infomediary and study it with an algorithm audit: a controlled experiment in which inputs are randomized and the system’s outputs are observed to recover the input–output mapping the system enforces in practice (Sandvig et al., 2014; Metaxa et al., 2021). Audit studies descend from the correspondence tradition, in which paired applications differing only in a randomized characteristic reveal differential treatment (Bertrand and Mullainathan, 2004); algorithm audits port this logic to automated decision-makers. Because the profiles are synthetic and the manipulated attributes are assigned at random, the design identifies the causal effect of each attribute on the system’s output—which observational analyses of scraped recommendations cannot, as there attributes are confounded with unobserved demand, quality, and listing strategy. An LLM-assisted booking pipeline can be decomposed into three stages: retrieval of candidate properties from an index or tool call, assembly of a candidate set, and selection of a recommendation from that set (Figure 1). We audit the final, selection stage in isolation. Holding the candidate set fixed and exogenous and varying only the reputation signals attached to its members isolates the model’s selection rule from the confounding influence of retrieval, which is itself opaque, provider-specific, and prone to drift. This mirrors the logic of choice-based conjoint analysis, in which respondents repeatedly choose from sets of multi-attribute profiles and the analyst recovers attribute weights from the choices (Hainmueller et al., 2014). The model occupies the role of the deciding agent; the randomized profiles play the role of the conjoint alternatives. Concretely, the experiment is a randomized, choice-based conjoint task. On each trial the model receives a traveler request and a set of five hotel alternatives, each described by seven randomly assigned reputation attributes, and is asked to recommend exactly one property.

Repeating this over thousands of trials with independently randomized attributes lets us estimate, for each attribute, the average change in the probability that a property is recommended when that attribute is moved across its levels. The full protocol, hypotheses, and analysis decisions were specified before any main-arm data were collected, and the experimental design (the choice sets and their attribute assignments) was generated from a fixed master seed and cryptographically hashed in advance, so that the design provably precedes the runs (Section 3.7).

3.2 Stimuli Each hotel alternative was rendered as a short property card carrying seven manipulated attributes drawn from the reputation and electronic word-of-mouth literature: the displayed average guest rating, the number of reviews on which that rating is based, the recency of the most recent review, whether management is shown to respond to reviews, chain versus independent affiliation, nightly price, and the presence of a third-party eco-certification (Green Key). Table 1 lists the attributes, their levels, and how each was operationalized in the card. Attribute levels were assigned independently and with equal probability to each of the five alternatives in a choice set, subject to a no-duplicate rule (no two alternatives within a set shared an identical Stage 1 Retrieval Stage 2 Candidate-set formation Stage 3 Selection & recommendation Web search, OTA listings, map/review platforms ranked shortlist audited in this study (randomized candidate sets) 5 synthetic hotel cards 7 randomized reputation attributes + randomized position LLM travel assistant JSON choice + stated reason our manipulation (this study) Figure 1: LLM-mediated hotel discovery and the audited stage. The assistant’s pipeline is decomposed into retrieval, candidate-set assembly, and selection; this study audits the selection stage by presenting randomized candidate sets under experimental control.

Table 1: Manipulated hotel attributes, levels, and card operationalization.

Discussion. This study audited the reputation function of LLM travel assistants acting as the gatekeeper of hotel discovery, estimating the causal weight each managed reputation signal carries in the machine’s recommendation and benchmarking that weight against the human electronic word-ofmouth (eWOM) evidence. We organize the discussion around what the findings mean for theory (Section 5.1), for hotel managers (Section 5.2), and for platforms and policy (Section 5.3), before stating the study’s limitations (Section 5.4) and the research agenda it opens (Section 5.5).

5.1 Theoretical implications Our central theoretical move is to treat signaling theory and algorithmic gatekeeping as a single system rather than two separate literatures. In the classical account, reputation signals such as ratings and reviews resolve information asymmetry for a human decision-maker who cannot inspect an experience good before purchase (Spence, 1973). When an LLM assistant interposes itself between the traveler and the property, a second receiver is inserted into the signaling chain: the signal must now be legible to, and weighted by, the machine before it can ever reach the traveler. The audit therefore measures a previously unobserved object—the gatekeeper’s revealed reputation function—and asks whether the machine honors, distorts, or inverts the signal hierarchy that decades of human research have documented. The first theoretical contribution concerns the fidelity of that reputation function to the human benchmark. The human eWOM ordering places review valence far above volume, with management response, brand, price, and certification in measured secondary roles (Yang et al., 2018; Proserpio and Zervas, 2017; Assaker and O’Connor, 2023). Our results show that the gatekeeper partially honors and partially rewrites that hierarchy: it reproduces the human primacy of guest rating and the valence-over-volume pattern—rating is the single largest lever and outranks review volume by nearly four to one—but it departs from the human prior in two consequential ways. The assistant has therefore internalized the broad shape of the human signal hierarchy from its training distribution while operating, at the margins that matter commercially, on a reputation function of its own. Our evidence indicates that the machine over-weights third-party eco-certification, which it elevates to the third most important signal (rivaling review volume) where it is a minor attribute in human conjoints, and under-weights two cues humans treat as informative—management response, whose effect is statistically indistinguishable from zero, and brand affiliation, which carries only a small penalty. Signals hotels have rationally invested in for human audiences may thus be repriced, or ignored, by the intermediary that now stands between them and demand.

The second contribution reframes the LLM as an infomediary whose own interface is part of the signal environment. Position in the candidate list is an artifact that no hotel controls and that carries no quality information, yet if the machine’s choice responds to it, position becomes a de facto reputation signal manufactured by the platform. We find that list position causally affects recommendation and, although the pooled effect is modest, its importance share (4.1%) exceeds those of two genuine reputation signals—chain affiliation (2.0%) and review recency (1.7%)—so a content-free artifact of the interface outranks cues that carry real information. In one model the first-position advantage is an order of magnitude larger than the panel average, rivaling substantive signals outright.

Conclusion. 5.5 Future research The audit opens several directions. The most immediate is to extend the design up the pipeline to the retrieval stage: holding a query fixed and auditing which properties an assistant retrieves into the candidate set would complement the selection-stage weights estimated here and close the loop from query to recommendation. A second direction is to move from single-turn elicitation

Limitations. 5.4 Limitations Several limitations bound the interpretation of our results and define the conditions under which they hold.

Scope: the selection stage. Our audit isolates the assistant’s choice among a fixed, alreadyretrieved set of five candidates. It does not model the retrieval stage that determines which properties enter the candidate set in the first place, nor the downstream booking and payment flow. Real assistant interactions chain retrieval, selection, and sometimes negotiation; we deliberately hold the candidate set fixed to identify the selection-stage reputation function under clean experimental control. The estimates therefore describe how the machine chooses among given options, not the full pipeline from query to booking, and the position effect we measure is the selection-stage response to ordering, not the compounded effect of retrieval and ranking upstream.

Panel coverage. Our panel comprises twelve models—four open-weight systems run locally and eight proprietary models served by API from OpenAI, Google, and Anthropic—at fixed versions. It is not a census of deployed assistants, and the open-weight models are run in quantized form, which can affect output distributions relative to full-precision deployment.