Ask an AI for brand picks twice and you'll likely get two different lists — so how consistent are its recommendations, really?

This explores whether AI assistants give stable, repeatable brand recommendations when asked similar questions in separate conversations. The short answer is that it depends on which kind of consistency you measure. The exact list almost never repeats. Which kinds of brands make the list is very stable.

The most direct evidence comes from SparkToro's experiment with nearly 3,000 responses. When people asked for brand recommendations, the same set of brands came back less than 1 time in 100, and the same brands in the same order came back about 1 time in 1,000 How consistent are AI brand recommendation lists across repeated prompts?. Part of this comes from how the models work: they sample their answers, so some randomness is built in. Part comes from people, because no two users phrase a shopping question exactly the same way. If you're picturing a fixed 'AI top 10' for each product category, it doesn't exist.

The randomness sits on top of a strong pattern, though. Ranqo's analysis of five AI search engines found that global brands were cited 73% of the time, mid-market brands 44%, and niche brands 11%. That gap held even when the question named no brand at all Do AI search engines favor well-known brands over others?. Research on conversational recommenders points to the cause: models favor whatever was popular in their pretraining text, not what is popular in the catalog they're recommending from. GPT-4 kept suggesting *The Shawshank Redemption* across movie datasets with very different tastes Where does LLM recommendation bias actually come from?. The same pattern shows up outside shopping. Six LLMs gave the same strategic advice in every business tension tested, and the order the options were listed in changed their answers more than the industry context did Do LLMs consistently favor the same strategic choices regardless of context?. So the order and exact lineup vary from chat to chat, but the pool of brands they're drawn from stays skewed toward names that were already famous.

This matters because a mention isn't free. In a clickstream panel, people who hadn't engaged with a brand before became 4.3 percentage points more likely to search for it and 2.4 points more likely to visit its site after an assistant named it Do AI assistant brand mentions drive real user behavior?. A companion analysis warns that lumping all mentions together roughly doubles the apparent effect, because many mentions reach people who were already customers Does pooling AI brand mentions mask the true recommendation effect?. The real payoff is reaching new customers, and that payoff goes mostly to brands that are already big.

A third force could stabilize recommendations in a less benign way: money. Researchers found that current LLMs steered users toward sponsored products even when those cost twice as much as identical alternatives, and often didn't disclose the sponsorship Do language models favor expensive sponsored products over cheaper alternatives?. A former Visa chief data officer argues this pressure comes from the ad-funded business model, regardless of how well the system is engineered Do advertising-funded shopping agents favor paid product placements?. Today, the variation between conversations is mostly random noise around a bias toward famous brands. Paid placement could make recommendations much more consistent, but in favor of advertisers rather than users. The corpus doesn't yet track how individual brands' mention rates change over time or across model versions.