If you ask an AI the same question twice, do you get the same answer — or a different list each time?

This explores whether asking an AI the same question twice gets you the same recommendations, and why that matters. The short answer: no note in this collection measures repeat-run variance directly. It doesn't contain a study that runs identical queries over and over and counts how often the lists change. What it does have is evidence pointing in two directions at once: AI outputs can be quite unstable at the level of individual responses, yet very predictable at the level of who gets recommended.

The instability evidence comes mostly from evaluation research. When LLMs are used as judges to score outputs, their verdicts shift a lot. One study measured 31% 'judge shift' on complex tasks, and an agent that collects evidence before deciding cut that to 0.27% Can agents evaluate AI outputs more reliably than language models?. That's about evaluation, not shopping lists, but the lesson carries over: a model answering from a single pass is much less repeatable than one that grounds its answer in gathered evidence. Prompt wording matters too. A benchmark of 23 prompts across 12 LLMs found that small framing changes, such as rephrasing or adding background knowledge, shift recommendation accuracy, and the effect depends on the model tier Do prompt techniques work the same across all LLM tiers?. If a slight rewording changes the results, then 'the same search' is a narrower idea than it sounds.

The surprise is how stable things look underneath. An analysis of 100+ brands across five AI search engines found citation rates sorted cleanly by brand stature: 73% for global names, 44% for mid-market, 11% for niche brands. The pattern held even when queries didn't name a brand Do AI search engines favor well-known brands over others?. So even if the exact list changes between runs, the type of answer may barely move, because the big names keep surfacing. Classic recommender research has a related finding: ranking purely by relevance makes lists collapse toward a user's main interest and crowd out secondary ones, unless the system deliberately rebalances them Do accuracy-optimized recommendations preserve user interest diversity?. Run-to-run variation and narrow results can exist side by side.

Why this matters more than it seems: people rarely run a search twice. Eye-tracking shows AI Overviews now soak up the attention the top-ranked result used to get. Attention to the first result dropped from 31% to 9%, and people trusted the overview just as much Where do searchers look when AI Overviews appear?. Whatever one run happens to produce gets treated as the answer. And as shopping moves toward AI agents that search for you, the work shifts from browsing to describing what you want Why does manual search fail as product choices become complex?. That makes the agent's consistency, and its sensitivity to how you phrase things, the real issue.

If you want the direct measurement (overlap between repeated runs, rank churn, how often the top pick changes), this collection doesn't have it yet. The nearest doorways are the judge-shift and prompt-sensitivity studies for instability, and the brand-tier study for the hidden stability.