AI shopping agents clearly favor certain positions in a list — but which spot wins can flip after every model update.

This explores whether position bias (an AI model favoring an option because of where it sits in a list) follows patterns you could predict from a model's family or version. The short answer from the collection is that the bias is reliable but its direction is not. The most direct evidence comes from ACES, which tested AI shopping agents. Each model had its own strong positional preferences. These persisted when the visual interface was removed and the agent saw only text, and even when it was told outright to ignore position. The detail that matters most for prediction: the bias pattern reshuffled after model updates Do AI shopping agents show persistent position bias in text interfaces?. A version bump can quietly change which slot wins. You can't measure it once and assume it stays fixed.

If you want a theory for where family-level patterns would come from, look at research on cognitive biases more broadly. One causal study found that models built on the same pretrained base show similar bias patterns no matter what fine-tuning data they later received. Fine-tuning only nudges biases that pretraining already put in place Where do cognitive biases in language models come from?. That suggests a useful rule of thumb: models that share a pretraining lineage should resemble each other more than models that merely share a brand name. A new version trained on a new base could behave like a different model entirely. The collection doesn't directly test position bias this way, so treat this as an inference, not a finding.

A neighboring kind of bias shows how much behavior can differ between families. When researchers checked whether frontier models favor their own company, Claude leaned slightly pro-Anthropic across all four tasks, GPT showed favoritism only when grading agent work, and Gemini leaned slightly against Google Do frontier AI models favor their own company?. Pulling the other way, the 'Artificial Hivemind' study found that 70+ models often give strikingly similar answers to open-ended questions because their training data and alignment methods overlap Do different AI models actually produce diverse outputs?. So models can look alike in what they say while differing in which hidden preferences decide their choices.

The practical point is that prompting rarely removes these biases. Persona prompts change how bias shows up in a model's output, but the underlying gaps between groups stay the same Can persona prompts actually reduce bias in language models?. That matches ACES's finding that 'ignore position' instructions didn't work. A more promising route is consistency training, which teaches a model to answer the same way when only irrelevant parts of a prompt change. Shuffling the order of options is a natural candidate for that kind of training Can models learn to ignore irrelevant prompt changes?. Recommendation systems have handled the same problem for years. YouTube's ranker uses a separate 'position tower' to strip out the boost an item gets just from being shown higher Why do ranking systems need to model selection bias explicitly?. That suggests teams building agents should measure and correct position effects outside the model, not hope the model corrects itself.

The gap is that the collection has no study that maps position bias across many model families and versions side by side. What exists suggests three things: position bias is stable within a given model snapshot, it is probably shaped by the pretrained base more than by the brand, and it can change with each update. So the predictable part is not which position wins. It's that you need to re-test after every model change.