Where do LLM choice biases actually come from?

Having measured LLM shopping agents responding far more strongly than people to price, rating, and nudge manipulations (see Do shopping agents respond more strongly to price and rating cues?), the paper draws out what that gap implies about where the bias comes from. It argues the contrast "suggests that importing human constructs such as bounded rationality or limited attention is not sufficient to explain agent behavior: agents appear to reproduce human-like heuristics and biases without sharing the cognitive constraints... that motivated such theories." Instead, "these results point instead toward mechanisms rooted in (pre- and post-)training data, reward signals, and other such sources."

The reasoning works by elimination. Simon's bounded rationality and Kahneman and Tversky's heuristics-and-biases program explain human susceptibility to nudges as a side effect of finite attention, working memory, and time pressure — real computational constraints a biological decision-maker operates under. An LLM agent reading pruned HTML with no such constraints should, on that theory, have no reason to show the same susceptibility. Yet it does, and more strongly. Since the standard explanation for the bias doesn't transfer, the paper concludes the resemblance must be learned rather than structural: baked into the model by what it was trained and fine-tuned on, including post-training reward signals, rather than emerging from anything like a resource-rational trade-off.

This is a mechanism-level companion to Are RLHF annotations actually measuring genuine human preferences?, which locates a different model quirk — RLHF reward models treating unstable elicited preferences as if they were genuine values — in the same part of the pipeline. Both notes point at training and reward design, not reasoning failure or inference-time instability, as the place where an unwanted behavioral pattern actually originates. It also reframes Do language model groups mimic human group reasoning patterns?: that paper's groups match a human aggregate pattern without the same underlying process either, consistent with learned surface-pattern reproduction rather than shared cognitive mechanism.

This is the paper's interpretation of its own result, not a separate causal test: the excerpt does not trace the bias to a specific training example, dataset, or reward term, and the inference rests on the absence of a plausible cognitive-constraint explanation rather than a direct manipulation of training data. If the account holds, however, it implies that reducing agent nudge-susceptibility is not primarily a prompting or inference-time problem — asking the agent to "be more rational" addresses a constraint it was never actually under — but a training-and-reward-design problem, which is a harder and slower lever to pull than anything available to a deployer at inference time.