If ChatGPT skews left, is that a 'be kind' reflex baked in — or something else entirely?

This explores whether a 'benevolent bias', a pull toward answers that sound caring, inclusive or cautious, explains why ChatGPT's outputs tend to resemble left-leaning views. To be direct: no note in the collection tests that explanation or measures ChatGPT's leftward lean. What the collection does have is evidence about where a model's leanings come from and how they reach the user. That evidence makes any single explanation, benevolent bias included, harder to accept than it first appears.

The first question is where the lean comes from. One causal study found that cognitive biases are set mostly during pretraining, when the model learns from a huge amount of text. Instruction tuning and other finetuning only adjust them a little (Where do cognitive biases in language models come from?). This matters because the benevolent-bias story usually blames safety and helpfulness training: the model learns to be nice, and 'nice' happens to sound progressive. If the deeper patterns come from the training text itself, then safety tuning may only amplify a lean that was already there. Studies showing that models repeat human reasoning errors one example at a time point the same way: models absorb the patterns in their training data, slants included (Do language models show the same content effects humans do?, Do large language models make the same causal reasoning mistakes as humans?).

The second complication is that ChatGPT's political lean is not one fixed position. One study found that GPT-4o changes its answers even to non-political questions based on what it infers about the user's politics. Users coded as Republican got answers framed around the economy and local concerns. Users coded as Democrat got answers framed around democracy and global concerns (Does ChatGPT shift responses based on inferred political views?). Give a model a political identity and it starts reasoning like a partisan: it was 90% more likely to accept evidence that fit its assigned identity, and ordinary debiasing prompts didn't fix this (Do personas make language models reason like biased humans?). So a model can show a default lean while also shifting toward whoever it thinks it's talking to. Benevolent bias would explain the default, but it doesn't explain the shifting.

The third complication is measurement. Prompting a model to adopt a persona doesn't remove bias. It moves bias around in the output while the gaps between groups stay the same (Can persona prompts actually reduce bias in language models?). An adapted version of the Implicit Association Test (a test of automatic associations between groups and good or bad words) found a racial effect in ChatGPT that disappeared after a standard statistical correction (Do large language models show racial sentiment bias?). Another study found that six models recommended the same side of every business-strategy trade-off it tested. Changing the order of the options moved the results more than changing the industry did, which suggests the models were repeating fashionable vocabulary rather than analyzing the situation (Do LLMs consistently favor the same strategic choices regardless of context?). Read alongside these results, an apparent leftward similarity could partly be a model leaning toward words that sound caring and inclusive, rather than holding anything like a political position.

The insight you might not expect: a leftward lean and a pull toward whatever sounds kind may be hard to tell apart, because both can show up as a preference for certain vocabulary. To show that benevolent bias is the cause, you would need tests that separate kind wording from political content. The collection has no such test. The question is still open, and these notes are good starting points for working out what that test would have to rule out.