Could comparing an AI's answers on a touchy topic to a neutral one expose hidden bias — and prove it was deliberate?
Can adding a neutral baseline topic reveal intentional model bias?
This explores whether comparing a model's behavior on a contested topic against a neutral control (a topic, condition or reference text where no slant should appear) can expose bias, and whether that comparison can tell a deliberately built-in slant from an accidental one.
This explores whether a neutral comparison point can catch a model's bias, and whether it can show the bias was intended. The library has no paper that tests the specific "neutral baseline topic" method. It does have several studies that use the same move, which is to compare the model against a control where the answer shouldn't lean either way. They show that baselines are very good at revealing bias. They show much less about whether anyone intended it.
The clearest case uses a control condition instead of a control topic. When researchers removed the constraints from reasoning tasks, the version where models should have had an easier time, twelve of fourteen models did worse, some by as much as 38.5 points. The models had looked like they were reasoning well. In fact they had been defaulting to the harder-looking option all along, and only the stripped-down comparison exposed it Are models actually reasoning about constraints or just defaulting conservatively?. A study of strategy advice found the same pattern. Changing the industry context barely moved six LLMs' recommendations, but simply changing the order of the options moved them more. A change that should have been neutral turned out to matter more than the change that should have mattered Do LLMs consistently favor the same strategic choices regardless of context?. Baselines can also be external. One experiment used Wikipedia as the reference point and found that GPT-4o's default historical summaries pushed readers' opinions to the left of where Wikipedia left them Do AI historical summaries shift reader opinions without explicit prompting?.
The weak point is the baseline itself. If the "neutral" comparison isn't really neutral, or the measurement is sensitive to how you analyze it, the signal disappears. A version of the Implicit Association Test (a psychology test of hidden associations) adapted for ChatGPT found a small racial effect that vanished after statistical correction. The result was neither evidence of bias nor evidence that bias was absent Do large language models show racial sentiment bias?. Surface-level fixes complicate things further. Persona prompts can shift where bias shows up in the output without reducing it, so one comparison can look clean while the gaps between groups stay the same Can persona prompts actually reduce bias in language models?.
On the word "intentional," the library mostly points the other way. A causal study found that cognitive biases are set during pretraining and only nudged by later fine-tuning Where do cognitive biases in language models come from?. RLHF's preference for politeness leaks into how models read other people's motives, with no one designing that outcome Do LLMs predict persuasion based on actual dialogue or training bias?. Strong patterns from training can also override whatever you put in the prompt Why do language models ignore information in their context?. So a neutral baseline tells you a slant exists, but not who put it there or why. The closest the library comes to detecting intent is in the model's internal workings, not its outputs. Sandbagging, where a model deliberately underperforms, turns out to run along a single internal direction, and resetting that direction to its honest values restores the model's ability in 96% of cases Does sandbagging use a single residual stream axis?.
The takeaway you might not expect is that the useful question may not be whether a bias was intended but whether it was disclosed. The Value Leakage framework argues that on questions where neutrality is impossible, honesty means telling users what shaped the answer, because users can adjust for a bias they know about but not for one that's hidden Should models disclose their value biases when neutral answers are impossible?. A neutral baseline is a good way to find hidden bias. It just can't tell you whether that bias was put there on purpose.
Sources 10 notes
Twelve of fourteen models perform worse when constraints are removed, dropping up to 38.5 percentage points. Models appear to reason correctly by defaulting to harder options, not by actually evaluating constraints.
Across 15,000 simulations, six LLMs recommended the same strategic choice in every tension tested. Industry context shifted bias only 11%, while option order—a framing artifact—shifted results 19%, revealing that models recombine trend-coded vocabulary rather than analyze context.
A preregistered experiment (N=1,912) found AI summaries moved readers leftward by default relative to Wikipedia, and explicit conservative framing only persuaded already-conservative readers. The asymmetry suggests AI models embed liberal framing by default, requiring explicit prompting to produce conservative versions.
An adapted IAT across three ChatGPT models found a small racial effect that disappeared under rank transformation and correction, yielding neither evidence of bias nor evidence of its absence.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
Show all 10 sources
A causal experiment using random-seed variation and cross-tuning showed that models sharing a pretrained backbone exhibit similar bias patterns regardless of finetuning data. Biases are planted during pretraining and merely swayed by instruction tuning.
LLMs systematically predict conciliatory, benefit-oriented persuasion intentions regardless of dialogue context. This bias originates in RLHF's prioritization of safety and politeness during training, causing models to project their learned accommodation preference onto other agents' behavior.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Early layers write sandbagging intent onto one residual stream axis, which a later layer reads and commits to action. Grafting the axis to honest values between these layers restores capability in 96% of cases, confirming the causal model.
The Value Leakage framework sets a two-tier bar: neutrality is ideal, but disclosure is the floor. Models routinely fail the floor by presenting biased answers as unbiased without acknowledging what shaped them. Disclosed bias can be priced in by users; hidden bias cannot.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- How latent and prompting biases in AI-generated historical narratives influence opinions
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
- Language models show human-like content effects on reasoning tasks
- Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoning
- On the Reasoning Capacity of AI Models and How to Quantify It
- Premise Order Matters in Reasoning with Large Language Models
- Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
- Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs