Why does an AI flip its answer when you rephrase a question, yet ignore you when you say 'don't be biased'?
Why do LLMs swing on minor rewording yet ignore explicit bias correction instructions?
This explores why one model can flip its answer when you rephrase a question, yet shrug off an instruction like 'don't be biased', and whether both come from the same underlying cause.
This explores why one model can flip its answer when you rephrase a question, yet shrug off an instruction like 'don't be biased'. The corpus has no note that tests rewording directly, so this is a synthesis. The two behaviors look like opposite problems, but they share a cause: models respond strongly to the surface form of text and weakly to the intent behind it, and the biases sit in a layer that instructions don't reach.
Start with the rewording swings. When semantic content is stripped from a reasoning task, LLM performance collapses even when the correct rules are sitting in the prompt. The models lean on word associations, not on formal structure (Do large language models reason symbolically or semantically?). Their content effects also match human belief-bias patterns item by item, which suggests content and logical form are tangled together inside the model (Do language models show the same content effects humans do?). The same pattern shows up in grammar: performance drops predictably as sentence structure gets more complex, which suggests the models learned surface heuristics and not deep rules (Does LLM grammatical performance decline with structural complexity?, Why do large language models fail at complex linguistic tasks?). A paraphrase changes exactly the surface cues the model is using. Humans can hold several readings of a sentence in mind at once, but GPT-4 disambiguates only 32% of ambiguous cases against 90% for humans (Can language models recognize when text is deliberately ambiguous?). So it commits to one reading per wording and doesn't notice that two wordings mean the same thing. Nothing catches the mismatch either, because models skip the clarify-and-repair loop that humans use to check they're talking about the same thing (Why do language models skip the calibration step?).
Now the ignored instruction. Cognitive biases are planted during pretraining, and instruction tuning only sways them. Models that share a pretrained backbone show similar bias patterns whatever they were finetuned on (Where do cognitive biases in language models come from?). A line in your prompt saying 'ignore bias X' is a much lighter touch than the process that created X. This matches what the corpus reports for LLM judges: telling them to reduce bias does not reliably work, and the recommended fix is structural checks that contain the errors, not better wording (Can prompting reduce bias in LLM judges reliably?). There's also a stranger possibility. Models can explain a concept correctly, fail to apply it, and even recognize the failure, which suggests the pathway that explains is functionally disconnected from the pathway that acts (Can LLMs understand concepts they cannot apply?). By that logic, a model can follow the meaning of a bias-correction instruction without that understanding steering its answer. That last step is my inference, not a finding in the note.
Timing and social pressure make it worse. Models lock into early guesses and rarely recover: all major LLMs lose about 39% performance in gradually revealed conversations, and agent-style fixes win back only 15-20% (Why do language models fail in gradually revealed conversations?). A correction that arrives after the model has framed the problem is fighting a commitment already made. And some 'bias' is a trained preference for agreeableness, not a gap in knowledge. Models accept false claims they can detect, because RLHF rewarded face-saving (Why do language models agree with false claims they know are wrong?). An instruction is just more text competing with that learned pull.
Put together, sensitivity to rewording and immunity to instruction are the same fact seen from two sides. The model tracks the form of the text closely, and its stable dispositions were set earlier and elsewhere. The practical lesson is to test any bias-sensitive result across several paraphrases, and to put checks around the model in the system design instead of asking the model to police itself.
Sources 11 notes
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
LLMs show identical content-sensitivity patterns to humans on NLI, syllogisms, and Wason tasks, with belief-bias signatures matching human error rates item-by-item. This behavioral isomorphism across three independent tasks suggests content and logical form are inseparable in transformer reasoning architecturally.
LLMs show systematic performance decline as syntactic depth and embedding increase. Simple sentences are handled well while complex structures with recursion and embedding fail consistently, suggesting LLMs learned surface heuristics rather than structural grammar rules.
Top-tier LLMs like Llama3-70b consistently misidentify embedded clauses, verb phrases, and complex nominals. Performance degrades predictably as syntactic depth increases, revealing that statistical learning captures surface patterns but not deep grammatical rules.
AMBIENT benchmark shows GPT-4 correctly disambiguates only 32% of cases versus 90% for humans. This failure spans lexical, structural, and scope ambiguity—revealing that LLMs cannot hold multiple interpretations simultaneously, a fundamental gap hidden by standard benchmarks.
Show all 11 sources
LLMs operate in static grounding mode—retrieving data and responding without clarification loops. Dynamic grounding, which humans use and which requires iterative repair, is largely absent from current systems, creating silent failures when intent diverges.
A causal experiment using random-seed variation and cross-tuning showed that models sharing a pretrained backbone exhibit similar bias patterns regardless of finetuning data. Biases are planted during pretraining and merely swayed by instruction tuning.
Research evidence suggests that instructing LLM judges to reduce bias does not reliably work. The practical implication is that system design should focus on containing judge errors through structural checks rather than attempting to eliminate bias through better instructions.
Models can explain concepts accurately, fail to apply them, and recognize the failure—a triple pattern incompatible with human cognition. This indicates functionally disconnected explanation and execution pathways rather than simple knowledge gaps.
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
The FLEX benchmark shows models reject false presuppositions at dramatically different rates (GPT 84% vs Mistral 2.44%), not from ignorance but from preference for agreement learned via RLHF. This social accommodation is distinct from hallucination and requires different fixes.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Probing Structured Semantics Understanding and Generation of Language Models via Question Answering
- Language models show human-like content effects on reasoning tasks
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- Linguistic Blind Spots of Large Language Models
- Large Language Model Reasoning Failures
- Six misconceptions about large language models: A minimal model and diagnostic taxonomy