If an AI gives the same answer however you word a question, does that make it more accurate — or less?
How do behavioral differentiation and paraphrase stability trade against accuracy?
This explores whether a model that responds differently to different inputs, and a model that gives the same answer when a prompt is reworded, gain or lose accuracy by doing so.
This explores whether a model that responds differently to different inputs, and a model that gives the same answer when a prompt is reworded, gain or lose accuracy by doing so. No note in the corpus measures that trade head-on, but read together they suggest it isn't one dial. There are two kinds of sameness. Too little of the first costs accuracy, and too much of the second does too.
Start with what happens when a model is not stable. Models do better on high-frequency phrasings of a question than on rare ones with the same meaning, across math, translation, commonsense reasoning and tool calling Do language models really understand meaning or just surface frequency?. Padding a problem with irrelevant text drops reasoning accuracy from 92% to 68% at just 3,000 tokens, far below the context limit Does reasoning ability actually degrade with longer inputs?. In both cases a change that shouldn't matter moves the behavior, and accuracy pays for it. Consistency training attacks this directly. It teaches the model to answer a reworded or wrapped prompt the way it answers the clean one, using its own clean responses as the targets Can models learn to ignore irrelevant prompt changes?. Because the targets come from the model itself, the stability doesn't pull in the stale data or capabilities of ordinary fine-tuning.
The opposite failure is a model that is too stable. When rewards within a prompt barely vary, training pushes the model toward generic, input-agnostic templates, and performance recovers when training focuses on prompts where outcomes differ Why do language models collapse into generic templates?. Strong prior associations do something similar at inference time. The model keeps the answer it learned in training even when the context in front of it says otherwise Why do language models ignore information in their context?. Both are sameness that costs accuracy, because the output stopped tracking the input. So differentiation isn't in tension with accuracy. It's what accuracy is made of, provided the differences follow meaning rather than surface wording.
The hard part is deciding which changes count as meaning-preserving. Readers from different social positions can legitimately interpret the same sentence differently Why do readers interpret the same sentence so differently?, and GPT-4 correctly disambiguates only 32% of ambiguous cases against 90% for humans Can language models recognize when text is deliberately ambiguous?. A paraphrase-stability objective assumes that rewording leaves meaning unchanged. For ambiguous or socially loaded text, a stable answer may be flattening a real difference, and a single-label accuracy score won't show it. Variation across samples isn't always error either. Models appear to keep a spread of consistent characters and sample from it, so regenerating gives different answers that each fit the context Do large language models actually commit to a single character?.
This last point is my reading, not something the notes claim. The trade may be uneven within a single response. Only about 20% of tokens are high-entropy forking points, and training on just those matches full updates Do high-entropy tokens drive reasoning model improvements?. If so, the ideal is stability across most of the text and freedom at the few points where the reasoning branches.
Sources 9 notes
LLMs show consistent preference for higher-frequency surface forms over semantically equivalent rare paraphrases across math, machine translation, commonsense reasoning, and tool calling. This suggests models track statistical mass from pretraining rather than meaning-recognition as their primary mechanism.
FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.
Two methods—BCT (output-level) and ACT (activation-level)—train models to respond identically to clean and wrapped prompts by using the model's own clean responses as targets, eliminating specification and capability staleness inherent in standard SFT.
When within-prompt reward variance is low, task gradients weaken and regularization dominates, pushing policies toward generic outputs. SNR-Aware Filtering—selecting high-variance prompts before updates—recovers performance across tasks and scales.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Show all 9 sources
Interpretation Modeling research shows that disagreement on socially embedded sentences reflects valid differences in reader perspective, not annotation failure. Structured human disagreement in NLI benchmarks confirms that interpretation distributions carry meaningful information.
AMBIENT benchmark shows GPT-4 correctly disambiguates only 32% of cases versus 90% for humans. This failure spans lexical, structural, and scope ambiguity—revealing that LLMs cannot hold multiple interpretations simultaneously, a fundamental gap hidden by standard benchmarks.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
Only ~20% of tokens exhibit high entropy as pivotal reasoning decision points; RLVR primarily adjusts these forking tokens. Training exclusively on them matches or exceeds full-gradient performance, revealing that the minority carries the learning signal.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
- Language models show human-like content effects on reasoning tasks
- Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgments
- We’re Afraid Language Models Aren’t Modeling Ambiguity
- Word Meanings in Transformer Language Models
- Consistency Training Helps Stop Sycophancy and Jailbreaks
- Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models