Even with randomness switched off, an AI can suddenly change course mid-conversation — why does 'deterministic' not mean 'stable'?
Why do even deterministic models show abrupt shifts across time?
This explores why a model with randomness switched off (temperature zero, fixed seed) can still change course suddenly over the length of a long conversation or reasoning chain. The corpus has little on the other reading, models drifting across versions or calendar time.
This explores why a model with randomness turned off can still change behavior suddenly partway through a task. The short answer from the corpus: determinism only means you get the same output every time from the same input. It doesn't promise smooth behavior as the input grows. Setting temperature to zero just fixes one draw from the model's probability distribution, and repeating that draw a hundred times shows consistency, not reliability Does setting temperature to zero actually make LLM outputs reliable?. If the frozen draw happens to sit near a tipping point, it stays there every run. Consistency checks have the same blind spot: a model that repeats the same mistake identically looks confident, not wrong Can agreement across samples reveal when models are wrong?.
The main reason for sudden shifts is that every token the model writes becomes part of its next input. Even when each step is deterministic, the conditions keep changing as the context fills with the model's own output. One study finds that once a model's earlier errors are in its context, it gets worse at an accelerating rate rather than a steady one. Making the model bigger doesn't fix this. Thinking models do better, apparently because they keep the contaminated history from steering their reasoning Do models fail worse when their own errors fill the context?. One design response is to stop carrying history at all. Atom of Thoughts reduces a problem step by step so that each state depends only on the current subproblem, not on the path taken to reach it Can reasoning systems forget history without losing coherence?.
A helpful way to picture this: the model doesn't play one fixed character. It keeps a spread of possible characters that are all consistent with the conversation so far, and that spread narrows as the conversation goes on Does an LLM commit to a single character or maintain many?. Greedy decoding always picks the top option. But when two interpretations are nearly tied, one extra sentence of context can flip which one is on top, and the visible behavior jumps. Nothing random happened. The ranking just crossed over.
Reasoning models make these jumps visible. They often drop a line of thought midway and switch to another, and simply penalizing the tokens that signal a switch improves accuracy on hard math Do reasoning models switch between ideas too frequently?. Some jumps are productive. In the hidden states of distilled reasoning models, researchers find loops where the model goes back to an earlier answer, and these line up with the documented 'aha moments' Do reasoning cycles in hidden states reveal aha moments?. Something similar happens inside a single token: how often predictions get revised between layers tracks how hard the model is actually thinking Can we measure how deeply a model actually reasons?.
What you might not expect: abrupt shifts in a deterministic model aren't noise to engineer away. They show where the model's internal state hits a decision boundary. That boundary can be a crash, like the snowballing errors above, or real reconsideration, like the aha loops. The corpus has a lot on why shifts happen within a single run. It has little on drift across model versions or over calendar time, so if that's what you meant, this collection won't take you far.
Sources 8 notes
Fixed seeds and zero temperature replicate the same output repeatedly, but that output remains one draw from the model's probability distribution. McDonald's omega testing across 100 repetitions reveals that consistency does not equal reliability.
The Consistency Veto suppresses answers that vary across samples but cannot detect systematic errors the model repeats identically. It carries strong signal on some queries but inherits a fundamental blind spot: agreement looks like confidence even when both are wrong.
Error accumulation in context causes non-linear performance degradation in long-horizon tasks. Model scaling does not fix this; only test-time compute through thinking models reduces the effect by preventing error-contaminated context from biasing reasoning.
Atom of Thoughts decomposes problems into DAGs and contracts them iteratively, ensuring each state depends only on the current problem—not prior steps. This memoryless approach eliminates historical baggage that bloats reasoning while maintaining answer equivalence.
Research shows LLMs don't commit to a single character but instead maintain a probability distribution over many consistent simulacra. Each response samples from this distribution, explaining why regenerations can yield different personalities while remaining consistent with prior context.
Show all 8 sources
o1-like models frequently abandon reasoning paths mid-exploration, wasting tokens on incomplete approaches. A decoding-only penalty on thought-transition tokens (TIP strategy) discourages switching, improving accuracy on challenging math without model fine-tuning.
Distilled reasoning models show ~5 cycles per sample versus near-zero in base models, and cyclicity correlates with accuracy. These cycles in hidden-state reasoning graphs directly map to RL-trained models' documented aha moments—moments when models reconsider intermediate answers.
Deep-thinking ratio (DTR) measures the proportion of tokens whose predictions undergo significant revision across model layers, correlating robustly with accuracy across AIME, HMMT, and GPQA benchmarks. Think@n, a test-time strategy using DTR, matches self-consistency performance while reducing inference costs.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
- Test-time Prompt Intervention
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
- Efficient Reasoning with Balanced Thinking
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
- Can Large Reasoning Models Self-Train?
- Atom of Thoughts for Markov LLM Test-Time Scaling