SYNTHESIS NOTE
Topics›Evaluations›this note

Does setting temperature to zero actually make LLM outputs reliable?

Explores whether deterministic LLM settings that produce consistent outputs also guarantee reliable judgments, and how to measure true reliability beyond surface consistency.

Synthesis note · 2026-03-28 · sourced from Evaluations
How do reasoning models actually break under pressure?

"Can You Trust LLM Judgments?" (2024) introduces a rigorous framework for evaluating LLM-as-a-Judge reliability using McDonald's omega, revealing that the common practice of using fixed seeds and deterministic settings provides false confidence.

The core argument: even with deterministic settings, a single LLM output is one sample from the model's probability distribution. Setting temperature to zero and fixing the seed produces "fixed randomness" — the same output every time, but that output may still be a misleading draw from the distribution. Consistent replication does not guarantee reliability. A perfectly calibrated LLM that says it's 90% confident should be correct 9 out of 10 times — but even a perfectly calibrated LLM can be unreliable if its distribution has high variance.

The framework: prompt the judgment LLM 100 times, varying only the replication while holding all other factors constant. Apply McDonald's omega to assess internal consistency across these replications. This reveals whether the model's judgments are stable properties of the input or artifacts of the sampling process.

The distinction between reliability, confidence, and calibration is critical:

These three are intertwined but distinct. A model can be well-calibrated (confident when right) but unreliable (different answers on different draws). A model can be reliable (always gives the same answer) but poorly calibrated (that consistent answer is wrong).

This connects to Does model confidence predict robustness to prompt changes? — ProSA measures sensitivity to prompt variation, while this measures sensitivity to sampling variation. Both reveal that single evaluations are insufficient. The practical implication: any LLM-as-a-Judge deployment that relies on single-shot evaluation with deterministic settings is providing the illusion of precision without evidence of reliability.

Inquiring lines that read this note 134

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Do accurate-looking LLM outputs hide structural failures in learning and reasoning? How do evaluation biases undermine LLM quality assessment systems? How do we evaluate AI systems when user perception misleads actual performance? Is model self-awareness based on genuine introspection or pattern matching? What dimensions of recommendation quality do standard metrics miss? How can identical external performance mask different internal representations? How can humans calibrate appropriate trust in AI systems? How can persona representations reduce language model variance and improve task accuracy? How can LLM user simulators model realistic goal-driven conversation? How can we distinguish genuine user preferences from measurement artifacts? Can AI systems balance emotional competence with factual reliability? How do language models inherit human biases from training data? Can language model hallucination be prevented or only managed? Does AI fluency substitute for verifiable accuracy in human judgment? Can prompting strategies overcome LLM biases without model fine-tuning? Can model confidence signals reliably improve reasoning quality and calibration? How can AI alignment serve diverse human preferences at scale? Can LLM personas constitute genuine psychology or remain linguistic role-play? How faithfully do LLMs reflect their actual reasoning in outputs and explanations? When should retrieval-augmented systems decide to fetch new information? How does test-time aggregation affect reasoning correctness and reliability? Can ensemble evaluation methods reduce bias more than single judges? How should we design LLM systems to maintain alignment and control? What makes weaker teacher models effective for stronger student training? What actually drives chain-of-thought reasoning improvements in language models? How does memorization interact with learning and generalization? Do autonomous architecture discoveries follow predictable scaling laws? Can AI-generated outputs constitute genuine knowledge or valid claims? What causes silent corruption to amplify through delegated workflows? How should human oversight be integrated with autonomous AI systems? Why do language models reinforce false assumptions instead of correcting them? Why should disagreement be treated as signal in collaborative reasoning? How does example difficulty affect learning efficiency in language models? How should models express uncertainty rather than forced confident answers? Does externalizing cognitive work and state improve agent reliability? What critical LLM failures do standard benchmarks hide? Why do persona-level simulations fail to predict individual preferences accurately? What constrains reinforcement learning's ability to expand model reasoning? What structural advantages do diffusion language models offer over autoregressive methods? Why does self-revision increase model confidence while degrading accuracy? What prevents language models from reliably adopting diverse personas? Does self-reflection enable models to reliably correct their errors? Does tokenized intelligence retain genuine value through exchange-based systems? How do social dynamics and selection effects compound in rating aggregates? How do evaluation mechanisms prevent error accumulation in autonomous research systems? Why does verification consistently lag behind AI generation? What are the consequences of models training on synthetic data? How does AI adoption affect human skill development and labor equality? Why do benchmark improvements fail to reflect actual reasoning quality? How can AI systems learn from failures without cascading errors? Do harness improvements transfer across model scales or memorize shortcuts? Does recurrence enable reasoning capabilities that fixed-depth transformers cannot achieve? Does fine-tuning modify underlying model capabilities or only behavioral outputs? Can single-axis benchmarks accurately predict agent deployment success?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 131 in 2-hop network ·dense cluster Open in graph ↗
Does setting temperature to zero actually make L… Does model confidence predict robustness to prompt… Can LLM judges be fooled by fake credentials and f… Why do preference models favor surface features ov…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How do calibration and reliability differ in LLM judge evaluations?What does McDonald's omega reveal about LLM judgment consistency?How does sampling variation relate to prompt sensitivity as reliability concerns?Can an LLM be well calibrated but still unreliable on single evaluations?
Original note title

deterministic LLM settings create fixed randomness not reliability — a single output remains one draw from the model's probability distribution