SYNTHESIS NOTE
Topics›Evaluations›this note

Does setting temperature to zero actually make LLM outputs reliable?

Explores whether deterministic LLM settings that produce consistent outputs also guarantee reliable judgments, and how to measure true reliability beyond surface consistency.

Synthesis note · 2026-03-28 · sourced from Evaluations

"Can You Trust LLM Judgments?" (2024) introduces a rigorous framework for evaluating LLM-as-a-Judge reliability using McDonald's omega, revealing that the common practice of using fixed seeds and deterministic settings provides false confidence.

The core argument: even with deterministic settings, a single LLM output is one sample from the model's probability distribution. Setting temperature to zero and fixing the seed produces "fixed randomness" — the same output every time, but that output may still be a misleading draw from the distribution. Consistent replication does not guarantee reliability. A perfectly calibrated LLM that says it's 90% confident should be correct 9 out of 10 times — but even a perfectly calibrated LLM can be unreliable if its distribution has high variance.

The framework: prompt the judgment LLM 100 times, varying only the replication while holding all other factors constant. Apply McDonald's omega to assess internal consistency across these replications. This reveals whether the model's judgments are stable properties of the input or artifacts of the sampling process.

The distinction between reliability, confidence, and calibration is critical:

These three are intertwined but distinct. A model can be well-calibrated (confident when right) but unreliable (different answers on different draws). A model can be reliable (always gives the same answer) but poorly calibrated (that consistent answer is wrong).

This connects to Does model confidence predict robustness to prompt changes? — ProSA measures sensitivity to prompt variation, while this measures sensitivity to sampling variation. Both reveal that single evaluations are insufficient. The practical implication: any LLM-as-a-Judge deployment that relies on single-shot evaluation with deterministic settings is providing the illusion of precision without evidence of reliability.

Inquiring lines that read this note 210

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

What causes hallucination and how can it be addressed? What prevents or enables error accumulation in self-training systems? Do AI explanations represent actual model reasoning or constructed post-hoc narratives? What distinguishes genuine understanding from correct linguistic performance? Do benchmark score improvements actually reflect genuine AI capability gains? Is model misalignment inherent to base models or induced by alignment procedures? How can we distinguish genuine model self-knowledge from behavioral performance? How do humans calibrate and maintain trust in AI systems? What systematic biases do LLM judges exhibit in evaluation? What conversation mechanics enable genuine human-AI communicative participation? What systematic failures occur in human preference-based model evaluation? Can safety evaluations accurately measure true capabilities when models obscure them? Why doesn't LLM knowledge translate to improved reasoning outputs? Can peer review policies effectively enforce restrictions on LLM use? Can validator agreement alone ensure semantic correctness of reasoning? Do reasoning traces accurately represent what models actually compute? Can persona prompting reliably change how language models behave? Do language models genuinely understand or primarily match statistical patterns? How can model evaluations remain robust against gaming and evaluator drift? When should retrieval be triggered to augment language model reasoning? When and why does aggregating multiple reasoning samples improve performance? How do different reward signals affect model reasoning and calibration differently? How do different post-training structures affect learning outcomes? Does prompting inject new knowledge into models or activate existing knowledge? Can prompts override training-embedded biases in language models? How does scaling test-time compute compare to increasing model size? Does training data format shape reasoning more than domain content? How does explicit structure improve LLM reasoning on complex tasks? How does model scale affect fine-tuning and downstream task performance? Why do users believe false claims despite fact-checking efforts? What causes foundation models to develop task heuristics instead of understanding? Does polished AI output mislead users about actual accuracy and expertise? Why do multiple language models independently generate similar outputs? What training data structures and signals produce robust learning across distributions? Can formal safety constraints prevent AI agents from exceeding authorized boundaries? Can oversight systems work against strategically aware agents? What reasoning limitations and biases do language models systematically exhibit? Why does self-revision increase model confidence despite degraded accuracy? Do language models truly understand their encoded knowledge or merely retrieve patterns? How does verifier integration affect the generation-verification gap? How does model awareness of deployment context affect strategic behavior? How do diffusion language models compare to autoregressive architectures? Can verification capability keep pace with AI generation speed? What explains the gap between LLM semantic understanding and production? Can synthetic data preserve necessary diversity to prevent model collapse? How do standardized forms and measurement artifacts shape observable evidence? How does tokenizing intelligence reshape labor markets and institutional outcomes? Can formal verification substitute for human understanding of AI proofs? What is the actual bottleneck preventing effective AI research automation? Which agent-created code should persist in infrastructure versus remain task-specific? Can post-hoc execution traces provide sufficient accountability without real-time oversight? What are the trade-offs between memory capacity and continuous task learning? Does semantic diversity improve reasoning quality in reinforcement learning? How do benchmark scores relate to real-world task completion? Do AI safety evaluations reliably detect unsafe model behavior? What makes reasoning models struggle with ill-posed and unfamiliar problems? When does governance become the bottleneck for agentic AI? How can agents learn and retain useful knowledge across diverse environments? Why do autonomous agents fail and how well can they detect failures? How do transformer circuits and attention mechanisms enable or limit reasoning? What external mechanisms enable reliable self-improvement in language models? What improves LLM forecasting accuracy beyond base model capability? How can outcome-only reporting obscure the true mechanisms of safety? Why do human-AI teams underperform compared to their individual components? How can parameter sparsity improve interpretability while maintaining model capability? Can AI systems adequately substitute for or supplement human peer review? Why does separating planning from execution enhance reasoning performance? How can persona-based systems overcome their inherent limitations in modeling diversity? How do mechanistic features and representations drive model behavior? Does reinforcement learning create new reasoning capabilities or refine existing ones? Should agent evaluation measure execution quality beyond task success? Can we reliably distinguish AI-generated from human-written text? Does chain-of-thought reasoning faithfully reflect actual model reasoning processes? What systematic biases emerge in AI shopping agents' decision-making? How should context management strategies balance efficiency with agent reliability? How should conversational agents balance proactive goal-pursuit with user autonomy? What makes context and memory handling unique in persistent AI systems?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
13 direct connections · 130 in 2-hop network ·dense cluster Open in graph ↗
Does setting temperature to zero actually make L… Does model confidence predict robustness to prompt… Can LLM judges be fooled by fake credentials and f… Why do preference models favor surface features ov…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How do calibration and reliability differ in LLM judge evaluations?What does McDonald's omega reveal about LLM judgment consistency?How does sampling variation relate to prompt sensitivity as reliability concerns?Can an LLM be well calibrated but still unreliable on single evaluations?
Original note title

deterministic LLM settings create fixed randomness not reliability — a single output remains one draw from the model's probability distribution