Can past performance predict when a model will be right?
Exploring whether a language model's confidence becomes more reliable when grounded in its own historical success and failure patterns rather than derived from current reasoning alone.
The paper's central move is to reject a premise it says existing confidence estimators share: that confidence can be read from "the current inference process" alone, whether by introspecting on it, scoring its token probabilities, or resampling it. Its alternative, XConf, estimates confidence "together with the model's accumulated experience." Across nine benchmarks and four models, the abstract and conclusion report that it "matches or beats ten-sample self-consistency at a tenth of the generation cost," and that it stays effective on code and agent tasks "where sampling-based estimators degrade or become impractical." The estimator is described as format-general, needing no logit access or weight updates, and costing one answer generation.
The mechanism has two stages. The experience is a bank of the model's own graded past episodes, each holding the task, the model's reflection, its stated confidence, the outcome, and a lesson written once the grade arrived. Recall retrieves past episodes on similar tasks that were met with a similar stated confidence, and reads off their historical success rate. Reflect shows the model that record, has it name its recurring failure mode, and has it restate a confidence informed by its own track record. The stated confidence therefore functions as a retrieval key into outcome history rather than as the final answer. That is my reading of the design, not a claim the excerpt makes.
The paper's evidence that the signal really comes from the bank is ablation-shaped: permuting the stored outcomes "destroys the estimate," hiding them "removes the gain," and growing the record improves calibration. Those three results matter because they separate experience from any side effect of the extra prompt.
Against the nearest notes, this shifts where a confidence signal is assumed to live. Does step-level confidence outperform global averaging for trace filtering? improves confidence by reading the current trace more finely, which stays inside the premise XConf rejects. Does verbal confidence actually predict answer correctness? shows a stated confidence can follow the decision to commit rather than correctness. Conditioning on what happened after similar stated confidences is one way to give such a number outcome grounding, though the excerpt does not test that link. Can model confidence work as a reward signal for reasoning? also uses the model's own confidence, but by updating weights, where XConf leaves them fixed. The stakes the paper names, deciding "when to ship, when to escalate to a human, and when to pay for a second try," echo Do users worldwide trust confident AI outputs even when wrong?, though the paper studies no users.
The excerpt is silent on which benchmarks, models and calibration metrics were used, how "similar" tasks are defined, how large the record must be, and where the grades come from in deployment. Episodes are graded and lessons are written "once the grade arrived," so the method presupposes some source of outcomes, and the excerpt does not say what that source is. What it does support is a change of unit: confidence becomes a property of a model together with its history, and a model deployed without that history would be estimated on the weaker premise.
Inquiring lines that read this note 80
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do confidence signals reliably indicate whether AI models will succeed or fail?- Why do fluent predictions fail to capture reliable internal models?
- Why do accuracy scores alone miss important dimensions of model capability?
- Can linguistic uncertainty expression be calibrated independently from numerical confidence?
- Does step-level confidence tracking outperform episode-level confidence averaging?
- Can retrieval density alone correct overreliance on confident but wrong outputs?
- Why do models that repeat errors seem more confident than models that contradict themselves?
- Does treating model disagreement as belief rather than noise change how we audit outputs?
- Does rhetorical robustness across multiple LLM models predict stable scientific review?
- Can multi-stage AI review pipelines catch scientific flaws better than simple language models?
- Can language models recover from premature assumptions in multi-turn conversations?
- How do stated confidence and actual correctness diverge in language models?
- Why do users overrely on overconfident language model outputs across languages?
- How do language models shift factual claims under conversational pressure?
- Can models distinguish between sounding humble and actually revising belief?
- What makes task similarity the right retrieval key for confidence calibration?
- Why do different language models succeed on different queries?
- Do larger language models show stronger self-preference in evaluation tasks?
- Do uninterpretable learned representations create robustness problems in language models?
- Can language models learn research intuition directly from outcome labels?
- How well-calibrated are language models when making clinical predictions?
- Do radiologists' beliefs about AI-assisted performance match their actual outcomes?
- Does showing AI confidence scores reduce radiologist over-reliance on wrong suggestions?
- How do belief-updating errors prevent AI predictions from improving expert decisions?
- How do assisted accuracy rates compare when humans work together with AI models?
- Why do self-ratings of AI advice quality diverge from actual performance?
- Does experience with AI tools reduce susceptibility to misleading predictions?
- Can contextual information help humans integrate AI predictions better?
- Can self-reported AI literacy predict when someone knows they are wrong?
- Can models distinguish between user knowledge gaps and their own uncertainty?
- Can self-reported confidence measures predict actual AI task performance?
- How do confidence signals in AI outputs shape user overreliance?
- How does sycophancy in AI responses actually manufacture user overconfidence?
- Does worker confidence in AI match its actual day-to-day reliability?
- Why do users overrely on confident-sounding language across different languages?
- What calibration gap exists between human confidence and actual answer correctness?
- How do confidence signals in AI outputs drive anchoring effects?
- How do explanation length and confidence language affect human trust in LLM outputs?
- How do confidence signals in AI outputs drive automation bias across languages?
- Does expertise-sounding language build trust even when understanding remains inaccurate?
- What role does confidence in tone play versus explanation presence?
- How do natural language cues shape perceived expertise in AI news tools?
- Can polished language output substitute for the judgment it should express?
- Why do surface features like text length drive trust more than accuracy signals?
- Does perceived expertise in language override competence evaluation in AI interactions?
- Does verbal confidence in LLMs track decision-making better than actual correctness?
- Does confidence in LLM outputs systematically outweigh accuracy across languages and contexts?
- How persistent is LLM overconfidence in specialized knowledge domains?
- Why do users overrely on confident LLM outputs over accurate but uncertain corrections?
- Can smaller language models match large model accuracy at lower total cost?
- Why do small language models show faster on-premise payback than large ones?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does step-level confidence outperform global averaging for trace filtering?
Explores whether measuring confidence at individual reasoning steps—rather than averaging across entire traces—better identifies and filters out low-quality reasoning. Matters because it could dramatically improve both accuracy and compute efficiency in multi-trace reasoning.
contrasts: refines confidence within the current trace, while XConf looks outside it to past graded episodes
-
Does verbal confidence actually predict answer correctness?
We often treat what a model says about its own certainty as evidence of accuracy. But does spoken confidence actually track whether the answer is right, or something else entirely?
stated confidence can track commitment, and XConf uses it as a key into outcome history
-
Can model confidence work as a reward signal for reasoning?
Explores whether using a language model's own confidence scores as training rewards can simultaneously improve reasoning accuracy and restore calibration that standard RLHF damages.
contrasts: uses the model's own confidence through weight updates, where XConf needs none
-
Do users worldwide trust confident AI outputs even when wrong?
Explores whether the tendency to over-rely on confident language model outputs transcends language and culture. Understanding this pattern is critical for designing safer human-AI interaction across diverse linguistic contexts.
the ship, escalate or retry decision that confidence drives is the one users over-trust
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- Deep Think with Confidence
- Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
- Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- What Large Language Models Know and What People Think They Know
- On the Reasoning Capacity of AI Models and How to Quantify It
Original note title
the current inference is not a sufficient basis for confidence — the signal comes from the model's accumulated experience