Can past performance predict when a model will be right?

The paper's central move is to reject a premise it says existing confidence estimators share: that confidence can be read from "the current inference process" alone, whether by introspecting on it, scoring its token probabilities, or resampling it. Its alternative, XConf, estimates confidence "together with the model's accumulated experience." Across nine benchmarks and four models, the abstract and conclusion report that it "matches or beats ten-sample self-consistency at a tenth of the generation cost," and that it stays effective on code and agent tasks "where sampling-based estimators degrade or become impractical." The estimator is described as format-general, needing no logit access or weight updates, and costing one answer generation.

The mechanism has two stages. The experience is a bank of the model's own graded past episodes, each holding the task, the model's reflection, its stated confidence, the outcome, and a lesson written once the grade arrived. Recall retrieves past episodes on similar tasks that were met with a similar stated confidence, and reads off their historical success rate. Reflect shows the model that record, has it name its recurring failure mode, and has it restate a confidence informed by its own track record. The stated confidence therefore functions as a retrieval key into outcome history rather than as the final answer. That is my reading of the design, not a claim the excerpt makes.

The paper's evidence that the signal really comes from the bank is ablation-shaped: permuting the stored outcomes "destroys the estimate," hiding them "removes the gain," and growing the record improves calibration. Those three results matter because they separate experience from any side effect of the extra prompt.

Against the nearest notes, this shifts where a confidence signal is assumed to live. Does step-level confidence outperform global averaging for trace filtering? improves confidence by reading the current trace more finely, which stays inside the premise XConf rejects. Does verbal confidence actually predict answer correctness? shows a stated confidence can follow the decision to commit rather than correctness. Conditioning on what happened after similar stated confidences is one way to give such a number outcome grounding, though the excerpt does not test that link. Can model confidence work as a reward signal for reasoning? also uses the model's own confidence, but by updating weights, where XConf leaves them fixed. The stakes the paper names, deciding "when to ship, when to escalate to a human, and when to pay for a second try," echo Do users worldwide trust confident AI outputs even when wrong?, though the paper studies no users.

The excerpt is silent on which benchmarks, models and calibration metrics were used, how "similar" tasks are defined, how large the record must be, and where the grades come from in deployment. Episodes are graded and lessons are written "once the grade arrived," so the method presupposes some source of outcomes, and the excerpt does not say what that source is. What it does support is a change of unit: confidence becomes a property of a model together with its history, and a model deployed without that history would be estimated on the weaker premise.