Why do explicit ratings fail to capture uncertainty in user preferences?
This explores why a single star-rating loses the *certainty* dimension of what someone actually prefers — and what's lost when you flatten a preference into one number.
This explores why a single star-rating loses the *certainty* dimension of what someone actually prefers. The corpus suggests the failure isn't that ratings are wrong, but that they're forced to do two jobs at once and end up doing neither cleanly.
The cleanest framing comes from collaborative filtering work showing that implicit signals — what you actually watched, clicked, or bought — naturally split into two paired magnitudes: *preference* (do you like it?) and *confidence* (how sure are we?). An explicit rating collapses both into a single scalar, so the information about how certain the estimate is simply has nowhere to live Can implicit feedback reveal both preference and confidence?. A five-star rating and a tentative five-star rating look identical on paper.
The number is also noisier than it looks. The same person rates the same item differently across sessions — drifting by multiple stars — because the score mixes genuine preference with temporal mood, anchoring, and personal rating style Why do the same users rate items differently each time?. And it isn't even independent: ratings are nudged by the ratings that came before them, with social-dynamics effects that compound over time Do online ratings actually reflect independent customer opinions?. So the uncertainty isn't only unmeasured — the act of rating actively manufactures some of it.
There's a deeper point lurking here, drawn from annotation and reward-modeling research: not all expressed preferences are the same *kind* of thing. Some responses are genuine preferences, some are non-attitudes (the person has no real opinion but answers anyway), and some are preferences constructed on the spot by the act of being asked. These only become distinguishable through *consistency across conditions* — exactly the signal a one-shot rating throws away Do all annotation responses measure the same underlying thing?. A forced rating can't tell you it was a coin-flip.
What's striking is how much of the corpus is about *recovering* the certainty that ratings discard. Some systems infer personalized reward functions through a handful of adaptive questions explicitly chosen to reduce coefficient uncertainty Can user preferences be learned from just ten questions?. Others let an LLM judge *abstain* rather than force a verdict on sparse evidence — and reliability jumps above 80% once it can say "I'm not sure" Why do LLM judges fail at predicting sparse user preferences?. The same instinct shows up in retrieval, where a model's own calibrated uncertainty beats elaborate heuristics at deciding when it needs more information Can simple uncertainty estimates beat complex adaptive retrieval?. The thread across all of them: preference is a distribution, not a point — and the best systems are the ones that refuse to pretend otherwise.
Sources 7 notes
Hu, Koren, and Volinsky show that implicit signals (watches, purchases, clicks) encode preference and confidence as two distinct dimensions. Explicit ratings collapse these into one number, losing information about certainty in the preference estimate.
Amatriain et al. found that the same user gives substantially different ratings to the same item across sessions, shifting by multiple stars. This noise stems from temporal inconsistency, rater-specific biases, and anchoring effects—making ratings reflect both preference and rating-behavior rather than stable preference alone.
Moe and Trusov decomposed ratings into baseline quality, social-dynamics influence, and error, finding that prior ratings meaningfully affect subsequent ones. These effects have both immediate sales impact and long-term compounding effects through future ratings, though high opinion variance can eventually dampen the distortion.
Behavioral science reveals that annotations contain genuine preferences, non-attitudes, and constructed preferences—distinguishable by consistency across measurement conditions. Treating them uniformly contaminates reward model training and downstream alignment.
PReF learns base reward functions from preference data, then uses active learning to select maximally informative questions that reduce coefficient uncertainty. Users can be personalized via inference-time reward alignment without weight modification.
Sparse persona information lacks predictive power for specific preferences, causing LLM judges to fail. Verbal uncertainty estimation recovers reliability above 80% on high-certainty samples by allowing abstention rather than forced judgment.
Calibrated token-probability uncertainty consistently beats multi-call adaptive retrieval on single-hop tasks and matches performance on multi-hop, using a fraction of the LM and retriever calls. The model's self-knowledge proves more reliable than external heuristics for deciding when to retrieve.