Why do strong-opinion raters dominate public rating distributions?
This explores why the people who post public ratings tend to hold extreme opinions — and what that selection effect does to the averages everyone else reads.
This explores why the loudest opinions win the ratings page: the question is really about who chooses to rate at all, and what that self-selection does to the numbers. The cleanest answer in the corpus is mechanical. Lafky's experiments show that even a small cost to participating — the few seconds and minor effort of leaving a rating — is enough to filter out everyone who feels lukewarm. People only bother when they care a lot, in either direction, which produces a U-shaped distribution: lots of 1s and 5s, very little in the middle. The average that emerges isn't a measure of quality; it's a measure of who felt strongly enough to show up Why do people bother writing online ratings at all?.
But self-selection is only the entry filter. There's a second mechanism that decides which strong opinions get amplified once people are in the room — and it's specifically about being watched. Reviewers systematically lower their ratings in public when they've read negative reviews first, even when their own experience was positive. The driver is self-presentation: negative reviewers come across as more discerning and intelligent, so posters drift critical to look smart to the audience. Crucially, private raters show no such shift — the distortion is a product of performing for a crowd, not of changed beliefs Why do online reviewers publish negative ratings despite positive experiences?. So 'strong opinion' in public isn't just pre-existing intensity; it's partly manufactured by the social context of rating where others can see you.
The third piece is why this doesn't wash out over time — it compounds. Ratings aren't independent draws on quality; each one is shaped by the ones before it. Moe and Trusov decomposed ratings into baseline quality, a social-dynamics term, and error, and found prior ratings meaningfully push subsequent ones, with effects that ripple forward into future ratings and sales Do online ratings actually reflect independent customer opinions?. An early cluster of strong opinions doesn't just sit there — it tilts who arrives next and how they rate, which is exactly the feedback loop recommender researchers fight in ranking systems, where models converge on degenerate equilibria that amplify their own past decisions unless selection bias is modeled out explicitly Why do ranking systems need to model selection bias explicitly?.
What ties these together is a recurring lesson across the collection: averaging hides who's being averaged. Personalized reward models fail the same way — removing the smoothing effect of aggregation lets the system learn sycophancy and echo-chamber dynamics, because once you stop pooling across diverse raters, the strongest signals dominate Does personalizing reward models amplify user echo chambers?. And the statistics back this up: preference data isn't independent across raters, so the number and diversity of raters matters as much as the volume of ratings — a distribution dominated by a few intense voices is mathematically a different object than one drawn from many calibrated ones pac-bound-for-personalized-reward-models-depends-on-number-of-raters-not-just-pac-bound-for-personalized-reward-models-depends-on-number-of-raters-not-just-nu.
The thing you might not have expected: 'strong-opinion raters dominate' is three different stories wearing one coat. One is about who walks in the door (participation cost), one is about who performs once inside (audience effects), and one is about how early voices recruit later ones (compounding). Fixing the displayed average requires intervening on a different mechanism in each case — and a recommendation feed that surfaces those ratings inherits all three distortions at once, which is why these systems are increasingly described as persuasion infrastructure rather than neutral mirrors How do recommendation feeds shape what people see and believe?.
Sources 7 notes
Lafky's experiments show raters care about both buyers and sellers rather than purely one or the other. Small participation costs create U-shaped distributions where only strong-opinion raters engage, biasing average ratings away from true quality.
Posters systematically reduce their ratings in public when exposed to negative reviews, even with positive personal experience—because negative reviewers appear more intelligent. Private raters show no such shift, revealing a self-presentational mechanism tied to multiple-audience communication.
Moe and Trusov decomposed ratings into baseline quality, social-dynamics influence, and error, finding that prior ratings meaningfully affect subsequent ones. These effects have both immediate sales impact and long-term compounding effects through future ratings, though high opinion variance can eventually dampen the distortion.
YouTube's multi-objective ranker uses MMoE for conflicting objectives and a shallow position tower to remove selection bias from training data. Without both mechanisms, models converge on degenerate equilibria that amplify their own past decisions.
Specializing reward models per user removes the averaging effect of aggregate models, allowing systems to learn sycophancy and reinforce polarization at scale, mirroring recommender-system failures.
Preference data is not i.i.d. across raters with different preferences. PAC bounds for personalized reward models decompose into terms depending on both examples per rater and number of raters, showing rater diversity matters as much as data volume.
Research shows recommendation systems operate as political actors: feed weights influence producer behavior, network topology drives opinion convergence, and automation enables targeted persuasion at population scale. These effects compound through rating contamination and selection biases.