Instead of describing someone in a profile, some systems learn their taste from a few quick choices. Does that close the gap?

This explores whether describing each user as a mix of shared base reward functions (reward factorization) avoids the gap between who a user is on paper (a profile) and what they actually prefer. The corpus doesn't name this problem directly, so that is my reading. The short answer is that factorization escapes the profile half of the gap but not the preference half.

The profile half is where it does escape. Reward factorization never asks who you are. It learns a small set of base reward functions from preference data and treats each user as a weighted mix of them. About ten adaptive questions, each chosen to be maximally informative, pin down the weights, and no model weights change (Can user preferences be learned from just ten questions?). Because the weights are fit to which real responses someone picks, nothing has to guess from a description of a person to what they would like. Text-summary approaches keep a profile but make it earn its place. PLUS trains the summarizer and reward model together, so the summary captures dimensions that predict preferences rather than ones a zero-shot summary would guess. Those summaries stay readable to users (Can text summaries beat embeddings for personalized reward models?). PRIME points the same way: abstract preference summaries beat retrieving specific past interactions (Does abstract preference knowledge outperform specific interaction recall?). That is evidence for compressing people into abstractions, not for factorization specifically. It also exposes a trade-off. A text summary can be inspected and corrected, while a vector of coefficients over base rewards is not obviously legible. There is a more radical route too: align to written principles with no preference labels at all (Can models learn behavioral principles without preference labels?). That aligns a model to a constitution, though, not to an individual.

The preference half stays open, because factorization assumes a person's answers are a clean reading of what they want. Behavioral science suggests otherwise. Annotation responses mix genuine preferences, non-attitudes, and preferences constructed on the spot, and they can only be told apart by checking consistency across measurement conditions (Do all annotation responses measure the same underlying thing?). A ten-question fit leaves little room for that check. If one answer was improvised, it gets baked into the coefficients as if it were a stable trait. So the profile-to-preference guess is replaced by a measurement-quality problem.

The numbers themselves are a second limit. Coefficients on scalar reward functions record how much you like something, not why or how you'd want it changed. Feedback carries an evaluative signal and a directive one, and scalar rewards capture only the first (Can scalar rewards capture all the information in agent feedback?). Critiques written in natural language have broken plateaus that numerical rewards couldn't (Can natural language feedback overcome numerical reward plateaus?). A user whose taste is contextual or explainable only in words is squeezed into a fixed set of base directions.

The better the fit, the more the concern shifts from misalignment to over-alignment. Personalized reward models remove the averaging effect of aggregate ones, and that lets systems learn sycophancy and reinforce polarization at scale (Does personalizing reward models amplify user echo chambers?). A precisely inferred coefficient vector is exactly what would feed that loop. The corpus has no head-to-head test of factorization against profile-based methods, so this is an argument from the pieces, not a measured result.