Why do personalized language models fail when profiles and preferences diverge?

The paper names a failure regime it calls profile-preference conceptual misalignment (PRCM): "observable profile cues and query-specific preferences lie in different concept spaces, making semantic retrieval inconsistent for personalization." Its stated target is preference reasoning, "inferring query-relevant preferences from user-related history," and its complaint about existing benchmarks is that they "largely assume that such preference can be retrieved from semantically related history." VIBE-Bench is built to remove that assumption. It has two psychology-grounded tasks, 3,504 personas and 12,239 dialogues, with a manually verified gold test set, and it "requires cross-concept preference reasoning beyond surface semantic overlap."

The abstract reports that current personalized LLMs "largely rely on shallow semantic correlations and fail to acquire robust cross-concept mappings." The discussion supplies the reasoning behind this. In the main, weakly supervised setting, a model sees only user histories, the current query, and end-task supervision. It must therefore identify task-relevant concepts, infer user attributes along those dimensions, and induce the mapping from one concept to another. The authors say this is the realistic setting because interaction data are scalable while concept and mapping annotations are "costly, domain-specific, and often unavailable," and in open-domain use "the relevant concept space may be unknown in advance." When the ontology and cross-concept knowledge are given explicitly, "Concept-Aware Reasoning substantially improves performance." That result comes from an extra experiment using persona-card labels to build template-based chain-of-thought rationales.

The claim sits close to Do user outputs outperform inputs for LLM personalization?, which argues that personalization is not RAG because the useful signal is not the semantic content of past queries. Both notes resist treating personalization as semantic retrieval, but they locate the problem differently. That note shows that which side of the history you use matters. This paper describes a case where even well-chosen history fails to help if the profile and the preference belong to different concepts. It also qualifies the How do personalization granularity levels trade precision against scalability? pattern, whose persona-level tier "requires domain knowledge for group definition." The concept ontology that Concept-Aware Reasoning needs looks like a similar dependency, though the excerpt does not draw that link. Approaches that infer per-user structure from paired data, such as Can user preferences be learned from just ten questions?, are not evaluated here, so whether they escape PRCM is untested.

The excerpt does not report the five baselines' names, the score gaps, the size of the Concept-Aware Reasoning gain, or how PRCM cases were constructed and verified. "Shallow semantic correlations" is the authors' diagnosis of the baselines, and the excerpt gives no analysis behind it. The Concept-Aware Reasoning result also depends on labels drawn from the same persona cards the benchmark was built from, so it shows what explicit concept knowledge can do, not how a deployed system would obtain it. What the excerpt does support is narrower and still useful. A personalization method that scores well on semantically related history has not been shown to handle cases where the cue and the preference belong to different concepts, and PRCM gives a way to test for that.