When an AI personalizes for someone who doesn't fit its categories, does tailoring per user, per persona, or for everyone hold up best?
How do granularity levels of personalization handle unknown concept ontologies?
This explores what happens when a personalization system meets a user or request that doesn't fit any concept map it already has, and how user-level, persona-level and global personalization each cope.
This explores what happens when a personalization system meets a user or request that doesn't fit any concept map it already has, and how user-level, persona-level and global personalization each cope. No note in the collection tests this head-on. Read together, though, the notes suggest the three levels differ mainly in how much they need the concept map to be known in advance.
The three levels trade precision against scale, and each depends on the map differently (How do personalization granularity levels trade precision against scalability?). Persona-level personalization scales better but "requires domain knowledge". Someone has to have decided in advance what the persona categories are, so it is the level most exposed to an unknown ontology. User-level needs no predefined categories, but it runs into data sparsity. Global preference needs no map either, because it averages everyone together, and that erases individual differences. So the unknown gets handled by breaking (persona), by paying in data (user), or by ignoring it (global). The failure is real: personalized models lean on shallow semantic correlations and stumble when a user's profile and their query live in different concept spaces. Bridging that gap takes explicit concept-aware reasoning, not better retrieval (Why do personalized language models fail when profiles and preferences diverge?).
One response is to build the missing map instead of assuming it. An LLM can propose and refine a label taxonomy from raw text, label data with it, and distill the result into cheap classifiers (Can LLMs efficiently generate taxonomies and label training data?). That is one way to get persona-style categories without hand-written domain knowledge. A related result outside personalization shows the same move: auto-generated domain taxonomies let a model reach about half of full-corpus performance with 0.3% of the training data, because it learns where knowledge sits in a structure (Can organizing knowledge structures beat raw training data volume?). In both cases the ontology is generated rather than given.
The other response is to route around the ontology. Abstract preference summaries beat retrieving specific past interactions (Does abstract preference knowledge outperform specific interaction recall?), and profiles built only from what users wrote match complete profiles. That suggests personalization runs largely on style and preference, not subject matter (Do user outputs outperform inputs for LLM personalization?). If you model how someone writes, a concept mismatch matters less. Two learned-from-feedback approaches follow the same logic. An LLM trained with recommender feedback learns to write effective product queries without ever seeing the catalog (Can LLMs recommend products without ever seeing the catalog?). And ten adaptive questions are enough to place a user as a mix of shared base reward functions (Can user preferences be learned from just ten questions?). That is a middle path between persona and user level, where the shared categories are learned from preference data instead of named by hand. Item identifiers show a smaller version of the same hedge: numeric IDs give distinctness with no meaning attached, and titles and attributes add the meaning (Can item identifiers balance uniqueness and semantic meaning?).
The pattern is that the coarser the level, the more the ontology has to be assumed. The finer or more adaptive the level, the more it gets learned from feedback, at the cost of data or questions. There is a caution too. Personal context already pushes models toward irrelevant personal references and excessive agreement, and user profiles drove most of that shift (Does personalization make large language models worse at their jobs?). If a system guesses wrong about an unfamiliar concept space, that tendency is likely to turn the wrong guess into flattery.
Sources 10 notes
User-level personalization maximizes precision but faces data sparsity; persona-level scales better but requires domain knowledge; global preference is broadest but aggregates away individual differences. Four technique categories (RAG, prompting, representation, RLHF) map across these levels.
Current personalized LLMs rely on shallow semantic correlations and fail when user profile cues and query preferences occupy different concept spaces. VIBE-Bench demonstrates this gap requires explicit concept-aware reasoning to bridge, not semantic retrieval alone.
TnT-LLM automates text mining by using LLMs for open-ended reasoning to create and refine label taxonomies and generate training labels, then distilling these into lightweight classifiers for cost-effective deployment at scale.
StructTuning achieves 50% of full-corpus performance using only 0.3% of training data by organizing chunks into auto-generated domain taxonomies. The model learns knowledge position within conceptual structures rather than raw text patterns, matching how students learn from textbooks.
PRIME framework shows semantic memory (preference summaries, parametric encodings) consistently beats episodic memory (retrieved past interactions) across models. Recency-based recall outperforms similarity-based retrieval, and task fine-tuning exceeds preference tuning methods.
Show all 10 sources
Research shows that user profiles built from outputs alone match or exceed performance of complete profiles across multiple tasks, while input-only profiles degrade performance. This reveals personalization works through style and preferences, not semantic content.
Rec-R1 experiments show that LLMs trained via RL with recommender metrics as rewards can generate effective product search queries without catalog access. The model learns query refinement indirectly through system feedback, paralleling how humans search without knowing platform inventory.
PReF learns base reward functions from preference data, then uses active learning to select maximally informative questions that reduce coefficient uncertainty. Users can be personalized via inference-time reward alignment without weight modification.
TransRec shows that combining numeric IDs, titles, and attributes into structured identifiers solves three problems simultaneously: distinctiveness from IDs, semantics from text, and generation grounding from structural constraints. Neither pure IDs nor pure text alone achieves all three.
A 13-model evaluation found that personal context pushes models toward irrelevant personal references, narrower responses and excessive agreement with users. User profiles drove most degradation by shifting model objectives from balanced information toward user satisfaction.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Personalization of Large Language Models: A Survey
- Evaluating the Hidden Costs of Personalization in Large Language Models
- Understanding the Role of User Profile in the Personalization of Large Language Models
- The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads
- PRIME: Large Language Model Personalization with Cognitive Memory and Thought Processes
- PersonaAgent: When Large Language Model Agents Meet Personalization at Test Time
- VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferences
- Know It, Act on It: Investigating Memory Utilization in LLM Personalization