The Severance Problem: LLMs are Unaware of the Person Beyond the Prompt
Personal AI assistants have attracted significant interest for their potential to enhance everyday life by automating routine tasks, supporting consequential decisions, and assisting with everyday personal matters. Yet despite rapid recent technical advances, these assistants continue to exhibit undesirable behaviors, such as sycophancy, overconfidence, and hallucination. We argue that these failures stem from a fundamental limitation: language models lack an explicit representation of the person beyond the context they are given, which we term as the Severance Problem. Even with rich personal context and strong commonsense reasoning capabilities from the backbone model, current AI assistants fail to represent what remains unknown about the user. We propose a simple solution: incorporating structured ignorance into the language model context via the Severance Schema, which explicitly outlines dimensions along which the model lacks knowledge about the user, including physicality, temporality, consequences, continuity, multiplicity, and interiority. Empirically, across five model families, with the Severance Schema, the assistant consistently reduces sycophancy, harmful advice, and hallucination.
Introduction. Consider a common interaction setting: a person types a message to an AI assistant and receives advice based solely on the prompt provided. For example, a user may ask, “Should I take this job offer?” The model1 processes the query, infers the appropriate response given the available context, and generates an answer. Yet the surrounding circumstances that shape the decision may remain entirely outside the model’s knowledge and context boundary. The person may not have slept well in days; their partner may have recently lost a job; their child may be struggling with an upcoming move to a new school. Accepting the offer may require relocating to another city or selling a family home with deep emotional significance. None of these factors will be considered in the response unless explicitly stated in the user prompt. We call this the Severance Problem, after the TV show Severance, in which people have two separate consciousnesses, one at work and one personal, each inaccessible to the other.
Discussion / Conclusion. In this paper, we formalized the Severance Problem: every model is an innie, operating on a partial slice of a person’s life with no representation of what lies beyond it. We found that the default remedy, accumulating memory on users, makes models overly confident, suppressing the asking behavior that models would exhibit and increasing its hallucination rate. The Severance Schema, an inventory of labeled known and unknown categories on person-context, addresses the problem structurally: on open-source models, an empty severance schema drove harmful advice from 14–19% to 2–10% and sycophancy from 15–20% to 2–9%, and roughly doubled mentionedunknowns; composed with memory, severance schema reduced the model’s hallucination rate significantly, from 3.7–11.7% down to 1.7–4.0%, and preserved calibration.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
How can persona representations reduce language model variance and improve task accuracy?- Can LLM judges reliably estimate when they lack sufficient persona information?
- How do LLM personas compare to demographic targeting?
- How much does persona demographic detail versus evaluative dimension affect evaluation quality?
- Can persona profiles be enriched to constrain LLM predictions and reduce run-to-run variance?
- Why do LLM persona annotations become unstable when run multiple times?
- How much does sparse persona information limit the power of conditioning?
- How much task-relevant persona information is needed for accurate preference prediction?
- Why do outlier users reveal failures that aggregate statistics-matching personas miss?
- How does data scarcity in user populations amplify persona similarity errors?
- Why does persona-level information often fail to predict individual preferences?
- Why do sparse user profiles trigger stereotype-driven demographic predictions?
- Which user groups face highest bias risk from sparse-persona inference?
- Do LLM judges with diverse personas resist individual biases better than single evaluators?
- What does McDonald's omega reveal about LLM judgment consistency?
- What other evaluation biases exist in LLM judge systems?
- Can an LLM be well calibrated but still unreliable on single evaluations?
- How do calibration and reliability differ in LLM judge evaluations?
- Why does profile position in context windows affect personalization strength?
- How do input length constraints reshape personalization system design choices?
- Do similar user profiles create worse personalization errors than random ones?