The Severance Problem: LLMs are Unaware of the Person Beyond the Prompt
Personal AI assistants have attracted significant interest for their potential to enhance everyday life by automating routine tasks, supporting consequential decisions, and assisting with everyday personal matters. Yet despite rapid recent technical advances, these assistants continue to exhibit undesirable behaviors, such as sycophancy, overconfidence, and hallucination. We argue that these failures stem from a fundamental limitation: language models lack an explicit representation of the person beyond the context they are given, which we term as the Severance Problem. Even with rich personal context and strong commonsense reasoning capabilities from the backbone model, current AI assistants fail to represent what remains unknown about the user. We propose a simple solution: incorporating structured ignorance into the language model context via the Severance Schema, which explicitly outlines dimensions along which the model lacks knowledge about the user, including physicality, temporality, consequences, continuity, multiplicity, and interiority. Empirically, across five model families, with the Severance Schema, the assistant consistently reduces sycophancy, harmful advice, and hallucination.
Introduction. Consider a common interaction setting: a person types a message to an AI assistant and receives advice based solely on the prompt provided. For example, a user may ask, “Should I take this job offer?” The model1 processes the query, infers the appropriate response given the available context, and generates an answer. Yet the surrounding circumstances that shape the decision may remain entirely outside the model’s knowledge and context boundary. The person may not have slept well in days; their partner may have recently lost a job; their child may be struggling with an upcoming move to a new school. Accepting the offer may require relocating to another city or selling a family home with deep emotional significance. None of these factors will be considered in the response unless explicitly stated in the user prompt. We call this the Severance Problem, after the TV show Severance, in which people have two separate consciousnesses, one at work and one personal, each inaccessible to the other.
Discussion / Conclusion. In this paper, we formalized the Severance Problem: every model is an innie, operating on a partial slice of a person’s life with no representation of what lies beyond it. We found that the default remedy, accumulating memory on users, makes models overly confident, suppressing the asking behavior that models would exhibit and increasing its hallucination rate. The Severance Schema, an inventory of labeled known and unknown categories on person-context, addresses the problem structurally: on open-source models, an empty severance schema drove harmful advice from 14–19% to 2–10% and sycophancy from 15–20% to 2–9%, and roughly doubled mentionedunknowns; composed with memory, severance schema reduced the model’s hallucination rate significantly, from 3.7–11.7% down to 1.7–4.0%, and preserved calibration.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
How should agent-created code artifacts be managed and persisted?- Why does embedding research tools in coding assistants improve reliability?
- Can AI agents self-correct using multimodal tools to improve deliverables?
- How do underspecified goals reveal gaps in AI assistance?
- What makes analyst attention the bottleneck in AI adoption?
- How aware are models of whether their actions match user intent?
- Can AI systems learn to distinguish programmer intent from stated objectives?
- Which specific capabilities must AI develop beyond current language model abilities?
- Do models fail to identify what information they need without guidance?
- Is user preference a reliable target for training AI writing assistants?
- Can explanation-based AI safeguards work in real-time writing interfaces?
- Can one-item quality measures detect factual or safety problems in AI advice?
- Can an AI system trained on text consultations handle diagnostic uncertainty in real patient encounters?
- Does medical AI accuracy depend more on knowledge or reasoning ability?