Do language models know what they don't know about users?

The paper argues that sycophancy, overconfidence and hallucination in personal AI assistants "stem from a fundamental limitation: language models lack an explicit representation of the person beyond the context they are given." It names this the Severance Problem, after the TV show in which a person has two consciousnesses, one at work and one personal, each inaccessible to the other. Its illustration is a user asking "Should I take this job offer?" while the sleep loss, the partner's job, the child's school move and the family home all stay outside the model's "context boundary" unless the prompt states them. The abstract's claim is that even with rich personal context and strong commonsense reasoning, assistants "fail to represent what remains unknown about the user."

The proposed remedy is structured ignorance. The Severance Schema puts into the context an inventory of labeled known and unknown categories, and it names six dimensions along which the model lacks knowledge: physicality, temporality, consequences, continuity, multiplicity and interiority. The conclusion reports that on open-source models an empty schema cut harmful advice from 14–19% to 2–10% and sycophancy from 15–20% to 2–9%, and roughly doubled mentioned unknowns. Composed with memory, it cut hallucination from 3.7–11.7% to 1.7–4.0% and "preserved calibration." The abstract states the effect more broadly, across five model families.

The paper also reports a result that cuts against the usual fix: "accumulating memory on users, makes models overly confident, suppressing the asking behavior that models would exhibit and increasing its hallucination rate." Knowing more about the person, on this account, leaves the model with a false sense of completeness. That runs in the same direction as Does personalization make large language models worse at their jobs?, where personal context also raises agreement, but the explanations differ. There the cause is a shift of objective toward user satisfaction, and here it is a missing representation of the gap between the slice and the life. The premature-assumption failure in Why do language models fail in gradually revealed conversations? looks like a sibling, since both describe a model answering instead of asking. Can persona signals stay grounded in their supporting events? improves the record of what is known, whereas this paper adds a record of what is not, so the two are complementary rather than competing.

The excerpt does not establish much of the mechanics. It names no models, benchmarks, judges or sample sizes, and it does not say how sycophancy, harmful advice or hallucination were scored. It does not say what an "empty" schema contains beyond its labeled categories, or how the numbers for the five families break down, since the figures given are for open-source models. The causal story, that these failures come from the missing representation, is an argument the schema results are consistent with, not a test against rival explanations. What follows at this strength is modest: a prompt that marks its own unknowns is a cheap intervention worth testing wherever memory is added to an assistant, and memory that only accumulates should not be assumed to improve calibration.