What makes multi-session context tracking harder than single-turn underspecification problems?
This explores why holding state across many sessions is a harder problem than resolving an ambiguous single request — and the corpus suggests the difficulty shifts from interpretation to consolidation, commitment, and selective forgetting.
This explores why tracking context across many sessions is harder than handling an underspecified single turn. The short version the corpus points to: single-turn underspecification is mostly an *interpretation* problem — you have all the evidence in front of you and need to resolve ambiguity. Multi-session tracking is a *retention and consolidation* problem, and those failure modes are structurally nastier.
Start with the single-turn case. When a user says something vague, the classic move is to not commit to one reading — maintain a belief distribution over what they might mean and update it. That's exactly the lesson from speech dialogue systems, where 15-30% recognition error rates made deterministic flowcharts useless and forced systems to carry probabilistic beliefs over intent rather than guessing Why do dialogue systems need probabilistic reasoning?. Hard, but bounded: the ambiguity lives inside one exchange.
Multi-session is harder first because the model has no stable way to *carry* what it learned. Research on the long-context bottleneck argues the real constraint isn't memory capacity at all — it's the compute needed to transform earlier context into durable internal state, a consolidation step that looks like test-time scaling Is long-context bottleneck really about memory or compute?. Architectures like Titans make the same point by physically splitting fast short-term attention from a compressed long-term memory that only stores 'surprising' tokens Can neural memory modules scale language models beyond attention limits?. In other words, persistence across sessions requires machinery that a single turn never needs — and just having a huge context window doesn't supply it, since long-context models still fail on structured, relational lookups that span the history Can long-context LLMs replace retrieval-augmented generation systems?.
Second, even when the prior context is present, the model may not honor it. LLMs don't commit to a fixed persona or stance — regenerate the same prompt and you get different, each-locally-consistent characters, because the model samples from a superposition rather than holding a position Do large language models actually commit to a single character?. And when in-context information conflicts with strong training priors, the priors quietly win; prompting alone can't override them Why do language models ignore information in their context?. Across sessions, those small drifts compound into a partner who keeps subtly forgetting who you are.
Third — and this is the part readers usually don't anticipate — long-horizon tracking is as much about *ignoring* as remembering. Models are trained on what-to-do instructions but not what-to-ignore instructions, so they happily engage conversational distractors and drift off-topic; closing that gap took explicit training on dialogues seeded with distractor turns Why do language models engage with conversational distractors?. The longer the thread, the more noise accumulates, and selective forgetting becomes the bottleneck rather than recall. So the real answer to the question: single-turn underspecification asks 'what did you mean?', while multi-session tracking asks 'can you consolidate it, commit to it, and filter it' — three separate failure points, each with its own missing machinery.
Sources 7 notes
Real-world speech recognition achieves 15-30 percent error rates in noisy environments, making deterministic flowchart dialogue systems unworkable. POMDP-based systems handle this by maintaining belief distributions over user intent rather than committing to single interpretations.
Research shows the bottleneck is not memory capacity but the compute required to consolidate evicted context into fast weights during offline sleep phases. Performance improves with more consolidation passes, following a test-time scaling pattern on harder reasoning tasks.
Titans architecture separates attention (short-term, quadratic) from neural memory (long-term, compressed), prioritizing surprising tokens for storage. The model outperforms standard Transformers and linear RNNs across tasks while scaling to 2M+ token contexts without quadratic penalties.
The LOFT benchmark shows LCLMs match RAG on semantic retrieval without explicit training, but cannot execute relational queries requiring joins across structured tables. Context length alone cannot bridge this gap.
Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Fine-tuning on just 1,080 synthetic dialogues with distractor turns significantly improves topic resilience, revealing that the gap is not model capacity but absent training signal. Models learn to follow what-to-do instructions but not what-to-ignore instructions.