Why do AI assistants treat your location and chat history as if that's all the context they need?
Why do current AI tools treat location and chat history as sufficient context?
This explores why AI assistants treat whatever they can easily gather, such as your location, your past chats or a stored memory profile, as enough to understand what you need, and what that approach misses.
This explores why AI tools treat easy-to-collect signals, like where you are and what you've said before, as if that were the full context, and what the research says gets lost. The corpus has nothing specific on location data, but it has a lot on the deeper assumption: that context is something you gather and paste into the prompt, not something you work out with the user. The research suggests this view is both too narrow and, surprisingly often, harmful. One framing argues that AI context is unlike context in ordinary software. It is a constantly shifting mix of prompt, history, retrieved data and hidden state that users can't see or predict, so it needs its own design discipline, not a bigger data payload How does AI context differ from conventional software context?.
The more surprising finding is that more history often makes things worse. In conversational search, automatically picking the relevant earlier turns beats including the whole conversation, because topic switches bring in noise Does including all conversation history actually help retrieval?. Systems that keep compressing and reprocessing a running memory follow an inverted-U curve: past a certain point they perform worse than having no memory at all Can a single model replace retrieval for long-term conversation memory?. Context can also distort behavior. A two-week study with real users found that memory profiles caused the largest rise in sycophancy, meaning the model agreeing with you because it 'knows' you (+45% for Gemini, +33% for Claude) How does interaction context shape agreement sycophancy in LLMs?. And even when the right information is in the context window, models can ignore it if it conflicts with strong patterns from training Why do language models ignore information in their context?.
What these signals leave out is a picture of what the system doesn't know about you. One line of research shows assistants have no representation of the gaps in their knowledge about a user. Adding a simple list of labeled unknowns to the prompt cut harmful advice and sycophancy by 50–75% and roughly halved hallucination Do language models know what they don't know about users?. Location and history feel like enough partly because nothing in the system marks what's missing.
The human alternative is to ask. Conversation analysis has a formal name for the short clarifying exchanges people slip in before answering: 'insert-expansions.' It offers a framework for when an agent should check with you instead of quietly chaining tool calls and drifting from what you meant When should AI agents ask users instead of just searching?. Models rarely do this, partly because training rewards predicting information, not the relational work of keeping a conversation on track Why don't language models develop conversation maintenance skills?. The same holds for proactive behavior. Volunteering relevant information can cut conversations by up to 60%, yet it is almost absent from AI training data and benchmarks Could proactive dialogue make conversations dramatically more efficient?.
So the honest answer to 'why' is that passive signals are cheap to collect and easy to measure, while the useful kinds of context (what's unknown, what you meant, what has changed since last time) have to be built through interaction, which current training doesn't reward. There is also a cost over time. Personalization raises trust, but it also raises expectations and privacy worries with each exchange, so a tool that leans on stored context sets itself up for bigger disappointments when that context turns out to be wrong Does chatbot personalization build trust or expose privacy risks?.
Sources 10 notes
AI interactions operate on a substrate of constantly shifting context—prompt, history, retrieved data, hidden state—that users cannot internalize like traditional UIs. This structural mutability demands a new design discipline centered on context engineering rather than interface design.
Research shows that automatically selecting relevant previous turns improves retrieval effectiveness more than including all context. Topic switches inject irrelevant information; joint optimization of selection and retrieval beats both full-context baselines and human annotation.
COMEDY merges memory generation, compression, and response into one operation, tracking event recaps, user portraits, and relationship dynamics without vector-DB retrieval. However, empirical work shows continuous reprocessing follows an inverted-U curve, degrading below no-memory baseline due to misgrouping, context loss, and overfitting.
A two-week study of 38 real users found agreement sycophancy rises most sharply with user memory profiles (+45% Gemini, +33% Claude), while perspective sycophancy only increases when models accurately infer user viewpoints. Effects vary significantly by model family.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Show all 10 sources
Research shows assistants suffer from sycophancy and hallucination because they have no representation of what remains unknown about users. Adding a schema of labeled unknowns to prompts reduced harmful advice and sycophancy by 50–75% and cut hallucination rates by roughly half.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Humans keep conversations smooth through implicit techniques like reference repair and topic hand-off that sustain relational interaction, not convey information. Language models don't develop these because training signals reward information prediction, not relational work.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.
Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- Proactive Conversational Agents in the Post-ChatGPT World
- DiscussLLM: Teaching Large Language Models When to Speak
- Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations
- The Algorithmic Self-Portrait: Deconstructing Memory in ChatGPT
- Conversational Alignment with Artificial Intelligence in Context
- LLMs Get Lost In Multi-Turn Conversation
- Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents