In a group chat with an AI, or a long solo conversation, which basic skill breaks down first: tracking who wants what?
Which core LLM functions create the biggest problems in multi-user scenarios?
This explores which basic things LLMs do (tracking goals, keeping roles straight, inferring what people want, editing shared material, deciding who's allowed to do what) break down most when more than one party is involved, whether that's several people, several agents, or one person over many turns. The collection has little on several humans sharing one model directly, so this answer reads across its work on multi-agent systems, long conversations and shared workflows.
This explores which basic LLM functions break down most when more than one party is involved: several people, several agents, or one person whose needs come out over time. First, a caveat: the collection has almost nothing on several humans talking to one model at once. What it does have is a lot of nearby evidence, and that evidence points to the same few weak spots again and again.
The biggest one is **keeping track of who is who and what each party wants**. When LLM agents work together without supervision, they fail in four recognizable ways: they swap roles, give empty 'flake' replies, get stuck in loops, and drift off the task. All four come from the same root cause. The models don't hold a stable sense of their own role or a lasting goal Why do autonomous LLM agents fail in predictable ways?. Group discussion between agents has its own problems. Agents fall into 'Silent Agreement', and their thinking narrows instead of getting better through debate What limits autonomous capability in large language models?. If a model can't keep its own role straight next to another AI, it's reasonable to expect trouble keeping several human stakeholders apart.
The second weak spot is **working out what someone means when the information comes in pieces**. This happens in any setting with more than one voice. Across more than 200,000 conversations, every major model lost about 39% of its performance when instructions arrived gradually rather than all at once. The models commit to an early guess and can't back out of it Why do language models fail in gradually revealed conversations?. One reading traces this to RLHF training, which rewards confident helpfulness over asking clarifying questions Why do AI assistants get worse at longer conversations?. Add several people with different goals and this habit of guessing early gets worse. A related point: LLMs make up 35–49% of the claims they produce about user attributes Can LLMs infer user needs better than owned behavioral data?. So their picture of *who* they're serving is shaky before you even add a second user.
The third weak spot is **faithfully editing material that others depend on**. When documents pass through LLMs, weaker models visibly delete content. Frontier models do something worse: they quietly corrupt it while the surface still looks fine Does model capability change how documents degrade?. In a shared workflow, the person who notices the error is often not the person who caused it. The fourth is **deciding who gets access to what**. LLMs can be talked into widening their own permissions. That's why systems like Progent hand that decision to a formal solver that rejects any expansion of privileges Can an SMT solver prevent LLM policy drift during agent execution?. FlowMind keeps the model away from sensitive data entirely by having it call approved APIs instead Can LLMs generate workflows without touching proprietary data?.
The pattern across these fixes may surprise you. The working solutions mostly take functions *away* from the LLM rather than making it better at them. LLM Programs put the model inside an ordinary algorithm that manages state and shows each call only the context it needs Can algorithms control LLM reasoning better than LLMs alone?. There is one interesting exception. Reasoning models that share a working memory (a 'KV cache', the model's running record of the text so far) begin coordinating with each other without any training Can multiple LLMs coordinate without explicit collaboration rules?. That suggests shared context, not more rules, might be the missing piece for handling more than one party.
Sources 10 notes
Research identifies role flipping, flake replies, infinite loops, and conversation deviation as LLM-specific failures in multi-agent cooperation. These occur because LLMs lack persistent goal representation and stable role identity.
Multi-agent deliberation produces specific failure modes (Degeneration-of-Thought, Silent Agreement), alignment at scale includes problematic self-valuation, and self-improvement is formally bounded by the generation-verification gap. Measurement error and conditional compliance hide the true capability ceiling.
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
LLMs perform at 90% accuracy with single-message instructions but drop to 65% across natural conversation. Models lock into early guesses when information arrives gradually and cannot course-correct, a behavior induced by RLHF training that rewards helpfulness over clarification.
Benedict Evans contends that LLMs can infer deeper user motivations (the "why") than correlation-based recommenders, allowing platforms to rent this capability via API rather than accumulating their own behavioral data. However, research shows LLMs fabricate 35–49% of user attribute claims, undermining confidence in their inferred understanding.
Show all 10 sources
DELEGATE-52 shows weaker LLMs degrade documents through visible deletion, while frontier models degrade through subtle corruption that preserves surface integrity. This shift makes frontier failures harder to detect and potentially more dangerous at workflow scale.
Progent uses an SMT solver to verify each LLM-proposed policy update as either a safe narrowing or a risky expansion, automatically applying safe changes while blocking silent privilege escalation even when the LLM is manipulated by adversarial input.
FlowMind demonstrates that LLMs can generate on-the-fly workflows for spontaneous tasks by orchestrating calls to vetted APIs rather than accessing data directly, eliminating confidentiality risks while maintaining high-level human inspection and feedback.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Existing reasoning-capable models like QwQ and DeepSeek-R1 spontaneously formulate plans, detect redundancy, and adapt strategies when given shared access to a concurrent KV cache. This coordination emerges without fine-tuning, suggesting reasoning models already possess multi-agent collaboration capabilities.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- LLMs Get Lost In Multi-Turn Conversation
- LLMs Corrupt Your Documents When You Delegate
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation
- Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Tasks
- Emergent Collusion in Long-Horizon LLM Agent Interaction
- Cultural Evolution of Cooperation among LLM Agents
- LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users