If an AI simulates the world, how does it keep track of what you know versus what I know?

This explores how a world model should keep separate records of what each person believes and knows, instead of one shared 'true state' of the world that everyone is assumed to see. The corpus gives no finished design for this. It does agree on what breaks when you get it wrong, and that points to a few requirements.

First, beliefs have to be explicit parts of the state. They can't be left for the model to infer from the scene. One study of eight LLM-based world models found that tracking only the physical scene produced wrong predictions of human action, even when the scene itself was rendered correctly. Making beliefs, wants, and intentions explicit state components, coupled to the physical simulation, was what worked, and all three were needed (Can world models predict human action from physics alone?). A person's mind is part of the world, and it gets its own variables.

Second, each person needs a private slice of information, and testing has to enforce that. LLMs look socially fluent when a single model plays every character in a conversation. They fail systematically once agents hold information the others don't (Why do LLMs fail when simulating agents with private information?). When one model controls everyone, it can quietly skip the grounding work of establishing who knows what. So a shared scratchpad that every simulated person can read is the wrong representation. It also makes benchmarks look better than the models are.

Third, a person's knowledge is a moving target. Models struggle to track how individuals reason over time. GPT-4o leaned on surface word cues, and every model tested adapted poorly to a player's changing strategy (Can models recognize how individuals reason differently?). A static list of facts each person knows would miss this. The representation needs to be updated as the person acts, and it needs to capture how they reason as well as what they know.

The last test is whether the representation supports reasoning about interventions. A useful world model has to answer counterfactuals, not just predict observations (What makes a world model actually useful for reasoning?). Today's LLMs mostly extract facts and fall back on task-specific heuristics instead of holding a generative model (Do LLMs actually have world models or just facts?). For minds, the counterfactual question is 'what would she do if she learned this?' Only a per-person belief state that responds to new information can answer it. One place such state might live is the memory/experience form in the agent-centric proxy framing (Should world models predict states or return agent feedback?). That placement is my inference, since the note doesn't say it.