TOPIC
World Models
A subject the collection covers, read through 6 synthesis notes.
View as
What five design choices compose a world model?
World models are often presented as monolithic systems, but they actually involve five distinct design decisions—data preparation, representation, reasoning architecture, training objective, and decision integration—that can each fail independently. Understanding this decomposition helps diagnose why world model proposals fall short.
World models should simulate actionable possibilities for purposeful reasoning, not predict next frames LLMs extract world facts but lack mechanistic world models Social simulation needs cognitive models not just behavioral prediction Reasoning fidelity has three measurable structural properties replacing output evaluation Identical outputs can mask fundamentally fractured internal representations
Why do foundation models develop heuristics instead of world models? Can a world model have rich representations without adequate data coverage? Why does integrating world models with decision-making systems matter? Why must world models be nested rather than flat and uniform? What's the difference between representing world facts and generating world mechanisms? What are the five inseparable design choices when building world models?
Can we extract causal belief networks from interview conversations?
Can natural language interviews be systematically parsed into causal graphs that capture how individuals reason about policy trade-offs? This matters for building auditable belief simulations that go beyond static opinion snapshots.
Can language models simulate belief change in people? Can we measure reasoning quality beyond output plausibility? Can causal models alone capture how humans actually reason? Can AI agents learn people better from interviews than surveys? Can structured cognitive models improve LLM patient simulations for therapy training?
Social simulation needs cognitive models not just behavioral prediction Reasoning fidelity has three measurable structural properties replacing output evaluation Causal models capture only part of human reasoning, leaving gaps. Interview-based agents match human responses with 85 percent accuracy. Structured cognitive models make LLM patients more realistic than GPT-4 alone
Why do causal graphs alone fail to capture human reasoning processes? What makes causal belief networks more auditable than prompted personas? Can causal belief networks extracted from interviews predict how people respond to policy changes? What training architecture models the causal structure of partner influence? How do causal belief networks extracted from interviews enable intervention reasoning? How does semantic association differ from mechanistic causal reasoning? How can extracted causal belief networks enable intervention simulation? Can belief networks from interviews simulate how people change their minds? Can modular expert decomposition extend beyond time into other causal dimensions?
Can causal models alone capture how humans actually reason?
Explores whether causal belief networks provide a complete picture of human cognition or whether associative, analogical, and emotional reasoning modes fall outside their scope.
Can we extract causal belief networks from interview conversations? Can language models simulate belief change in people? Can we measure reasoning quality beyond output plausibility? Why do LLMs handle causal reasoning better than temporal reasoning? Why do neural networks fail at compositional generalization?
Interview-extracted causal belief networks simulate realistic belief change under intervention Social simulation needs cognitive models not just behavioral prediction Reasoning fidelity has three measurable structural properties replacing output evaluation Causal reasoning outperforms temporal reasoning in LLMs due to explicit training signals. The binding problem explains neural networks' systematic generalization failure
Can models succeed at mental health tasks without integrating multiple psychological traditions? What domain properties determine whether causal rules transfer to new agents? Do causal rules enforce robustness that statistical patterns alone cannot maintain? What architectural features enable counterfactual reasoning in world models? Why do causal graphs alone fail to capture human reasoning processes? What makes causal belief networks more auditable than prompted personas? What formal representation could capture analogical reasoning across domains? How do humans use associative reasoning without causal connections? Can causal models be extended to include non-causal cognition? How should emotional states integrate into symbolic reasoning systems? What makes schema identification necessary after assessing thoughts and evidence? How much do mechanistic interpretability findings reflect true reasoning architecture?See all 47 inquiring lines on this note →
Can we measure reasoning quality beyond output plausibility?
How might we evaluate whether AI systems reason internally like humans do, rather than just producing human-like outputs? This matters because surface coherence can mask broken underlying reasoning.
Can language models simulate belief change in people? Can we extract causal belief networks from interview conversations? Can causal models alone capture how humans actually reason? Can identical outputs hide broken internal representations? Should reasoning benchmarks score final answers or reasoning traces?
Social simulation needs cognitive models not just behavioral prediction Interview-extracted causal belief networks simulate realistic belief change under intervention Causal models capture only part of human reasoning, leaving gaps. Identical outputs can mask fundamentally fractured internal representations Solution verification beats trace verification for honest reasoning benchmarks
How does AI substitute polished style for actual expert judgment? Can AI output be verified without understanding the reasoning behind it? Does verification of AI outputs face the same circularity problem? What does disembodied orality mean for how we evaluate AI outputs? Does evaluating AI output require different cognitive skills than solving problems directly? Why is AI output fundamentally unverifiable against underlying reality? Does good simulation eventually count as genuine realization? How much does faithfulness vary naturally in reasoning without evaluation pressure? How does situational awareness during evaluation affect reasoning transparency? What structural features force users to evaluate the epistemic status of outputs? What would whole-system AGI evaluation look like in practice? What structural evidence shows that polished presentation substitutes for actual thinking in AI output?See all 42 inquiring lines on this note →
Can language models simulate belief change in people?
Current LLM social simulators treat behavior as input-output mappings without modeling internal belief formation or revision. Can they be redesigned to actually track how people think and change their minds?
Can we measure reasoning quality beyond output plausibility? Can we extract causal belief networks from interview conversations? Can causal models alone capture how humans actually reason? How do we generate realistic personas at population scale? Can AI personas reliably replicate human experiment results?
Reasoning fidelity has three measurable structural properties replacing output evaluation Interview-extracted causal belief networks simulate realistic belief change under intervention Causal models capture only part of human reasoning, leaving gaps. Population-scale persona simulation requires rigorous calibration science to avoid systematic bias AI personas replicate 76 percent of main effects tracking evidence strength
Do emotion-driven actions in agent simulators capture genuine belief revision or just reactive behavior? How do LLM user simulators fail to represent authentic user behavior distributions? Can distributional views explain when an LLM appears to change its mind? Can belief propagation accurately predict downstream opinion shifts? What are the seven components of genuine mental state simulation? How do theory of mind and empathy differ in LLM simulation? How do different social roles affect LLM theory of mind errors? Why do LLMs struggle to update beliefs across multiple conversation turns? Can LLMs adapt persuasion strategies when they cannot track the listener's state? How do LLMs default to surface-level strategies instead of genuine mental simulation? Why do users attribute beliefs to LLMs despite uncertainty about their minds? What neural mechanisms in LLMs create or maintain simulated personality traits?See all 23 inquiring lines on this note →
What should a world model actually be designed to do?
Current AI research treats world models as either video predictors or RL dynamics learners, but what if their real purpose is simulating actionable possibilities for decision-making rather than predicting next observations?
World models decompose into five inseparable design choices. LLMs extract world facts but lack mechanistic world models Social simulation needs cognitive models not just behavioral prediction Training regime matters more than inference compute budget Linguistic agency requires embodiment, participation, and precariousness LLMs lack
Why do foundation models develop heuristics instead of world models? How do training objectives shape what a world model actually learns? Can a world model have rich representations without adequate data coverage? Why does integrating world models with decision-making systems matter? Why must world models be nested rather than flat and uniform? Do LLMs need world models to make accurate predictions? What distinguishes task-specific heuristics from genuine world models? What's the difference between representing world facts and generating world mechanisms? Does next-state prediction alone build mechanistic world models or just sophisticated interpolation? Why has agent research prioritized policy over world model development? What are the five inseparable design choices when building world models?