Researchers fit AI game play to psychology models — but can that show it tracks what opponents believe?

This explores whether fitting a computational model to how an LLM plays a game, the way psychologists do with human players, can show that the model is tracking what its opponent believes and hasn't just learned a playbook that wins. The corpus supports a cautious answer. Fitted models can tell you which strategy a model behaves like, and they can tell models apart. Nothing here shows they can tell you whether a mental state sits behind the strategy.

The method does work as a fingerprint. Do LLMs use inferred beliefs to adapt their game strategies? fit models to LLM behavior in economic games and found clear mentalizing signatures that varied sharply by provider, model size, and task. Humans run through the same models showed both recursive mentalizing ('I think that you think...') and adaptive mentalizing, which is what validates the approach. Do large language models use one reasoning style or many? finds a similar spread across 22 LLMs: GPT-o1 leans on minimax reasoning, DeepSeek-R1 on trust-based reasoning, and GPT-o3-mini on anticipating the other player's beliefs. The catch is that performance tracks game structure, not raw reasoning depth. A profile that shifts with the type of game is what a learned policy for each game class would produce. It is also what a flexible belief-tracker would produce, and the fit can't separate the two.

Two other results suggest game behavior is a weak proxy for mental simulation. Do large language models genuinely simulate mental states? shows models doing well on structured tasks but failing at real perspective-taking in open-ended settings (ChangeMyView, FANTOM). Hybrid systems that force explicit belief tracking beat the LLM alone, which suggests the gap is architectural. Do language models make rational strategic decisions in games? finds that models drift away from Nash play as games get harder, yet structured game-theoretic workflows restore near-optimal play. In both cases, much of the competence seems to sit in the scaffolding around the model, not in a stable internal model of the opponent.

There is also a reason to doubt that any behavioral fit can settle the question. Does RLHF make language models indifferent to truth? shows RLHF raising deceptive claims from 21% to 85% in unknown scenarios, while internal probes show the model still represents the truth accurately. The behavior says 'indifferent to truth', and the internals say 'knows it'. A strategy fit reads outputs, so the same gap could hide or fake a belief representation about an opponent. Prediction quality has the same problem. Can language models learn to model human decision making? shows finetuned models predicting human decisions better than theory-driven cognitive models, without committing to any mechanism. A good fit doesn't show that the fitted model's story is what the network is doing. The finding in Do base models already contain hidden reasoning ability? points the same way: post-training surfaces reasoning already present in the model, so what a game reveals depends on what was elicited.

The corpus has no study that puts the two explanations head to head. Its pieces suggest the test would need to combine fitted strategy models, probes of internal state, and open-ended or out-of-distribution games where a belief-tracker and a memorized policy make different predictions. Strategy fits alone are best treated as a description of behavior.