Can current benchmarks really measure theory of mind in conversational agents?

A survey of multi-party conversational agents (MPCAs) argues that Theory of Mind (ToM) — "the ability to understand, reason, and build beliefs about what participants might be thinking or feeling at each step of a conversation, even if those thoughts and feelings are not explicitly expressed" (citing Premack and Woodruff, 1978) — is essential for building socially intelligent agents that can hold group conversations, team meetings and other collaborative dialogue. Yet the survey reports that current evidence for ToM in LLMs is unsettled: "studies show that scaling up LLMs does not always improve ToM reasoning," even as "some LLMs achieve near-human performance" on standard ToM tasks. A newer benchmark, ToMATO, "reveal[s] gaps for strong models like GPT-4."

The survey locates the uncertainty in two places. First, benchmark validity: techniques like MindDial and benchmarks like MuMA-ToM and KokoMind "show promise," but "the benchmarking is done on synthetic datasets, thereby not emulating realistic multi-party settings for ToM" — so performance on them may not predict performance in a real group conversation. Second, construct validity: even where LLMs score well, "there is an ongoing debate as to whether these models truly reason or simply memorize patterns about mental states," citing Ullman (2023) and Kosinski (2024). The survey also notes "there is little work in integrating ToM capabilities into other MPCA tasks" — State of Mind Modeling, Semantic Understanding and Agent Action Modeling have mostly been studied separately — and states as the authors' own intuition, not a tested finding, that such integration would improve MPCA performance.

This sits adjacent to How do users actually form intent when prompting AI systems?, which locates a related gap — tracking a single user's evolving intent — inside two-party dialogue; this survey's ToM gap is the multi-party analogue, tracking several participants' beliefs and intentions at once, and finds it even less settled, since the field lacks non-synthetic benchmarks altogether. It also qualifies Can tracking dialogue dimensions simultaneously reveal hidden conversation patterns?'s claim that multiple simultaneous dimensions can be tracked and visualized: for the mental-state dimension specifically, this survey suggests the tracking itself (ToM) remains an open capability problem, not just an open visualization problem.

The excerpt does not report what fraction of the 70-plus reviewed papers actually test ToM, nor does it give a benchmark score for MPCAs specifically — the GPT-4 gap on ToMATO is a general ToM benchmark result, not a multi-party conversation benchmark result. It also does not resolve the reasoning-versus-memorization debate it cites; that is reported as an open question in the cited literature, not a position the survey itself takes. The implication is narrower than LLMs lack ToM: the evidence supports only that ToM performance is inconsistent across benchmarks and that no realistic multi-party ToM benchmark yet exists to settle the question.