AIs training only against each other can settle into alien conventions — a tiny dose of human data steers them back.

This explores a subtle failure of pure self-play: when agents learn by playing against each other, the game usually has *many* stable outcomes (equilibria), and nothing in the reward signal says which one is right. A self-play driving policy might converge to a coordinated but alien convention — everyone swerves left, everyone tailgates efficiently — that is internally consistent yet incompatible with how humans actually drive. The equilibrium-selection problem is exactly this: the math doesn't prefer the human-compatible solution over the many other stable ones. Human data solves it by acting as a tiebreaker. The striking result is how *little* is needed — Can human data steer self-play RL toward human-compatible behavior? shows that thirty minutes of human demonstration, used as a soft regularizer on top of self-play rewards (not as the primary training signal), steers the whole system toward the human-compatible equilibrium with 2500x less data than imitation learning would demand.

The important design move there is that the human data is a *regularizer*, not a teacher. This is the inverse of the usual failure mode of learning from demonstrations. When human data is the primary signal, you get the ceiling problem: Can agents learn beyond what their training data shows? shows agents trained purely on static expert datasets can never exceed what the curators imagined, because they never interact and never learn from their own failures. So the corpus stakes out both ends — self-play gives you competence but no guarantee of *which* competence; pure demonstration gives you human-compatibility but caps ability. The thirty-minutes result is interesting precisely because it keeps self-play's open-ended skill discovery while borrowing just enough human signal to pin down the equilibrium.

What's worth knowing is that other work reaches human-compatible equilibria *without* any human data at all — by manipulating the training population instead. Can agents learn cooperation by adapting to diverse partners? shows that training a sequence-model agent against a sufficiently *diverse* set of co-players makes cooperation emerge on its own: mutual vulnerability to exploitation creates pressure that selects the cooperative equilibrium without hardcoded assumptions. So diversity in the opponent pool and a sprinkle of human data are two different levers on the same problem — both are ways of selecting among equilibria that raw self-play leaves underdetermined.

The stakes of getting the *wrong* equilibrium show up elsewhere in the corpus. Why do multi-agent systems fail to coordinate at scale? finds agents settling into coordination failures — agreeing too late, or adopting a strategy without telling their neighbors — that are stable but bad. And Why do LLMs fail when simulating agents with private information? shows that a lot of apparent multi-agent 'competence' is an artifact of one model secretly controlling everyone; the moment agents hold genuinely private information, the convenient shared equilibrium evaporates. Human data, in this framing, isn't just about politeness — it's a cheap anchor to the one equilibrium that stays coordinated with the humans already in the environment.

The takeaway you didn't know you wanted: the value of human demonstrations may not be the *skill* they contain at all, but the *coordinate* they provide. Thirty minutes carries almost no strategic knowledge, yet it's enough to tell an otherwise self-taught system which of a thousand valid worlds to live in.