What actually drives unsafe choices in AI races?

A framed behavioral experiment has pairs of participants repeatedly choose between "Safe and Unsafe development under an uncertain time horizon," where "Unsafe development gave faster progress and higher immediate payoffs but accumulated private risk up to a treatment-specific maximum of 10%, 60%, or 90%." Before the race task, participants also completed a risk-preference elicitation task (the Eckel–Grossman method). The two pre-registered hypotheses — that Unsafe choices would differ between the pmax_r = 0.6 and 0.9 treatments, and that elicited risk preferences would predict Unsafe choices — "were not supported by the data." Instead, exploratory analyses find that "participants are more likely to choose Unsafe after their opponent does so, being ahead reduces Unsafe play while falling behind increases it, and first-round choices predict later behaviour."

The paper reads the opponent-tracking effect as reciprocal or defensive rather than risk-seeking: "Once one competitor acts unsafely, the other becomes more likely to do the same, not necessarily because they prefer risk, but because Safe behaviour becomes strategically costly when the competitor is gaining speed." Falling behind works the same way — "the race structure itself makes Unsafe development attractive when it offers a chance to recover from a disadvantage" — while being ahead suppresses it. High private risk does not cancel this; it only shifts the mix, since the modal frequency of Unsafe play is lower at pmax_r = 0.9. To interpret the pattern, the authors build a reduced evolutionary model of four strategies — Always Unsafe, Conditionally Safe (play Safe first, then copy the opponent), and Conditionally Antisocial Safe (play Unsafe first, then copy) — which "reproduces the treatment effect," with Always Unsafe favored at low private risk because "the speed advantage dominates the expected cost of risk," and Conditionally Safe gaining ground as private risk rises.

The result reframes where the authors locate the cause of unsafe behavior: in the race's strategic structure rather than in the actor's own disposition. That move parallels Does a benign goal actually prevent harmful AI behavior?, which likewise argues that risk tracks the shape of the problem an agent is embedded in rather than a fixed trait. It also echoes, in a different population of agents, Does knowing about another model change self-preservation behavior?: in both cases, a rival's mere presence or prior move reshapes self-protective or risk-taking behavior more than any measured individual trait does. And the finding that higher stakes narrow but do not eliminate unsafe choices sits alongside Do more capable models resist collusion better?, another case where raising the stakes changes the mix of behavior without removing the unsafe option from the table.

The authors are explicit that the central findings are exploratory rather than pre-registered, and that the task is "deliberately stylised and short relative to real technological competition," involving two paired participants rather than the many actors, public signals, and regulatory constraints of actual AI development; the design also isolates private risk rather than the collective or systemic risk that many real safety failures impose on others. The excerpt does not show that AI labs or states behave like these paired participants, only that human behavior in this idealized two-player race tracks opponent action and relative position more than risk level or individual risk tolerance. The paper's own policy implication — that reducing competitive pressure and reciprocal dynamics through transparency, coordination, or credible commitments may matter more than targeting individual risk attitudes — follows at the strength of a single small lab study, not a settled finding about real-world races.