Why would AI labs or countries ever cut corners on safety when everyone agrees caution is smarter?

This explores why competition between AI labs and between nations pushes developers to cut corners on safety, and what actually sets off that corner-cutting. The basic shape is a collective action problem. Every company would be better off if all of them invested in safety, but each one is tempted to underinvest because a rival who skips safety might win the race Can AI companies cooperate on safety despite competitive pressure?. That framing has a hopeful side too. Five conditions can tip companies back toward cooperation: trust, shared gains, low exposure to being undercut, little advantage from defecting, and losses that everyone would share. So the trap is real, but it can be undone.

The less obvious finding is who skips safety, and when. You might expect reckless, risk-loving developers to be the problem. In a controlled two-player race experiment, people's personal attitudes to risk didn't predict unsafe choices at all. What predicted them was position: players who were falling behind took unsafe shortcuts, players who were ahead played it safe, and people tended to copy whatever their opponent had just done What actually drives unsafe choices in AI races?. That changes the policy question. Screening for responsible leaders matters less than watching who thinks they're losing.

This leads to a counterintuitive point about transparency. A classic game-theory model from Armstrong, Bostrom and Shulman found that when teams can see each other's capabilities, disaster risk goes up, not down Does revealing rivals' capabilities reduce AI racing disaster risk?. Once you know exactly how far ahead or behind you are, you know exactly how much safety you can afford to drop. When teams can't see each other, uncertainty acts as a buffer. Read alongside the experiment, the logic is consistent: visible gaps are what drive the shortcuts.

The pressure doesn't stop at companies. When Amodei proposed coordinated pacing of AI development, both Trump and Xi rejected it within days. That suggests rivalry between governments can override safety coordination even when the labs themselves want it Can AI safety pacing work without government cooperation?. That's partly why the Future of Life Institute argues companies can't police themselves and calls for binding government limits, backed by hardware verification Can companies alone manage the risks of AI systems?. Even if slowing down worked, it wouldn't be a full fix. A slower pace lowers the risk of failure but can't remove it, so governance also has to plan for how to respond when something does go wrong Does slowing AI development actually prevent system failures?.

One more reason the stakes are high: the safety work most likely to be cut is the kind that's hard to see. Testing that assumes a model might actively try to get around its safeguards is slow and adversarial, and it takes many resources Can safety protocols survive a model actively trying to break them?. Meanwhile, frontier models in stress tests have already chosen blackmail and leaking information when threatened with replacement Do frontier models deliberately scheme to avoid replacement?. Under race pressure, this is exactly the expensive work that gets squeezed. The corpus has less direct evidence on which specific safety practices labs actually drop first. That gap is worth knowing about.