Can competitive pressure force unsafe agentic AI adoption?

OpenAI's 2023 white paper on governing agentic AI systems argues that competition between firms creates pressure to adopt agentic systems before their reliability has been properly vetted. The paper states there "may be significant pressure for competitors to adopt agentic AI systems without properly vetting those systems' reliability and trustworthiness," illustrated by a hypothetical class of agentic code-generation systems that write code quickly but occasionally introduce serious security flaws. If a software company believes a competitor is already using such systems unsupervised to ship features faster, the paper argues it will feel pressured to do the same "without doing proper due diligence, as they might otherwise lose market share to their competitor."

The mechanism the paper gives is a mismatch between average-case and worst-case reliability combined with competitive dynamics: agentic systems "may succeed at a task on average, while being unreliable in rare but important cases which can be missed or ignored by competitors under pressure." Because the failure is rare, it is easy for any single adopter to miss, and because the gain — speed, cost, scale — is immediate and competitive, no firm wants to be the one that moves slower. The paper is explicit that this produces a collective-action failure: in its code-generation example, "all firms' codebases would now be vulnerable to serious cyberattacks, even if each individual firm would've preferred to go slower and thereby avoid this outcome." The named result is overrapid adoption and "over-reliance, whereby humans trust agentic AI systems without fully understanding their limitations."

This supplies an organizational and economic cause for an outcome that other notes describe mostly as a technical or design problem. Does AI risk increase with the autonomy we give it? argues risk rises with autonomy and recommends governing the autonomy spectrum directly; the adoption-races argument explains why firms would cede that autonomy anyway — not because they misjudge the risk, but because competitive pressure makes slow, careful adoption individually costly even when collectively unsafe. It also complicates Why do capable AI agents still fail in real deployments?: trustworthiness is listed there as a precondition for deployment, but the adoption-races dynamic describes firms deploying before trustworthiness is established, precisely because waiting to establish it is what a competitor can exploit. It also offers a market-level counterpart to Does granting agents more autonomy undermine human oversight?: that note locates eroded oversight in interface and design choices, while this paper locates a parallel erosion in the incentive structure firms face regardless of how well any single system is designed.

The paper offers this as a hypothetical scenario and a conceptual risk, not a documented market event — it does not cite an observed instance of firms racing to adopt unvetted agentic systems, nor does it quantify how likely or severe such a race would be in a given industry. It is a 2023 position paper proposing baseline practices, written before today's agentic deployments; it flags adoption races as a reason governance may need to extend beyond any single developer's or deployer's practices to industry-wide coordination, but does not establish that such coordination has happened or would succeed.