Does AI risk increase with the autonomy we give it?

"Fully Autonomous AI Agents Should Not be Developed" makes a monotonicity claim that is stronger than the usual hedged caution: the more control a user cedes to an agent, the more risks to people arise. The most extreme form — full autonomy with no human-defined constraints — is where the lack of constraint lets a single failure impact multiple human values at once. The argument is grounded not in speculative superintelligence but in the ethics literature and current product marketing, mapping benefits against risks across levels of delegation. The historical anchor is the 1980 nuclear false alarm, where automated systems reported 2,000 inbound Soviet missiles and only human cross-verification between warning systems caught the error — a concrete case where autonomy without a human check-point nearly proved catastrophic.

The load-bearing move is the cost-benefit asymmetry: the authors find no clear benefit to agents that operate outside human-defined constraints, but many foreseeable harms. Therefore the recommendation is not "no agents" but a governed spectrum of autonomy, with clear distinctions between levels to aid task delegation, governance, and development. This complements the empirical curve in Does targeted human intervention outperform both full autonomy and exhaustive oversight? — that note shows where to keep humans, this one supplies the normative argument for why the top of the autonomy ladder should stay unbuilt. It also sharpens Does machine agency exist on a spectrum rather than binary? by adding a value judgment to the spectrum: not all levels are equally worth reaching.