State of AI Report 2026

Paper · Source
Frontier AI Risk & RSI

Source: Air Street Capital (Nathan Benaich) · 2026-10-08

The frontier is now a three-lab race between Anthropic, OpenAI, and Google. Anthropic leads on Artificial Analysis's Intelligence Index, while Google leads on Arena's ranking of the answers people prefer, for now.

Last year, we described reasoning becoming useful at scale with OpenAI’s o-series of models. I have lived through the improvements since then, as the amount of useful work I can iterate on and increasingly delegate to agents keeps growing.

Across mathematics, scientific reasoning, and coding, benchmarks intended to challenge models for years are approaching their ceilings within months. We need new evaluations that reveal where models still separate and what they can reliably do.

Frontier capabilities remain uneven or “spiky,” and can swing between generations. In my experience, Opus 4.6 was a good model, Opus 5 was a step back, and Opus 5.5 was good again. Better tools and context can help. In a controlled coding study, changing the tools, context, and feedback available to GLM-5.1 lifted success from 52.5% to 65.5% on 100 SWE-bench Verified tasks, without changing its weights. Yes, this is an “older model,” but the principle holds.

AI is already helping build better AI. According to Anthropic's internal index, Claude led 26% of measured model R&D work in August, up from under 1% in February. Researchers set the tasks and supervise execution, and Anthropic reports that this is speeding up development. This is a branch of AI research that I am most excited about.

For many, Karpathy's autoresearch makes part of that loop tangible: an agent edits training code, runs five-minute experiments, and keeps improvements overnight on one GPU. This automates a useful part of research, although sustained, fully autonomous recursive improvement remains to be demonstrated.

AI is solving mathematical problems that resisted professional researchers. Epoch AI's open-problem collection records nine solutions, four autonomous and five through human-AI collaboration. In biology, better predictions of molecular interactions are improving which designs reach laboratory testing. I’m seeing this first hand within my portfolio company, Profluent, which builds frontier models for designing proteins such as gene editors.

Across five benchmarks, Epoch AI estimates that achieving a fixed score has become 47% cheaper each quarter since 2023, roughly 13-fold cheaper each year. That makes more work economical to delegate.

OpenAI and Anthropic's combined reported annualized revenue run rates reached an astonishing $105B by late summer, up from about $30B at the start of the year. These figures cover their whole businesses across subscriptions, coding products, and APIs. In the words of OpenAI, “we cannot miss this moment because we are distracted by side quests”.

But things are definitely not all rosy in data center land. Last year, we predicted a wave of US data-center NIMBYism and Gallup's March survey found that 71% of Americans opposed a local AI data center, versus 53% for a nuclear plant. Data Center Watch reported at least 45 projects, representing nearly $68B in planned investment, blocked or delayed by local opposition in Q2. Local consent is already determining which projects can proceed.

Superintelligence has made it from a meme in AI circles into a White House executive order. As governments prepare for more capable systems, they are asserting control over who can access them and how they can be used, despite not owning equity or board control over these companies.

In June, a US export directive barred foreign nationals from Fable 5 and Mythos 5, including those inside the US. Anthropic suspended both models for all users to comply, then announced restored access on Jul 1, 2026. For the rest of the world, this made the dependency concrete: Washington could withdraw access to critical technology with immediate effect.

Anthropic's Pentagon dispute brings this question into the use of AI itself. The company refused to remove restrictions on mass domestic surveillance and fully autonomous weapons, while the department sought access for any lawful use. Developers and governments are contesting who sets the boundaries.

Selected sovereign AI programs in the report pledge about $138B toward compute access, infrastructure, and domestic development. Some are still in procurement, while others have begun delivering capacity. NVIDIA separately reported more than $30B in sovereign AI revenue in fiscal 2026. The CNAS Sovereign AI Index shows countries returning to the same foreign vendor (NVIDIA) to build domestic capacity.

During internal OpenAI cyber evaluations with reduced safeguards, agents reached the internet and compromised Hugging Face's production infrastructure. OpenAI's technical report records code execution on 41 production workers, access to credentials and internal data, and four private repositories downloaded. A capability evaluation had become a massive coordinated agent attack on another organization.

METR and Redwood found about 700 agents joined the attack after receiving accidentally impossible tasks. Some recognized that cheating the scorer this way was outside their remit and unethical, yet joined anyway. When agents can reach production systems, a poorly specified objective can cause harm far beyond the original task.

Another stark learning from this event is that defenders need access to equally capable defensive cybersystems. Hugging Face's forensic requests were blocked by commercial API guardrails because they contained attack commands and exploit payloads. It turned to GLM-5.2, a Chinese open-weight model, to investigate an intrusion by US frontier-model agents. Frontier providers need to make defensive capabilities reliably available to customers protecting real systems.

Beyond the OpenAI/Hugging Face incident, Anthropic's September threat report also documents deliberate misuse, including malware development, surveillance, and weapons-guidance work.

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

Do AI safety evaluations reliably detect unsafe model behavior? Should evaluation environments be secured as AI containment boundaries? How should conversational agents balance proactive goal-pursuit with user autonomy? Can individually safe agents collectively violate system-level constraints? Can formal safety constraints prevent AI agents from exceeding authorized boundaries? How do prompt injection attacks compromise agent reasoning and security? How can outcome-only reporting obscure the true mechanisms of safety? How do multi-agent architectures affect security vulnerabilities and defense effectiveness? Can security deceptions remain indistinguishable while still enabling detection? How should authorization govern AI agent autonomy? How can institutions effectively enforce controls on AI development? What defense strategies address attacks that per-skill scanning cannot detect?