Did smaller, cheaper AI training runs use a wider mix of chip types than the biggest, most expensive ones?
Did non-frontier AI training spread across more chip designs than frontier training?
This explores whether AI models trained outside the frontier (smaller, cheaper, less cutting-edge runs) were spread across a wider variety of chip designs than the biggest frontier training runs, which seem to have concentrated on a few chip types.
This explores whether training below the frontier spread across more kinds of chips than frontier training did. The short answer: the collection can't settle this. Its hardware evidence comes from Epoch AI's analysis of 149 frontier models only, so it describes one side of the comparison and says almost nothing about the other.
What it does show about the frontier is striking. Before NVIDIA's V100, each chip design trained about 2.1 frontier models on average. After the V100, that average rose to 9.3 per design Did AI chip designs consolidate after the V100?. Frontier training didn't just grow. It piled onto a small set of chips, which fits a move from scattered academic experiments to large industrial training runs where labs standardize on proven hardware. The same chips also stayed in use longer: V100-era and newer designs kept training frontier models for a median of about 3.9 years after release, compared with 2.6 years for older designs How long do leading AI chips stay in frontier training?. Fewer chip designs, each used more heavily and for longer, is the frontier pattern.
The premise behind your question is a reasonable guess: if the frontier narrowed, did everyone else stay spread out? That would make sense. Smaller labs, universities and hobbyists tend to use whatever hardware they can get, including older GPUs, consumer cards and cloud instances of different generations. But the collection has no chip-level data on non-frontier models, so this stays a hypothesis here, not a finding.
There's a related point that makes the question more pressing than it first looks. Frontier-level ability no longer always requires frontier-scale training. One note shows a 3-billion-parameter model reaching top-tier math and coding scores through careful post-training rather than sheer size, at least on tasks where answers can be checked automatically Can small models match frontier reasoning without massive scale?. If capable models can come from smaller runs, the hardware those runs use, whether more varied or not, starts to matter for who can build competitive AI. The collection doesn't yet connect these two threads. For now, the solid finding is about frontier consolidation, and the non-frontier comparison is an open gap in the collection.
Sources 3 notes
Epoch AI's analysis of 149 frontier models shows post-V100 chip designs averaged 9.3 trained models each, versus 2.1 for pre-V100 designs, indicating a shift toward large-scale industrial training runs on narrower hardware choices.
Epoch AI's analysis of 149 frontier models shows V100-era and newer chips have a median frontier lifespan of 3.9 years from release to last use, though pre-V100 designs lasted only 2.6 years median.
A 3B model trained with curriculum SFT and multi-domain RL reaches 94.3 AIME26 and 80.2 LiveCodeBench scores matching much larger systems. The result is bounded to verifiable tasks with checkable ground truth, where RL can provide clean reward signals.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Leading AI chip designs are used for around four years in frontier training
- The rising costs of training frontier AI models
- PostTrainBench: Can LLM Agents Automate LLM Post-Training?
- AI Compute Architecture and Evolution Trends
- The plunging price of thought
- The Price of Progress: Price Performance and the Future of AI
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
- RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs