Google and Amazon build their own AI chips — could custom hardware break Nvidia's grip on frontier AI training?

This explores whether custom chips and new model architectures could undo the narrowing of frontier AI training onto a few chip designs. First, a limit: the corpus has no paper that studies custom ASICs directly, so it can't say whether TPU-style chips are winning share. What it does have is evidence about where the hardware bottleneck sits, and that changes the question in a useful way.

The consolidation is real and large. Epoch AI's analysis of 149 frontier models found that chip designs released after the Nvidia V100 each trained an average of 9.3 frontier models, against 2.1 for earlier designs Did AI chip designs consolidate after the V100?. That points to industrial-scale training runs settling on a few proven platforms. Their cost data, though, suggests the design of the processor itself may matter less than people assume. High-bandwidth memory (HBM), the stacked memory placed beside the processor, rose from 52% to 63% of AI chip component spending between early 2024 and late 2025. HBM spending nearly tripled while spending on the logic, the part a custom ASIC actually redesigns, stayed flat Is memory becoming the dominant cost in AI chips?. So a company can design its own processor and still depend on the same tight memory supply as everyone else. Consolidation may simply move from chip design to memory supply.

The model-architecture research points the same way. Several lines of work argue that memory is now the main scaling constraint. One synthesis argues that restructuring how models store and access memory now pays off more than adding parameters Has memory architecture replaced parameter count as the scaling frontier?. MobileLLM found that on phones it is cheaper to run the same transformer block twice than to fetch a second set of weights, because moving weights costs more than computing with them Does recomputing weights cost less than moving them on mobile?. Selective layer looping uses the same idea at training time, reusing layers instead of adding new ones and matching standard models with 3.3× fewer training FLOPs Can looping layers beat adding depth in diffusion models?. These designs reward hardware that holds weights close and recomputes cheaply. That is the kind of specific workload profile a custom chip can be built around, so new architectures could open room for specialized designs.

The strongest case for more hardware variety may be a change in where compute gets spent. Snell et al. showed that a smaller model given more inference-time compute can match a larger model on hard prompts Can inference compute replace scaling up model size?. Separately, PEFT adapters let one shared base model serve millions of lightweight personalized variants Can lightweight adapters replace millions of personalized models?. Both move the workload away from huge one-off training runs, where consolidation is strongest, toward serving and inference. Inference workloads are more varied and more cost-sensitive, so they leave more room for different chips. One plausible outcome is that frontier training stays consolidated while inference spreads across many chip designs.

The takeaway: the useful question isn't really "custom chips vs. Nvidia." It's whether anyone can escape the memory bottleneck. On the corpus's evidence, a new processor design doesn't by itself; changes in model architecture and a shift of compute toward inference might.