A decade of AI research called most individual neurons hopelessly tangled — then one researcher found 90%+ do one clear job. What changed, the models or the microscope?

This explores why neuron interpretability, meaning whether you can say what a single neuron does, went from looking nearly hopeless to looking mostly solved. The corpus gives two answers. Interestingly, both come from the same researcher, and they point in different directions. For large language models, Cammarata says the models didn't change, but our tools for inspecting them did. His audits of models up to trillions of parameters found that roughly 90–97% of neurons do one main job or close variations of it. That is a sharp break from 2023 benchmarks that put the figure near zero How interpretable are neurons in today's large language models?. On this account the neurons may have been fairly clean all along, and earlier methods just couldn't see it.

The second answer comes from vision models, and there the architecture did matter. In InceptionV1, an older image model, changes to the architecture raised the share of interpretable neurons from almost none to 99–100% Does recursive generalization explain polysemantic neurons better than superposition?. This is also where the bigger idea sits. The usual story for 'polysemantic' neurons (single neurons that respond to several unrelated things) is superposition: the network crams more concepts than it has neurons into the space it has. Cammarata argues instead that messy neurons come from a learning failure he calls recursive generalization, where concepts branch off one another during training. If that's right, messy neurons aren't a law of nature. They are a bug you can engineer away.

Other work in the corpus supports the idea that clean internals can be built in deliberately. Training transformers with sparse weights, meaning most connections are forced to zero, produces small circuits whose neurons map to simple concepts. Knocking those circuits out confirms they really do the work, though nobody has yet kept this up beyond tens of millions of parameters Can sparse weight training make neural networks interpretable by design?. Separately, ordinary networks seem to split compositional tasks into separate subnetworks on their own, and pretraining makes that split more reliable Do neural networks naturally learn modular compositional structure?. So the structure researchers hope to find may already be forming during training.

There is also a reason for caution, and it's the part most readers won't expect. Neurons that are easy to read don't prove the model is organized sensibly. Research on 'fractured entangled representations' shows that networks trained the standard way can give exactly the same outputs as a cleanly organized network while being badly scrambled inside. You only see the difference when you perturb the weights and watch what breaks Can identical outputs hide broken internal representations? Can AI pass every test while understanding nothing?. A 90% neuron score tells you whether the parts are readable. It doesn't tell you whether the way they're put together makes sense.

The main takeaway is that 'interpretability' is partly a measurement, not just a property of the model. The jump may come from better tools (the LLM case), better architecture (the InceptionV1 case), or both. The corpus doesn't yet have independent replications of Cammarata's figures, so treat the 90%+ number as a strong claim from one research program, not a settled consensus.