Why do some neurons stay sharply focused on one idea while others blur several unrelated things together?

This explores whether recursive generalization explains how neurons end up with single, interpretable jobs. In the corpus, the argument actually runs the other way. Cammarata uses recursive generalization to explain the *failures*, the neurons that respond to several unrelated things. Clean neurons are what you get when that failure is avoided. The standard story says polysemantic neurons come from superposition: the network has more concepts than neurons, so it packs several concepts into each one. Cammarata says that's mostly not what happens. Instead, during training, features grow outward like branches, each generalizing from the one before, and sometimes a branch drifts into unrelated territory inside the same neuron. The key point is that this is a fixable learning failure rather than a hard capacity limit. When architectural changes were made to InceptionV1, an older image model, the share of interpretable neurons went from almost zero to 99–100% Does recursive generalization explain polysemantic neurons better than superposition?.

Here's the surprise. The same researcher audited language models up to trillions of parameters and found that roughly 90–97% of neurons already do one main job, or close variations of it. In 2023 the benchmarks put that number near zero. He credits the change to better analysis tools, not to changes in the models How interpretable are neurons in today's large language models?. Put the two findings side by side and you get an uncomfortable possibility. Some of the "messiness" that motivated superposition theory may have come partly from the tools we used to look, and less from the networks themselves.

Other lines of work reach clean structure by different routes. Pruning experiments show networks naturally split compositional tasks into separate subnetworks: knock one out and only its matching function breaks. Pretraining makes this split more consistent Do neural networks naturally learn modular compositional structure?. Other researchers force cleanliness on purpose. Training transformers with sparse weights, where most connections are zero, produces small circuits whose neurons map to simple concepts. The catch is that nobody has yet kept that interpretability above tens of millions of parameters Can sparse weight training make neural networks interpretable by design?. Kolmogorov-Arnold Networks redesign the basic building block so interpretability is built in from the start Can learnable spline activations beat fixed MLP designs?. So clean neurons can show up on their own, be encouraged by the architecture, or be imposed by constraints. Recursive generalization is one explanation for why the natural route sometimes goes wrong.

The nearby debate about compositionality, meaning whether networks build meaning by combining parts, adds a useful caution. Simple networks can learn to generalize compositionally through scale alone, as long as the training data covers enough combinations. Success is predicted by whether each part can be read cleanly from the network's internal activity Can neural networks learn compositional skills without symbolic mechanisms?. But transformers often succeed by memorizing chunks of computation they saw in training, and they fail on genuinely new combinations Do transformers actually learn systematic compositional reasoning?. That leaves an open question: does a neuron that does one main job reflect real understanding, or a well-organized memorized pattern?

The corpus has limits here. Cammarata's recursive-generalization account rests on a single source and was shown on a vision model. No note directly tests whether the same branching process explains the 90–97% clean neurons in language models. So the honest answer is that recursive generalization is a promising explanation for why neurons *fail* to stay single-purpose. It has not yet been shown to explain why they succeed.