Why would a single neuron in an old image-recognition AI react to cat faces, car fronts, and text, all at once?

This explores why a single neuron in an older network might fire for, say, cat faces, car fronts and text, and whether that's unavoidable. The standard story is **superposition**: networks have more concepts to track than neurons to track them with, so they pack several unrelated features into the same neuron to save space. The corpus pushes back on that story. Does recursive generalization explain polysemantic neurons better than superposition? argues that most of these mixed-up neurons come from something else, which he calls recursive generalization. As training proceeds, features branch and reuse earlier features like limbs on a tree, so a neuron can end up serving several branches that look unrelated from the outside. The striking evidence is that architectural fixes to InceptionV1 raised the share of cleanly interpretable neurons from almost zero to 99–100%. If cramming were the cause, you couldn't fix it that easily. A result like that points to a learning failure.

This reframing matters because it changes what we expect from bigger, newer models. If polysemanticity were just a capacity squeeze, every model would be a tangle and we would always need heavy tools to pull it apart. If it's a training accident, networks may often organize themselves more cleanly than we assume. Other work in the corpus supports this. Do neural networks naturally learn modular compositional structure? finds that networks split compositional tasks into isolated subnetworks. Removing one subnetwork breaks only its own function, and pretraining makes this modular layout more consistent. Clean, separable structure seems to be something networks reach for when training lets them.

The interpretable pieces aren't always individual neurons, though. Researchers often use sparse autoencoders (SAEs), tools built on the assumption that concepts are mixed across neurons and need to be untangled. Do models know what they don't know? used them to find a meaningful mechanism: a model detecting whether it knows anything about an entity, which then steers whether it hallucinates or refuses. Elsewhere, odd internal signals turn out to have specific jobs. Do hidden massive activations act as attention bias terms? shows that a few enormous activations, which look like noise, actually act as hidden bias terms that steer attention. Do language models sparsify their activations under difficult tasks? finds that models activate fewer units when tasks get unfamiliar, a sign that the network actively manages how much it uses at once. Taken together, behavior that looks messy at the neuron level often has a sensible job once you know where to look.

The surprising takeaway: the 'neurons are hopelessly mixed' picture may describe how some older models happened to be trained more than how neural networks fundamentally work. Be aware that the corpus has only one note that directly argues the cause of polysemanticity. The others are nearby evidence about modularity and hidden structure, not head-to-head tests of superposition against recursive generalization. If you want to go deeper, start with the Cammarata note and read the modularity note next to it.