Can tweaking a network's wiring alone cure neurons that each mean several unrelated things, no special training trick needed?

This explores whether changing a network's design, rather than adding a sparsity penalty during training, can produce neurons that each respond to one concept instead of several unrelated ones (polysemanticity). The most direct evidence in the collection says yes, and it also questions why polysemanticity happens in the first place. The usual explanation is superposition: a network has more concepts to store than neurons, so it packs several into each one. Cammarata argues that this mostly isn't what's happening. The argument is that polysemantic neurons come from a branching, recursive kind of generalization during training, which is a learning failure that can be fixed. The evidence is striking. Architectural fixes to InceptionV1, an image model, raised the share of interpretable neurons from almost zero to 99–100% Does recursive generalization explain polysemantic neurons better than superposition?. If that holds up, the mixing isn't a fixed cost of storing many concepts in a small space. It's a side effect of how the model learned, and design choices can steer it away.

Other work points the same way from a different angle: networks seem to organize themselves into clean parts without being told to. Pruning experiments show that when a task is made of distinct sub-steps, networks often handle each sub-step in its own isolated subnetwork. Switching off one of these subnetworks breaks only its matching function Do neural networks naturally learn modular compositional structure?. Pretraining makes this separation more consistent across architectures. So the tendency toward tidy internal structure is already there, and the open question is what gets in its way.

Sparsity also turns out not to need imposing from outside. Language models make their own hidden states sparser when a task gets unfamiliar or harder. This happens in specific, consistent places in the network and seems to help keep performance stable rather than being a sign of breakdown Do language models sparsify their activations under difficult tasks?. That suggests a model can find sparse, selective representations by itself when the situation calls for it, which is the behavior explicit sparsity training tries to force.

The collection has a clear gap here. The InceptionV1 result is about a vision model, and none of the notes here tests a specific architectural change on large language models to see whether neurons become cleaner. Some nearby design choices look promising but are untested for this purpose, such as deep-and-thin layouts that build concepts up layer by layer Does depth matter more than width for tiny language models?. What you can take away is a change in framing. If polysemanticity is mostly a learning failure rather than a storage limit, then the tools now used to untangle neurons after training may be working around a problem that better design could prevent.