Does recursive generalization explain polysemantic neurons better than superposition?

Nick Cammarata, in a research write-up published by Goodfire, argues that the polysemanticity long taken as evidence for the superposition hypothesis was mostly caused by "several ordinary, mostly fixable issues" rather than a second learning principle. He writes "the cause of polysemanticity in early model organisms wasn't superposition, but several ordinary, mostly fixable issues" and that "what looked like arbitrary packing was mostly a motif I'll call recursive generalization, which is much less concerning." His central claim, captured in the essay's title, is that "neural networks are already clean" — legible at the level of individual neurons, without first needing to be decomposed by a second model such as a sparse autoencoder.

Cammarata's mechanism for most apparent polysemanticity is recursive generalization: a neuron grows the way "a tree branch sprouts sub-branches," picking up related but non-identical senses over training rather than folding together unrelated concepts for lack of room. His example is a neuron that detected the string "iy," which grew to also detect "diy" and then "do it yourself" — a semantic drift with no literal string overlap. He revisits the field's most cited counterexample, InceptionV1's neuron 4e:55, previously described in Zoom In (which he co-authored) as an arbitrary "OR(cat, car)" unit; he now reports that architectural changes aimed at helping deep layers learn better — with no change to model size or training data — pushed interpretable neurons in that last layer from "almost none" to "99-100%." He concludes the model was never "knotted"; its overgrowth was a fixable ordinary learning failure, not evidence of a second, superposition-based learning principle.

This argument pushes directly against the premise behind Can sparse weight training make neural networks interpretable by design?, which treats legible neurons as something a model must be trained toward by explicitly zeroing out most weights, trading capability for it. Cammarata's claim, if it holds, removes the need for that trade: ordinary dense and mixture-of-experts models, with better tooling, are reportedly already landing at high single-concept interpretability (detailed in the companion note, How interpretable are neurons in today's large language models?) without being trained toward sparsity at all.

The essay is a company research write-up, not a peer-reviewed paper, and Cammarata flags his own account as partial: he calls the broader claim "highly speculative," and the InceptionV1 retraining is a single case study on one architecture family rather than a controlled test across model types. Whether recursive generalization explains most polysemanticity in the field's harder cases — rather than InceptionV1 and the hand-picked neuron examples described here — is not established by this excerpt; the implication the essay draws, that mechanistic interpretability may be far more tractable than assumed, should be read as Cammarata's working hypothesis rather than a settled finding.