Nick Cammarata: “The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread … Neural networks are already clean”
translated by AI, reviewed by the editors
· Science · interpretability researcher at Goodfire who started the automated neuron interpretability project at OpenAI, in the essay “Neural Networks Are Already Clean” (a working hypothesis, not peer-reviewed; a companion paper is promised); Neel Nanda and other researchers pushed back
original
“I think I have good news about the nature of neural networks. After many months studying neurons and circuits in image and language models, I've come to believe a network's own mechanisms are its legible building blocks, and that the outlook for mechanistic interpretability may be far better than I'd feared. The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread. What looked like arbitrary packing was mostly a motif I'll call recursive generalization, which is much less concerning. This means we can study networks directly, instead of first untangling them. Neural networks are already clean.”— Nick Cammarata, Goodfire, 15.10.2026 ↗
backs
Transformer+Web text → Large language model (2018)
verification
✓ checked against the source
The original is given verbatim and only as an excerpt — as much as the meaning needs; “[…]” marks a cut. The translation is ours.
¶ cgij8q2um