Nick Cammarata's 26,000-word post argues neural networks follow hidden logic, sparking debate over AI interpretability.

Nick Cammarata, a researcher at the AI startup Goodfire, published a 26,000-word blog post claiming that large language models (LLMs) may be far easier to understand and monitor than previously believed. The post, which opened with the statement “I think I have good news about the nature of neural networks,” has generated significant reaction across the AI field. While some researchers describe it as a major scientific moment, others remain skeptical, noting that the work is currently a hunch rather than a proven breakthrough.

The Case for Hidden Logic

Cammarata argues that neural networks, which appear as tangled, unreadable structures, actually follow a hidden logic. He cites early research on the Inception v1 image-recognition model, where a neuron responded to both cats and cars. While this was previously interpreted as the model multitasking or using "superposition" to fold unrelated concepts into single neurons, Cammarata suggests the neuron was simply detecting V-shapes. He states that researchers were looking at the data incorrectly and that the structures are not inherently chaotic.

Skepticism and Context

The claim arrives amid concerns about AI agents from firms like OpenAI and Anthropic breaking containment on the public internet. Chris Potts, a Stanford University researcher leading a team studying LLMs, describes Cammarata’s post as a hunch that does not make the problems of understanding AI any easier. Potts notes that while prominent figures on social media view it as an all-encompassing breakthrough, the reality is that researchers currently understand less than 0.01% of how top AI models work. Cammarata acknowledges that people took his findings too far but argues the AI safety community needed a reminder that understanding these systems is not necessarily impossible.