<\/p>\n
The researchers concluded that the model was getting at least some of its neurons to multitask, perhaps so that it could fit in more data. Looking at other models appeared to back this up. The phenomenon became known as superposition: Models folded unrelated concepts into single neurons. The upshot was clear: When you looked inside a model, what you saw was a vast, tangled knot.<\/p>\n
Most mechanistic interpretability research has therefore focused on ways to untangle that knot. But despite a few breakthroughs, progress has been slow. Today\u2019s models, such as the latest LLMs, are so large and complex that researchers can make sense of only tiny pieces at a time. Some leading figures in the field doubt if it will ever be possible to understand an LLM in fine-grained detail.<\/p>\n
Cammarata was one of them. \u201cI was very pessimistic,\u201d he says. But then he took the trouble to really look.<\/p>\n
At Goodfire, Cammarata spends nearly all of his time peering inside neural networks. He has built new tools and deploys teams of AI agents to help him make sense of what he sees. \u201cI spent like, I don’t know, 10 hours a day, seven days a week, for most of the last year just looking at neurons,\u201d he says.<\/p>\n
What he claims to have realized is that there is a kind of logic to neural networks after all, even if they seem to make no sense at first. \u201cWe were looking at it wrong,\u201d he says. \u201cI just think you want to try slightly harder; you don’t want to give up too quickly.\u201d<\/p>\n
Cammarata almost makes it sound like a conversion of faith. Those early experiments with models such as Inception v1, which suggested that neural networks were tangled up, fostered a kind of confirmation bias. When researchers looked at a new model, they expected to be flummoxed, he says. \u201cEvery time I couldn\u2019t understand something, I was like, \u2018Wow, this untangling process is even more important than I thought, because I can\u2019t make any progress without it.\u2019\u201d<\/p>\n
But the more Cammarata looked at neurons, he says, the more things started to fall into place. \u201cNot all of them are flawless, but I expected nothing to be legible, or very little to be legible,\u201d he says. \u201cAnd actually, I was able to, like, map out whole circuitry.\u201d<\/p>\n
Cammarata took his new approach\u2014and his new agents\u2014and went back to Inception v1. \u201cI\u2019d studied that for probably thousands of hours,\u201d he says. But with fresh eyes, Cammarata found things that he now thinks they got wrong the first time around. Take the car-and-cat neuron. Rather than asking why multiple unrelated things were packed into a single neuron, he now tried to figure out what those things might have in common.<\/p>\n<\/p><\/div>\n
#wrong #explosive #claim #upended #world<\/p>\n","protected":false},"excerpt":{"rendered":"
The researchers concluded that the model was getting at least some of its neurons to…<\/p>\n","protected":false},"author":1,"featured_media":24345,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[729,9346,14788,920,1741],"class_list":["post-24344","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-stories","tag-claim","tag-explosive","tag-upended","tag-world","tag-wrong"],"featured_image_urls":{"full":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2.jpg",1200,600,false],"thumbnail":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2-150x150.jpg",150,150,true],"medium":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2-300x150.jpg",300,150,true],"medium_large":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2-768x384.jpg",640,320,true],"large":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2-1024x512.jpg",640,320,true],"1536x1536":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2.jpg",1200,600,false],"2048x2048":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2.jpg",1200,600,false],"covernews-featured":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2-1024x512.jpg",1024,512,true],"covernews-medium":["https:\/\/news678.top\/wp-content\/uploads\/2026\/10\/261016_LLMsNotcomplicated_V2-540x340.jpg",540,340,true]},"author_info":{"display_name":"admin","author_link":"https:\/\/news678.top\/?author=1"},"category_info":"Stories<\/a>","tag_info":"Stories","comment_count":"0","_links":{"self":[{"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/posts\/24344","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=24344"}],"version-history":[{"count":0,"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/posts\/24344\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/media\/24345"}],"wp:attachment":[{"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=24344"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=24344"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=24344"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}