The human brain is a neural network that operates on less than 100 Watts of energy. So it’s obviously not physically impossible.
There’s no scaling law that invalidates general intelligence, only ones that invalidate the exact, unmodified techniques we use today. At any time someone could find a more efficient way to do things and turn the whole world upside down.
I figured that’s what he meant.
Seems like a bit of a leap to me.(edit: slight wording adjustment).
Just because we understand one part of something and can recreate that part, however seemingly fundamental, in isolation, doesn’t mean we can recreate the entire thing using only that.
It’s like saying we have evidence that fluid pipe systems can replicate a circulatory system. There’s so much more to it, systems within systems, and when it comes to the brain and intelligence, we understand far less than we do about the circulatory system.
Maybe that’s a bad parallel but hopefully my point comes across. (I say this as a total layman)
what’s the difference between a single neural network and multiple neural networks? are BERTs multiple neural networks working together in concert, if you count the encoder and decoder separately? the different layers of a MoE? certainly VLMs count.
You jest, but only parts of the brain are responsible for decoding auditory signals. It doesn’t require weighted input from the whole brain to process these signals. In theory, one could surgically remove Brocha’s and Werniche’s areas, and they would function completely independent of the rest of the brain.
My understanding is that you cannot identify a region of chatgpt that identifies plant leaves and separate it from the rest of the neural network without destroying its function. In this sense, the entire set of billions of weights is required for each function that chatgpt does.
The brain has many separate components that function independent of each other, each of which is a fully functional neural network, all working in tandem.
The human brain is a neural network that operates on less than 100 Watts of energy. So it’s obviously not physically impossible.
There’s no scaling law that invalidates general intelligence, only ones that invalidate the exact, unmodified techniques we use today. At any time someone could find a more efficient way to do things and turn the whole world upside down.
I figured that’s what he meant. Seems like a bit of a leap to me.(edit: slight wording adjustment).
Just because we understand one part of something and can recreate that part, however seemingly fundamental, in isolation, doesn’t mean we can recreate the entire thing using only that.
It’s like saying we have evidence that fluid pipe systems can replicate a circulatory system. There’s so much more to it, systems within systems, and when it comes to the brain and intelligence, we understand far less than we do about the circulatory system.
Maybe that’s a bad parallel but hopefully my point comes across. (I say this as a total layman)
Exactly. I would argue that brains are multiple neural networks working together in concert, however.
what’s the difference between a single neural network and multiple neural networks? are BERTs multiple neural networks working together in concert, if you count the encoder and decoder separately? the different layers of a MoE? certainly VLMs count.
A neural network network?!
You jest, but only parts of the brain are responsible for decoding auditory signals. It doesn’t require weighted input from the whole brain to process these signals. In theory, one could surgically remove Brocha’s and Werniche’s areas, and they would function completely independent of the rest of the brain.
My understanding is that you cannot identify a region of chatgpt that identifies plant leaves and separate it from the rest of the neural network without destroying its function. In this sense, the entire set of billions of weights is required for each function that chatgpt does.
The brain has many separate components that function independent of each other, each of which is a fully functional neural network, all working in tandem.