Simply because its interior is a black box doesnāt mean we donāt understand how we built that black box, or how it operates and functions.
Wait a sec. I think weāre saying the same thing here. I guess depending on what you mean by how it operates and functions. Iāve said multiple times we understand the math and the code. We understand how values propagate through it because again, thatās all the math and code people wrote. What we donāt understand is how it uses that math and code to actually do thinks that seem intelligent (putting aside the point of whether it is or is not intelligent). If thatās what youāre arguing then great, weāre on the same page!
I also canāt inspect the electrons moving through my computerās CPU. Does that mean we donāt understand how computers work? Is there intelligence in there?
Well, I donāt have the equipment to look at electrons either (I donāt think that tech exists), but I can take a logic probe and get some information that I could probably understand, or someone who designs CPUs could look at the gates and whatever and tell you what they did and how they relate to whatever higher level operations. Youāre bringing up something completely different here. Computers are not a black box at all. LLMs are-- you just said that yourself.
No, that is not my main objection. It is your anthropomorphization of data and LLMs
Iām not anthropomorphisizing them. What are you talking about? I keep saying they donāt work like human brains. I just said I donāt think theyāre sentient or conscious. I said they donāt have agency.
I think youāre getting caught up in trying to define what intelligence is; but I am simply stating what it is not.
How do you know what itās not if we canāt define what it is?
It is not a complex statistical model with no self-awareness, no semantic understanding, no ability to learn, no emotional or ethical dimensionality, no qualiaā¦
Juryās still out on whether human brains are complex statistical models. I mean (from here)ā¦
Our brains have learned, through evolution and experience, the statistical properties of our natural environments and exploit this knowledge when performing perceptual tasks.
I donāt make any claim to understanding neuroscience, and I donāt think that article is saying for sure we know that.
Anyway, in-context learning is a thing for LLMs. Maybe one day weāll figure out how to have them adjust their weights after training, but thatās not happening now (well people are experimenting with it).
New research is showing they do have semantic understanding.
They donāt by themselves have self-awareness, but a software framework built up around them can generally do that to some extent.
They do understand emotions and ethics. Someone built a fun GPTrolley web site a while ago. I think it died pretty quickly because it was too expensive for them, but it had GPT 3(?) answering Trolley Problem questions. It did (in my memory of it) like to save any āAGIā on one track over humans, which was amusing. They donāt have emotions, no. Does something have to have emotions to be intelligent?
And no, Iāve said all along they arenāt conscious, so no qualia. Again, is that required for intelligence?
This is the crux of the problem: it is not a āsquareā to a computer because a āsquareā is a human classification. Your thoughts about squares are not just more robust than GPTās, they are a different kind of thing altogether. For GPT, a square is a token that it has been trained to use in a context-appropriate manner with no idea of what it represents. It lacks semantic understanding of squares. As do all computers.
No. A square to GPTs is not just a token. Itās associated with some meaning. Iām not going to re-hash embedding and word vectors and whatever since I feel like Iāve explained that to death.
If youāre saying that intelligence and understanding is limited to the human mind, then please point to some non-religious literature that backs up your assertion.
Iām disappointed that youāre asking me to prove a negative.
Iām literally not. āIntelligence is limited to the human mindā is not a negative.
The burden of proof is on you to show that GPT4 is actually intelligent. I donāt believe intelligence and understanding are for humans only; animals clearly show it too. But GPT4 does not.
I feel like Iāve laid out my argument for that mostly through the Microsoft and Max Tegmark papers. Are you saying intelligence is only the domain of biological life?
Hereās a question-- are you conflating āintelligenceā with āgeneral intelligenceā like AGI? I find a lot of people think āAIā means āAGI.ā It doesnāt help that some people do say those things interchangeably. I was just reading a recent argument between Yann LeCun and Yoshua Bengio and they were both totally doing that. Anyway, I donāt at all believe GPT4 is AGI or that LLMs could even be AGI.
For an overview of how many different kinds of LLMs function, hereās a good paper: https://arxiv.org/pdf/2307.06435.pdf
Looks like a great paper-- I hadnāt seen it yet. I know how LLMs are constructed (generally-- while I could go and write some code for a multi-layer neural network with back propagation without looking anything up, I couldnāt do that for an LLM without looking at a diagram of the layers or whatnot).
My argument?
Iām not saying itās thinking or has thoughts. Iām saying I donāt know the answer to that, but if it is it definitely isnāt anything like human thoughts.