Loading summary
Transcript63 lines
- [00:01]
Geoffrey Hinton
For me, that's the most important question in neuroscience. How similar is the way the brain learns to how these large language models learn? And at a very abstract level, I believe it's quite similar.
- [00:15]
Jascha Monk
And now the good fight with Jascha Monk. My guest today is none other than Geoffrey Hinton, widely known as the godfather of AI. Geoffrey has invented some of the key architectural innovations that have made the current AI boom possible. For example, he was one of the people who figured out how to use the principle of back propagation in such a way as to allow deep learning. He is also the winner of the 2024 Nobel Prize in Physics for foundational discoveries and inventions that enable machine learning with artificial neural networks. Well, my agenda for this conversation was twofold. Firstly, for him to explain to us what that means. What is machine learning with artificial neural networks? And what exactly was his so important contribution to this field? Building on the recent episode with David Bau, I think of this as my 101 and 102 of understanding AI. And Jeffrey is, I think, very clear in helping us think conceptually about what this way of producing information and of enabling machines to think about the world actually consists in. Well, the second part of a conversation may be of interest to you, even if you're not that interested in the details of how AI works. We're asking all of the big questions. Is artificial intelligence actually intelligent? Is it going to take the jobs of a lot of people? How should we think about the different kind of risks that are involved in AI? Is it just about bad people doing things to us? Or is it also about bad AI doing bad things to us? And it's in the final part of a conversation, which is reserved for paying subscribers, that we really try to answer that final question. Why is it that Geoffrey Hinton takes concerns about AI destroying humanity very seriously? Why is it that but the goal architecture of AI systems and the evolutionary advantages of any AI system that is trying to reproduce itself creates such a serious risk for humanity. To listen to that part of the conversation, to support what we do here, to stop hitting these annoying paywalls at the end of the podcast, please become a paying subscriber. Please go to yashamonk.substack.com that's yashamonk.substack dot com. Geoffrey Hinton, welcome to a podcast.
- [03:25]
Geoffrey Hinton
Thank you for inviting me.
- [03:26]
Jascha Monk
You are known as the godfather of AI. AI has gone through these strange periods, periods when there was great excitement about AI even in the past. And then these AI winters when people thought that the Technical requirements for making AI work didn't yet exist. Or perhaps that the whole concept just was misguided and it wasn't ever going to work out in any way that's useful. Tell us about why it took so many run ups, so many attempts to get to the huge AI boom we have now, and the way in which AI, you know, whatever its future is, and I hope that we'll have a chance to talk more about that towards the end of our conversation, clearly is now integrated in all kinds of useful processes in the world.
- [04:10]
Geoffrey Hinton
So during the last century there was an approach, there were two approaches to AI. Basically the main approach that almost everybody took was to base it on logic. So they said, what's special about people is their ability to reason. They focused on the ability to reason and logic was their model for that. And that basically didn't work. It might have worked, but it didn't work very well. And that led to some AI winters. There was an alternative approach, which, which started in the 1950s with people like von Neumann and Turing, who unfortunately both died young, that was to base AI on neural networks, the biological inspiration rather than the logical inspiration. And the alternative approach said rather than trying to understand reasoning, you need to understand things like perception and intuition and motor control. And the way they work in the brain is by changing connection strengths between neurons in a neural network. And so we ought to try and understand how that's done and worry about reasoning later. And it turned out in the beginning of this century, that approach suddenly started working much better than the logic based approach. And almost everything we have now that's called AI, is not the old fashioned AI that uses logic, it's the new fashioned AI that uses neural networks.
- [05:31]
Jascha Monk
And everything is always obvious with the benefit of hindsight. But I guess if you are trying to figure out how to build a machine that is intelligent, from first principle, the logical approach would have seemed very intuitive, that we need to teach it that two plus two is four, and we need to teach it how certain physical things about the world work, and we need to teach it the basic rules of logic and then we throw a bunch of computation at it and it can come to conclusions that we might not be able to come to. Why is it you think that that approach failed? And what is it about this alternative approach that you were so key in championing that turned out to be more generative of useful technology?
- [06:13]
Geoffrey Hinton
So human thinking you can sort of divide into sequential, conscious, deliberate, logical reasoning, which involves effort, is what Danny Kahneman calls type 2. And then there's immediate intuition, which doesn't normally involve effort. And what the people were focusing on, who believed in symbolic AI, was type 2. It was conscious, deliberate reasoning without trying to solve the problem of how we do intuition, how we do analogies, how we do perception. And it turns out it's much better to start with how you do those things, which many animals can do too. They can do perception, they can do motor control. Once you solve that, then reasoning comes next. But they started basically from what was distinctly human rather than starting from sort of basic biology. How do other animals do it? Obviously we're just a jumped up ape and you need to understand how animals think.
- [07:24]
Jascha Monk
Yeah. One of the interesting things is that when we think about what is intelligence, we have a sort of bias where we tend to think about what makes us as human beings distinct from other species. What is that extra little last mile of intelligence that we have that those other animals can't have. But actually, a lot of that intelligence is built on top of skills that actually are incredibly complex. But they don't seem as remarkable to us because, you know, a cat has it, a lion has it, a dog has it, an elephant has it, right? Which is how to perceive what's going on in the world around us, how to make certain basic calculations about where to place our foot so we don't fall into an abyss. You know, how to, you know, perceive when a predator is approaching. All of those things are not what Kahneman calls Type 2 systems. And when we say, you know, what makes us intelligent, that's probably not the first question that comes to us because we share that with so many other animals. But that is actually in some ways the more remarkable achievement. And then you can go on to say, okay, so what do we need for that last extra mile?
- [08:25]
Geoffrey Hinton
Let me give you an example of a piece of thinking that you can't do with logic, you can do with intuition. And for most males in our culture, the answer is obvious. Turns out it's not so obvious for females in our culture, but for males in our culture, you'll find the answers obvious. Suppose I give you a choice between two alternatives, and neither alternative makes sense. Both alternatives are clearly nonsense, but one seems better than the other. So alternative one is that all dogs are female and all cats are male. An alternative two is that all dogs are male and all cats are female. Now, most males in our culture find it obvious that all dogs are male and all cats are female. That seems more natural. Dogs are loud and noisy and they chase after cats. And if you ask and that's just immediate. You didn't do any reasoning about that. It just felt right. It felt less wrong that way around than the other way around. So why is that? And you can't explain that with logic.
- [09:25]
Jascha Monk
The other example is presumably certain rules of language for perhaps those, you know, I know exactly how something like universal grammar fits into that. But the fact that we know it is the little warm red house and not the warm little red house or the warm red little house, you know, there's a particular kind of order of
- [09:46]
Geoffrey Hinton
that may depend on what language you speak. So I don't actually believe in universal grammar, and these large language models don't believe in it either. So the large language models are doing something that Chomsky would have said was impossible. In fact, he still says it's impossible, which is they start off with no innate knowledge of language, they just see a lot of language and they end up knowing grammar extremely well, so they didn't have to have any innate knowledge.
- [10:12]
Jascha Monk
So whatever the explanation for that is, and I didn't mean to make this a debate about universal grammar, it is both true that a competent speaker of English will put adjectives of size, color, kind, et cetera, in a particular kind of order without logically, it's not like you're thinking, oh, which adjectives should go where if you're learning the language, if you're not a native speaker, you may have learned the rule that grammarians over time have deduced and then think, oh, damn, does the little go before the blue or does the blue go before the little? As a 10 year old competent speaker of a language, you do it automatically and it turns out that ChatGPT does it automatically in some kind of way too. Whatever exactly automatic here means. But it doesn't. What isn't the case is, as you might have thought, that somebody gave ChatGPT the rules of in the English language, this kind of adjective goes before that kind of adjective. And yet ChatGPT, from all of the data that's thrown at it, deduces, oh, this is where I have to put the adjective of size, and this is where I have to put the adjective of color.
- [11:14]
Geoffrey Hinton
Yes, but what it shows is that you don't need any innate knowledge of language. You just need to see a lot of language and have a, have a fairly universal learning mechanism, which is just the opposite of what Chomsky said.
- [11:26]
Jascha Monk
Oh, that's very interesting. So Chomsky argues that there's certain kind of presets that get pushed in one direction or another, and that this is what allows us to do this. And what you're saying is that isn't necessary. All that's necessary is these neurons in our brain seeing a lot of data and detecting the patterns in that data without ever being explicitly told what that pattern is. Is that roughly right?
- [11:48]
Geoffrey Hinton
Yes, exactly. This example with cats and dogs shows that we have strong intuitions about things without even thinking about them. And the question is why? And the answer, according to people who work on neural networks, is that you have a representation of cat. So the meaning of the word cat is a large bunch of activated features, each feature corresponding to a neuron that's active. And so a cat is a thing that's animate, it's, it's hairy, it's about the size of a bread box, it's domestic, or it might be domestic. Dogs are another big bunch of features that overlap a lot. So dogs are quite similar to cats. But if you ask about the similarity between a cat and a woman versus a cat and a man, cat is more similar to woman for males in our culture, and dog is more similar to man. And so you could do analogies that way. It's just obvious that cat's more similar to woman than it is to man, and dog's more similar to man than it is to woman. And so that's what's going on. When you instantly know which way around it, it seems natural. That's very different from logical reasoning.
- [13:01]
Jascha Monk
And so explain to us how that works in the human brain and how you inspired in some ways by your knowledge of neuroscience, in thinking about a way of teaching machines those kinds of things without putting those hard coded logic rules into the machine in a way that has proven to not work.
- [13:26]
Geoffrey Hinton
So it's probably easiest to start by explaining it for doing visual perception. Once I've explained how you learn to do visual perception, then it's relatively trivial to see how you could learn language. So let's start with visual perception. And let's suppose that you have a lot of images that contain a bird and a lot of images that don't contain a bird. And you want to build a neural network that when you put in an image of a bird, it will activate the output that says bird. And when you put an image that's not of a bird, it'll activate the output that says not bird. And you have layers of neurons that are going to detect various kinds of features. So. And the kinds of features they detect were inspired by research on the brain, looking at what neurons in the brain get excited by so if you think what the task is, suppose we had a thousand by a thousand image, and let's suppose it was just a gray level image. To keep things simple, no colors for now, you've got a million numbers which tell you the brightness of each pixel in that thousand by thousand image. And so if you think of it in computational terms, I give you a million numbers and you have to say bird or not bird. And obviously those individual numbers aren't very helpful because a bird might be, it might be an ostrich about to peck you on the nose, or it might be a seagull in the far distance. They're both bird and obviously they're very different. So somehow you have to be able to deal with huge differences in what kind of bird it is, what pose it's in how big it is, where it is in the image, but still get all the birds and don't get any of the non birds. So the first thing you do in a vision system is you try and detect little bits of edge all over the image. And I'm going to tell you how a neural net would detect a little bit of edge. Suppose you had a little column of three pixels, and next to it on the left, and next to it on the right, another little column of three pixels. So three pixels vertically in a column and three pixels vertically next to them. So total six pixels. And you want to detect when the three pixels on the left are brighter than the three pixels on the right, because that would be an edge, a little piece of edge. So what you could do is you could have a neuron whose inputs come from those pixels, and you give it big positive inputs from the pixels on the left and big negative inputs from the pixels on the right. So if a pixel on the right is bright, it sends a big negative input to the neuron saying please don't turn on. If a pixel on the left is bright, it sends a big positive input saying, please turn on. So now if the pixels on the left and the pixels on the right are of equal brightness, the big negative input cancels out the big positive input and the pixels doesn't say anything. But if the pixels on the left are all bright and the pixels on the right are all dim, you get big positive inputs saying please turn on from the left. And you don't get anything from the right because all those pixels are turned off, they've got no values. And so this neuron turns on. So if you set the connection strengths, the weights on the connections that tell each pixel how to vote for whether the neuron should be on or off. If you set them right, you can make a little something that detects a little piece of edge. So to begin with, let's not worry about how we would learn this. Let's worry about how we would hand design it. So I've shown you how to hand design something that detects when the three pixels on the left are a bit brighter than the three pixels on the right. Now, you need to do that in all positions in the image. So you're going to need hundreds of thousands of these guys, and you need to do it in all different orientations in the image. So you're going to need millions of these guys, and you probably need to do it at all different scales. You need ones that detect small, sharp edges, like when you're reading black print on a white page. You also need ones that detect big fuzzy edges, like when you're looking at clouds, because clouds sort of had edges, but they're all fuzzy. So we've now got tens of millions of these neurons that are good for detecting edges anywhere in the image, at any orientation, at any scale. That's going to be our first layer of feature detectors. And when we put in an image, a small subset of those will get active, which will tell us where all the edges are in the image. That's still not very good for detecting birds. If I just tell you I've got a little piece of vertical edge here, is it a bird? Well, that doesn't really tell you whether it's a bird. So we're now going to have another layer.
- [18:24]
Jascha Monk
You have to understand how all these edges interrelate in some kind of way.
- [18:27]
Geoffrey Hinton
Exactly. So we're going to have a second layer of feature detectors that take as input these edges. So you might, for example, have a feature detector that's looking for a little row of edges that slope up slightly and a little row of edges that slope down slightly. So maybe three edges in a row that slope up and three edges in a row that slope down and that join at a point. And you need detectors like that all over the image, because if you see something like that, it might just be the beak of a bird. You don't know for sure it's the beak of a bird, but it's a little bit of evidence that it might be the beak of a bird. We might also have neurons in that layer that detect six edges that make a kind of ring, because if you detect something like that, it might just be the eye of a bird. Okay. So in the next Layer we'll detect things like possible beaks and possible eyes and maybe possible feet, maybe have something that looks for something that looks a bit like a chicken's foot and maybe also neurons that look for something that looks like the tip of the wing of a bird. Okay, so we have a whole bunch of neurons that are now detecting little features of birds that are typical of birds. Now in the next layer we might look for combinations of those. So we might, for example, in the next layer have a feature detector that looks for a possible beak and a possible eye that are in the right relative locations to be the head of a bird. The eye is kind of above the beak and slightly to one side. And so you have neurons that are looking for that all over the image. You can an awful lot of neurons to do this, but fortunately we have billions of them.
- [20:11]
Jascha Monk
So that's all very useful. But let me ask a few very simple questions, both to avoid misunderstandings and to get you to clarify some things. But I may get wrong, other listeners may get wrong. So the first is that the way you're describing this at the moment, it still feels a little bit like somebody is inputting a set of rules to the system. Right? It sounds a little bit like somebody is saying birds have beaks. And so therefore, and beaks look roughly like this. And so therefore we're going to design this system from first principles to look for beaks and to alert us when there's beaks and so on. But somehow the system learns to pick up features of birds by itself in the same way that ChatGPT didn't have somebody explain to it first go the adjectives of size and then go the adjectives of color or whichever way around it is, it picked up on that by itself. Right? So how is it that this system is picking up by itself? And oh look, it seems to be that I've seen these thousand pictures of birds and these thousand pictures of non birds, but and these thousand pictures of the bird have something in common which is some beak like feature. And so that's what I'm going to start looking out for. Right? You are not telling AI this. AI is somehow deducing that from the data that's been thrown at it. How does it do that?
- [21:22]
Geoffrey Hinton
Okay, so in order to explain that, it's good to start by saying if I was building it by hand, what would I build? Because we need to know sort of what the target of learning is. So I'm describing to you how I will build multiple layers of features so I could detect A bird. And I got to the layer where you're looking for a combination of a beak and an eye, and that might be the head of a bird. And in that layer you might have lots of things that detect the wing of a bird or the leg of a bird or the head of a bird. And then if you see several of those things, it's a pretty good bet it's a bird. So those things to begin with, the intensity of an individual pixel wasn't evidence for a bird. It didn't really tell you anything about whether it's a bird or not. And even when you got a little bit of edge that doesn't tell you whether it's a bird. And even if you get two bits of edge that join and make a potential beak, that's a little bit of evidence it might be a bird, but not very good evidence because there's all sorts of other things that join in a kind of beak like shape, the corner of a table seen at an angle, for example, that makes a shape like that. But once you start seeing the eye of a bird and the beak of a bird, and you see other combinations like that that are obvious features of birds, then you're beginning to get good evidence as a bird there. So I've sort of explained to you what kind of system we want to build. We want these layers of features, and in each layer you're detecting combinations of the features in the layer below until you've got combinations that are quite specific to birds. And you can say it's a bird. But the question is, how do you learn all those connection strengths? How do you learn to have a detector that has big positive inputs from three little bits of edge sloping down and three little bits of edge sloping up like this? How do you decide that those six bits of edge should have big positive weights to this detector, and all the other features you detected should have no weights to this detector. They're all irrelevant. You're just looking for those six features, those six edges. Okay, so now I'm going to explain. I'm going to start by explaining an obvious way to do it that clearly is hopelessly inefficient, but gives you an idea of what's going on. So this is really three stages to explain how it learns. First, what is it you're trying to learn? Secondly, let's understand a really dumb way of doing it so we can get the feel of what's going on, and then I'll show you how to do it better. So the really dumb way of doing it is this. You start with all these layers of neurons and you put random weights between the neurons so you have connection strengths from one layer to the next. And they're all just random numbers, some small positive numbers, some small negative numbers. And now you put in an image of a bird and you see what it outputs. And if you, if it's got random numbers, it'll say 50% is a bird and 50% is not a bird. So that's not much use to you.
- [24:17]
Jascha Monk
No.
- [24:17]
Geoffrey Hinton
You could ask the following question. Suppose I took one of those connection strengths, just one of them, and I changed, I made it slightly bigger. Clearly the output will change very slightly
- [24:32]
Jascha Monk
and you can check whether it got better or worse.
- [24:34]
Geoffrey Hinton
Exactly. So I change one of the connection strengths slightly and I say, does it now say 50.001% chance it's a bird and 49.999 chance it's not a bird, in which case it got better or did it get worse? Although supposing it was a bird, and if I take an image of a non bird, obviously I would like that change to make it more likely to say it's a non bird and less likely to say it's a bird. And you might think that you now had enough evidence to change the connection strength a little bit, but actually you don't because for this particular image it turns out that increasing that connection strength a bit helped. But it may not help on all images. It may make it worse on other images. It may be. There's lots of other bird images where if you increase that connection strength, it makes it less likely to be a bird.
- [25:24]
Sponsor 1
Eczema is unpredictable, but you can flare less with ebglis, a once monthly treatment for moderate to severe eczema. After an initial four month or longer dosing phase, about four in ten people taking ebglis achieved itch relief and clear or almost clear skin at 16 weeks. And most of those people maintain skin that's still more clear at one year with monthly dosing.
- [25:44]
Sponsor 2
EBGLIS Lebricizumab LBKZ, a 250mg injection, is a prescription medicine used to treat adults and children 12 years of age and older who are weigh at least 88 pounds or 40 kilograms with moderate to severe eczema, also called atopic dermatitis, that is not well controlled with prescription therapies used on the skin or topicals or who cannot use topical therapies. EBGLIS can be used with or without topical corticosteroids. Don't use if you are allergic to Eglis. Allergic reactions can occur that can be severe. Eye problems can occur. Tell your doctor if you have new or worsening eye problems. You should not receive a live vaccine when treated with Ebglis. Before starting Eglis, tell your doctor if you have a parasitic infection.
- [26:18]
Sponsor 1
Ask your doctor about evglis and visit evglis.lilly.com or call 1-800-lilyrx or 1-800-545-5979.
- [26:24]
Jascha Monk
Right, so perhaps this image is of a bird against a sunset. And so the color is mostly purple. And what you actually taught the system is that if a color is mostly purple, then you should say it's a bird, but that's actually going to make it less likely on average, to guess correctly. Is that the kind of example you have in mind?
- [26:41]
Geoffrey Hinton
That's exactly right. So what you need to do is you need to show it a whole bunch of examples. You take a sort of a random collection of examples, a few hundred of them. And for this few hundred examples, you say, did changing this connection strength, did increasing it a bit improve things or make things worse? And if it improved things, you increase connection strength. If it made it worse, you probably decrease the connection strength. Okay, so we just did a little experiment. What we did was we take a few hundred images and we see whether changing this connection strength by a little bit, increasing it by a little bit, improves things or makes them worse. And if it improves things, we increase it by a bit.
- [27:22]
Jascha Monk
And when we do this, is this a primitive form of what we call learning? I know that obviously, when we talk about AI, we always talk about learning.
- [27:29]
Geoffrey Hinton
That would be a learning algorithm, and that's a kind of evolutionary kind of learning algorithm. It's like you're making a little mutation and seeing whether that helps, and if it helps you keep it. The problem is you've got in your brain, you have 100 trillion connections in a big neural net. You might have hundreds of billions of connections. And you have to do this for each connection just to increase it by a little bit or decrease it by a little bit. And each time you do, each time you do one of those experiments, you have to run it on hundreds of images to see if it really does help. So this is going to be incredibly slow. If you got even, even if you only had a billion connections, you'd have to run 100 images through a billion connections, through all these layers, just to decide on whether you should increase one connection strength a little bit. So it would work in the end if you kept doing that for billions of years, you would in the end get a neural net that was good at recognizing birds.
- [28:34]
Jascha Monk
And this is not an abstract problem, Right. Because for a lot of the early stages of AI development, one of the basic problems was that you could make these machines learn stuff, but it took an incredible amount of computing power, and not enough computing power was available. Obviously, even today, computing power is one of the constraints on getting even more intelligent systems. But sort of this thought about, okay, we have a basic set of ideas about how to allow these neural networks to learn stuff, but we're constrained by resources was really important. And so a key part of your work, as I understand it, was to try and think about, okay, how can we design these learning processes in a way that is more efficient, sufficiently efficient, so that with the computing power verse available, even at a time which was much more limited than today, we're going to get to somewhere that is potentially useful. So how do you adjust this learning process? How do you transform this learning process to make it, you know, not prohibitively computation intensive?
- [29:29]
Geoffrey Hinton
Yes. So even with all the compute power we had today, that particular learning algorithm where you try changing one connection at a time and seeing if it helps, that would still be completely hopeless. It's just much too inefficient. Okay, so what you'd like to do is figure out for all of the connection strengths at the same time, whether increasing them a little bit or decreasing them a little bit helps. So you'd like some way of computing for every connection strength at the same time, whether to increase it a little bit or decrease it a little bit. And if you could do that, you go, if there's a billion connections, you go a billion times faster than the dumb algorithm. And there is an algorithm called backpropagation that does that. And roughly the way it works is this. You put in an image, you run it forward through your layers of feature detectors to decide whether it's a bird or not a bird. And suppose it says 55% it's a bird, and 45% it's not a bird. And let's suppose it was a bird. So you'd like it to increase. Increase that 55% to decrease the 45%. And so you take the discrepancy between the output of the network and the output you would like it to give. So you'd like it to give 100% as a bird. It actually said 55% is a bird. So it's like a 45% discrepancy between what it said and what you'd like it to say. And you take that difference and you send it backwards through the network through the same connections. And there's a way of sending it backwards, which is sort of obvious if you do calculus, and if you don't do calculus, don't worry about it, There's a way of sending that information backwards through the network. So that one sweep backwards through the network from the output back all the way to the input, you can compute for every single connection whether you ought to increase it or decrease it. And then you change all billion connections at the same time. And so you go a billion times faster. And that's called backpropagation. And that really works.
- [31:33]
Jascha Monk
And so backpropagation just means literally sending it back through the system. Right. That's what the word back propagation refers to in this context, I take it.
- [31:40]
Geoffrey Hinton
You propagate it backwards through the system. You propagate this error backwards through the system. And what you're really doing is trying to figure out, for every neuron in the system, should I make it a little bit more active or a little bit less active? And once you know that, whether you should make it a little bit more active or a little bit less active, you know how to change its incoming connection strengths to achieve that.
- [32:00]
Jascha Monk
So let's do a little back propagation of ourselves. I'm going to try and say back to you what I think I just heard. And my understanding is that back propagation is one of the real contributions that you made to this field.
- [32:10]
Geoffrey Hinton
Well, let me just correct that. Lots of people invented back propagation. Our main contribution, the contribution of Rommel, Hart and Williams and me, was to show that back propagation would learn the senses of words and would learn interesting representations.
- [32:25]
Jascha Monk
Thank you for the clarification. I don't want to overstate your very significant contribution. So, you know, we're trying to figure out if it's a bird or not. You feed it an image, it's telling you 55% likelihood that it is a bird, and say, okay. One way of thinking of what it does to send this result back through the system is to say, okay, what would all the neurons have looked like if it had come back with the answer 100%? And based on that, you then adjust the weights to say, all right, that seems closer to the kind of setup we should be having. Is that roughly on track, or did I completely misstate that?
- [33:05]
Geoffrey Hinton
No, that's roughly on track, but it's not quite on track because you're not trying to solve the problem of how do I change the weight so I get exactly the right answer. You're trying to solve the problem of how do I change the connection strength. So my answer's a little bit better. So you said, we're trying to figure out how we should change the neurons. So it says 100% Bert. We're not really trying to do that if it says 55% Bert. We're trying to figure out how to change the connection strength. So it says 55.001% Bert. In other words, asking the question, how would you change connections to make it just a tiny bit better? That's what calculus is all about.
- [33:43]
Jascha Monk
I see. Okay, great. And then, you know, once with the other people who made contributions to this, the importance of backpropagation became clear. How far along to the basic conceptual basis of contemporary artificial intelligence were you, and what was the bridge? What were other elements that still needed to be pioneered and championed to be able, along with obviously more computing power and more resources and so on, to get to the stage where we have the degree of artificial intelligence that we've been able to reach today?
- [34:17]
Geoffrey Hinton
So in 1986, we showed that the backpropagation algorithm could learn the senses of words in a very simple toy example. And we were very optimistic. We thought, you know, we figured out how to make things learn layers of features, how to. How to make them learn how to do vision, and we'll be able to make them learn how to do language. We've solved it. Everything's going to be wonderful. And it was okay for some things. It was, for example, pretty good at reading the zip codes on envelopes, the postal codes. It was pretty good at reading the numerical amounts on checks. At one point, it read the numerical amounts on, like, 10% of the checks in North America. And that was in the 80s and early 90s. But it wouldn't scale up to recognizing real objects in real images. You know, being able to say it's a birdie, even if it's a seagull in the distance or an ostrich in your face. And we didn't know at that time what the problem was. And the problem was mainly we didn't have enough data and we didn't have enough compute power. But if we'd said that at the time, people would have said, yeah, yeah, you're just saying if you had a bigger one, it would work. That's an easy thing to say. And they did indeed say that. And so it was kind of embarrassing to say, yeah, but if we only have one that had a Thousand times as much data and a thousand times as much compute. Maybe it will work better. In fact, that's what we really needed, was a million times as much data and a million times as much compute. And then it worked really well. Now, there were other technical advances, but the main advances were getting much faster compute and much more data. And the much more data came from the Web, and the much more compute came from GPUs, particular Nvidia GPUs, which are easy to program. When I say easy to program, they're a pig to program, but much easier than most parallel systems.
- [36:17]
Jascha Monk
And presumably one of the reasons why data is so important in all of this is that we have assumed in this example that we have an image of a bird for which we have the answer as to whether or not it is in fact a bird, right? So if you didn't have something against which we can measure the system and something against on which to base the accuracy of the predictions of this model, and therefore the system by which we adjust the weights of different neural connections, then all of this learning algorithm is not going to work. And so, you know, we need lots and lots and lots of images about which we are reasonably confident that they are birds and that they are not birds. Is that right?
- [37:00]
Geoffrey Hinton
Yes. So with computer vision for a long time, we didn't have a big data set like that. We needed a data set with millions of images that were accurately labeled or fairly accurately labeled, and we didn't have it. Someone called Fei, Fei Li, who was the kind of junior professor, realized that if we had a big labeled database like that, a data set, it will make a huge difference to whether neural nets could do vision. She didn't actually necessarily think it would be neural nets, but she did think it would make a huge difference to whether computers would get to be good at doing vision and recognizing objects and images. And so she went to a lot of effort to build a huge database, and that was crucial. The digitized images were there on the Web, but they also needed somebody to provide labels for them all. Now, you don't get the same problem in language. And the reason you don't get the same problem in language is because you use the next word as the label. So you say, I've seen a string of words. They're the input from this string of words. I've already seen. Can I predict the next word? And of course, the next word is part of the data. You don't need anybody to tell you what the next word is. When someone gives you a document you see all the next words given each context. So the nice thing about language, and the reason you can have trillions of examples with language is because you don't need someone to give you labels. There is research using neural networks on language where you say, does this movie review have a positive sentiment or a negative sentiment towards the movie? And someone has to hand label that. And for a while, people did a lot of research like that. But if you just try and predict the next word, that's called self supervised because the data itself contains the label. Now, you don't need all those human labelers.
- [38:50]
Jascha Monk
So you said earlier that by something like the late 1980s, you might be able to correct me on the exact date. The basic conceptual foundations for contemporary artificial intelligence were in place. And the truth of it at that time was we just needed more compute and more data. But that sounded off. It sounded like making excuses for, you know, why the system isn't yet working as well as it might one day. And yet it turned out to be true.
- [39:17]
Geoffrey Hinton
Sorry. There was another reason. It wasn't just that they didn't believe a bigger one would work. The symbolic community was convinced that if you started with random connection strengths and you just adjusted them like this, you get trapped at local optima. A bit like if you're in a mountain range and you just go uphill, you'll end up at the top of a small foothill. And if you keep trying to go uphill, that's as far as you can go. You have to be willing to go downhill to get to the top of Mount Everest. Now, it turned out they were wrong. It turned out on a normal landscape in three dimensions, that'll happen. You'll get trapped at a local optimum on the top of a foothill. In these neural nets, you may not get to the very best set of connection strengths, but you will get to a very good set of connection strengths. If you don't get to the top of Mount Everest, you'll get to the top of some nearby very high peak. And people didn't know that. That's just an empirical result. And it was a big surprise to the symbolic AI people that if you just kept creeping along, improving the weights to make the answer a little bit better, you would learn incredibly impressive things.
- [40:32]
Jascha Monk
So perhaps let's stick with this contrast for just one moment, because I think to a lot of people, it seems that human intelligence is more like the symbolic AI. People might predict that the way we reason about the world is that we have these rules of logic and we're applying them and we're doing these calculations based on these rules of logic, and that's how we reach a firm and logical conclusion. And one of the ways to attack current AI systems is the idea that they're just stochastic parrots, that all they are is predicting the statistical likelihood of the next word being something. And I take it with a lot of skepticism of the symbolic AI community about whether or not you were going to get somewhere with the approach that you helped to champion was that that is just not how you get to real intelligence. Now, of course, you yourself are actually quite inspired by neuroscience in many ways, and by our understanding of how neurons in our brain work and the way in which human minds learn is different from AI in many important respects, but seems in some ways actually more analogous to being neural networks themselves and to getting a bunch of data and trying to predict which response to that has given me reinforcements in the world and which response to that hasn't given me reinforcements in the world.
- [41:53]
Geoffrey Hinton
Hang on, you keep saying things that I believe are deeply wrong and I don't know when to interrupt you. You said about three of them so far in this question.
- [42:00]
Jascha Monk
Please interrupt me and explain those things to me.
- [42:04]
Geoffrey Hinton
Okay, let's start with the stochastic parrots. So the people who talk about stochastic parrots are typically linguists, strongly influenced by Chomsky, who believed that language is basically innate. And Chomsky was adamantly opposed to statistics. He thought it's discrete rules and that's how language works. And statistics is just sort of silly. And that's not how language works. And it turns out he's completely wrong, according to me. So I can't let you get away with sort of that. And also the idea that just predicting the next word can't possibly be how you learn language, that's also deeply wrong if you think about it. If you want to do a not very good job of predicting the next word, you can just use simple statistics. So for example, you could keep a big table of phrases and if you see the words fish and you could look in your big table and you could see actually fish and chips occurred a lot. So chips is a pretty likely next word because we've seen lots of occurrences of fish and chips. That would be simple co occurrence statistics. And the people who talk about stochastic parrots, that's their model of statistics, that's what they're arguing against. But that's not at all how these neural nets work. They don't really understand how they work, particularly Chomsky. So if you think about it, suppose you want to do a really good job of predicting the next word. Not just a kind of moderately good job by keeping a table of how often particular phrases occur, but a really good job, the best job that could be done. To do that, you have to understand what the person's saying. So if I design a system that's going to end up doing a really good job of predicting the next word, the only way it can do that is by understanding what was said. And what's impressive is that training these big language models just to predict the next word forces them to understand what's been said. In particular, if the next word is the first word of the answer to a question, and the context is the question, if you don't understand the question, you're not going to be very good at predicting the answer. So the stochastic parrot people don't seem to understand that that just predicting the next word forces you to understand what's being said.
- [44:28]
Jascha Monk
I'm not sure that we disagree on this. Actually, I was trying to give voice to that critique, but also to say that it seems to me that actually there's something about the ways in which models of artificial intelligence today engage in learning that seems actually more similar to the human brain. And it strikes me when I speak to friends of mine who are neuroscientists, but we still don't fully understand how the human brain works. But I guess what I wanted to ask you is how similar do you think the learning mechanisms of these AI models today are to what's going on in the human brain? In some ways, the inspiration of that was partially to understand how neurons work together in the human brain. That's why that metaphor is there. That's why we're talking about neurons and neural nets in the context of AI. Do you think that the basic mechanisms that are going on in a neural net that is being fed with a bunch of data and that is learning how to interpret the question so that it can give that answer is the same kind of thing that's going on in a human baby when it is learning to maneuver around the world and to eventually answer the questions that its parents put to it, or do you think that there is a fundamental difference between those two things?
- [45:41]
Geoffrey Hinton
Okay, so that's a huge open question. It's probably, for me, that's the most important question in neuroscience. How similar is the way the brain learns to have these large language models learn? And at a very abstract level, I believe it's quite similar. And the abstract level is the large language models have a way, this back propagation algorithm of figuring out for each connection strength, should I increase it or should I decrease it to make this whole system work better? That's actually called the gradient. Which direction should you go in to improve things? Now, the brain probably has the same thing, but may not get the gradient in the same way. We don't know how the brain figures out for each connection strength whether to increase or decrease it. But what we do know from these large language models, that is that if you could get that information, which we get using backpropagation in the large language models, then you can build very impressive systems just by trying to predict the next word. So we know that if you get the gradient, you can learn very effectively. We don't know how the brain gets the gradient. There've been many, many attempts to try and show how your cortex, the newer bit of the brain, actually can get these gradients so that it can learn the way the large language models do. Nobody has been highly successful in that. There've been many theories, some of them moderately plausible, but none of them work really well. Hopefully, eventually someone will figure it out. There are some reasons for believing the brain might have a different algorithm. And I'll now give you the reason for believing it might be different from backpropagation. So what back propagation is figuring out is if you have a lot of experience, like trillions of examples and not many connections, like maybe a mere trillion connections, you have 100 trillion. These large language models, the biggest ones have about a trillion, but they have trillions of examples. So they have many more examples than connections. So what they're trying to do is squeeze lots of knowledge into not many connections, trillions of bits of knowledge into only a trillion connections. Our brain is very different. We only live for about 2 billion seconds. We don't have, we don't have trillions of experiences, we just have a few billion experiences. So we've got lots and lots of connections, connections to spare, but not much experience. So our brain has to deal with a rather different regiment where you're limited in experience but not limited in connections. Whereas back propagation is very good when you've got lots of experience but you're limited in the number of connections. So they're somewhat different problems they're solving.
- [48:32]
Sponsor 1
Starting a business can seem like a daunting task unless you have a partner like Shopify. They have the tools you need to start and grow your business. From designing a website, to marketing, to selling and beyond. Shopify can Help with everything you need. There's a reason millions of companies like Mattel, Heinz and Allbirds continue to trust and use them. With Shopify on your side, turn your big business idea into sign up for your $1 per month trial@shopify.com specialoffer you
- [49:00]
Jascha Monk
know, if one of the constraints of how to make AI models more smart than they are at the moment is that we may run out of high quality data that clearly data is incredibly sparse because it's so important to us processed, then is it imaginable that we might be able to emulate some of the mechanisms that the human brain uses in order to extract so much understanding of a world from relatively less informational input?
- [49:28]
Geoffrey Hinton
Yes, that is possible. It is possible that the brain's using some other way of getting gradients that's not quite the same as backpropagation, and that might let you learn faster. But I think a more promising approach at present for artificial intelligence is to see how you can deal with the data limitation. So there's areas in which we don't worry about shortage of data, and that's for example, AlphaGo or AlphaZero that plays chess. Nobody there worries about a shortage of data. They did to begin with. So to begin with, when they make go playing programs with neural nets, what they did was to get the neural net to copy the moves of experts. And you only have so many moves by experts. I mean, with the popularity of chess now you have billions of moves, but it's still probably not trillions, or maybe it's only a few trillion, but nobody worries about that when you're training a chess program or a Go program because it generates its own data. So what happens in things like AlphaGo is it plays against itself. So there's a neural net, there's two neural nets that sort of give it its intuition. And I'm going to talk about chess because I know chess much better than I know Go. And I imagine most of the viewers of this podcast know more about chess than Go. So we're making a chess playing program. It has one neural net that can look at a board position and can say, how good is that position for me? And he just looks at it and says, hey, that's good for me. It has another neural net that can look at a board position and say, ooh, this will be a good move to make. So if you know a little bit about chess, if the other guy has a backward pawn, it's very good to put a knight just in front of that backward pawn. It stops the pawn advancing, and there's no other pawns that can take it. So that's a very natural square to put a knight on. That's a little bit of intuition about what a good move is. But in, of course, Alphazero, it has hugely more sophisticated intuitions. Those are the two neural nets. And the question is, how does it train those neural nets? Well, it plays against itself. And so it does what's called Monte Carlo rollout, which is roughly, if I go there, then maybe he'll go there, and then I'll go here, and then I, oh, whoops. And then I'll be in a terrible situation. And so from that, you can figure out, well, you shouldn't go there. Your. Your neural net that said, what a good move might be suggested. You go in a particular position, you do Monte Carlo rollout. So you try many, many times. If I go here, he goes there. If I go here, he goes there. And you discover that even if you try many times, it always seems to end that you lose if you do that move. So that's a bad move. You thought it was a good move, but it's a bad move. So that Monte Carlo rollout is what gives you your information about whether it's a good or a bad move. And so now you can modify the neural net that was previously saying, oh, that's a great move. And now you modify it and say, oh, that's not such a great move. So the neural nets get trained by using the results of this Monte Carlo rollout, which is the sort of like conscious, explicit reasoning. You know, if I go here, he goes there, I go there. That's very sequential in people. Chess players can do it very fast, but it's still fairly sequential. And that's what's used to train the intuition. And it's like that for a lot of what we do. So you have. You have intuitive beliefs, and then you do some reasoning. And of course, doing the reasoning, you're using your intuitions to do the reasoning. And as a result of your reasoning, you discover your intuition was wrong. And so you go back and you revise your intuition. Now, that was an example where you didn't need anybody external to give you training examples. So most people have a lot of beliefs, and if they were to do some reasoning, they'd discover those beliefs aren't consistent. Something's wrong somewhere. Either the way they do the reasoning is wrong, or one of those belief, their belief in the premise is wrong, or their belief in the conclusion is wrong, but something's wrong. Somewhere, so they need to change something. So as soon as you've got something like reasoning working, you can generate your own training data. That's a nice example of what people in Maggad don't do. They don't do reasoning and say, look, I have all these beliefs and they're not consistent. It doesn't worry them. They have strong intuitions and they stick with the intuitions even though the intuitions are inconsistent. It's very annoying for people who believe in reasoning. And reasoning is very important for tuning your intuitions. And so that's one way you can get training data without having to have other people provide it. It's what's used in chess and go already. And it's very good in closed worlds. So for mathematics, for example, mathematics is a kind of closed world. You can make some conjectures about what might be true and then you can try proving them and you can have some conjectures that seem very plausible to begin with and then you can do a bit of reasoning and discover they must be wrong. You might have a conjecture that maybe there's a biggest number. So suppose you're a five year old and you think, well, there must be a biggest number. And then you think to yourself, but if I added one to that, I'd get an even bigger number. Oh, so there can't be a biggest number. That's an example where you didn't need training examples, you just need a little bit of reasoning. That's one way how AI is going to get around this data limitation. And the large language models I think are already doing some of that. Demis Hassabis, I know, believes in that method of getting yourself a lot more training data without needing external data.
- [55:38]
Jascha Monk
Oh, that's very interesting. Just on the point you were making earlier. I remember arguing with somebody once where I thought I had a very convincing logical argument. And they said, we either can believe this or really that, you know, on a pain of inconsistency, you have to accept this conclusion. And then they said, well, I choose inconsistency. And it's very said so explicitly that it is very infuriating. But that is not. You can't do that. That some people say, well, you know, I don't mind, I care more about having belief X and having belief Y and having belief Z than I care about having a consistent worldview. And that makes it very hard to argue with such people.
- [56:12]
Geoffrey Hinton
There's a name for that. There's a name for that. The name for choosing inconsistency is faith. And the Whole Enlightenment was about choose reason over faith. And we're losing it.
- [56:24]
Jascha Monk
Indeed. And we're at the tail end of the Enlightenment unless we can help it and fight back. Just to go to one other point, you were saying earlier that there's this moment when you believe, and some others believe, that if we have more computation and more data, then we're going to be able to make progress. And some people disbelieve that. It seems to me that there's now a point where there's a question about how fast the continuing progress of AI is and whether we're going to get to much smarter systems in two years or in five years, and perhaps even to something like artificial general intelligence, just by throwing more data at it, Just by throwing more compute at it. Perhaps by some smaller innovations, like figuring out better ways in which, as we were just discussing, some of the systems can themselves create the data on which they've entrained themselves, or whether it would need a real change, a real, let's say, a more revolutionary change in how some of those learning algorithms work or how these systems are able to take learnings from limited amounts of data. What do you think is the truth of this? In 10 years, in 20 years, are we just going to have very rapid linear improvement or even exponential improvement in the intelligence of these AI systems by throwing more compute at basically the same architecture? Or do you think that we're going to need real changes in architecture in order to make a significant leap forward from where we are today?
- [57:51]
Geoffrey Hinton
Okay, so nobody knows for sure, but what we've seen so far is for quite a long period, just scaling things up made them work better. And that's still the case. But there's problems scaling it up because you need huge amounts of computer and huge amounts of data. But we know that scaling it up will make it work better. We may just have problems scaling it up. We also know that new scientific ideas, new architectures like Transformers will make it work a lot better. In 2017, Google figured out Transformers. People at Google and published it and chat GBT was basically based on using Transformers. We can reasonably expect that there will be more scientific breakthroughs like that. We don't know what they'll be or when they'll occur, because if we knew that, we'd have done them. We can also expect there'll be lots of engineering advances. So over the last few years, the engineering's got much better. So you see things like Deep SEQ that may have cheated to some extent by distilling things from bigger models, but there's always room for better engineering. And it's very young, it's only been going a few years, this stuff. So there's lots of room for engineering improvements that will make everything much more efficient, and that may ultimately be the way we deal with needing much more compute. So there is a school of thought which has been around for a while. Its most vocal proponent is probably Gary Marcus, which is that we used to have symbolic AI, which he really believed in. This all about having symbolic expressions and rules for manipulating them. And we need to go back to that to make serious progress in reasoning. Now, that hasn't been the case so far. So if you look at the progress in reasoning that's been made, it's not like you have some special internal symbolic language. So what symbolic AI basically believed, I'm simplifying it, but it believed, if I give you a sentence in English, what you need to do is turn it into a sentence in some special language, some special internal symbolic language that's unambiguous, and you can then operate on that expression in the symbolic language, using rules to derive new expressions. And that's what logic is, and that's how reasoning is going to work. Well, reasoning in these models is working now quite well, and it doesn't work like that at all. There is no special symbolic internal language inside. It's just activations of neurons in these neural nets. The only symbolic language is natural language. They are symbols, but they're at the input and at the output. And if you look at how they do reasoning, they do reasoning by they predict the next word and then they look at what they predicted, and then they predict a word after that. And so they can do thinking like that. You give them a context, and by predicting words, they kind of get a scratch pad to do thinking. They can see the words they predicted, and then they can in effect reflect on those words and predict more words. And that's what thinking is in these things. And that's why we can see them thinking. And it's not at all like the symbolic way of doing it. They are producing symbols, but they're symbols at the level of the input and output, not some special internal language. So my own view is that people who want hybrid systems that consist of sort of neural nets for the input and output and symbolic AI for doing the real reasoning, they're trying to cling on to the past. And I have an analogy for what they're doing. Suppose you took someone who manufactures gasoline engines and you said, look, electric motors are actually better. There's all sorts of things about electric motors that make them better than gasoline engines. And after a while, the car manufacturer agrees with you and says, okay, okay, I accept that electric motors are better. So here's what we're going to do. We're going to use the electric motors for injecting the gasoline into the engine, which is what they actually do. That's called fuel injection.
- [61:59]
Jascha Monk
That obviously is a nonsense.
- [62:01]
Geoffrey Hinton
Yes, well, it's actually quite helpful to do free oil injection, but it's not the main point, right? It's trying to hang on to your gasoline engine and have your electric motor as well. And that's what I think these hybrid systems are like.
- [62:17]
Jascha Monk
Thank you so much for listening to this part of the conversation. In the last part of our exchange, Jeffrey explains why you don't have to imagine a machine suddenly jolting awake and deciding it wants to destroy humanity for that to be real. Existential Risk from these AI Systems if you want to understand exactly why people like Jeffrey take the existential risk from artificial general intelligence so seriously, this part of a conversation will help to explain this important question to you. In order to listen to this part of the conversation in order to support this podcast, please go to yashamung.substack.com and become a paying subscriber. That's yashamung.substack dot com.
- [63:19]
Geoffrey Hinton
Sa.