Steve Garrett (4:49)

The nerve cell will go. When we come to think about neurally inspired artificial systems, the fact that it's sending a pulse or it isn't is the critical insight that gives us all the information processing power that we use now.

Alok Jha (5:02)

From the earliest attempts to model the human brain in silicon, the systems we.

Dawood Dasu (5:07)

Were building were so dumb and so weak and so difficult to train to.

Alok Jha (5:12)

The technologies that enabled those models to be scaled up.

Daniel Glaser (5:16)

Imagenet was the turning point of AI's history, recognizing how critical it is to use big data.

Alok Jha (5:25)

We'll hear why. Finally, around a decade ago, a AI got astonishingly good.

Daniel Glaser (5:31)

All of a sudden things are working and people pay attention to what we do. We have a number of examples where computer vision systems can beat human experts.

Alok Jha (5:40)

At their own game and how those systems just kept on getting better.

Steve Garrett (5:46)

The Change from, say, GPT2 to GPT3 was huge. The change from GPT3 to GPT4 was huge.

Ainsley Johnston (5:53)

I did not think the large language.

Alok Jha (5:55)

Models would work as well as they are.

Ainsley Johnston (5:57)

I just thought, I can't just throw.

Alok Jha (5:58)

The whole Internet at it and be.

Ainsley Johnston (6:00)

Able to get next word prediction and have that seem like a human. You know what, I was dead wrong.

Steve Garrett (6:05)

You can't.

Alok Jha (6:08)

If you want to understand the origins of artificial intelligence, it's best to start with the second of those words. So our first question in this series is what is intelligence? To figure out exactly how the human brain works. Let's pick up where we left off with our correspondent Ainsley Johnston.

Ainsley Johnston (6:42)

The UK Biobank Centre just outside of Manchester scans patients brains seven days a week as they work towards their goal of imaging 100,000 people. Steve Garrett, the imaging programme manager, explained the process that participants undergo.

Steve Garrett (7:00)

We are in the imaging clinic. We've got participants in here who are coming for around a four to five hour visit.

Ainsley Johnston (7:08)

Are they doing tests and things on the computers?

Alok Jha (7:11)

We have a touchscreen questionnaire where they.

Steve Garrett (7:13)

Give research really comprehensive answers to anything about their health and lifestyle. But they also do the cognition tests. Who wants to have a seat?

Ainsley Johnston (7:22)

Oh sure, I'll have a seat. Yeah, great. That's Joanne Norris, one of the health research assistants at the clinic.

Steve Garrett (7:29)

We explain obviously that they're going to go through a 25 minute time section on some games, puzzles and memory tests as well. So have a read in the yellow and then when you're ready, press that smiley next button.

Ainsley Johnston (7:41)

Okay?

Daniel Glaser (7:41)

Okay.

Ainsley Johnston (7:43)

Okay. The game will have three pairs. Right? Okay. So I can see six cards in front of me and now they've been turned over and so I've got to find. Okay, this one was a man and this one was a man. I think this one was a square. Oh no. Oh no, that's embarrassing. Okay, this one is a kite and this one is a kite.

Alok Jha (8:03)

Great.

Ainsley Johnston (8:03)

Square. Square. Okay, great. Okay, so I'm being asked to add the following numbers together. 1, 2, 3, 4, 5. Okay, so that equals 15. If Truda's mother's brother is Tim's sister's father, what relation is Truda to Tim? Truda's mother's brother, right, that's Truda's uncle is Tim's sister's father. That's Tim's father. Truda's uncle is Tim's father. Then Truda must be his aunt, I think. Oh God.

Dawood Dasu (8:47)

Ainsley.

Alok Jha (8:48)

At this point, some of our participants asked for a pen and paper.

Ainsley Johnston (8:51)

I think I need a pen and paper. I feel like we've probably got enough of this and I think I'm probably embarrassing myself.

Alok Jha (9:01)

Uh oh.

Ainsley Johnston (9:07)

These tests are about a lot more than just making fun of journalists though. The scores from each of the tests help to paint a unique picture of participants cognitive abilities. This is powerful data for researchers, particularly in combination with the biomedical data that's about to be collected.

Steve Garrett (9:25)

And then they'll go and get changed. And after they're changed, one of them will go for their brain scan.

Ainsley Johnston (9:33)

At the end of a corridor full of warning signs for a strong magnetic field is the brain MRI machine. These machines look like giant donuts. The participant lies down on a bed and then their head and shoulders are moved inside the bore of the scanner. We entered the control room next door. From here, a radiographer controls the scanner, checks the quality of the brain images that are being collected and makes sure that the participant is happy and comfortable.

Alok Jha (10:03)

Task is starting now.

Ainsley Johnston (10:06)

Angela Emmons, one of the radiographers, took me through the process.

Steve Garrett (10:10)

It's a half hour scan of the brain. First 25 minutes. You just need to keep nice and still. Then there's a task coming up. The task is just to look at the brain when it's actually working. We run an earlier sequence when they are at rest and then just run two minutes of that when they're undertaking a game of Snap. We show them a series of shapes and we show them a series of faces. Runs about two and a half minutes and then when that comes to an end, they've got about another two minutes left in the scanner.

Ainsley Johnston (10:44)

While the participants in the scanner, what can you see in the control room?

Steve Garrett (10:47)

Lots of images come up. The images come up in real time. We check the resolution, make sure we've got good images, participants settled, and then just follow that through the sequences.

Alok Jha (11:02)

This gives you an idea of intelligence.

Ainsley Johnston (11:05)

That's Dawoud Dasu, who we heard at the start of the podcast.

Alok Jha (11:08)

You can look at how much of that variation amongst participants is explained by genome data. You can look at our imaging data. You might even be looking at history as well. So lifestyle, job, diet and things like that. You could look at all of that as well and I'm sure somebody will figure out a way of looking at.

Steve Garrett (11:28)

All of that together.

Ainsley Johnston (11:32)

Using the biobank data, scientists have discovered that having a larger brain, and in particular a larger frontal cortex associated with higher intelligence, there are also certain patterns in how different parts of the brain communicate with each other that can predict people's scores on cognitive tests. There's still a lot of variability in intelligence that scientists can't explain using these measures of the brain, though. But access to enormous data sets like the UK biobank is allowing scientists to pick apart how the tangle of neurons inside our heads have enabled us to Develop vaccines, send a man to the moon, and even create AI.

Alok Jha (12:23)

Lots of researchers from around the world use data from the UK Biobank and other sources to investigate brain intelligence. But intelligence in human brains is not something that's easy to pinpoint. There isn't one bit of the brain that's responsible for it, for example. And the more you get into it, the harder it gets to define what intelligence even is. So let's take a step back and look at how the brain works at a more basic level. To do that, I spoke to Daniel Glazer. He's a neuroscientist at the Institute of Philosophy, part of the University of London. He works at the intersection of neuroscience and AI.

Steve Garrett (13:05)

We know a lot about how the brain is structured, and we know a lot about how it works in the sense of how the molecular level works. I can tell you in exquisite detail about the structure of the individual neurons. And at the level of the whole brain, I can tell you what the front does and what the back does. What I can't tell you is what difference at the microscopic level makes the difference at the macroscopic level. So although I know all of these levels of description of the brain, I can't give you a coherent story that tells you how the overall behavior derives from all this exquisite detail that I do know about the molecules.

Alok Jha (13:40)

Let's go into a bit of that exquisite detail, then. Just describe the anatomy for me and how the anatomy functions.

Steve Garrett (13:47)

So brains are collections of neurons which are nerve cells. And while nerve cells exist throughout the body, pain detectors and all sorts of things like that, in the brain, they're all clumped together in a big wadge. And the principal property that almost all nerve cells have is that they use electricity to send signals over a distance. And there's two things that derive from that. So one is that these cells are often elongated. So most cells in the body are kind of roundy, clumpy. They have a shape like that. Nerve cells, characteristically, have got a long, extended process, which we tend to call an axon. And you really can think about this extended process like a wire. And like a wire, nerve cells send information along this long process using electricity. So nerve cells are signaling devices that get, if you like, information from one bit of the cell to the other bit of the cell along a long bit called the axon, and they use electricity to do that.

Alok Jha (14:37)

Just in terms of how that manifests in sensing the world. Just explain to me how a network of these cells smells something or learns something.

Steve Garrett (14:46)

I think to understand how this works. You can actually go back in evolution around about 70 million years. You could use chemicals to send information. Ooh, there's something nasty there. Pull back, and you could retract your feelers. But that only works at very short distances. And for animals and cells to get bigger, organisms to get bigger, they needed to communicate information about smells, about predators, about food over longer distances. And so what evolution did, if we can say it that way, about 70 million years ago, is to use some of these proteins that were being used for signaling within cells and wire them up to an electrical signal. And then at the other end, they turned them back into chemical information, which they then used to set off other cells in the network. And that insight, interestingly, which was about signaling, is paralleled in the evolution in human terms of what we would call telegraphy. If you want to send reliably a signal of long distances, you want to be using some kind of code, for example, Morse code. And so the first transatlantic cable used pulses. Dit dit dit da da da, which could be reliably read out at the other end. And it turns out that 70 million years ago, evolution came up with the same insight. So the critical thing about nerve cells is that they use electricity to signal, but that code is not a kind of more or less, more or less, more. You know, it's not a continuously modulated signal. It's pulses. And this transmission of information by pulses, it either fires or it doesn't, is a critical thing you need to know about nerve cells. When we come to think about neurally inspired artificial systems, the fact that it's thresholding, it's sending a pulse or it isn't, it's doing a yes, no firing pattern. It is the critical insight that gives us all the information processing power that we use now.

Alok Jha (16:20)

And it's kind of in a very sort of crude way, kind of a digital signal in that respect.

Steve Garrett (16:24)

Not crudely at all. It's a digital signal in the sense that the information is a 1 or a 0. The cell either fires or it doesn't. A good way to think about nerve cells probably is not so much to start with in the brain, but to think about flexing a muscle. You want to send a signal from your spinal cord to your muscle in your arm. If you want the muscle to contract more alok I'm going to sound like a neuron for a second, the nerve cell will go ba ba ba ba ba ba ba ba. If you want to contract a little bit, it will go ba ba ba ba ba. Similarly, if you have a pain receptor and something's a bit painful, you'll get that. If something's really, really painful, the nerve cell signals that by going, ba, ba, ba, ba, ba, ba, ba, ba. And so the rate coding, we would say the rate at which things fire, and there can be more subtle codes. It's a yes, no signal that contains information over time rather than an amplitude modulated smooth signal as you might have in the nuances of your voice.

Alok Jha (17:17)

So in your brain, the neurons are very close together. They exist in networks which represent all sorts of functionality and memory, et cetera. So in your brain, how do the brain cells, the neurons work together to learn something, whether it's a language or what a predator looks like or whatever else.

Steve Garrett (17:37)

When neurons are connected to each other individually or in networks, there is a strength of the connection, so you don't get the same bang for your buck from each of the cells that connects into a particular cell. So imagine you've got a cell and it's got thousands of other cells connecting into it. Those thousands of other cells are firing, but each of the pulses from those cells does not give you the same input to the cell that's the target. So we can control the amount of input that you get from a cell by the strength of what we call a synapse, the wire that comes in. And if you like to take an analogy from humankind, you might ask all of your mates for a restaurant recommendation, but you're going to pay more attention to one of your friends who's good with food or likes that kind of cuisine or knows the city than another. So they're all saying, pizza, burger, we should go to that Indian place, we should go to that Asian restaurant, whatever. But listening, and you might say, well, I hear all of those inputs, but I'm going to upregulate one and down regulate the other other. So that's the strength of connections in learning. If it turns out that the restaurant you chose was a good one, from your experience, you go to the restaurant, it was great. You'll say, oh, that restaurant was amazing. And then you say, who has it recommended that restaurant? You go, alok, well, do you know what? Next time I'm looking for a recommendation for a restaurant, I'm going to upweight Alok's signal compared to that other guy who, you know, didn't recommend thousands.

Alok Jha (18:57)

Yeah.

Steve Garrett (18:58)

So things that fire together, wire together. When a neuron fires, it says, okay, I got excited, now I'm asking what was the input that got me to the place? I Am and I'm going to subtly up regulate those inputs so that in future the ones that got me to this good place are more likely to get me going again.

Alok Jha (19:15)

That learning that strengthening the connection at a chemical level, what's happening at a.

Steve Garrett (19:19)

Chemical level, there are neurotransmitters which change, generally speaking, the structure of the dendrites. So there are things called spines that basically allow each neuron that fires to release more neurotransmitter to that cell. So it changes the neurochemistry and to a small extent the neuroanatomy. It really does change the microstructure of the neurons so that you get more input to a particular cell from the cells that fired previously.

Alok Jha (19:47)

Let's zoom out. People always ask this question about intelligence and are human brains intelligent? Where does that come from in all.

Steve Garrett (19:53)

Of this, by the way? If you look it up online or elsewhere, I mean, as I would with any respected interview and interviewer, I looked it up on Wikipedia before I came out this morning. And if you look up on Wikipedia, what is intelligence? It said it's that thing which humans are good at. Right. That's a bit facetious, but there is a sort of circularity. And so, for example, when we look for intelligence in animals, or indeed in plants, there's some nice stuff about forests being intelligent. Basically, if you just speed up forests, then they kind of think things through and they're generous and they look after each other and they feel pain when their fellows are trying chop down and so on. When we say that when we look for intelligence in animals, broadly speaking, we're looking for things that they do that are like things that we do. Right? So I can do better than this. But actually, as a starting point, intelligence is what we think like.

Alok Jha (20:37)

And so just break that down. What does intelligence mean? Even if we can't define it exactly, what are the kind of components of what we think of as intelligence?

Steve Garrett (20:44)

So intelligence is the ability to think things through, and the evidence for that is that you can apply it to different domains. You can abstract things to look at something and see their structure, to apply it to other things, to bring knowledge of different domains to bear on certain things. That requires kind of memory and breadth of reference and understanding. It turns out that language is quite a useful tool in helping one to be intelligent. So it's difficult maybe to imagine a human or a creature that doesn't have any kind of symbolic, abstract thought like language and is still intelligent. It seems to be very helpful to do that. Although, you know, when we Start to look at other organisms like octopuses. They exhibit behaviors which you might think of intelligent, they solve problems, they learn from experience, they thinks things through, they try stuff and try things again differently from that. And they probably don't have internal, you know, language of thought. It's interesting, Alok, if you think of any given thing. So, for example, the ability to project into the future, to think about a future plan, think of that planning as an intelligent thing. The problem is, as soon as you write down a single thing that's about intelligence, you can usually find an animal that does that particular thing. Right. So if you want planning, go for corvids like crow, like creatures we call crows.

Alok Jha (21:50)

Intelligent people say all the time.

Steve Garrett (21:51)

Quite so. And that's because they share a thing which we think of as intelligent ourselves, which think the ability to plan, the ability. So, for example, when crows hide stuff, if they're observed hiding a thing by another crow or in sometimes a different species, they'll kind of wander away. And then when they're sure the person who saw them hide the piece of food is gone, they'll go back and move the food hiding place to a place somewhere else. Now, why would you do that? Is because you kind of have thought through that when your back is turned, if you don't come back soon, the person who saw you hiding it is going to come and move it. So we used to think that only humans could do that. The problem is, once you, as you're encouraging me to do, alok, once you define a single thing which is, yeah, do you know what intelligence is? That I can probably find you an animal that can do something like that. What I can't find you an animal that can do is all the things that we count of as intelligent. But that's a bit circular again, because we call them intelligent, because we do them.

Alok Jha (22:37)

Yeah. So it is a bit reductive and it's not at all comprehensive in the way that, you know, you can define intelligence. But as scientists, you want to try and test hypotheses, you want to try and measure specific things in this sort of slightly confusing world. So in terms of intelligence in humans, what are the ways that neuroscientists or others would try and measure that or test it?

Steve Garrett (23:01)

So we can certainly look at what's going on in people's brains when they do things that we would consider intelligent. And we can also particularly do that in the bits of brains of which we have more than animals that are less intelligent than we are. So we can learn by looking at the bits of the brain which are different in us from monkeys and we can draw out the circuits which enable us to do that kind of complex thought. I do think that intelligence is something that allows us to manipulate objects. It's very rare for somebody to be just intelligent without using some kind of external system, even if they've internalized it. So language would be an example of an external system which you put in your head. But actually smart people use tools. Well, while we're talking a lot, we've got somebody very friendly in the room who's operating some complex sound recording equipment and. And you're using a Mac to structure your thinking and look at the questions. That's intelligence. We use these prosthetics. And actually again, when we come to think about large language models and the contemporary developments in AI, one of the things that intelligent people like us do is to make good use of these tools. Now we also fool ourselves that they might be intelligent too. But nobody thinks that their phone is intelligent really, but they use it to enhance their own intelligence if they're smart. Or it can defeat your intelligence by too much scrolling, but you can use it to extend yourself by judicious use of Wikipedia on the fly or storing information in a helpful way. And this ability to use tools is something that we observe in the history of man, actually when these frontal lobes developed as something that is a marker of a time when our intelligence probably really took off.

Alok Jha (24:28)

It's interesting with the phone example, actually, isn't it? A mobile phone that's connected to the Internet, basically a small computer has memory. It has some sorts of reasoning capabilities too, these markers, as you say, of intelligence. But it doesn't have all of the things, it doesn't plan or abstract things in the way that humans do. But I guess it's a different type of intelligence in that respect. But we would never call it intelligent.

Dawood Dasu (24:48)

You're right.

Steve Garrett (24:49)

In general. That's right. I think it's an interesting question about ascribing intelligence is worth pondering for a second. Fast forwarding to LLMs, artificial neural networks like large language models and machine learning. I think our inevitable ability, we can't turn it off to make them seem intelligent, allows us to use these tools more effectively. It doesn't mean they are intelligent, but treating them like they're intelligent enables us to engage with them in more effective ways. And when we come to ask, as I'm sure you will alok, whether these machines are smart or not, we must always beware of this innate capacity of humans to ascribe intelligence to others and to machines, and that will mislead us when we try to make judgments about the new machines that we've built.

Alok Jha (25:31)

All right, well, we've talked about the difficulty of defining human intelligence. We've talked about the difficulty of actually trying to understand it at all the different levels, from the sort of whole level to the cellular level. And there's clearly huge amounts still to learn. But I guess if we try and understand where all of this knowledge leads into how to do the artificial bit of the artificial intelligence. So when we're talking about computer scientists who were looking for ways of being inspired about intelligence to make artificial versions of it, was it a good idea to try and build artificial intelligences on the human brain? I suppose it's the only way they had, right?

Steve Garrett (26:08)

Well, when computer scientists tried to make smarter machines, one of the observations that they made is that maybe what's important about the way that humans think is the wet stuff is the neurons. And so we can ask, what are the properties of neurons that they lit upon and how did they implement them? And actually, they did go right back to basics. So to understand a neural network in the sense of computers, that's the way that most machine learning algorithms work. You really just start with a neuron. It's a device which takes inputs from a bunch of other neurons. Not all the neurons affect it to the same extent. Those are called weights. This is true of a tiny little worm. Each of the neurons that comes onto another neuron excites it to a different extent, and it works out on the base of those inputs, whether it's past a threshold for excitement or not. And if it does, it goes boom. And that ping, that spike goes to the next ones, taking that architecture and layering upon it a learning rule which, as we said before, things that fire together wire together. So by adjusting the weights between the neurons to up regulate things that tended to make things fire in a good context, those two simple insights give you quite a powerful computational learning machine. Now, when we talk about these neural networks, they're actually being implemented in digital architecture. So, funnily enough, you've got a good old fashioned digital computer like the kind that works in your desktop PC or in your phone, but it's running a simulation of these very simple neurons. And again, if you think about the exquisite micro architecture of human neurons, it would take, you know, years to describe even a single human neuron. So, no, we abstract it into some inputs, some weights, a firing pattern. So this very simplified neuron is at the basis of all of the artificial neural networks that underlie machine learning and current AI.

Alok Jha (28:12)

Next, we'll continue that thought and move from human cells to to silicon chips and look at the first attempts to create artificial versions of the human brain. And one of the godfathers of modern AI will tell us about the first time his computer system showed some of the skills that Dan Glaser has been telling us about. That's all coming up. First, though, just a quick reminder that this is a free episode of Babbage. To continue listening to our special series on AI, you'll need to sign up to Economist Podcast Podcast plus and now's the perfect time to do so. We've got a sale on subscribe for less than $2.50 a month. But hurry. The offer ends on Sunday 17th March, and as a subscriber you won't only have access to all of our specialist weekly podcasts, you'll be able to join us for Babbage's first ever live event following the conclusion of this very series that's going to be held on Thursday, April 4, where we're going to answer as many of your questions as we can on the science behind artificial intelligence. Don't miss out. You can submit your questions, check the start time in your region and book your place by going to economist.com aievent all one word. The link is in the show Notes.

Ainsley Johnston (29:40)

My dad works in B2B marketing.

Steve Garrett (29:42)

He came by my school for career day and said he was a big roas man.

Ainsley Johnston (29:46)

Then he told everyone how much he.

Steve Garrett (29:48)

Loved calculating his return on ad spend. My friends still laugh at me to this day. Not everyone gets B2B, but with LinkedIn you'll be able to reach people who do. Get a $100 credit on your next ad campaign. Go to LinkedIn.com results to claim your credit. That's LinkedIn.com results. Terms and conditions apply. LinkedIn the place to be to be.

Alok Jha (30:17)

Today on Babbage, we've heard about how the brain works, and we're trying to unpack how computer scientists who wanted to build intelligent systems were inspired by what neuroscientists had already found. But rather than building an artificial version of a physical nerve cell, computer scientists wanted to build virtual ones. And that leads us to the next step as we build our understanding of the science behind modern AI. Question 2 what was the first artificial neuron? To answer this question, we travelled across the Atlantic Ocean to Boston in Massachusetts. Main street, which carries traffic after crossing the Charles river from central Boston, is awash with Offices of some of the world's biggest tech firms. Google, Facebook and IBM. This part of the city has been called the most innovative square mile on the planet. Companies are lured in because the area is dominated by two institutions, Harvard University and the Massachusetts Institute of Technology or mit. Few places on the planet have played a more central role in the evolution of modern artificial intelligence.

Daniel Glaser (31:46)

Our quest to mathematically think about intelligence and model our brains goes back to 1943, where Warren McCulloch and Walter Pitts introduced the concept of neural networks.

Alok Jha (32:02)

Daniela Ruz is the director of the MIT Computer Science and Artificial Intelligence Laboratory, also known as csail.

Daniel Glaser (32:10)

And they published the first mathematical model that at that time was believed to capture what is happening in our brain.

Alok Jha (32:20)

If the way that neurons work in the brain can be explained by mathematics, then the brain's network surely could be replicated using computer code. Professors McCulloch and Pitts thought that machines with brain like architecture could have a lot of computational power.

Daniel Glaser (32:38)

The early artificial neuron was a very simple mathematical model. You had a computational unit that took as input data from other sources, maybe other units. The input was weighted by parameters. And then inside the artificial neuron, the computation was very simple. It was a thresholding computation. Essentially, if the sum total of what came in was larger than given threshold, the neuron output 1, otherwise the neuron output 0. So the computation was discrete and very simple, essentially a step function. You're either above or below a value.

Alok Jha (33:24)

Neurons in the human brain also operate using discrete functions, which Dan Glaser mentioned earlier. They either fire or they don't fire. A psychologist at Cornell University called Frank Rosenblatt went on to develop this model to create an artificial neuron, a mathematical function that he called a perceptron. At first, the perceptron seemed promising. After learning some examples, perceptrons could do some basic things, giving a yes or no answer to an input that hadn't been previously analysed by the machine. Let's say you fed the model some data about the strength and speed of athletes in a sports team. Learning from those two variables, the model could answer whether or not a new athlete would be likely to be accepted into a team. As the field matured, however, flaws in the perceptron became clearer. Because perceptrons only worked like a single artificial neuron, they couldn't be trained to recognize patterns that were more complex. What about, for example, athletes who were neither particularly fast nor strong, but had really good technique?

Daniel Glaser (34:32)

In 1969, Marvin Minsky and Seymour Papert co authored Perceptrons which is a book that demonstrated that mathematically, if all you have is a single layer neural network, then you could only compute linear functions. And if you can compute a linear function, then you can have a closed form solution. There's no need for machine learning. And actually, this work triggered the first AI winter because people lost faith in what would be possible.

Alok Jha (35:05)

It became clear that if artificial neural networks were to work, they'd have to have more layers of perceptrons to deal with the complexity of the real world. During the AI winter that Daniela mentioned, funding dissipated and interest in the very idea of creating artificial neural networks dwindled. There was very little Progress until the 1980s, but some researchers did persist within that time, and they found other pathways to creating intelligent machines.

Daniel Glaser (35:39)

In 1950, Alan Turing introduced a very important benchmark for machine intelligence in his paper Computing Machinery and Intelligence. This is the paper that introduced the Turing test, the idea that you know whether a machine is intelligent or not if you aim to converse with it, and if you cannot tell whether on the other side you're talking to a machine or to a person. So there have been many efforts to build intelligent machines that meet the Turing test. In particular, at mit, one of the earliest systems called ELIZA was introduced. And ELIZA enchanted people for a long time.

Alok Jha (36:26)

Eliza was one of the first so called chatterbots, which took the world by storm. In 1966, its creator, Joseph Weizenbaum, put the bot in the role of a psychotherapist. A user would type a message on an electric typewriter and the machine would reply.

Daniel Glaser (36:48)

Men are all alike. In what way?

Steve Garrett (36:51)

They're always bugging us about something or other. Can you think of a specific example?

Alok Jha (36:57)

Well, my boyfriend made me come here.

Ainsley Johnston (37:00)

Your boyfriend made you come here?

Daniel Glaser (37:02)

He says, I'm depressed much of the time.

Ainsley Johnston (37:05)

I am sorry to hear you are depressed.

Steve Garrett (37:08)

It's true.

Daniel Glaser (37:09)

I am unhappy. The early AI systems used what is called a lot of rule based systems. So you need to solve a problem, you identify some rules, and then you write a program that makes decisions according to the set of rules. This is sometimes called good old fashioned AI.

Alok Jha (37:33)

Eliza didn't use an artificial neural network and it didn't learn from its inputs. Instead, the language model recognized keywords and reflected them back in the form of simple phrases or questions, supposedly modeling the kind of conversation that you'd expect with a therapist. It was almost like a mirror.

Daniel Glaser (37:52)

ELIZA did not pass the Turing test.

Alok Jha (37:54)

Which was in fact the point. The researchers behind the bot designed ELIZA to show how superficial the state of human to Machine conversation really was, but. But in reality, it had the opposite response. People became engaged in long, deep conversations with the computer program. You know, that's really incredible. It's as if it really understood what I was saying.

Steve Garrett (38:17)

But it doesn't, of course.

Daniel Glaser (38:18)

It's just a bag of tricks.

Alok Jha (38:20)

Oh, I get it.

Dawood Dasu (38:21)

It hasn't the faintest idea what I'm talking about.

Alok Jha (38:25)

Eliza was not an intelligent machine, but it made people stop and think about what the world might be like if artificial intelligence did come along. It was perhaps also the first time that humans showed how willing we all are to believe that computers could be intelligent if they spoke to us in our own language. It's another example of what Dan Glaser described earlier as the innate desire of humans to anthropomorphise everything in the world around us. Of course, in the decades since Eliza, chatterbots became chatbots. And that's not all.

Daniel Glaser (39:01)

These days, our conversations with chatbots easily pass the Turing test.

Alok Jha (39:10)

But how did the skills of chatbots that we see today emerge from the primitive AI of the 1960s? What was it that made the theory of artificial neural networks actually work in practice? At its core, the answer lies in the insight that artificial neurons had to be layered on top of each other, like neural networks are in the human brain. And so at the end of the 1960s, researchers came up with the idea of the deep neural network. The deep learning revolution that came several decades later happened in no small part thanks to three scientists who would later become known as the godfathers of AI.

Dawood Dasu (39:51)

The systems we were building were so dumb and so weak and so difficult to train.

Alok Jha (39:57)

That's one of the so called godfathers, Yoshua Bengio. He is a computer scientist at the University of Montreal, and he was a key figure in the development of deep learning.

Dawood Dasu (40:06)

What got me really excited when I started reading some of the early neural net papers from the early 80s is the idea that our own intelligence with our brain could be explained by a few principles. Just like think of how physics works. Could it be possible that we would do something similar for understanding intelligence, and of course, take advantage of those principles to design intelligent machines? And in fact, it goes also in the other direction because there are experiments we can run in computers that we can't run on, on real brains. And so the work we've been doing in AI is informing also theories of how the brain works. So it's a two way street. So that synergy and that idea that maybe there is an explanation for intelligence that we can communicate as a Scientific theory is really what got me into this field.

Alok Jha (41:03)

Talk to us about what the challenge was in trying to model the human brain in silicon.

Dawood Dasu (41:09)

Well, we didn't try to model the human brain in silicon because that would have seemed too daunting a task. Instead, we looked at the simplest possible models that come from neuroscience and see how we can tweak them. In the early days when I was doing my PhD, we were trying to use these systems to classify simple patterns like shapes of characters or phonemes. Using the sound recording of me saying R E O, can a neural network, which is this very simplified calculation inspired by neurons in the brain, can a neural network learn to distinguish between those different categories of objects in the input? I've been working on this from the mid-80s to the mid-2000s.

Alok Jha (42:04)

What were some of the first things you tried to do with the neural networks to sort of prove that they could be useful?

Dawood Dasu (42:10)

So in the 90s, I worked on these pattern recognition tasks, both speech and image classification and industrial applications emerged. For example, I worked on a project to use neural nets for classifying amounts on checks to automate the process of making sure that a check you deposit at the bank has the right amount. And that was actually deployed in banks in the 90s and processed a large number of checks. All of the approaches that had been tried before didn't do very well because there is so much variability between people. We write in different ways. So it was not trivial. And it is something that had a lot of economic value already. To address that challenge.

Alok Jha (43:05)

Next week we'll look at exactly how artificial neural networks allowed machines to learn. And we'll also examine the clever maths that allowed all of this to happen.

Steve Garrett (43:16)

People realized that if you could insert a middle layer, which is sometimes called a hidden layer, these systems could actually compute many more functions.

Alok Jha (43:27)

That's next time on Babbage. Thanks to Daniel Glaser, Daniela Ruz, Yoshua Bengio, the economists Ainsley Johnston and all of the people she spoke to at the UK Biobank. And thank you for listening. To follow the next stage of our journey to understand and modern AI. Subscribe to Economist Podcast plus find out more by clicking the link in the show notes. Babbage is produced by Jason Hoskin and Kunal Patel with mixing and sound design by Nico Rofast. The executive producer is Hannah Mourinho. I'm Alok Jha and in London. This is the Economist.

Ainsley Johnston (44:23)

My dad works in B2B marketing. He came by my school for Career.

Steve Garrett (44:27)

Day and said he was a big roas man.

Ainsley Johnston (44:30)

Then he told everyone how much he.

Steve Garrett (44:32)

Loved calculating his return on ad spend. My friends still laugh at me to this day. Not everyone gets B2B, but with LinkedIn you'll be able to reach people who do. Get $100 credit on your next ad campaign. Go to LinkedIn.com results to claim your credit. That's LinkedIn.com results. Terms and conditions apply. LinkedIn the place to be to be.