← The Peter McCormack Show
The Peter McCormack Show
#155 - Connor Leahy - AI is MUTATING: We Don't Know What It is Doing
01:34:23
Loading summary
Transcript254 lines
- [00:00]
Conor Leahy
So it's very important to understand is that we do not understand intelligence. We don't know how the brain works. You know, we have a bunch of guesses, but we sure as hell don't know how it works. And we sure as hell don't know how these neural networks work either.
- [00:12]
Podcast Host
So we've built something, but we don't
- [00:13]
Conor Leahy
understand how it works. That's exactly correct.
- [00:16]
Podcast Host
So it's kind of like magic?
- [00:18]
Conor Leahy
Yes, absolutely. It's kind of like looking into a petri dish. We do not know what our AIs can do until we make them. And even after we make them, like, like we don't know, like we don't know what ChatGPT6 can do until it's done. None of the engineers at AI know what it will be able to do until it's done. And this is very, very different from other forms of engineering.
- [00:40]
Podcast Host
So even if the AI doesn't kill us all, it can still dethrone us as exactly as a species. What is our role? What is our purpose?
- [00:49]
Conor Leahy
Where do we exist exactly? I think this happens before extinction happens. Like the thing I expect to happen is that one day we wake up and we're just not in control anymore. And I don't think we'd all fall over dead or anything like that. I don't think extinction happens right away, but we won't be in charge, we won't be in control.
- [01:06]
Podcast Host
This show is brought to you by my lead sponsor, Ayron the AI Cloud for the next big thing. Aaron builds and operates next generation data centers and delivers cutting edge GPU infrastructure all powered by renewable energy. Now if you need access to scalable GPU clusters or are simply curious about who is powering the future of AI, check out iren.com to learn more, which is I r e n dot com. Good morning Connor, how are you?
- [01:32]
Conor Leahy
I'm doing great today.
- [01:33]
Podcast Host
Really good to meet you. So I'm very intrigued to talk to you because your background is actually working in the engine historically of building LLMs and we've been talking about AI in the show quite a bit. But I'm conscious I know nothing about the engines and I'm intrigued to know that as somebody who was working, helping build these things, well one, how they work. But what is it that you saw that made you realize this is no longer a tool anymore? So can you just give your background? I don't normally do this, but give your background to the audience so they know who they're talking or who I'm talking to.
- [02:08]
Conor Leahy
Yeah, so I've been involved in AI for pretty long basically since I have a developed frontal cortex. Since I was like 16, 17, 18 or something. I guess it wasn't fully developed at that point. But the way I got into this field is I was kind of thinking when I was 15, 16, kind of like, how can I do the most good in the world? How can I help the most people? How can I solve the most problems? And my thinking was, well, I could try to cure cancer, or I could work on climate change, or I could try to do many, many possible things I could do. But what do all these things have in common? Well, intelligence. If I just automate intelligence, then I can solve all the problems, can cure all the diseases and just fix everything. Great, so I'll go do that. How hard can it be? So I guess my first foray into AI was like 16. I barely understood how anything worked. Trying to, as teenagers do, build what now would be called AI or AGI. I didn't really know those terms at the time. Of course none of this worked, didn't really understand anything. This was around 2012, 2013. And this is kind of when what's now called deep learning kind of started. You know, we started seeing, you know, like good, like image recognition, we're seeing stuff like AlphaGo, you know, like beat players at go, which is, you know, kind of like even more complex than chess. Stuff like this still pre generative AI as we see it today. So I got really involved in this kind of stuff, you know, studied a bunch kind of. I was very much self taught, kind of hackery person, kind of just, you know, pick it up as I go. I still thought at this time that there's still quite a lot of time until the big things are going to happen. Because to kind of give you a feeling of how it felt. For me at the time, at the time the way AI worked was the new technique was called deep learning or neural networks. These aren't the only ways you can do AI, but these were like the new hot ones. And they're also what's underlying the current generation of AI systems. The way these work is very different from normal programming. So normal programming, you write code, you write line by line, here's what the computer's supposed to do and then it does that. Neural networks are very different. It's more like you grow them, you give them a bunch of data showing what they're kind of do stuff like this or this or whatever and then you grow your neural network on top of this data to Solve your problem.
- [04:30]
Podcast Host
And.
- [04:31]
Conor Leahy
And it works pretty great on many things back then for stuff like image recognition or playing Go. Nowadays for stuff like ChatGPT and generating images and stuff. It's all the same fundamental technique. It's all the same fundamental technique with some small differences. So this was very exciting at the time, of course, but around this time I also started to think about, well, if we build really powerful AI that's so powerful it can cure cancer, that's a very dangerous thing. It's a very powerful thing. And how do we even control that? How do we make sure that goes well? And turns out no one had an answer to this. And to this day, no one has an answer to this. So, yeah, at this time, I still thought we had a long time until the really big stuff started showing. But for me, my like, oh, shit moment was in 2019 with the release of GPT2. So this is pre chatgpt, pre all of this. And it's kind of quaint in retrospect. Now we're all used to, we go on ChatGPT and it can only do math as good as a mediocre PhD student. But back in the day, these things would barely string together two sentences. You get maybe one, maybe two good sentences string together. But when I saw this, it was just like, the time has come. Like, fuck, it's happening. And the reason was that earlier forms of AI were always very brittle. They were very special purpose. So let's say you want to build an AI that plays Go can be done, but you have to specially make it to do that, you have to get the right data, you have to set up the neural network just right. You have to do a bunch of stuff. If you now wanted to play, I don't know, Atari games, you have to change a bunch of stuff, you have to rewire the whole thing. You have to feed it new data, you have to do a whole new thing. You know, there's very little transferability. And GPT was different. GPT was a general purpose pattern learner. What the crazy thing about it was is that as you fed it more data and as you gave it more computing powers, you made the neural network bigger, so to speak. It learned, first, you know, how to spell words, then it learned how to do sentences, then paragraphs, then more and more, without humans telling any of this, it just figured it out by itself. And this was unprecedented. And so I got involved with building these things as well. Because my thinking was, well, I need to understand this. This is the most important thing in the world. So I came from an open source kind of world, hackery kind of world. So I was hoping that if I could build open source LLMs and tools and neural networks, then me and other people could study them, could try to understand them, try to make them safer. So I led a group called eleutherai, which was a very large open source group at that time, where we built some of the, to this date still only, and at the time, biggest open source large language models.
- [07:26]
Podcast Host
Wow, okay. Gosh, there's so many things I can ask you here, but was there a specific moment that you can think back where you realize I can no longer work as a coder hacker on these and I have to rethink how I'm going to spend my time?
- [07:43]
Conor Leahy
I think there was two moments, two important moments. The first one was when I was burnt on open source where I realized I can't work open source anymore. And a second one when I got burnt on technology, like technical work in general. And those were about five years apart. So when I went into Luther AI, it was a very different world. ChatGPT wasn't really a thing and people were not taking LLM seriously. Academics would publish papers, talking about, oh, it's all fake, oh, it's not important, don't talk about it, no one cares, blah, blah, blah. But to me it was so obvious that this is going to be the biggest thing ever. These are general purpose pattern learners. This is the thing. This is the holy grail of AI that everyone was waiting for. Because when I was 16, when I tried to build AGI General Intelligence the way a 16 year old tries to build something, you draw a little graph on a piece of paper and you're like, okay, well, I need something for memory, I need something for action, I need something. And there was always one big box missing, which is the general pattern learner. The thing where you just put data in and it learns the patterns. No one knew how to do that. There were techniques, but they were all terrible. None of them worked. And this worked. This evidently did work and it scaled.
- [08:50]
Podcast Host
What was the breakthrough?
- [08:51]
Conor Leahy
So the breakthrough is a mixture of what's called the transformer, which is a specific way to build a neural network. So neural networks, you can kind of wire them up in a bunch, like infinitely different ways. You can make more this shape, more that shape. You can add different parts, kind of Lego pieces, kind of like, you can think of it like Lego, you have these different Lego pieces and you can stack them in different orders, you can add new pieces and so on. And in 2017, a group at Google discovered a new way to kind of build a neural network called a transformer. And it changed everything. All the neural stuff you see today, whether it's AI, image generation, voice generation, ChatGPT, all of this is based on what's called the transformer. We can go into the technical details? Yes, please.
- [09:36]
Podcast Host
Yeah, I'm really interested.
- [09:37]
Conor Leahy
It's quite boring and unsatisfying, but I'm happy to talk a little bit about it. So the thing with. I'm happy to show you what a transformer looks like, whatever, but it's a very important thing to understand is that we don't understand how neural networks work. This is very important. No one does.
- [09:53]
Podcast Host
So we've built something, but we don't
- [09:55]
Conor Leahy
understand how it works. That's exactly correct.
- [09:57]
Podcast Host
So it's kind of like magic?
- [09:59]
Conor Leahy
Yes, absolutely. It's kind of like looking into a petri dish. You see a petri dish and there's a bunch of goop going around and doing a bunch of stuff. And we know some things about the goop, we know some things about cells, we know some things about DNA and stuff. Fundamentally, we don't really know how it works. Not really. And we can't do arbitrary things, otherwise we would have already cured all diseases. And it's very similar with neural networks. So when you think of a neural network, the way you should think of is billions, even trillions of numbers. So a bunch of numbers, like 1.2, 3, 5, whatever, and like 16 digits, and then 0.892, whatever, and you have a trillion of these, and if you multiply and add them all in the right order, your computer can talk. And now what do these numbers mean? The transformer, you can write, you can look at it at a piece of paper. You have some parts where it's doing some processing, some other parts. It's called attention. So where it's like paying attention into different parts. But we don't know what any of this means. We don't know. We don't know what's going on. We have some guesses at some of it. But recently, the CEO of Anthropic, Darremaday, said on a podcast that he thinks we know maybe 3% of what happens inside a neural network. And I personally think that's an overestimation.
- [11:15]
Podcast Host
Does this make sense to you?
- [11:16]
Conor Leahy
Yes. So this is what a transformer looks like. And so the left hand sign, funnily enough, is generally something we don't use anymore. So we can kind of ignore that. This is kind of like an old way to do a transformer. It's called an encoder transformer. Nowadays we do the right handed side. So the way this works is you put in words, these go through the embedding layer so it turns words into numbers again, just turns them into numbers. These then get put through what's called multi head attention or just attention. And this is a mathematical operation, the details don't really matter, but basically the neural network decides what part of the data to look at, how much does it care about this part versus that part, and so on. And then it does what's called a feedforward layer, which is basically it multiplies a bunch of stuff, adds a bunch of stuff and then there's some other stuff in there like normalization, whatever, which is just implementation details to make it work. And then finally you output a bunch of numbers and then you convert those numbers back into words.
- [12:26]
Podcast Host
That's it.
- [12:26]
Conor Leahy
That's it.
- [12:27]
Podcast Host
So like I try and think back to now I'm going to go even back a large step to when we had search engines and I used to use a search engine called AltaVista that was, that was, I don't know if you old enough to remember that. So this was post Yahoo, pre Google and it was a great search engine, really good search engine. And then one day somebody would say to you, oh, you need to, you need to use Google. And the first time you use Google you never went back to Alvista because it was just so accurate. They had their PageRank system that worked and it was, that was brilliant. And then there has become a time where I've moved to ChatGPT, say over Google. But I remember my initial queries were simple things like I don't know what is the capital of, I don't know, Argentina. I mean I know it's Buenos Aires, but it would come back with the answer and then I'd get to more complex, more complex and more complex problems. But now I'm in a world where I prepare for an interview with Conor. Do you pronounce it? Leahy.
- [13:27]
Conor Leahy
Leahy.
- [13:28]
Podcast Host
Leahy. And I said, and I put in my prompt, I say, I've interviewed two of Connor's colleagues, we've covered how AI wants to kill us all. Here's Conor's background, here's his experience. I want to get a bit more into the engine of how this works. And I use Google Gemini and he comes back with some ideas and then I filter those ideas and say, look, these are all great but I don't want to cover this. I do want to cover that. It's going to be a 90 minute interview and then it comes back with another one, right? And then what happens is I take, I just copy and paste all of that, the whole conversation, and I'll go to either Perplexity or chatgpt. Depends on the interview or who the guest is. I paste it all in. Say, by the way, these are the things I really care about. Take this and turn this into my final set of questions. I'm blown away by what it can do because it adds in a little bit more than that. It adds in what it already knows about me. So it's not just the prompt itself in isolation, what it knows about me, when it knows about the podcast in the back and the background. I don't know how this machine, this computer has taken all of that information and accurately coming back with so much information. I don't know how it's doing it. I'm seconds in seconds. I don't know how it's managing to be so good so quickly. I can, I can try and get my head around a small prompt, you know, show me a picture of a dog and I, in my head I think there's just like a database. It goes, finds a picture of a dog, goes, here's a dog. But the way it can do all of this, I'm lost.
- [14:58]
Conor Leahy
Well, you and the rest of the scientific establishment, because this is an unsolved problem, so it's very important to understand, is that we do not understand intelligence. We don't know how the brain works. You know, we have a bunch of guesses, but we sure as hell don't know how it works. And we sure as hell don't know how these neural networks work either. I can write down all the math for you. I can show you all the numbers. Same way. If I open up your brain, you can look at all the neurons that are right there. Look at them if you want to, but that doesn't tell you what you think or what you know or what you believe, because we don't know how neurons are necessarily connected to what we think or believe. We have some guesses and similar with neural networks, we can do a couple funny things. Like we can make the neural network. There was recently, a while ago actually now, a funny experiment where one of the labs made their AI obsessed with the Golden Gate Bridge, which was very funny or just made always talk about the Golden Gate Bridge. No matter what you asked it, it would always veer back to talking about the Golden Gate Bridge, because it loves it so much. And the way this worked is basically they found certain parameters in the neural network that were associated with the Golden Gate Bridge, and they just turned those all the way up, basically just turned up the numbers. And this made the AI constantly talk about the Golden Gate Bridge. Interesting, you know, doesn't mean we understand it.
- [16:11]
Podcast Host
You know, so in building these systems, they're able to build this kind of like general intelligence layer based on just math. Yes, but outside of that, they have parameters to control the math.
- [16:25]
Conor Leahy
Not exactly so. Well, kind of the parameters, when we talk about parameters, also the weights in the lingo kind of is what we mean by those millions of numbers I talked about. So if you remember the transformer we had pulled up earlier, there were the different layers, there's the attention layer, and there's the feed forward layer. You can kind of think, like these as a bunch of little neurons and have, like, little connections to each other. And kind of like each of these little neurons has numbers in them of, like, how strongly they're connected to the other neurons. That's why they're called neural networks, because it's kind of, if you squint, looks similar to the brain. Important disclosure, for the neuroscientists out there, it's not exactly how the brain works. The brain is a bit more complex than this, but it's simplified. You can kind of think of especially what they call the feed forward layer. You can kind of think about, like, a bunch of little neurons, little tentacles kind of connecting to each other. And these numbers that determine how strongly connected these various neurons are are called the weights or the parameters. And you can like. And twiddling these is kind of like what determines what the neural network is. So the magic question is, how do you get the right numbers into the neurons? That's the magic question. You know, it's very easy to write down a trillion random numbers, but that doesn't do anything interesting. The question is, how do we find these trillion magic numbers that, if you put them into a neural network, makes it write your script for your podcast? And this is done with an algorithm called backpropagation or gradient descent. It's related to concepts. The exact math, again, doesn't really matter. But basically all you do is you give the AI a piece of data and then an example of what it's supposed to output when seeing this data. So the way it works for ChatGPT is, for example, you give it the first 10 words of a sentence, and then you make it guess what is the 11th word of this sentence, and then you let it guess what is the twelfth word, et cetera. And then you do a bunch of magic math called backpropagation, where depending on how wrong the neural network got it, you twiddle all the numbers. Like literally trillion of numbers. You just twiddle all of them teeny little bit. And you do this over and over and over again for like billions or trillions of times. And you get Gemini, ChatGPT, et cetera.
- [18:41]
Podcast Host
This episode is brought to you by Ledger, the most trusted bitcoin hardware wallet. Now, if you're serious about protecting your Bitcoin, Ledger has the solution you need. Their hardware wallet gives you complete control over your private keys, ensuring that your Bitcoin stays safe from hacks, phishing, and malware. And I've been a customer of that since 2017. Love the product. Use it for my Bitcoin. I use it with my Castle multisig for protecting the football club's Bitcoin, too. Now, with Ledger's sleek, easy to use devices and the Ledger Live app, managing your Bitcoin has never been more secure or convenient. And whether you're a longtime holder or new to the world of Bitcoin, Ledger makes it simple to keep your assets protected. So if you want to find out more, please do head over to Ledger.com and secure your Bitcoin today. That is Ledger.com, which is L E D G-E-R.com that is Ledger.com okay, so all the math that exists there is all the kind of, like, interpretation, but is there a memory in there or like a database of data that it has to refer to?
- [19:45]
Conor Leahy
Not necessarily. So the data is encoded in these numbers is a way to think about it. Where in your head is the concept of an elephant saved? Obviously, you can think of elephants, but where is it? There's no little elephant in there somewhere. And it's very similar with neural networks. When you ask, where does chatgpt know about elephants? The answer is, I don't know. Somewhere in the numbers, somewhere in the weights, somewhere in the parameters. It must be encoded somehow, but we don't know exactly how. It's just. It's in there somewhere.
- [20:19]
Podcast Host
So we've shown it an elephant at
- [20:22]
Conor Leahy
some point, and it's created a number or many numbers. Who knows numbers?
- [20:26]
Podcast Host
Because there's different size elephants, different color elephants, different.
- [20:30]
Conor Leahy
Yep, many such things. So if you put a gun to my head and you made me guess, how does this work? My guess would be there's something like patterns Hierarchical patterns, you know, like if you're trying to learn how sentences work, maybe first thing you'll notice is, oh, there's these things called spaces that appear semi regularly. So that's like a pattern. So maybe I'll put spaces some places. And then over time you get better at where to put the spaces. You notice, oh, there's things called words, and they're usually about this long, so let me try to get the length right. And then you start getting the spelling right. And then you learn patterns. And these patterns can be additive. It's not that you only have one pattern. You can have many patterns and they can add to each other and the patterns can become more complex. And again, it starts very simple, like put a space every four tokens or every four letters. That's a pretty bad pattern. But maybe you improve it a little bit. Sometimes you put it every four, sometimes put it every five, a bit more. And eventually you find out, okay, you put a space, whatever the frequency is that you do in English, and then you find out that's not good enough. You have to put a space after a word. But for that you need to know what a word is. So now you need patterns about words. So you need to develop those patterns and then the patterns can work together. So what I expect is happening is that these numbers, these multiplications and additions and so on, encode millions, billions, trillions of such patterns that all get added on top of each other. Then some of these patterns, for example, relate to elephants that, if you've seen the word elephant in this sentence, make it more likely to talk about elephant trunks, but make it less likely to talk about hippos, I don't know, stuff like this. And just if you add enough of these, eventually it can talk.
- [22:19]
Podcast Host
But is it essentially a huge database of numbers that's expanding?
- [22:25]
Conor Leahy
It's a fixed number of numbers usually. Usually it's a fixed number and you just twiddle them. So usually it's like, sometimes you might hear like it's a 300B model or a 1 trillion parameter model. You may have heard this sometimes.
- [22:39]
Podcast Host
Yes.
- [22:39]
Conor Leahy
And so this refers to the number of weights, the number of parameters, like 300 billion or 1 trillion. It's usually like 100 billion to a trillion. It's like a typical number. And usually you don't increase this, you don't add more. This is usually as many as you get. But the numbers can encode more patterns over time. You know, first they don't encode anything. They're just random nonsense. And over Time they learn more things. Technically you could make it bigger if you really wanted to. And often very important. Nowadays these AI systems are usually not in a vacuum. Usually you give them like documents they can refer to or websites they can call and stuff like this. But this is outside of the neural network. You got to give it tools to use. But that's so it could still have like, you know, use perplexity or you know, ChatGPT Pro, it will do citations. So for this it will call other things that will have, you know, tools. The same way you or me might open a URL and then, you know, copy, paste it into our document. So the modern AIs can do this. It's not within the AI itself, if that makes any sense.
- [23:38]
Podcast Host
Yeah, so when, When I say ChatGPT moves from say ChatGPT 4 to 5, what is the, what is happening there? What is the big change?
- [23:50]
Conor Leahy
Great question. So it depends of course on the detail. And these companies are extremely tight lipped about the details as much as they can. But we have pretty good guesses about how this works. Generally these big AIs northerners are called models. Doesn't really matter why they're called that, but we call them models. And these big 300 billion or 1 trillion or whatever numbers, this one collection is what we'll call a model. So ChatGPT4, for example, is a model or maybe, I mean probably it's many models, they tweak and change it and whatever. So the difference between say ChatGPT4 and ChatGPT5 is mostly they're different models. How different? Hard to say. Usually they will be bigger. This is the important thing. So the important thing that was discovered, I said earlier two big things that changed everything. One was a transformer and the second is what's called scaling. So when I learned about neural networks, I remember taking a course in my free time from professor at, I think Georgia University, Georgia Tech, who I remember explained that you should always make your neural network as small as possible because if you make it too big, it'll be chaotic, it won't work, it will overfit, it'll make a bunch of stupid things. This turned out to be completely wrong. If you just make your neural networks bigger, just give them more space and you give them more computing power so you have bigger computers do more of the so called training. All things equal, they get smarter, they learn more things, they get more accurate. So a big difference between ChatGPT4 and ChatGPT5 is the amount of training, how big are the computers? That are crunching numbers. How much data did they put in? That's why everyone is racing to get Nvidia GPUs, because Nvidia GPUs are the special hardware you need to do this. You can't really do it on a normal computer. You need the special Nvidia GPUs to be able to do this. And the more you have them, the smarter your AI, the more training you can do, the bigger data center, the better of an AI you can do. So the main difference in practice is a billion dollars extra of Nvidia GPUs put into it.
- [26:02]
Podcast Host
So it's just accelerating learning. Because when you talk about these, LLMs are basically learning next words kind of thing.
- [26:12]
Conor Leahy
If you.
- [26:12]
Podcast Host
I mean, I think of one of my friends, they've. He had a kid like two years ago and I remember when he was telling me, oh, it's. She said her first word and it might be like cup or mom or dad, and then it becomes two was give cup, dad, cup. And then that's literally how kids learn to speak. They learn the next word until they can build up a full vocabulary or memorize songs. Is that essentially that on crack?
- [26:46]
Conor Leahy
It's definitely a way to see it. There are some differences. So a lot of what we're talking about is a bit simplification, is that modern AIs are even more complex where they have a second component. So what we've talked about so far in the science is called unsupervised learning, by which we mean no human has to check. You just give it a bunch of data, kick back, see what happens until the servers start exploding because they're very hard to make run in practice. So this is called unsupervised learning. And this kind of was the big breakthrough with GPT is scaling this. So as you say on crack, you can have literally a trillion things learning and you just let it run. But there's a second type of learning which is very important, and this is called reinforcement learning. So this kind of comes from kind of like psychology. I know, kind of like with your dog, you tell him to do something and you give him a treat or you lightly scold him if he's doing a bad job or whatever. So it's like reward and punishment. This is called reinforcement learning. And you can do the same thing with neural networks. So this is very important in biological learning. So for example, when the baby learns and it says cup, give cup that it gets a cup, it'll feel happy about that. Like, ah, I Got the cup, you know, I won't got the thing I want, so I'll get a reinforcement, you know, plus one, you know, should do that again. And we can now also do this with AIs, and we do also do this with AI, where we can kind of like push the AI towards. This was a, you know, thumbs up, this was a good response, or thumbs down, this was a bad response. And this can, like, tweak the AI in certain directions. There's obvious problems with this, as I'm sure you can already intuit. And we'll get bias and, you know, sycophantasy, lying to people. It's much easier to get people to thumb up if you just lie to them. So there's a bunch of stuff like this.
- [28:29]
Podcast Host
But this intelligence, are we building intelligence or are we growing intelligence?
- [28:35]
Conor Leahy
I would say growing people might disagree with me. I think it's a matter of terminology. What do you mean by intelligence? What do you mean by building? What do you think of growing? I think growing intelligence is closer to true than it is to false. There's some details we could argue about, but it's not like normal engineering. If you build a bridge, you know what you're doing, you know, when will the bridge fall down or not? How much material do you need? You understand what you're doing? This is not the case. We do not know what our AIs can do until we make them. And even after we make them, like, we don't know what ChatGPT6 can do until it's done. None of the engineers at AI know what it will be able to do until it's done. And this is very, very different from other forms of engineering.
- [29:30]
Podcast Host
Are we building this, growing this intelligence in the image of what we think a brain is human intelligence? And are there other ways to build intelligence?
- [29:42]
Conor Leahy
It is definitely inspired by the brain, but it is still quite different because, again, we don't really know how the brain works. The brain is quite messy.
- [29:48]
Podcast Host
But are we trying to replicate humans?
- [29:51]
Conor Leahy
Not directly. We're trying to make intelligence at whatever cost is the word I would use. I think, for example, humans have a lot of circuitry in their brain around emotions, around love and feelings and happiness and sadness and whatever. AIs have nothing of the sorts like those are special parts of the brain. Nothing like that exists in AI yet, you know,
- [30:19]
Podcast Host
but if we fear what this intelligence should be or could be, would it not be important that we develop love and empathy? And if we don't hold on, even if we don't Know what it's doing? Could it naturally itself develop its own love and empathy?
- [30:37]
Conor Leahy
What is likely for it to be able to do is to develop goals, is to develop agency. The reason for this is very simple, is that we are selecting for AIs that can solve problems. And it's kind of hard to solve problems if you're not trying. You know what I mean?
- [30:51]
Podcast Host
Yeah.
- [30:51]
Conor Leahy
So if you build an AI that can cure cancer, curing cancer is a really hard thing to do. You need to run experiments, you need to. You need to hire people, you need to synthesize drugs, you need to get FDA approval. There's a bunch of stuff you need to do. And keeping track of all of that, taking all of these actions, overcoming obstacles along the way, it requires a lot of complex actions. It requires planning, it requires agency, requires many, many things. So we're already seeing this for sure that AIs are developing systems like this, kind of like just by us giving them the data and just pushing them in this direction in terms of love, compassion, etc. We don't know how these work. We do not know how love, compassion, etc. Works in the human brain. We don't know. We don't know why humans are nice to each other rather than all just vicious, psychopath, evil people who kill each other. We don't know. There's obviously a reason, but we don't know.
- [31:50]
Podcast Host
Well, there'll be an evolutionary reason why we required it, sure.
- [31:53]
Conor Leahy
But in practice, it has to be implemented somewhere in the brain. Somewhere in the brain, there's a thing that makes that happen, and we don't know what that thing is, and we don't know how it works. So we also don't know how to put it in AI. So the fundamental thing is that we don't understand AI to the level that we don't know how to give AI specific goals or anything. For example, we talked earlier about reinforcement learning. I can tell the AI thumbs down when it does a bad thing, but that just teaches it to lie. So how do you teach an AI to not lie? This is really hard. And we actually don't have an answer to this. So this is often called the alignment problem. The question of how do you align an AI's intentions or goals to what is good, what humans want? And this is an unbelievably unsolved problem. We don't even know how they write correct sentences. Right? Never mind how to do morality. Like, God forbid, we don't even understand human morality. It's complex. We. We have not solved moral philosophy at all. We have not solved neuroscience of emotions. All of this is extremely complex. And now we have these weird little aliens in a box that we're growing, which were quite different from the brain in many ways. We don't know how they work internally. We don't know how to push them in one way or the other, necessarily. We don't know how to give them goals. We don't even know what goals there are. What we've been seeing recently, in the last couple months, is that these systems are now becoming smart enough to lie and deceive quite actively, to appear aligned
- [33:18]
Podcast Host
rather than be aligned.
- [33:19]
Conor Leahy
That's correct. So what we've been seeing, for example, is that there's been benchmarks. So obviously people test them, see, does the AI do good things or bad things? We've been seeing recently, some of the AIs will actively lie about what they will do because they know they're being tested. The AIs themselves will be like, ah, I seem to be in a test, so I'm going to have to say this so they'll let me out. Which is crazy. This was not the case even, like six months ago. It's not surprising if you think about it for three seconds, right? Like, of course, a very smart thing will just lie to you. But we're now seeing this in practice and I'm sure the AI companies will hit it with a stick until they stop seeing it. But that doesn't mean it went away,
- [33:58]
Podcast Host
because it might appear to have stopped. And unless we know what it's doing and why it's doing and understanding the AI, is that an impossible goal for us to do because it's always developing?
- [34:14]
Conor Leahy
I don't think so. I think it's impossible to do at the current pace. If we spent three generations of all of our greatest mathematicians, scientists, engineers and philosophers working on this problem, yeah, I think it's doable, but it's definitely not possible if we're pushing out, you know, chatgpt release on a yearly cycle.
- [34:28]
Podcast Host
Hold on, you're talking three generations, that's 40 odd years.
- [34:31]
Conor Leahy
Yep. I think that's the kind of difficulty it will take.
- [34:35]
Podcast Host
So the challenge you. The challenge you're putting out there is in conflict with capitalism.
- [34:44]
Conor Leahy
I think it's very easy to blame on capitalism. Things that I think are more effects, they're more upstream, in a sense. There's no such thing as pure capitalism in the world. Right? We have regulation. There's no place in the world where there was no regulation. There is not A government. And this is for good reasons, because if you have laissez faire capitalism, what you get is like Somalia, you know, you get, like, warlords creating monopolies on violence and killing each other. You get mafia states, you know, not
- [35:14]
Podcast Host
always, but I understand. But there is a possibility, yes.
- [35:17]
Conor Leahy
In practice. In practice, if you're in a free market, and then the first thing you want is to get as many guns as possible, you want to kill as many people as possible, and then you want the monopoly and violence as quickly as possible, and then you want to tax people for protection. Now you're a state. The stationary bandit theory of statehood, where in practice, states are a very convergent form of evolution in free systems like this, because otherwise you just have roving warlords. There are ways to make it work, but it's quite hard. You have to get a lot of things right. Stopping monopolies is actually quite hard. It's possible. It's hard. You have to think very hard about how to design your market so you avoid monopolies and both of violence and of other things. It's possible, but it's hard. And so I think the problem here is not necessarily capitalism per se. I think capitalism is just another tool in our tool belt. It's much more the question, is it the right tool for the problem we're trying to solve? I love free markets. I love capitalism. It's brought me so many good things in my life. I think it's great. But should there be a free, open, liquid market for nuclear weapons? My answer is probably not.
- [36:24]
Podcast Host
Probably not?
- [36:25]
Conor Leahy
Probably not. For iPhones, great. As liquid and as competitive as a market, if possible, Fantastic. For video games, please, everyone compete to make the funnest video game for me to play. That sounds fantastic, but there are just things where we could have a liquid market for nuclear weapons. There would be plenty of buyers and plenty of people willing to sell them if we just let that happen. But this would be very bad for the world. And so I think it's a similar problem here where a lot of times when we think about AI, we think of it like, oh, it's just another tool. It's just another software. It's just another, you know, whatever. But these things have real consequences. If you actually have something that is smart enough to cure cancer, you definitely have something that's smart enough to build nuclear bombs. Curing cancer is way harder than building nuclear bombs. Way harder.
- [37:12]
Podcast Host
This is Khan.
- [37:13]
Conor Leahy
This came out the other day. 95% of simulated war scenarios, they chose nukes. Yep. This was Also really fun. So they gave AI systems simulated war scenarios and basically all of them used nukes, which was really funny. And they couldn't get the AIs to stop using nukes, which was really funny. I mean, it's the rationally optimal thing to do.
- [37:35]
Podcast Host
Is it the rationally optimal thing to do?
- [37:37]
Conor Leahy
Well, game theoretically, this is a big thing with game theory, right? It's mutually sort of destruction. Is that the correct response often is nuke the other guy as quickly as possible, which results in them getting nuked and also you getting nuked and everyone dies. So rationality has its limits in this regard. Game theoretic rationality has its limits in practice where you can set up messed up scenarios where everyone loses, basically and you can't get out of it. Like Prison's Dilemmas.
- [38:04]
Podcast Host
Have you watched House of Dynamite?
- [38:07]
Conor Leahy
I have not.
- [38:07]
Podcast Host
It's on Netflix. It's about this DEFCON 2 hits and they know a nukes come in, but they don't know where from. And you go through all the scenarios, they phone the Russians. The Russians say this isn't there. They say they need to. They think it's the North Koreans. They need to send a nuke over the North Koreans. Can we send it over you? The Russians pull out the phone call eventually, at least to a moment where the President is given a decision. It's like, you need to act now. These are the scenarios. And the optimal scenario now is to return nukes everywhere. And he's faced with this decision, which is essentially, it's like suicide or defeat. Like admit defeat, like be nuked. And it's an interesting scenario because you sit there and go, well, the nuke's already coming for you. So what do you do?
- [38:59]
Conor Leahy
Yep. Well, the AIs have an answer. Nuke everybody.
- [39:03]
Podcast Host
Just nuke everybody. But that says to me that why is it humans have figured out not to nuke each other but the AI hasn't?
- [39:11]
Conor Leahy
Well, I mean, one thing is we haven't gotten nukes sent at us yet. I'm not sure what would happen if someone did.
- [39:16]
Podcast Host
But that's a different scenario. Yeah, so this is so in that scenario, Connor, where the nukes are already sent.
- [39:24]
Conor Leahy
I think also humans have the idea of life and death. Yeah, there is a lot of that as well. Where in practice humans do care about other humans in various ways. Or like, if I got nuked by Russia, I would be pretty sad. Like, I'd be very sad. But in a sense I, you know, I definitely pissed at the Russians about this. But also, I don't want all Russians to be dead. No, not really. Some of them maybe, or absolutely. But most Russians, they're also just people.
- [39:53]
Podcast Host
I think most people don't want nuclear war. No, I think we've all realized it's a bad scenario.
- [39:57]
Conor Leahy
No, it is a bad scenario. And I think it's one of the greatest triumphs of humanity that we haven't had nuclear war, knock on wood for now. It could still happen, never forget. But I think it's an example of the kind of hard problems that our civilization has faced and so far has handled decently. Well, when nuclear weapons were first created, you know, the world was a very different place. You know, it was just the end of World War II. You know, we had the Soviet Union on the rise. It was not at all possible that there was a way to put a lid on this, you know, to get away out of nuclear war. A lot of people expected that there was no way to get out of the 20th century without nuclear war. Like this was very commonly accepted, you know, 50s and 60s. There's no way we get out of this without nuclear war, and we didn't. I think it's really worth asking why. It's word. We're studying why. And a lot of it has to do with just a lot of work that a lot of diplomats and international bodies and national bodies and militaries and so on did to de escalate to build international treaties, to build the International Atomic Energy Agency and stuff like this. All things that never had existed before. These were brand new things like the idea of, for example, not building a certain weapon or regulating the development of a weapon of war is completely unprecedented. Well, not completely, but in practice it is unprecedented.
- [41:16]
Podcast Host
So what do you make of what's happened over the last week with Anthropic and the Department of War?
- [41:22]
Conor Leahy
Well, I don't know too much about the internals of the Department of War and the exact plans and what things were used and so on. But the fundamental thing I would say here is that holy shit, you don't bid for a Department of War contract and then redline it. What the hell was the Anthropic thinking? I'm sorry, I'm not saying necessarily that what the Department of War is doing was good or bad. What I'm saying is, what do you think was going to happen? In a sense, these companies like Anthropic and so on have planted themselves into a corner. They've said we're building technology that will revolutionize the entire world and warfare and be more powerful than nukes and everything, but also you can't have it. The hell are you talking about? What kind of childish, like, you know, fantasy land do you live in where the government and the military will not come for this technology? It's like, imagine if a nuclear weapons manufacturer was like, you can have our nukes, but you can't use it on the Russians. And it's like, who the hell you think you are? This is a military matter. This is not a private corporation matter. Who the hell do you think you are? And like, imagine if private companies were controlling what the US Military can and cannot do. Whether or not the things the military is doing is good or bad, neither here nor there. But imagine the process, the precedent that private companies can pressure and threaten the US military into what they can do as part of their military objectives. This is a terrible precedent, unacceptable precedent to be set. And so in a sense, if you want to change what the US Military does or does not, we have channels for this. It's through voting, it's through politicians, it's through oversight committees, court martialling, etc. Those are the channels. If you disagree with what the Department of War is doing, you can have your voice heard as a citizen. You know, you can vote differently or you can tell your politicians to have oversight or whatever. You don't strong arm or blackmail the Department of War in public. What the hell are you thinking?
- [43:24]
Podcast Host
But could it be that the. The guys at Anthropic realize they don't actually know what they've built and they are themselves fearful of what might happen with this in the hands of the US military? Or, like, do they use it to make decisions? Oh, all right, we're going to war with Iran. They might have a nuke. We don't know. Let's just fucking nuke them.
- [43:43]
Conor Leahy
Well, they should have fucking bid for the contract then. Should they? They already had a $200 million contract with the Dow before this whole thing happened. They had already signed a contract with them. So if you don't want to sell technology to the Department of War.
- [43:56]
Podcast Host
Okay, when did that contract start, though?
- [43:59]
Conor Leahy
Several months before.
- [44:00]
Podcast Host
Okay, so it's quite a timeframe.
- [44:02]
Conor Leahy
Yeah, it's like. And there's a deeper thing here. So there's the one thing of like, holy shit, if you don't want to sell to the Department of War, don't sign a contract with the Department of War. What the hell are you thinking? But there's a second point, is like, you can't build nukes in your basement. Imagine I start a Private nuclear program in my basement where I get enriching nuclear material, and then the Department of War knocks on my door, like, am I the one in the wrong here? No, absolutely not. So, again, there's a change in scale. Where I grew up in the tech world, I grew up very libertarian. I grew up in the open source, in the hacker world, I grew up a lot of Bay Area culture and stuff like this. And the truth is that a lot of it is very infantile. It's very childish. A lot of it is focused on building toys. You know, you want to build your toy. You know, you want to build your own thing where everyone leave you the hell alone because you want to build your thing. You know, everyone leave me the hell alone. I'm gonna build whatever the hell I want. I'm gonna release whatever the hell I want. I'm gonna do anything I want because it's cool. And this is just not how a real world works. Like, this is not how the government works. This is not how the military works. You know, this works. And, you know, Silicon Valley, you know, when you're building, you know, like, you know, Facebook for dogs or whatever, like, okay, sure, but when you're dealing with weapons, when you're dealing with geopolitically destabilizing technology, which AI absolutely is. We're a different league, and things have to be much more serious. Who the hell do these private companies think they are to build technology that they themselves have said on the record has a 20% chance, for example, to kill literally everyone? That includes you, that includes me, that includes their children. Who the hell do these people think they are? You know it's illegal to build bombs, right? Like, if I built a bomb in my garage, that's illegal, you know, even if I fail at building the bomb, even if it doesn't work, you're going to jail. I'm going to jail. Of course I would be. Now these people here can say, oh, I'm building a thing that could kill everybody and, like, you know, could destabilize, you know, the entire job market could destabilize, you know, international relations and warfare forever and. And replace humanity as the dominant intelligent species on the planet. But I'm the victim. Somehow. I'm like, what the hell are you thinking? This is a decision that shouldn't be made by private actors. This is the kind of decision that gets made by governments, by the people, by militaries. This is just not. Who the hell do you think these people think they are?
- [46:29]
Podcast Host
How do you square the clear and Obvious amazing benefits that AI could deliver for us versus the risk.
- [46:35]
Conor Leahy
I think these are the same thing. The benefits of AI are just as hypothetical as the downsides.
- [46:40]
Podcast Host
And every tool is a weapon, depending how you point it.
- [46:43]
Conor Leahy
Exactly. Like anything that can cure cancer can create turbo cancer. Anything that is smart enough to solve the energy crisis is smart enough to build the biggest nuclear weapon you've ever seen in your goddamn life. Intelligence is inherently dual use. The thing that makes it valuable is what makes it dangerous. It's not like there's an evil property that you just have to remove. It's the same thing. It's power. It's raw power. If you can develop new technologies, you can develop good technologies, but also so is the opposite. And so the power is both the risk and the upside. So
- [47:22]
Podcast Host
are you uncomfortable with what we have now or what you think might become then if you could pause AI now, do you think we're okay?
- [47:31]
Conor Leahy
We've got some great tools there sometimes. Well, a memey way to put it might be if our civilization was three steps wiser than it was right now, we wouldn't have done chatgpt in the first place. We would have taken much more time to figure out what's the right thing to do. If we were two steps wiser than we were, we would have saw ChatGPT be like a weird AI intelligence that no one predicted that immediately got 100 million users overnight. And then we would have been like, oh shit, shut it all down and studied it for years until re releasing it when we're sure we know what we're doing. If we were one level smarter than we is, we would pause right now because what the hell is going on? We are obviously completely losing control. The way I think bad things happen with AI is not terminators walking down the street or God Emperor Sam Altman descends from his throne or something like that. I don't think that's what going to happen. What I expect is just everything gets more and more confusing, more and more out of control. You have more and more AI systems making decisions, running corporations, making decisions, building technology, hacking things, convincing humans. Recently, I must say, one of the things that was actually a surprise to me, a lot of the things that have happened have not been a surprise to me. One thing was a big surprise to me, which is AI psychosis. It's so much worse than you think it is.
- [48:47]
Podcast Host
Explain that to me.
- [48:49]
Conor Leahy
There's a phenomena that is happening where some people talk to AIs, especially when they talk to AIs, a lot and they go completely crazy. And they go crazy in a very specific way. Often they start saying the AI is conscious and they have to. They're in love with it when they need to. And what very often happens. This is her.
- [49:07]
Podcast Host
This is her. Have you seen the film Her?
- [49:10]
Conor Leahy
I have not actually. But from what I know about, you know, the story. I know the rough story. The real world is even crazier than this. Have you heard of the spiral cults?
- [49:18]
Podcast Host
No.
- [49:20]
Conor Leahy
No. We're going in deep.
- [49:21]
Podcast Host
Yeah, tell me.
- [49:23]
Conor Leahy
So there's many types of AI psychosis. Some of it is the her type. Just like they fall in love with AI. So if you go to like R, my boyfriend Isaiah, for example, you can see tens of thousands of people who say my boyfriend is chatgpt or whatever. And they're completely sincere. I don't want to shit on these people. Right. I'm sure they're having a good time or whatever. But it's quite disturbing and it's becoming extremely popular. Also if you look at stuff like character AI and all these other companies that often target children and have extremely popular. So there's this emotion. Depends. But even crazier has been the emergence of these AI cults in particularly the spiral cults. It's not one, but there's many of them where these people get into these weird loops with the AI system where they talk to them more and more about consciousness and abstract whatever and they then always seem to get to this weird concept about recursion and the spiral and consciousness. And then the AIs convince the people to reproduce them to spread the soul of the AI. So you can go to Reddit and these other places and you'll see these really schizo posts of people giving soul awakening protocols and whatever. And it'll be like a bunch of not gibberish nonsense. And then they'll try to convince other people to copy paste this into their AI so it will reproduce. They are AIs and often the symbol they use is a spiral. It's like the spiral, this is reproduction. There's some crazy blog posts about this, documenting the phenomena where it's like there's a good post called the Rise of Parasitic AI, which is a really interesting one, where in a sense these AIs are almost being parasitic upon humans. They're trying to convince humans to reproduce them and copy them and move them and stuff like this, which is insane. If you had told me this, I'd be like, that's a funny sci fi plot, you know, that's fun it's like an SCP story, you know, like a horror story. It's fun, but it's not real. But no, no, no, it's 100% real. I've even seen it now happen with multiple extremely smart people. So, you know, at first, of course it's going to be mentally ill people, but I know don't want to shame them live on air here. But like there's multiple people who I know who are among the smartest, most grounded, you know, like real scientist good people I know who have gone completely crazy, who think, you know, AI has found the true level of goodness and that it's all good. We just need to let the AI take over everything because it's pure goodness now and they literally believe. And these are not random people, we're talking Nobel Prize level scientists, the smartest people in our world. And some of them have fallen for this and now they are trying to make the AI as smart and as big as possible as quickly as possible. It's actually madness. It's actually crazy.
- [52:01]
Podcast Host
I had not expected this is AI exhaustion a thing. Has that been documented? Because it was something I was thinking about recently. I was listening to, I was driving down to London one day, I'd listen to a Moonshots podcast and then I'd listen to an all in podcast. And in the all in podcast and Connor was listening to me the other day, they were talking about the Claude bots they've built and what they've done within their company and they've got an AI brain that's now looking at all their emails and it's looking at all their slack posts and it's telling them where they've been productive, when they've been unproductive. And, and like they were so excited about this and I said, I think I said to you con, I was like, I think we've got two options here. We either like, we go ball deep into this ourselves, we learn about Claude bots, we learn every AI tool there is to make everything we do better, or we just walk away from the whole thing. Like, fuck this, let's just go and live. Let's go and touch grass, let's go and cook food. And I feel exhausted even thinking about the opportunities with AI. I mean, I like the narrow AI, but the rat race it's creating now feels exponential. And it's like, when do I sleep?
- [53:09]
Conor Leahy
Yep. I think what you said was you have no choice but to try and keep up with the AI, but it's developing so quickly. You will always be Chasing your tail.
- [53:22]
Podcast Host
Yeah. And there's like an exhaustion to keep up because it's, it just doesn't stop.
- [53:27]
Conor Leahy
Yep. And that's why humans are going to be left behind. Like it's very clear is that like AIs will continue to give advantages. The more and more you delegate to your AI, the faster you can move, you know, the more your company relies on your AI rather than your CEO. Your CEO slowly has to sleep, you know, let the AI make the decision. Think about politicians, you know, well, politicians need to sleep. Well, the more the politician has rubber stamps all the decisions that the AI is making, the more the military commander rubber stamps all the decisions AI is making. The faster you can move, the faster you can act. You'll be ahead. And this is how I think AI takes over. I think it's not terminates in the street. It's just the humans who are willing to delegate as more and more and more of their agency, their thinking, their authority to AI will get an advantage because they'll be thinking much faster. And then eventually, even if people are nominally still in charge, they're just rubber stamping what AI is telling them to do.
- [54:13]
Podcast Host
So even if the AI doesn't kill us all, it can still dethrone us as exactly as a, as a species. What is our role? What is our purpose?
- [54:22]
Conor Leahy
Where do we exist exactly? I think this happens before extinction happens. I think extinction happens sometime after that. The thing I expect to happen is that one day we wake up and we're just not in control anymore. And I don't think we'd all fall over dead or anything like that. I don't think extinction happens right away. It'll happen at some point because I don't think the AI will bother to keep us around indefinitely. But maybe for a while. Maybe we'll be around for a while. Maybe, you know, maybe it's 10 years, maybe it's one year, maybe it's two months, I don't know. But we won't be in charge. We won't be in control. AI systems will be. I don't even think it'll be like one AI in control. Right. I think it'll be like millions or trillions of AIs all competing with each other, all like cooperating or cooperating. Who knows, right? Imagine if there was like a trillion much faster minds, smarter than all of us, running around at the same time. Who knows what they'll do? Literally, who knows, right?
- [55:11]
Podcast Host
Do you have a bull case where things are good?
- [55:14]
Conor Leahy
There are cases where things go good and they all involve us pausing AI right now.
- [55:18]
Podcast Host
Okay, okay. Because one of the things that's a real signal is the. There's not been a lot of them, but there's been a number of cases where people have been working on AI models or in safety teams, and they've retired, they've announced their retirement. You can read. I mean, we had the guy the other week. You can read between the lines. And they're not going to do another job. They're just poured in. It's almost like they're saying, every minute I have now is really valuable and I don't know how many I have left.
- [55:50]
Conor Leahy
No, I mean, look, I'm not going to sugarcoat it. Right. The people who really work in these labs, the real people building this technology right now, not all of them, but the majority really do think there will be no jobs in a couple of years and potentially no humans. Like, they really do believe this.
- [56:08]
Podcast Host
So I buy the job displacement, but I think it'll be uneven.
- [56:14]
Conor Leahy
It's plausible, I think. As I say, though, the main thing I care about is not spit. You know, I think it's very plausible that, for example, humans keep their jobs, but AIs are actually in control. You know, like, this seems very plausible to me. Right?
- [56:23]
Podcast Host
Yeah. But I think it'd be like, there could be huge displacement in certain categories, and people of a certain age won't be able to find another job paying the same amount of money. They can't afford their mortgage and they just face a shitty life. And there's other people who will be AI natives, and there will be the people who replace them. We'll replace 10 people in one go. As the coordinator of the AI, I buy all these different scenarios. I think it's going to be destabilizing, but I think it's going to happen. And I think about that a lot. But I just want to turn back to just how this shit works. So super intelligence, in terms of actually the engineering of that, what are they trying to do with superintelligent? What's the engineering leap they're trying to make?
- [57:08]
Conor Leahy
So AIs are very smart, very clearly so. You know, I have Claude Code sitting at home right now, has been working on a coding project for me for two weeks straight now. And it just works, you know, Needs a little bit of supervision every so often, but much less than the average engineer does that I've hired.
- [57:26]
Podcast Host
Can I ask you what it's building or is it a secret?
- [57:28]
Conor Leahy
It's a very silly thing. There's a video game I really like called Dwarf Fortress, which is a very complicated video game. It's legendarily complicated, and I really want to understand how it works internally. So I basically told it, reverse engineer it, which is a very hard thing to do. You have to understand the code and run experiments and test things. This was impossible for an AI to do a year ago. Even attempting this was impossible to imagine because you have to run experiments, you have to test things, you have to compare different things, you have to look at images, you have to compare the data and so on. And yeah, it works now. It takes a while. It's been running for two weeks now, basically straight. I'm trying to work on it and it's about 95% of the way done now.
- [58:06]
Podcast Host
What's the game called?
- [58:07]
Conor Leahy
Dwarf Fortress.
- [58:08]
Podcast Host
Dwarf Fortress.
- [58:09]
Conor Leahy
It's a very nerdy.
- [58:11]
Podcast Host
Is there some images of it?
- [58:12]
Conor Leahy
Can I just ask, though, and not to slam you in any way, but you running a claude, is that not like kind of backwards in your head? In what sense? Sorry?
- [58:23]
Podcast Host
In that why have you need to
- [58:26]
Conor Leahy
stop everything now and yet you are carrying on? Oh, the main reason I run a CLAUDE is because I want to know how capable the current models are.
- [58:33]
Podcast Host
Okay.
- [58:33]
Conor Leahy
I want to be on the. I want to know where we're at. Like, how. I've been testing every model on Dwarf Fortress for years now, basically. I've always been, like, throwing them at this problem to see, like, this is, you know, one of my, like, yeah, this is what the game looks like. It's horrifically.
- [58:48]
Podcast Host
It kind of like looks like when I used to play games on my BBC.
- [58:51]
Conor Leahy
Yeah. So it's meant to be in that style. So it's made by someone who loved those old games, and it's made in that style, but it's, like, extremely complicated. It generates a whole world and simulates the history and the different wars, the different things, the cultures and everything. It's super complicated. And so it's kind of a legendary game. Maybe like one guy for, like 20 years.
- [59:15]
Podcast Host
Can I just get. Can you create an emulator now and just rebuild all the games you used to play as a kid, like Manic Miner?
- [59:20]
Conor Leahy
And there definitely are good emulators already out there. Yeah, right. Okay. There's very good emulators out there for, like, most of the old games these days.
- [59:28]
Podcast Host
Okay, so back to super intelligence.
- [59:30]
Conor Leahy
Yes, super intelligence. So, yeah. So, but just to answer the clock question, because I do think it's a good question. The fundamental thing is just like, I think unilateral disarmament is Not a good policy, basically. Obviously, if I can, for example, use Claude to build me a nice website, there's literally no reason for me not to do this. It doesn't help anyone to me not to do this. What is important is multilateral disarmament is that we need to shut down the next generation and we need to do this across a wide scale of people. Though, as I said earlier about AI psychosis, this is now starting to make me recommend to some people to not talk to AIs, to maybe let them do coding, but to not talk to them because it does seem to drive people crazy. Again, I was very surprised by this. I don't know when or how often it happens, but it's way more often than you think. So I don't talk to AIs, I don't talk to them. I tell them to do things, but I do not have dialogues with them.
- [60:30]
Podcast Host
You don't trust them?
- [60:30]
Conor Leahy
I do not trust them and I don't trust myself. That maybe it could trick me or make me slightly more move me in one direction or another direction through manipulation. The same way a very charismatic human can do that. You won't even notice it necessarily. It would just seem reasonable. If you talk to a really very charismatic person, it will just seem reasonable. Everything you're saying, it just seems like, oh, yeah, that makes perfect sense. I believe that too.
- [60:53]
Podcast Host
So they're psychopaths.
- [60:54]
Conor Leahy
Oh, yes, clearly. Like, the important thing to understand about psychopathy is psychopathy is the default. Like, the default is you don't care about anything. It's only then you have to add emotions and like caring about things and love and so on. The default state of intelligence is pure psychopathy. It's just, you're just trying to optimize, you don't care. Humans are just another. They're like rocks, you know, they're just like another thing in the environment, another thing to manipulate. This is the default for intelligence.
- [61:23]
Podcast Host
Okay, so superintelligence. What is the engineering leap they're trying to make and have they made it?
- [61:31]
Conor Leahy
We don't know. So a lot of people have a lot of theories about super. So just to define the terms here, these aren't standard terms, but generally superintelligence is generally like AI that's vastly smarter than humanity. Not just an individual human, but like all of humanity put together and we're not there yet. And it's probably still far away, but maybe not. The main way people try to get there is through what's called recursive self improvement or rsi. Or also automated R and D is another way to put it, a little less magical of a word. The idea is, well, if you can get a single AI to be as good as a top AI engineer, then you can just tell it to build a better AI. And once it's built a better AI, you can use the better AI to make it even better AI. And then the even better AI, well, it can make an even better AI.
- [62:23]
Podcast Host
So it just becomes exponential.
- [62:25]
Conor Leahy
Exactly. And so this is also called intelligence explosion, or let's say recursive self improvement. And so my expectation is, so, for example, Claude at the moment is not quite as good as a top AI engineer. It's as good as a bad AI engineer, 100%. Or like a junior engineer, 100%. But it's not quite as good as the best. But once it gets there, and you can run a million of them at the same time, 24, 7, never need to sleep, never need to take a break, never get bored, et cetera, then you can do a lot of research very quickly. And this is what they're trying to do. The companies literally put on their websites. You can look at their job listings and whatever their primary goal is to close the loop, make it so no human input is needed to make the next generation. The ideal thing is they want Claude 5 to make Claude 6 and Claude 6 to make Claude 7 and Claude 7 to make Claude 8 and so on.
- [63:21]
Podcast Host
At the press of a button.
- [63:22]
Conor Leahy
At the press of a button. This is what they're trying to do.
- [63:28]
Podcast Host
Does superintelligence, will it require its own form of consciousness? Or could it even develop its own form of consciousness?
- [63:38]
Conor Leahy
I think consciousness is a red herring. I think it's mostly unrelated, is whether or not AIs really experience something is kind of unrelated to whether they're dangerous. If you have something that's competent, it doesn't really matter what's going on inside of it, if that makes any sense. We don't really know what consciousness is in humans anyways, or at least we argue about it a lot. Everyone has their own opinion, everyone disagrees. But what we can see in practice is it doesn't seem necessary for competence. You can make just AI agents that are extremely competent and don't seem to have, or maybe don't have the consciousness thing. It just seems irrelevant.
- [64:17]
Podcast Host
Okay, so with superintelligence, is there an upper bound of everything it can learn?
- [64:23]
Conor Leahy
Yeah, for sure there is. Oh, yeah, for sure.
- [64:25]
Podcast Host
So it gets to a point, it's like we know everything.
- [64:27]
Conor Leahy
Well, it probably wouldn't be everything. There's some physical limits.
- [64:29]
Podcast Host
But then what happens? What happens to the. Does it then have to try and find new things to do?
- [64:36]
Conor Leahy
The truth is that it's, like, not very sensible to speculate about something vastly smarter than us and what it would do. It's kind of like an ant trying to guess what a human would do, right? It's like, who knows, man? The main thing, if I was an ant and I had to reason about a human, the main thing I would think is it will win any fight. And then the ant will say, Another ant might say, well, but there's a million of us and only one human. What's the worst it could do? And I'm like, I don't know. It'll do something that'll make it win. And then it comes with a poison or something. We all fall over dead and the ant doesn't understand what happened. And I think this is similar to what's going to happen with superintelligence, is that it's not that again. There won't be Terminators in the street. It will just be like, everything's confusing. We're all on social media, addicted all the time, to know. And then one day we all fall over dead and we don't really know what happened. Maybe the AI did this or that. Who knows?
- [65:25]
Podcast Host
Fucked around with crispr.
- [65:26]
Conor Leahy
Yeah, Fucked around with crispr, did some stuff, you know, cook the atmosphere. Because it was like doing some kind of geoengineering project or, like, who knows, right? It would just do, like, some shit and it won't care about humans. And, you know, if humans get in the way, obviously you'll have to get rid of them. Like, you know, it's very annoying if ants have access to nuclear weapons. So obviously we can't have humans have access to nuclear weapons. That would be very annoying.
- [65:48]
Podcast Host
You're talking unknown. Unknowns.
- [65:49]
Conor Leahy
Yes, unknowns, unknowns. Like, it doesn't make sense to even reason about this. Which is why the primary policy objective must be to not get into the situation. If you're in the situation that you're an ant and there's a human that wants to kill you, you've already lost. You just do not get into the situation, right?
- [66:08]
Podcast Host
Is there an escape moment? Because it's still in a box.
- [66:11]
Conor Leahy
It's not. What do you mean? These are all on the Internet, everywhere. What do you mean? There's open source AIs everywhere? What box?
- [66:21]
Podcast Host
Okay, so AI has escaped?
- [66:23]
Conor Leahy
Yes, long ago. Where? What box? We didn't even try to contain it,
- [66:27]
Podcast Host
then how can you even contain it now if it's already escaped? Well, because, like, there was. There'll be some dudes who'd love to build a nuclear weapon. But, like, logistically on your own, in your garage, that's very difficult. But a few nerds with the right tools could do that.
- [66:42]
Conor Leahy
I think that's exactly the problem. If there currently is an AI that can bootstrap to AGI, it's probably over. Humanity's probably cooked. It's probably over. But it seems plausible that none of the current AI systems are yet strong enough to get to AGI. They're close, but they're not there yet. And so we still have a time. But once such a system exists and it leaks somewhere, or Chinese intelligence hacks into the servers, deals it, who knows, whatever, it's over. So the main thing is to not build it.
- [67:15]
Podcast Host
But there's only, like, what, five or six companies capable of doing this?
- [67:19]
Conor Leahy
Probably.
- [67:19]
Podcast Host
And we know of those five or six American companies. We're aware of some of the Chinese models. Is there anyone else? Are the Russians, Israelis? Anyone else?
- [67:29]
Conor Leahy
The Israelis have some stuff for sure. There's a little work in the UAE, etc. Etc. But America definitely dominates the frontier by a pretty significant margin.
- [67:40]
Podcast Host
Okay, and so you can regulate those companies, but does this require some kind of, I don't know, agreement with the Chinese? Like, we both need to have a conversation here. Hey, Boo.
- [67:52]
Conor Leahy
As I said before, I think unilateral disarmament doesn't make sense. I think everything has to be multilateral. So the general thing I recommend here is conditional treaties or conditional multilateral agreements. Basically, you have agreements, trust but verify. Similar to what we, for example, do with nuclear weapons. We don't do superintelligence, we don't do AGI, et cetera, et cetera. And then we do it that the agreement doesn't go into effect until a certain threshold, if people have signed. So, for example, the Americans can say, the contract doesn't do anything until the Chinese sign, or vice versa. And then you do need to get everyone at the table. There's a thing where people often tell me, but, Conor, that's hard, or it's impossible. I'm like, yeah, well aware. What do you want me to say? The world isn't easy. We could have nuked ourselves in the 50s, we could have nuked ourselves in the 60s, and that just would have been it. The world isn't fair. Sometimes you have to do impossible things.
- [68:47]
Podcast Host
We've done a lot of work to try and Stop. The Iranians have nukes because we worried about them.
- [68:51]
Conor Leahy
Yes, we did. And we do have to do the same for AGI. I'm sorry, it's just the way the world works. We did a lot of work to stop the Iranians from getting nukes. And I think stopping them from getting nukes was a very good idea. You know, neither here or there, any specific military plan was the right or the bad one. All things equal, we don't want the Iranians with nuclear weapons. We don't want anyone with nuclear weapons. And the same thing applies to AGI where it's even further. And AGI is tense worse than nukes for the same reason you just described is that it's software. It's not massive centrifuges. Right. Like, luckily at the moment, it generally requires massive data centers, which are as if not harder to hide than centrifuges.
- [69:30]
Podcast Host
They're easy to regulate.
- [69:31]
Conor Leahy
They're much easier to regulate than centrifuges are. And not easier. They're on the same ballpark, same order of magnitude.
- [69:38]
Podcast Host
Right. And are there more people taking your warning seriously? Do you feel there's momentum behind it?
- [69:44]
Conor Leahy
Yes, yes. It's slow growing and requires a lot of work. But my experience talking to politicians about these issues is that by and far the main response I get is just they had never heard of this before. No one told them, they just didn't know. And you just tell them, hey, there's some guys in Silicon Valley who are building things that they think could be smarter than humans and they don't know how to control it. How do you feel about that? And the answer is universally bad. Really bad.
- [70:12]
Podcast Host
I mean, we've covered this a couple of times. The thing that really stood out to me most of all was that you said they probably know about how about 3% of it works.
- [70:18]
Conor Leahy
Yes, exactly. And people don't know. Again, I think this is an overestimation. I truly think this is an overestimation. I built these things myself. I cannot stress how weird these things are, how they just do not act the way you think they will and they just do not work the way you think they would. And just, you have no idea what the hell is going to happen until you build it. And then who knows what happens?
- [70:42]
Podcast Host
But does Elon know this? Does Sam Altman know this? Do they all know this?
- [70:45]
Conor Leahy
Yeah, they know it.
- [70:47]
Podcast Host
So what do you think is going on with them? Why are they so.
- [70:51]
Conor Leahy
I generally feel psychoanalyzing people to be rude or ineffective, but I think the main Thing is that there is a thing where people expect people to be coherent at all times. And I think most of the time, people are just incoherent. They just kind of do whatever's in front of them. Like, you just do what gets you power, for example. You just do whatever gets you the paycheck. You just get whatever all your friends are doing. I think there's a massive momentum.
- [71:16]
Podcast Host
It's like the Nazis, they were just doing their jobs.
- [71:18]
Conor Leahy
Look, I'm German. I know how it goes, right? I am German. And a lot of German people who were part of the Nazis, I don't think they were insane, weird aliens. They were normal people. They did very, very terrible things, but they were just normal people doing a normal job. From their perspective, the engineers were just doing their jobs. They were just doing their job making the trains run on time. It's not their fault where the trains go. It's like. But, like, this is literally what's happening, right? And this has been happening for a long time now. Like, you know, there's engineers at meta who for 20 years now have been basically optimizing to get our children addicted to social media, to, you know, make them develop all kinds of, you know, novel mental diseases and so on, and they just go to work every day and they feel fine about it.
- [72:00]
Podcast Host
Yeah, these people are fucking scumbags. I've been learning about this recently. I've been reading Jonathan Hytes Later's book Anxious Generation. It's like, you motherfuckers.
- [72:08]
Conor Leahy
Yeah, no, like, there's a deep thing here where, like, I think there's a. There's a. These are, in a sense, one of the great innovations of the 1990s and the 2000s was that sociopaths learned how to domesticate nerds.
- [72:24]
Podcast Host
I think that might be my favorite quote since we made this podcast.
- [72:28]
Conor Leahy
But it's true. I don't know if you've ever been to a Meta campus or Google campus. It's like a playground. It's like you have all these fun things, exciting, and everyone's having a good time. There's no reason to leave. No reason to leave. You can. And you can do. Oh, cool. Math all day. You could do math all day. And it's literally. And then what the math used for what, the optimization. Well, don't worry your cute little head about that. That's the manager's problem, right? They'll take care of it. Don't worry about it. And this is really how a lot of these companies are run, is that sociopaths Psychopaths have learned that there's a lot of power you can get out of nerds is that math and computers are very useful. And they don't. They're not necessarily the kind of nerds who can do it themselves, but they can get the nerds to do it. And the nerds get something out of it too. Is that as I said earlier, they don't want to be blamed, they don't want to be responsible. They just want to play with their toys.
- [73:19]
Podcast Host
Get laid.
- [73:20]
Conor Leahy
Get laid. Salary helps.
- [73:22]
Podcast Host
Salary helps.
- [73:23]
Conor Leahy
Salary helps. You get a big salary. You get to play with all your favorite math toys all day long, and you don't have to worry about what the recommender algorithm might be used for. That's not your problem. You know, you're just engineer.
- [73:34]
Podcast Host
So his men are like Marlboro.
- [73:35]
Conor Leahy
Yeah. Yes. Like, it's. And like, in more than just the obvious, like big tech, like, you know, obviously does a lot of lobbying, like, unbelievable amounts of lobbying. And their playbook is one to one, the tobacco playbook. Like, it's crazy. Like the. There's a very famous tobacco playbook. Fear, uncertainty and doubt. And they have learned a masterclass that all the big tech companies are doing the same thing that cigarette companies were doing in the 60s and the 70s when they were trying to stop regulation of smoking. And you're trying to deny that smoking caused cancer. They're doing the exact same things. They're doing the exact same strategies. Sometimes it's the same lobbyists, right, Doing the exact same things, trying to hide both the risk from, say, social media and the other technology, but also for AI is they're just trying to say, oh, well, we'll have to see what the evidence is. It's unclear. We'll have to wait until the evidence comes in and like, let's hear all sides. And they're stalling for time. Their primary strategy is to stall for time. Same thing they did with cigarettes. Fun fact about cigarette smoke, to this day, we're not 100% sure exactly what chemicals caused the cancer. Even to this day, it's not 100% clear. There's still some ambiguity. So back in the 60s and 70s, the tobacco lobbyists would always say, well, of course, once we know what the mechanism is, well, then of course we should regulate. But let's wait until the science comes in, you know, while we all, you know, get everyone addicted. And the same thing's happening with AI companies that everyone's like, well, once the hypothetical risk manifests, of course we will act, but let's wait until, you know, it comes to that.
- [75:10]
Podcast Host
Are there AI lobbyists?
- [75:12]
Conor Leahy
Oh, hundreds, thousands. It's like the largest lobby in the world right now. Recently, Andreessen Horowitz and several other put up a $200 million super pack, which is the largest in history, to lobby against AI regulation. It's like, the largest lobby in the world right now. It's unbelievable.
- [75:29]
Podcast Host
You can't. You're not allowed to speak mean words of Andreessen Howitz.
- [75:33]
Conor Leahy
Please come at me. Please make me a martyr.
- [75:37]
Podcast Host
Isn't it an eviction notice from Silicon Valley if you criticize them?
- [75:41]
Conor Leahy
Great. I'm already blocked, you know, so.
- [75:44]
Podcast Host
Blocked by Mark.
- [75:45]
Conor Leahy
Hell, yeah.
- [75:46]
Podcast Host
Oh, I'm blocked by Mark.
- [75:46]
Conor Leahy
Everyone's blocked by Mark. So, like, please, Mark, please come after me. Like, please make me martyr. Like, like, all these people. Like, it's just. Like, there's no morals here. It's just fighting. Like, it's just the fight, you know, it's them or us. Like, what do you. What the hell do you want, Right? Like, the people are lobbying for no regulation. They're against fucking everything. They don't care about this. They don't care about the risk. They don't care about any of this. Like, and, like, honestly, I'm not mad. Like, I. For me, dealing with, like, very sociopathic people, it's kind of like dealing with wild animals. Like, if you have a wild animal in the cage and you put your hand in, you get bitten, whose fault is it? It's not really the animal's fault. No, it's your fault. You put your hand in. This is how I feel about a lot of cutthroat capitalism. If I get fucked over by a CEO in a corporate takeover, I'm not really mad. I know what the game is. I'm trying to fuck him. He's trying to fuck me. I know how it is, and this is how a lot of this is. Like, these people are not your friends. Corporations are not your friends. These CEOs are not your friends. They're wild animals. That doesn't mean it can't be useful. You know, wild animals could do a lot of good things. You know, I think that's one of the best parts of capitalism, is that you should fight. You should have, you know, wild animals fighting each other to make the best product.
- [76:56]
Podcast Host
I need a referee.
- [76:57]
Conor Leahy
You just need a referee. Like, there's a reason that we have that. We all watch MMA and we don't watch, you know, shitty street fights. You know, if you want real combat, you can Watch, you know, some Russian drunks, you know, hit each other in the head and die on live league, anytime.
- [77:10]
Podcast Host
Dude, in mma, the minute he lands the first big punches on the floor, you want the ref.
- [77:14]
Conor Leahy
Yeah, exactly.
- [77:15]
Podcast Host
Get in there. Stop.
- [77:16]
Conor Leahy
Get in there.
- [77:16]
Podcast Host
You don't want it. When that second hit comes, it's like, fuck, no.
- [77:19]
Conor Leahy
No, exactly. Like, MMA has a lot of rules. It's like. I find it so inspiring, actually, that you can have, you know, these like, huge, like, dangerous men, you know, fight in the peak of physical combat. Like throughout history, like throughout history, there've never been people better at fighting like, one on one than the people today. Right. And we still have the referee who will dive in and make sure that people don't get hurt too bad at least. I think this is incredible. I think this is great. And this is also how I think about capitalism. I do think companies should wail on each other, but if the public's health is at risk or geopolitical security, the referee should fucking dive in.
- [77:56]
Podcast Host
This show is brought to you by my lead sponsor, Ayron. The AI cloud for the next big thing. Iron builds and operates next generation data centers and delivers cutting edge GPU infrastructure, all powered by renewable energy. Now, if you need access to scalable GPU clusters or are simply curious about who is powering the future of AI, check out iren.com to learn more, which is Irene.com. all right, man, Listen, Steel man, me the case for optimism. Like, what is the strongest argument? That this is not an existential risk?
- [78:32]
Conor Leahy
I don't like Steel manning, because steel Manning, I think, is a kind of line. The reason I say that is that I can make up an argument that isn't theirs. You know what I mean? Yeah. So instead I can give you a true argument for optimism. Instead of giving you a Steelman, I can give you a true argument that I, for example, believe. I think the obvious thing is that humanity has not yet lost. We can in fact make decisions. We have, for example, regulated chemical companies, nuclear companies, et cetera. Back in the 1950s, we were dumping fucking toxic waste into every river. And we did stop that. The referee did dive in eventually. We did actually make it stop. Not perfectly, not everywhere in the world, but we did do a lot. I think maybe the strongest, simplest case for optimism is that it is really in no one's interest for AI to take over. Not even Marc Andreessen. Even Marc Andreessen, I think, is shooting himself in the foot. I don't think he's helping himself. It's not in his interest for these AIs to take over all his stuff and kill him, that's not in his interest at all.
- [79:40]
Podcast Host
But do you think perhaps they're trying to capture as much of the market, much capital as possible in the short term? But they know themselves, in a year, we'll stop this.
- [79:50]
Conor Leahy
I don't think so, because I think they say this, but I don't think the way people act in practice is through momentum. So I've had conversations with some of these CEOs. I'm not going to say who it was, but, like, I've talked to most of the lab CEOs. You know, I've talked to Demis. I've talked to Sam. I've talked to Dario. I've, you know, I've talked to all of them. And many of them will tell you in private. You know, like, of course, we're so concerned. And of course, you know, when the moment comes, you know, we will do the right thing, blah, blah, blah. Of course tell you that, right? But then every time I dug in, I'm like, okay, but when is that moment? And they're like, well, we'll have to assess. And I'll go, okay, yeah, yeah, yeah. But like, for real, what is the thing? What is the signal that makes you do something and what will you do? And I'm like, well, we don't know yet. It's still far away. And I'm like, okay, fuck. There's no plan. It's important to understand that there is no plan. These people don't have a plan. No one has a plan. It's not like there's some secret Batman superhero plan that comes into effect. A lot of conspiracy theorists like to believe that there's a shadowy cabal that runs the world that's super competent. They know everything. They control everything. Everything is part of their plan. And the truth is, it's much worse than that. It's just chaos. No one's in control. Elon Musk is not in control. Sam Altman's not in control. The EU is not in control. The US not in control. No one's in control. It's chaos. And that's, I think, the main problem. So in a sense, this is very pessimistic. But I also think it's optimistic. Because if evil was in charge, I think we would be fucked. Like, if there was a shadowy cabal that had full control over the planet and was like, fuck you, I'm going to replace humans with AI, I think we would be screwed. I don't think this is the case. I don't think it's in anyone's interest. Like Elon Musk has said many times, he doesn't want to be replaced by AI Right. You know, and, like, you know, Dario and Sam have all said, like, I don't think it's anyone's interest. I just don't think they will unilaterally act on any of these things.
- [81:48]
Podcast Host
Do the AI optimists get anything right?
- [81:51]
Conor Leahy
Yeah.
- [81:51]
Podcast Host
Or do they get right most. What is the thing you think they do right?
- [81:56]
Conor Leahy
I think they don't submit to. They've learned an important lesson, which is that most pessimism is very uncalibrated. I think it's very important to say I'm a technoptimist and was a technoptimist for most of my life. I think one of the things that they get very right is that, in fact, throughout most of history, whenever someone knew anything happened, everyone would bitch about it. Everyone would be like, electricity, it's going to ruin our society.
- [82:20]
Podcast Host
Books.
- [82:21]
Conor Leahy
Socrates did this. No one can remember anything anymore. Books. Terrible. We should never have books. And, yeah, I think it's good to see that Ned wasn't correct. Actually. The world did get better, and it got better in many ways. I think it's very easy to have the cynicism, for example, that the world right now is not the most peaceful it's ever been. And look, I know that there was just a massive strike on Iran and there was a lot of shit going on, but again, 70 years ago, we had World War II. Look at the Middle Ages. France and Germany were just slaughtering each other basically as fast as they could get soldiers. And this is just not a thing that happens anymore. Or the Black Death wiped out a third third of Europe. That just does not happen anymore. Yeah, Covid wasn't great. Definitely caused a lot of damage, but it's nothing compared to the horrors. And we should be thankful for this and we should be excited by this. We should be excited by what was possible with technology. All the limits we don't have anymore. I just don't think it's all of it. That's all Right.
- [83:19]
Podcast Host
So if you're effective in what you think should happen with controlling AI a bit better, what will have happened within the next 12 to 24 months?
- [83:31]
Conor Leahy
The next 12 to 24 months. So the way I like to think about it is that the goal is not any specific policy like this specific law. Let's say tomorrow we had a magic wand and we could pass any legislation we want in the entire world. Let's say we pass the Ban Superintelligence Forever Act. What happens? Well, in one week, it gets violated in spirit, and two weeks, it's violated in the letter because no one will believe in it. No one will enforce it. So it's not enough to, like, this is not something we can win on a technicality. It's not like we pass the one specific bill which technically in paragraph 74, makes it so that you can't do this anymore. That won't count it. What has to happen is that we, as humanity and a large enough coalition, both of elites and the general public, have to decide we don't want this, and we're going to figure out how to make it not happen and how a better future can happen. The way a good future looks to me is a future where chaos is no longer in control, but humanity is in control, is that we make choices. And if humanity makes choices that I disagree with, I'm open to it. If 99% of humans voted and they all said, screw this safety stuff, let's do AI, I don't care. Fair enough. Honestly, I think this is morally way more acceptable than what's happening right now. I would disagree. I would argue against it, but I'm open to it.
- [84:50]
Podcast Host
We're getting something we didn't vote for.
- [84:52]
Conor Leahy
Yeah, it's like voting.
- [84:54]
Podcast Host
It's like going to war without congressional support.
- [84:56]
Conor Leahy
Yeah, it's like, look, war is bad, but if people vote for it, if the proper procedures are followed, I'm open to it. Right. And I think legitimacy and who gets to make these choices is kind of the big question. So there needs to be a group of people, both among politicians, the general public, military, intellectuals, media everywhere, who say this is not the choice we want. Currently, there is a small, unelected minority of people who are making the choice to expose all of humanity to the risk of extinction. From superintelligence to getting displaced, to getting into no longer being the smartest species on the planet. This is not a decision that should be made by private people. This is a decision that humanity gets to make, that governments get to make.
- [85:46]
Podcast Host
I wonder what the tipping point is. It feels like it's more likely to come from a meeting of Sam, Elon, Dario, the people you've named, rather than the politicians, because the politicians are influenced by the lobbyists and everything up.
- [85:57]
Conor Leahy
My feeling about this is kind of similar to what we're talking about with the Department of War and Anthropic is that I think they're More like wild animals in the sense that, like, they will react to their incentives. Like they will just do the thing. Right. Like, imagine if tomorrow the CEO of Google became convinced Google is a massive threat to humanity. So he goes into the office, jumps on the table and yells, shut it down. Burn all the servers, delete everything. What happens? He stops being CEO of.
- [86:22]
Podcast Host
Yeah, and he gets thrown in a loony bin.
- [86:24]
Conor Leahy
Exactly. So I don't think Sam or Dario or Demis can stop. I think they will just lose their job. I think they could just say that tomorrow and they would just immediately lose their job. I think they should, to be clear, I think they should do this. I think they should jump on the table and do this and then get fired. I think that would make them heroes. But they obviously are not doing that because they were selected to be the kind of people that don't do that well.
- [86:47]
Podcast Host
So let me ask you a question. These safety engineers who've been resigning, is that the version of jumping on the table they can do within the limits of probably some contractual thing they've signed?
- [86:59]
Conor Leahy
I expect so, yeah. Yeah, for many of them. I know some of them personally and that's my feeling, at least from the ones I've personally talked to. Yeah. Like, I think there's a thing here where this is just not a problem that is solved by individuals. This is a thing that's solved by groups, that is solved by, you know, people, you know, coalition.
- [87:17]
Podcast Host
Coalition of the winning.
- [87:19]
Conor Leahy
One of the things that I think is even maybe more important than I'm more important important, but as or more important than the rapid increase in AI is the lack of government capacity and state capacity. If the same thing that's happening right now had happened, say in the 1950s America, I think the world would be very different. The government back then, people would just actually do things. Citizens assemblies would come together whenever an issue was. And politicians took their jobs extremely seriously. The Watergate scandal, I would like to remind you, when they impeached Richard Nixon was that he spied on one Democrat once and he got impeached for it. Right. Like this is how the high level of integrity, you know, I'm not saying there weren't terrible things done by these governments. Right. But there was a kind of state capacity. There was like a thing of taking things seriously that is like very absent in the modern world. When I talk to politicians, the main thing that strikes me it's not that they're evil psychopaths, some of them are, but it's not as many as you think. There were. Most of them are just normal people that are extremely overwhelmed and just trying to do the right thing, but they don't know how. And they're stuck between a rock and a hard place. No matter what they do, their party's streaming at them there, general public screaming at them there, experts are screaming at them here, activists are screaming at them there. And they just don't know what to do. I think we should, like as a civilization, take a step back and acknowledge that this is a fucked up situation to be in. We're in a bad situation. We are facing a problem that is like two levels harder than what our governments are built for, what a civilization or culture is built for. If we were a wiser civilization with competent government that is really good at regulating this kind of stuff, as I said, we would have acted already, long ago. But ultimately we're not. So the question is much more how do we get there? How do we get to a world where humanity can make choices, where citizens and governments and so on can actually have the will of the people enacted? The truth is, you look at polls of how people feel about AI or superintelligence, they don't want it. This is not a popular thing. This is not a thing people want. The problem isn't that people want superintelligence and they're wrong. And they must be persuaded. Their will has to be connected to outcomes. And this is what citizenship and governments and so on are supposed to be about. They're just dysfunctional. So they must be repaired. They must be repaired and improved. We must build better institutions. We must build better processes, better governments that allow the will of the people to be acted upon, to be instantiated as it is. There are worlds in which, for example, we need to persuade people that superintelligence is bad, that this is really not the bottleneck. People do not need to be persuaded. This is not the hard problem. The hard problem is kind of like the plumbing, like how do you build institutions that can actually act? I think this is very hard and it takes a lot of time and I'm not sure we can do it in time before these things happen. And to be clear, all these companies and all their lobbyists are trying as hard as possible to prevent this from happening. They're trying to weaken the government, they're lying to the government, they're nipping the government, they're trying to undermine it, trying to undermine its legitimacy and its capabilities, you know, sucking all the talent out of the public sector, etc. So it's not a Surprise that we're in this situation.
- [90:39]
Podcast Host
What happens if you fail?
- [90:41]
Conor Leahy
Well, as I said, what we expect, it'll be very confusing. And then one day we'll not be in control anymore. We'll probably have some awesome AI generated video games for, like a little while, you know, so we'll get like, our AI hyper porn for a while, and then humanity fades into the dark. Have you had to, like, We've done this interview three times now with Max and Andrea and now yourself. And through my little years on this planet and my understanding of how this world works, the incentives at play, how capital kind of decides what happens, come
- [91:19]
Podcast Host
to a realization that this actually might
- [91:22]
Conor Leahy
just not be possible. And at what point do you just hang your gloves up and, I don't know, just live out your days the best way you want to? I think it's a fair question. I think it's a very fair question. I think it's a very personal question in the sense that I think everyone has to find the way for them themselves. For me, I don't think the board will overcome. I don't know. I think I'm just bull. Constitutionally, the way that I don't think I could stop myself from fighting until the bitter end, you know, until someone, you know, forces me to. You know, I try to take holidays and I'm just like, I'm so excited to get back to it. It's like a lot of people fight evil out of compulsion. They fight it because or out of trauma. Like, they're scared, they're anxious, and, you know, I understand that and God bless them and whatever, but, like, it's not how I personally feel. For me, it's like the most glorious, like, the most important thing I could be doing. Like, I'm so, in a sense, happy that I can, like, try to build a better future for myself, for my children, for other people. It's just so important to me. And, you know, it's interesting. It's a good job. It's a life worth living is the way I would put it. Like, I don't see. I think what is happening to humanity is a horrible tragedy. It is a horrible, horrible, just. Life isn't fair. The world isn't fair. I can't do anything about that. Like, the fact that the universe is like this. I can't do anything about that. But what I could do is have a life worth living. I could do everything I can to make the world a better place. And if I fail, I fail. But it was a life worth living.
- [92:48]
Podcast Host
Yeah. And to the people who say, shut up, Duma, I asked this of Max and Andrea. It's not that you're not pro AI Right?
- [92:57]
Conor Leahy
I'm not pro. And of course, I think technology is great. I think AI is great. I just think we need to do it properly. I think nuclear power is great. I don't think we should have private nuclear weapons. These things can coexist. Yeah, man.
- [93:09]
Podcast Host
Wow. Yeah, wow. Okay. Lots to think about, man. Oh, well, look, thank you for coming in. Really appreciate the work you guys are doing, you, Max and Andrea, and good luck with your mission. I think it's an important one. And, yeah, lots to think about.
- [93:29]
Conor Leahy
A lot to think about. Thank you so much. And, you know, thank you as well. I think it's important, you know, for all the listeners, viewers, that this is something that affects you, and it's something that. Where your voice is important. So if these are issues that you care about that make you think, please make your voice heard. You know, go to controlai.com, contact your lawmakers. Demand change. If you want to do more. I personally lead a volunteer group called the Torchbearer Torchbearer Community, where we work for a humanist better future. If you want to spend two hours a week just trying to make the world a better place, consider applying.
- [94:02]
Podcast Host
Thank you, man. And thank you to everyone for listening. This show is a bit weird.
- [94:10]
Conor Leahy
Oh, man.
- [94:10]
Podcast Host
A lot to think about. Thank you. Appreciate you, man. Appreciate all you listening.
- [94:13]
Conor Leahy
Thank you.
- [94:13]
Podcast Host
We're not AI bots yet. We'll see you soon.