A (3:00)

I'd also add that just in terms of who's coming to the table with the best bargaining position, the United States is. So if we carve the world into thirds, we'd be giving away more than we already have and more than we already influence, including Russia as one of the three participants in this. Well, that's really, really kind to Russia. And even if it's direct thirds, China might just want to bully their neighborhood. The United States does not want to bully neighborhoods. I mean, they shouldn't want to bully neighborhoods. But the United States is the only one where our ideals are additive. That very last thing that David Frum said, even if you don't believe in democracy, or even if you think the United States has a bit of a hypocritical record on human rights, the liberty of trade is a very good answer for why we don't give in to spheres of influence. Because the Chinese may or may not be happy enough to trade with their partners. They certainly want to trade with us, but they're not into liberating anyone through trade. And even if this isn't our intention, this is what happens. And so the United States retreating is not just a surrender. It's poor tactics, it's poor strategy. And not for pie in the sky ideals, but pies made in Jamaica, shipped to the United States, shipped further to Japan with material sourced from Indonesia. That kind of pie on the ground and on the sea and in supply chains, it makes us all a lot better off on the show today. Well, you think that's one big issue of our time. Let's talk AI and let's talk about it with AM Andy Mills. He's put together the last invention of podcast. It is an excellent story, an excellent history, an excellent debate podcast, and an excellent explainer through its eight episodes. And he takes you through all the key issues to orient you so that if, like me, you listen to the podcast, you are now very informed in the contours of the debate. That will define us for the next, I don't know, ever. Quarter century. Some of the people Andy quotes say, well, there might be a quarter century for whoever our successor is, but it's not going to be humans. All right, let's orient our machines, our podcast machines, our portable audio players. Listen to me and Andy Mills talk. You know why we make New Year's resolutions? Well, it's because what we did in December kind of destroyed us. All those holiday parties and the skipped workouts and maybe some late nights decorating the tree. So, you know, you have to get a bit healthier and maybe you're thinking things like gym or sleep. That's hard. What's not hard, but depends on you recognizing how important it is, is to get the right amount of healthy water. And that starts with COVID Pure. When you think about the garbage that might be in your water and therefore in your body, you're starting off behind the curve before you even begin. So Cove Pure changes all that immediately. Their technology is certified to remove up to 99.9% of contaminants. Pretty much anything that isn't water. You know, pfas and microplastics and pharmaceutical residue and fluoride, it all gets removed. This is the purest water you can get. And there are a couple points about COVID Pure that when I extol their virtues, I don't talk about enough. One is with the touch of a button, you get hot water so you don't have to put on a tea kettle. Tea kettles are nice seeming and sounding, but they burn a little bit and they take many minutes. The hot water is great and also you might be tempted to install a complicated system right into your pipes. But if you rent, that's not even an option for you. So this is really the best way to purify your water and by extension your body if the house you live in is not your own. I use Cove Pure every day, many times a day, and it is quite delicious. I will attest to that. Cove Pure makes it so easy to get pure water with the push of a button. So this year, make a New Year's resolution that sticks. Improve your health with clean water. Right now you can get $200 off for a limited time if you use my link covure.com the gist here comes the spelling portion of the show. C O V E p u r e.com the gist to start this new year right I'm here to talk about performance in bed. No, not monkeys jumping on it. Not the North American Snoring championships, not folding a fitted sheet. You know, performance in bed. I'm here to suggest you take control of ED with personalized treatments made with doctor trusted ingredients. It's 100% online. It's HIMS. You can access all of this if prescribed. They offer treatment options ranging from personalized products to trusted generics that I'm going to say this is important and might help with performance. Just knowing this that cost 95% less than brand names if prescribed. It's not a one size fits all kinds of care. It's a real doctor, it's a real medical professional. They will put your health and goals first. Real medical providers to make sure that you get your results. Don't get intimidated. ED is really quite common and the way HIMS goes about attacking it and not just it, but a lot of other things is the most efficient, most streamlined way to to get you what you need. Simple online access to affordable care for ED, hair loss, weight loss and more. Visit himss.com the gist that's himss.com the gist for your free online visit himss.com thegist Featured products include compounded drug products which the FDA does not approve or verify for safety, effectiveness or quality. Prescription required. See website for details. Restrictions an important safety information. Actual price will depend on product and subscription plan. We're here with Andy Mills, the brilliant podcaster behind the brilliant podcast. The last invention, which of course refers to the spinning jenny. Is it the loom? Is this the last invention?

C (9:32)

You've got it.

A (9:33)

So the last invention is a I and that. Wait, what's the phrase that hits on at least two levels? The guy who invented the phrase, I think meant we've perfected it, but now I take it to mean this could end us all.

C (9:48)

Well, IJ Goode, true thinking machine. I believe it was a ultra intelligent thinking machine. Maybe the last invention man ever need make.

A (9:59)

Need need.

C (10:00)

Right? And this guy, he was friends with Alan Turing and he earlier in his career hadn't been like super obsessed with Turing's obsession and actually creating a thinking machine, what we would today call a true artificial AI or an AGI. But as the idea turned into a field and as the field started to make its early discoveries, I.J. goode wrote this paper where he was essentially predicting what's called the intelligence explosion. And he starts off this paper that he wrote in 1965 saying, maybe we need to make this and maybe it would be the greatest thing ever, because if you could create a true artificial general intelligence as we'd Call it today a true ultra intelligent thinking machine, something that could think at least as good as we can, then that thing could build the next thinking machine, which would build the next thinking machine. And then you'd have this like, exponential.

A (10:57)

Explosion of insight and thought and power.

C (11:00)

And capacity, and that we would never need to invent anything again because it would invent all the future inventions. Yeah. And at the time in 1965, you got to think that this is the early environmentalist movement. This is people who are worried about the Cold War nuclear winter. People are worrying about existential threats. And this technology, this dream of this ultra intelligent machine was seen as like, maybe something we would have to build to protect ourselves against other existential threats. But he also ends the essay by saying that if we don't ensure it's docile, if we don't maintain control of that machine, then essentially the last invention man ever need make would be the last invention. Very dramatic.

A (11:51)

Yes. So we're gonna get to, as the show gets to an episode, 6, 7, and 8, the three schools of thought about this, the implications of AI. But let's define terms. It's necessary. And you just threw a term out there a couple times, AGI. So AI we all understand to be artificial intelligence. AGI, as you said, is artificial general intelligence. Adding the word general is pretty anodyne, but it changes everything, actually, in terms of what it means.

C (12:22)

Right. This is a phrase that mainly became popularized by this guy Shane Legg, who's one of the founders of DeepMind, now acquired by Google. They're behind the Gemini chatbot and a lot more. But the general is important because it's saying right now we have automated algorithms that are recommending YouTube videos. Perhaps someone right now is listening to this podcast because an AI system thinks it will like it. Sure. And we've been interacting.

A (12:54)

God bless that. All we're trying to do with these videos, this is to try to latch to the AI.

C (12:59)

These are the AIs.

A (13:00)

We show it a little leg. Yeah.

C (13:03)

So to distinguish the AIs we have now from the benchmark that they are going for, they throw this word general in there. And this is far more exciting than the phrase AGI makes it sound. The goal is to create an automated system that is as intelligent, as capable of learning new things as a very smart human. They believe that once we hit that generalized intelligence, that benchmark is for many of them, a hinge moment in human history, that there will be a before and after, much like the discovery of electricity, the discovery of fire, and this is why there is so much hype, but also so much fear associated with artificial intelligence. Because it used to be this long off dream, this idea that maybe one day, even Alan turing in the 1950s, he believed maybe one day likely one day many, many years from now, we could create something like this. But because of the recent developments in the past seven to three years, even the people who were poo pooing this idea, even the people who are making fun of the technologists who thought it was possible in our lifetimes, all of them have been so surprised by the rapid developments that they're seeing that now some of them are saying, this could be here by 2028. This could be here by 2030. This. Like we may only have 3, 5, 7, 10 years to prepare our societies for this massive change.

A (14:41)

Are the developments via the mechanism they put their fingers on, which is that the machines can learn from themselves. Is that why the acceleration, not to borrow a phrase there, is that why the acceleration has occurred? It's like proof of concept. Once the machines start learning from this themselves, we get exponential growth.

C (15:01)

That's the dream. Like that is what they all are wondering when we'll hit that. When will we hit the moment where it is largely teaching itself how to be more intelligent? Yeah, we aren't quite there.

A (15:14)

Is that the singularity? Is that what they mean by the singularity?

C (15:16)

Some people call it the singularity, yeah.

A (15:18)

But there's another. Isn't there another singularity out there?

C (15:21)

There's also the singularity. That is when the line that separates humanity from our technology is crossed. And where there's. This is the. The transhumanist singularity.

A (15:34)

Right. It's great that the word singularity has dual meanings. Fantastic.

C (15:37)

Well, another meaning, because it comes from the black hole.

A (15:39)

Right.

C (15:40)

It's. It's a phrase borrowed from.

A (15:42)

When you get close to the black hole, time stops operating, as we would think it would. So there's another really important part of the entire concept of AI which is unlike every other invention, say the loom, the spinning jenny, you keep coming back to the steam engine. Big ones, big inventions. The people who made it knew how it worked. That's how they made it. And it really is the case that AI, unique among inventions, is a black box. They don't know. They don't really know how it works.

C (16:16)

Right. I mean, as a reporter, this is one of the reasons it's such a captivating story, because you have the people who dedicated their lives to chasing after this dream, to building this technology, and who spent most of that time in obscurity, in failure, in disappointment, sometimes in ridicule. And right now, those people are in a new debate. The debate is called, all right, we just made rapid and shocking progress. What do we do next? Some of them are excited and they think, let it rip, let's get faster. We'll get into the debate that they're having. Let's bring this on as quick as we can. Others of them have quit their jobs to go out and warn the world that we aren't prepared for what's coming. And some of them take it even further, and they think we have to stop this technology from taking another step forward. And the people who are most concerned, they point to the fact that no one exactly knows how the current models we have, which everyone agrees aren't dangerous in the existential sense. We don't even know how they work exactly. And yet we know that they're growing in their capabilities and their intelligence. We know that we're plugging them into everything from our web browsers to our national security systems. And they're trying to raise a flag and say, hey, pay attention to this. Don't just think about the cheating in high schools. Don't be too distracted by the day in, day out political dramas that they believe that when we look back on this year, we will wonder why this wasn't the center of every single debate we had.

A (18:05)

So what do they mean exactly by they don't know how it works? Because I think about algorithms and there was a lot of attention a few years ago, YouTube algorithms were recommending more and more extreme videos after you watched a video.

C (18:18)

And I also did a podcast on that called Rabbit Hole. It's right. It's a New York Times. Very good.

A (18:22)

So the people who made the algorithms didn't initially say, oh, yes, that's what we meant. They had to kind of take a second and say, wait, why did this happen? Then they go back, back, engineer it, and they could figure it out. So you could argue they didn't know that it would happen, but then they went back and figured out how it worked. Are you saying with AI they can't and figure out how any of these things came to be Kind of.

C (18:47)

I mean, it's a more, it's, it's not just more extreme than what happened with the algorithms. Although in many ways the algorithms were our test drive for the big race we're in now. And there are some similarities. It's an exponentially more complex system to the point where what the LLM chatbots, which are central to all of the big AI companies and their excitement right now.

B (19:12)

Right.

A (19:13)

This is what we interact with when we go on chatgpt. The large language models trained on the corpus of every Reddit and every New York Times, and they stole every book.

C (19:22)

And yeah, yeah, also a great story, very, very fascinating story, but there's no way that anyone could go through the billions and billions and billions of numbers that are central to the operating system underlying these LLMs. Right. It is an unfathomable amount of data that they are consuming and that's running their quote, unquote intelligence. And we just lack the ability to comprehend the details of what's happening inside of them. And that's not controversial. There's a lot that's to be debated, but there's no one who thinks we have a comprehensive, good understanding of how they're working, how they're making some of the capabilities. In fact, it's one of the most exciting fields in education and academia right now. If you are out there wondering what you could study in the world so that you're prepared for the job market disruption from AI, you can get a job studying what the hell is this AI doing? How is it working? And it's a fascinating part to cover as a reporter because in many ways what these labs are doing is on the one hand, you can think of it almost like part of the lab is trying to increase the intelligence and the capabilities of these AI systems. Another part of the lab is saying, how does this work? Why did it do that? It's a game of translation. I think of it almost like that movie with Amy Adams arrival where they are trying their best to communicate with an alien intelligence. The difference is the alien intelligence is being made by the very lab that is then trying to translate the inner.

A (21:05)

Working so they get. They don't crack the code. But it's not like the people who are tasked with cracking the code have struck out all the time. They get some insight.

C (21:15)

Yeah.

A (21:16)

So let's go to something in the show. And this was a major watershed in AI. The they, as we know, AI is better or ch. Computerized chess beat humans all the time. And AI allowed jeopardy to be won by the computer. And then the game Go was the next in line. And Go is supposedly more intuitive and more inherently human and harder to crack. And AI Go beat the greatest human in Go. And I think it was move 37 that. Yeah, yeah. So in move 37 was this crazy move and everyone thought it was a terrible move. And it turns out to be a Brilliant move. Just take that. Have they gone back and been able to crack the code on move 37? Did they figure out why it was a brilliant move? Did they. Did other GO players say, ah, that gives me an insight how to use such a move. Did they even figure out how the computer got to move 37? Not at the time when it shocked everyone, but in the years since.

C (22:14)

Well, the short answer is not really. Obviously with I've said this. Well, the short answer is not exactly. When you try and put together what are the milestones that moved the technology world from their deep skepticism that we were ever going to see anything like AGI in our lifetimes to this moment we're in now, where they are investing so much time and so much money in it that it feels as if they, if they fail, that is going to affect the entire world economy. Right. What is responsible for that conversion? That flip, one of the milestones is move 37 from DeepMind and their AI system, AlphaGo. And yes, it is a, it's a great moment in the story in the podcast because it surprises people not just by the fact that it beats the world's best Go champion. AI's had beat the world's best chess champion before, but it's specifically the kind of technology that it's based on, which sounds like a sci fi movie when you hear it in detail. And the fact that it did something that people thought was a completely original move and some people would even describe as having an original thought.

A (23:33)

Yeah, yeah.

C (23:34)

And this is something that obviously the DeepMind team is going to study. They wrote papers about it, they assigned some of the world's best PhDs to try and figure out what was going on, what led to this. And it's not like they learned nothing. But it is true that at the end of the day, it remains a mystery. And it's one of the things that's so intriguing about it. But if you talk to the people who are involved, they would say that this shouldn't surprise us because the strategy of the AI researchers that have found, you know, this success in this kind of AI, that we're all dealing with the hype and the fears around right now, they've been trying to model it off of the human brain and human intelligence. And of course, no one knows exactly how human intelligence works. So in a way, the mystery is evidence that they're on the right track.

A (24:31)

Yeah.

C (24:32)

It's not mysterious how a calculator works, and a calculator will never be AI. The fact that the interworkings of this remain a mystery to some of them is a sign that they're on the right track.

A (24:45)

So this, to me, if I were to give my recommendations and Ted Cruz listen to me, or the people who make the recommendations or implement them, I would say do whatever you can to crack that code, too. There's a. There's a phrase of unwieldy phrase in the show about mechanisms. I think of it like the clear box right now. AI Is a black box. We have to make that box translucent, transparent. So whatever it takes to understand that if Robert Fulton had no idea how the steamship worked, that would be a problem. And Oppenheimer and the bomb. Right. There were questions about that. As you bring up in the show. Will this, once it goes, once it's first exploded, lead to a chain reaction that destroys humanity? A good thing to study. That's where I would. Because it's very hard to stop all of progress, but I would just try to understand progress a lot. And that is Mike Pesca's solution for AI but now, as to some of the people who are even more studied on this than I am, you talk about the doomers, the accelerationists in this middle camp. Your phrase, the scouts. The doomers are famous people within the world who've won big prizes. And they're doing things like warning us that there is a 20% chance that this destroys humanity. How would it destroy humanity?

C (26:03)

Well, just, you know, the doomers, many of them have a 98% sure higher likelihood. And some of the criticism of them is that they're a little bit too certain.

A (26:15)

There's. There's no limiting factor. There's no limiting factor to how high you put those odds. Right. And like, if it doesn't happen, I don't know, you just go and say, oh, the guy who said 20 and the guy who said 98. It's not like the guy who said 98 is shunned at parties, but the guy who said 20 is like, you might have been right. Anyway. Yeah. So you talk to these guys, and I understand their concerns. And it's a black box, and it's unbelievably powerful, and it could end humanity. And you did this on the show. Well, how could it end humanity? And we got a bunch of different answers. So what were the most plausible to you about how it. How a machine, which I guess you can unplug. Quoting. I'm quoting the general from War Games and the Whopper. Just unplug the damn thing. Yeah. What are the plausible explanations for how.

C (26:56)

It ends humanity well, my job, thankfully, is not to engage in predictions, but from the people who are the most concerned, who I've interviewed, most of them are pretty humble about the fact that, like, no one can get granular on their details. They keep coming back to this very simple argument that they're making, which is that if you create something that is more intelligent than you are and you don't know exactly how it works, and you don't have control over that thing, how do you think it will go? Well, that's their big question. And then they build off that to try and paint a scenario to help us connect with their fears. And one of them that I find, you know, not too hard to imagine is one where the AGI we were talking about becomes an ASI, an artificial super intelligence. So if an AGI, as an AI system, is capable as a super smart person, an ASI or a super intelligence, that is when the AGIs make themselves and make the copies and make themselves make the copies, and they hit this level where they are now as capable and intelligent as a civilization. And that if that leap happens, of intelligence and capability, just look around the planet right now. How many species are being controlled and dominated by species of lesser intelligence? They say not many. Right? The reason that human beings have such a powerful reign on the planet is not because we're the biggest and the strongest and got the best fangs. We have brains and we work together. And they say that these systems may become more intelligent than we are. They definitely can work together and they can share their knowledge way easier than we can. We have to, like, make podcasts and listen to podcasts and read books, Right? This is all very time consuming. They can just send their intelligence, quote unquote, across their system. And they predict that it just may be the case that as they get more and more intelligent, they are not interested in us.

A (29:07)

Right?

C (29:07)

And it's kind of fun to think about. I hate to say that, but it is kind of fun to think about. And I think it's one of the reasons that people like to listen to the Doomers is because it stretches your mind in a way that's like a philosophy class in undergrad. But just imagine how hard it would be to explain to our caveman ancestors the kinds of things we care about today, the kinds of things we worry about today, the anxiety that we feel about all these small social sins that we commit and our status and standing. Think about a basketball game and how much we pay people who are amazing at basketball and how much we pay to go see Them. It'd be hard to explain to our caveman ancestors why this is all so important to us. And they are saying we are much closer in intelligence to our caveman ancestors than we are likely to be to these artificial super intelligence.

A (30:04)

Yeah. And we'll be back in a minute with more of Andy Mills. You know, when I'm seen about town wearing my camel hair double breasted overcoat and they have two, but mine's in the more camel y brown Carmel type color. I draw stairs and it's because much like their responsible down hooded parka, no camels were actually injured. But I create quite the show. I cut quite a figure and I am warm. Quince has unbelievable cold weather coats for men and women and everyone who is even smaller and maybe having a little bit of mucus dribbling out their nose. Put them in a quince, they'll look better, they'll feel better, they'll be warmer. They have men's Mongolian cashmere sweaters, wool coats, leather suede overcoats. It's impressive. They have, like I said, the down jackets and the camel hair and Italian leather outerwear. They cut out the middleman. Sorry, Middleman. You can still qualify for an excellently priced quint product. Costs as a middleman or woman, you are still in America or Canada. But because the middleman is cut out, there are no markups. They give you unbelievable quality. The same quality as luxury brands at a fraction of the price. Sounds like a cliche about how Quint does it, but that really is how they do it. So you'd be a fool not to take advantage of the fact that middlemen are getting cut out left and right. If the middleman economy has to suffer, I mean, shouldn't you be the beneficiary? Plus middlemen will be warmer. You got to see some of their Italian leather outerwear that the middlemen are wearing. Now refresh your winter wardrobe with quince. Go to quince.com the gist for free shipping on your order and 365 day returns now available in the cold land of Canada as well. That's quincy.com/the gist. Free shipping and 365 day returns. Quince.com/the gist. Everyone deserves to be connected. That's why T Mobile and US Cellular are joining forces. Switch to T Mobile and save up to 20% versus Verizon by getting built in benefits. They leave out. Check the math@t mobile.com switch and now T mobile is in US cellular stores.

B (32:35)

Savings versus Comparable Verizon plans. Plus the cost of optional benefits, plan.

C (32:38)

Features and taxes and fees vary.

A (32:39)

Savings with three plus lines include third line free via monthly bill credits.

B (32:42)

Credit stop if you cancel any lines.

A (32:44)

Qualifying credit required. And we're back with Andy Mills talking about the last invention. Let's get to the concept of worthy successors. I also.

C (32:59)

Are you familiar with the worthy successor theory?

A (33:03)

Tell me of AI. Oh, by the phrase, I can tell what it is.

C (33:08)

Well, this is a growing movement and lots of people in technology.

A (33:11)

This is crazy. This is crazy.

C (33:13)

But they believe that it may be our destiny, our duty as human beings to create the next superior intelligent species.

A (33:26)

Well, it's not a species, it's a computer. It doesn't have.

C (33:30)

Well, this is an interesting debate that I didn't know about before I did the podcast.

A (33:33)

It doesn't have a soul.

C (33:35)

It doesn't have meat yet. Although some of them think that that is a matter of time as well. But they would say that we got here from being single celled organisms. What a crazy, hard to believe journey we've been on through biological evolution. And a lot of them don't think that it is outside the realm of possibility. In fact, many of them think it's likely that we could create an intelligent, conscious species that just goes through a digital evolution and much faster, and that it might be better than us, that it might be more ethical than us, that it will not be burdened by the biological evolutionary envies and desires and the violent tendencies that we have because of our evolution. Right. We lived in very different circumstances in the past where maybe some of these now considered antisocial behaviors were not just forgiven, but encouraged. And they think we might create a digital species truly worthy of handing over Dominion to. Sure. And that it will do a better job than us. And that we should be excited about that. And that in a way, our humanity, our legacy, will live on through this intelligent system that they believe will populate not just the planet, but the universe. And it may sound crazy.

A (34:56)

Yeah.

C (34:56)

I guarantee you, if you're hearing about this for the first time right now, this is far from the last time you're going to hear about this. These people are growing. It's a small subculture, largely Silicon Valley and other places in technology, but it is growing rapidly. And some of the people who subscribe to it are among the billionaires who are funding this AI investment we're seeing. Okay.

A (35:20)

Computers are nicer than people. Right now, my. Right now, my. My car doesn't engage in war. I don't think it's a better species than A person, it is still a machine. But okay, I understand that the guys in Silicon Valley, some of them are drawn. Yeah, yeah, yeah. To interesting ideas. I haven't had the breakthrough where I believe that yet. So the answer is that these doomers tell you it could happen by them, I guess, shutting down all of our power and then controlling our killer drones and, you know, shooting us. Or one of them said something like, it could happen by AI developing better pornography and more arresting media and then. And creating more fake news. So we don't, we don't know what's real and fake. And to me, that said, all right, this is what you're articulating is just a basket for current anxieties.

C (36:15)

I'll push back on that a little bit. I think what we assume they mean when they say that is that they are actively the AI systems, we are giving them evil motives and saying they are going to manipulate us with this super porn that they create that's even more addictive and even more perhaps bad for our souls and psyches than current day porn. And they're going to use that to make us docile. Right. A lot of these AI doomers, they're not saying it's going to be intentional. They're saying it's more like an Aldous Huxley situation where we are so entertained by these technologies, we are so distracted by these technologies, we create all of these incentives for the technologies and the people who are behind them to capture more and more of our consciousness that we essentially sleepwalk into the day when we've given the machines so much power. Right, but this is that they are now in control.

A (37:20)

This is my point. This is not an AI story. This is a general anxiety about the state of the world story. This is Aldous Huxley and this is Wally, and this is amusing ourselves to death. And this is not totally something to dismiss, but, oh, if I accelerates this trend, then we will become the people who are taking Soma. Is that what it is in Brave New World? Okay, sure. But I have to discount, though they're experts and might have won all the touring awards, I kind of say to myself, all right, what's going on here is you're worried about certain other things. And you, to your great credit, you talk about the general idea of there's been doomerism with every form of technology. And I thought this was a good answer. I forgot which of these doomers you asked it to. And he said something like, don't examine the psychology of the doomer in the abstract. Think about the actual arguments the doomer is putting forward and that was good enough. But I do have to come back to, you know, there was doomerism and there I guess still is in certain circles around peak oil and King Hubbard, remember, remember this concern. It wasn't even that long ago, oh my God, we're going to run out of oil. Now it seems like the idea is, thank God, we're going to run out of oil. And Paul Ehrlich and the population bomb, that seemed absolutely plausible. We're going to, you know, Malthusian resources will outstrip our means of production. I think, I think the environmentalism is a lot more legitimate. And I do. It's very hard for me not to put the AI doomers in that camp just because we've seen. I'm not even talking about the crazy God hates us is sending a comment. Comet, I'm talking about. I understand technology. I am an expert in technology. I've seen the trend lines. This isn't going to end well. That is a constant among every nation. Technology that captures our imagination.

C (39:17)

Well, one of the reasons that I wanted to do this story is because I have a allergy to predictions of apocalypse. I grew up a fundamentalist Christian in the Midwest and in some ways there's a lot of great aspects of growing up that way and I'm grateful for those. But there are also a number of things I left behind that I'm grateful to never engage in again. And one of them is this fear that I had at a very young age. I lived with a fear that we were on the verge of the end of the world, the return of Christ, the saved going where the saved go and the damned, which I feared I was going where the damned go. And I don't think it's good to live with a constant fear of apocalypse, whether it's the climate change or the AI apocalypse. But I also want to examine my own biases and the fact that there are a number of people who understand the technology far better than I do who were taking seriously the existential risk of this technology. I wanted the opportunity to sit down with them and give them a chance to make their best pitch. I will. As a human being, I have to admit I still can't quite get to existential threat. I am not walking around day in, day out with that fear of apocalypse. But I have been convinced that AI is not a hoax, it's not fake. That the technology, whether it's in this 5, 10, 15, 20 year window that a lot of them think it is, or Whether it's more like 100 or 200 years, it is likely to be a hinge moment in human history. And I think it is good that a number of people, even if their concerns sometimes slip into a certainty that can be off putting, I'm glad that we are having this robust debate. I think that it is good for societies to engage in debate, even when they sometimes slip into exaggeration. I am trying with this podcast to tell people that this is the debate that's happening among the handful of people who work in these technology spaces and have all this power. And I think it should take up far more of the public's imagination than a lot of the shit we debate about. Because it may turn out, even if they're just half right, it may turn out that we wished we would gotten in on this debate a lot sooner to help have some sort of an influence on the impact it's going to have on our society. And that said, if it turns out that they're totally wrong, and 20 years from now, we are looking back on the Last Invention podcast. In a world where AI really didn't become much more than what it is right now, I still think it will be this interesting time capsule to say this is what we thought was going to happen.

A (42:25)

Now, we were talking a little bit about the doomers, and I had a critique which of their doomerism. First we talked about how would the doom come. Fair enough. And then I kind of psychoanalyze them a little bit. Now, the deal with the accelerationists is you don't need anyone with a degree to psychoanalyze them. Almost all of them are very literally invested in AI. They have a lot of money to make if AI succeeds. And you get into this in the podcast, yes, it's true, but they will argue that bad guys are trying to make money on it. So I'm a good guy trying to make money on it to counter the bad guys. I talked to a guy who was, who is in charge of an institute that really thinks about this, and he is optimistic and he says that, look, there are other things that humans have invented that we've stopped ourselves from destroying ourselves. And the two examples are nuclear weapons, which you deal with in your podcast, and biological weapons. Also gene splicing. Like there is a agreement in the world not to do human gene splicing and we don't. And that that is held. And so I asked him, can anyone make a trillion dollars on gene splicing? It was like, no, that's. That's the Thing, no one's making that much money on nuclear weapons and no one's making that much money on, on gene splicing. So given the enormous economics, how much do you discount? And then we'll get into the specifics of what the acceleration is. Bring it on. More AI is better for everyone. How much do you discount their assessment?

C (43:58)

I want to push back on this idea out there that in some ways in here, in this room with you, Mike, I think it's a bit too cynical to think that at the core of this technology we have the age old story of capitalists looking to make money, that this is John D. Rockefeller with AI Oil. It's a far more interesting and complicated story. The truth is that many of the people who are right now leading these AI labs trying to make AGI, trying to hit this benchmark as fast as they can, they truly believe it is dangerous. And yet they truly believe that the potential for it to be this absolutely amazing life saving human life altering technology is likely and capable if it's done safely, if it's done right. Their motivation, yes, of course they've got a number of complex reasons for this, but their motivation at least in part is almost more like a religious component where they think if the Chinese government if the wrong for profit company just wanting to go as fast as they can and jumping over all the safety training hurdles along the way, they think that it could be catastrophic. And so for them acceleration is salvation. I know that sounds a little cheesy, but it truly is. And whether or not you agree with them, I think you have to acknowledge that they have a genuine belief and it's also a self serving belief.

A (45:46)

No doubt there are employees within the companies that have that belief. But does Sam Altman? Does Elon Musk?

C (45:51)

Yeah, I think that at least there's compelling evidence that should make us consider it's true. I mean, one of many pieces of evidence that's out there is the leaked emails from the leadership team at OpenAI that came out when Elon Musk left OpenAI and then later on sued Sam Altman and Ilya Suskover and the leadership at OpenAI. And in that deposition, the public got to see their private emails going back to 2017 and how they were talking about what they were building when they said they were building AI. And in those emails they're arguing about like they are worried that this, this technology in the wrong hands could lead to a global dictatorship. That this technology in the wrong hands could kill the human race.

A (46:41)

Right.

C (46:42)

And that is how they Were talking privately when no one was looking.

A (46:45)

The Manhattan Project had those same kind of discussions. Oppenheimer thought about that. I think there are a lot of people now that would say if they could wish the nuclear bomb out of existence, and also maybe some stipulation about ending the war, US War with Japan, pretty quickly, we'd be in a better situation.

C (47:02)

Right.

A (47:03)

Just because when you have these terrible weapons, it increases the risk. I also think about bioweapons researchers like the entire idea of gain of function research. Like, of course you have to do it. We know how bad it is. But then there are all these downsides from doing it. Right. Either perhaps a lab leak or a rogue agent. Like, it was probably better if it just didn't exist. So, yes, that is my.

C (47:25)

And also, if they were completely cynical about it.

A (47:28)

Yeah.

C (47:30)

And this is a point that Bill Gates brings up a lot. But look at where they are investing time and money. There is no federal regulation that is telling the AI industry that they need to do safety training.

A (47:43)

Right.

C (47:43)

Which is a bit shocking at this point, especially since Sam Altman went to Congress and told them regulation. But there is not right now a law that tells them you need to be safety training these models for X amount of hours or X amount of weeks before you release them to the public. And yet they are doing that anyway. They're investing a ton of money in that anyway. And it's because they think it could be bad, that it could cause harm. And you could say cynically. Well. And then people won't want their product.

A (48:11)

Well, it's not cynically. It's good capitalist theory. If you put a product to market that has all these downsides, the product will be less popular. Like, if the people who made leaded gasoline knew that it would have all these downsides, they would probably have been better off making unleaded gasoline from the jump. But I want to ask you about Sam Altman. So it's great in the series, his 2023 testimony before Congress. 2023. And they loved them. And Dick Durbin loved them and John and Senator Kennedy of Louisiana loved him. They. He blew them away. Because no one has ever come before Congress and said, please regulate me. Watershed moment. Cut to two years later. The concern was, China's gonna beat him. And he totally changed his tune. It's like, I don't know about all this regulation. What went on in between.

C (49:04)

There was a change, not just with Sam Altman, but with Congress.

A (49:08)

Yeah.

C (49:09)

That when he first shows up in D.C. and he meets with these lawmakers, they are.

A (49:15)

Oh no. Oh no. We've talked to Andy for a while, but he has more to say, more great stuff to say. And if you subscribe to Pesca plus, you could hear it all. It's subscribe.mike pesca.com for one reasonable fee, you get all of our bonus episodes, all of our bonus content and the show without any ads. Or if you just want the show without any ads and forget the bonus stuff, it's also available to you at subscribe.mike pesca.com. And that's it for Today's show for 2026. Corey Wara is the producer. Jeff Craig runs our socials. We have a pretty great new website with a lot of features that Astrid Green contributed greatly to Catherine Sykes. She's still doing the just list, which, you know, it's no longer discounted. We told you all about it. But please go and check that out@mike pesca.substack.com and still running it all even in this late year. Michelle Pesca, CEO of Peach Fish Productions, improve. And thanks for listening.