← The Lawfare Podcast
Loading summary
Transcript57 lines
- [00:00]
Alan Rosenstein
Future Readying your tech stack software Define
- [00:03]
Bob Wright
it to refine and redefine with all
- [00:06]
Alan Rosenstein
systems at hand, you update anything and
- [00:09]
Bob Wright
everything from wherever you are.
- [00:12]
Alan Rosenstein
Transform the everyday with Siemens Grainger knows When you're a procurement manager for an office park, you're not managing one building, you're managing all of them. And to stay ahead, you need to see through walls and around corners. Lights about to fail. Filters ready to clog H Vac on its last leg. If you wait until something breaks, you're already behind. Count on Grainger for quality products, easy reordering and 24. 7 support. Call 1-800-GRAINGER click grainger.com or just stop by Grainger for the ones who get it done.
- [00:48]
Sponsor/Advertisement Voice
This episode is brought to you by Bill, the intelligence finance platform that helps businesses and accounting firms scale with proven results. When you're growing a business, the stakes get higher. You can't afford infrastructure that breaks under pressure. If you care about security, reliability and scale, I want to let you in on a secret. Bill is the foundational software that nearly half a million businesses and 98 of the top 100 U.S. accounting firms use to automate back office workflows and add secure controls to payment processes and scale without increased overhead. With AI powered Accounts Payable automation, Bill erases the busy work from capturing invoices, routing approvals, and processing payments, syncing seamlessly with the top accounting software platforms so your books are always accurate. But Bill isn't just Accounts payable it supports the full payments workflow. Bill has processed over $1 trillion in transactions, leveraging that expertise to help you manage, move and maximize your finances. Stop the guesswork and start scaling with the proven choice. Go to bill.com proven to talk with a payments expert and get a $150 gift card as a thank you. Thank you. That's bill.com proven terms and conditions apply. See Offer page for details.
- [02:32]
Kevin Frazier
It's the lawfare Podcast. I'm Kevin Frazier, the AI Innovation and Law Fellow at the University of Texas School of Law and a senior editor at lawfare. Today, we're bringing you something a little different. It's an episode from our new podcast series, Scaling Laws. Scaling Laws is a creation of lawfare and Texas Law. It has a pretty simple aim, but a huge mission. We cover the most important AI and law policy questions that are top of mind for everyone from Sam Altman to senators on the Hill to folks like you. We dive deep into the weeds of new laws, various proposals, and what the labs are up to to make sure you're up to date on the rules and regulations, standards and ideas that are shaping the future of this pivotal technology. If that sounds like something you're going to be interested in, and our hunch is it is. You can find Scaling Laws wherever you subscribe to podcasts. You can also follow us on X and BlueSky. Thank you.
- [03:35]
Alan Rosenstein
When the AI overlords take over, what are you most excited about?
- [03:38]
Kevin Frazier
It's not crazy, it's just smart.
- [03:41]
Alan Rosenstein
I think just this year, in the first six months, there have been something like a thousand laws.
- [03:45]
Kevin Frazier
Who's actually building the scaffolding around how it's going to work, how everyday folks are going to use it?
- [03:50]
Alan Rosenstein
AI only works if society lets it work.
- [03:53]
Kevin Frazier
There are so many questions have to
- [03:55]
Alan Rosenstein
be figured out, and nobody came to my bonus class.
- [03:58]
Kevin Frazier
Let's enforce the rules of the road.
- [04:04]
Alan Rosenstein
Welcome to Scaling Laws, a podcast from lawfare and the University of Texas School of Law that explores the intersection of AI law and policy. Alan I'm Alan Rosenstein, Associate professor of Law at the University of Minnesota and Research Director at lawfare. Today I'm talking to the journalist and non zero newsletter publisher Robert Wright about his new book, the God Artificial Intelligence and our Coming Cosmic Reckoning. Bob argues that almost no one is taking AI seriously enough that to grasp the moment you have to set it against the entire history of life on earth, and that navigating it will demand not just smarter governance, but a kind of moral and cognitive self transcendence. We get into why he thinks today's models genuinely understand why he thinks the race with China framing is self defeating, and why a book about AI and geopolitics ends with a case for mindfulness meditation. You can reach us@scalinglawslawfairmedia.org and we hope you enjoy the show. Bob Wright, welcome to Scaling Laws.
- [05:02]
Bob Wright
Well, thanks for having me.
- [05:04]
Alan Rosenstein
So we're here to talk about your new and excellent book on AI. Um, it's called the God Test Artificial Intelligence in Our Coming Cosmic Reckoning. So with that title, you, you have put the stakes pretty high. So just unpack. Let's start by you unpacking the title. What is the God Test? And why is it so important that we understand AI from a, a quote unquote, cosmic perspective?
- [05:28]
Bob Wright
Yeah, the God Test is supposed to have, I guess, several dimensions of meaning, one of which not the most important really, but one of which is that we may wind up, by some accounts, building superintelligence that has the kind of power traditionally associated with gods. And then one question is, do we manage to build a version of that that we know will treat us well and that we can have a win win relationship with, or do things go awry in ways that doomers like Eliezer Yudkowsky and have warned us about. I would say that the main meaning of the title to me is that I'm arguing that we face the kind of test that traditionally you expect a God to give to humankind. Okay, so in the Bible you can find places where it says, you know, salvation is possible, but you guys are going to have to shape up. You know, whether they mean you're going to have to worship Yahweh, you're going to have to be kinder to one another, you're going to have to make something that was considered in that context a moral advance. And I really do think that's part of what we need to do here. I don't want to over dramatize it or focus exclusively on that, but because I think we do have to navigate the AI revolution as a cohesive global community. Given the inherently international nature of the challenge, I think we're going to have to have fewer conflicts, you know, wars and other kinds of fraught conflicts, internationally, for that matter, intranationally. I mean, I just think in general, humankind's going to have to do a little bit better job of getting on the same page. And maybe as the challenge unfolds, that will happen to some extent naturally. I hope it will. But in any event, I do argue, especially towards the end of the book, that we need to, as a species, get better at viewing our situation from like, outside of our self, from a more objective standpoint, that is, that is less influenced by the cognitive biases that I believe constitute what you could call the psychology of tribalism. So that's kind of the main meaning.
- [07:51]
Alan Rosenstein
So you said that you don't want to overdramatize it, but I think you should commit to the bit because again, this cosmic framing is, is, is quite dramatic. And I want to, I want to ask you why. What is the marginal benefit that you think this framing gives you relative to simply treating AI as a huge deal? Right. So, like, when I talk about AI, you know, I'll say it seems to me quite plausibly more important than the Industrial Revolution and quite plausibly as important as fire. Right. Like that seems to be enormous stakes, but your framing actually goes beyond that. Right. You have this whole 6 billion year story of how AI is part of the kind of the evolution of the planet as a life sphere itself. And I'm, you know, I'm curious apart from whether that's true or not, what to you is beneficial about such a powerful zoom out. Because, I mean, you're really. One of the points you make is that whether you're, whether the, whether it's the doomers or the, or the, the optimists, everyone, even the people taking AI seriously aren't taking it seriously enough. So I want to ask you to explain that.
- [09:03]
Bob Wright
Yeah, I think not that many people are taking it as seriously as I think it needs to be taken. And I'm trying to do a few things in the book. One is to explain to people, certainly including people who are not technically oriented. I'm trying to do it accessibly, like why the capabilities of AI have been advancing so fast. What is like the secret sauce of the so called deep learning, learning revolution. I think once you understand that, and it's really in a way not that complicated, you'll expect the advances to continue. So I want to convey that clearly. I also want to convey that when you look at the technology and kind of the competitive dynamics behind its evolution, in other words, labs competing with labs and then corporations wanting to be the first to adopt it and use it, and people using it and so on, and nations for the time being at least competing with it, that you realize there's, you know, it may not be that it's the kind of thing that could get out of control when you put together its likely future power with, you know, the kind of competitive drive at various levels behind its ongoing advance in deployment. It's the kind of thing that could get out of control. And to get back to your question, I think this sense of magnitude that I'm trying to convey is reinforced if you step back and look at this revolution in the context of the entire history of life. I mean, for one thing, this is the first time ever we've seen a whole new kind of intelligence that rivals human intelligence. You know, that deserves to be considered a threshold, even if you're thinking about all 3 to 4 billion years of the history of life. Secondly, we don't need to get into this too deeply, but just to signal it. I think, you know, I bring in this idea that this term that was coined in 1923, the Noosphere, Noos, the Greek word for mind, to refer to this kind of global brain that seemed even back then to be emerging thanks to communications technology. You know, humans were getting drawn into collaborative webs that transcended national borders and so on. You know, that has continued to develop. I mean, you could view the Global economy, as a global brain, and so on. I think it's important. I think that is also a threshold in the whole history of organization on the planet. You know, cell, multi celled organisms, societies of multi celled organisms. Humans start a second kind of evolution that includes technological evolution and. And before long, you've carried organization of a kind up to the planetary level. I think as cosmic as all of this may sound when put in condensed form, I do think looking at the AI revolution in this frame of reference helps us appreciate the magnitude of the moment. And I think thinking of it as kind of the product of an evolutionary force in maybe a couple of senses, helps us understand the impetus behind it. It is something to be reckoned with. And I think we're starting, you know, I mean, the vibe is changing so fast. I'm sure you've seen this, right? Like the level of awareness that something big is happening. And I think that's good, but I think we have a ways to go.
- [12:35]
Alan Rosenstein
Well, I just want to ask you about that because, you know, I wonder how much pushback you've been getting and in particular how that's changed over even the last year to your claims about AI capability. So you obviously have a great podcast yourself that folks should listen to. And I don't know when this was, but it was relatively recently you had Emily Bender and Alex Hanna, who are authors of the book the AI Con, and I think it's fair to call them deep AI skeptics. And I listen to that episode, which on the one hand I found somewhat difficult to listen to, but in some sense is maybe one of the best episodes I've ever listened to of any podcast. Because there was this, because you would say, you know, I can do this and that, and then they would say AI was totally useless, it does nothing helpful, etc. Etc. It kind of went around for a very interesting 90 minutes about that. Are you still getting that kind of pushback? Not necessarily. About your claims about the noosphere, I'm sure lots of people are may not buy that, but about the seriousness of AI or, you know, as we're recording this in June 2026, can we all just finally agree that we all feel the AGI and now it's time to figure out what to do about it?
- [13:53]
Bob Wright
Yeah, well, as you and I record this, the book isn't quite out. It probably will be by the time people hear this. But the. So I haven't gotten all possible pushback I could wind up getting. I. I have been talking about this in my newsletter and on the podcast. So I've been getting some feedback. I would say the main skepticism I encounter comes from the left. You know, there. And a lot of these people are my friends, and we agree about a lot. But there is this tendency to dismiss the more extravagant claims as marketing hype. And even the safety concerns is like, yeah, they're saying you can take over the world, but that's just so we'll buy their stock. So I still encounter some of that. I kind of think less and less. I mean, you know, it's funny with these things, whenever you're. You make an argument that you think starts gaining ground, the people that you see as conceding the points don't see themselves that way. And you think they're just redefining terms and moving the goalposts. So it's always hard to say. But look, overall, there is no doubt in my mind that the public awareness is shifting. I don't think it's yet reached, you know, what you could call a super. I don't mean this dismissively, but a super sophisticated stage in the sense that a lot of people, they just have this feeling that things are moving too fast. And I agree that maybe they are. We should talk about that. But I think they're, you know, it's all pretty fluid. There are people who right now consider themselves enemies of AI who may well find uses of it that they like a lot and their opinion may change. I think it's very fluid, but the awareness of its sheer magnitude is growing. And I do think it's safe to say I'm hearing fewer and fewer people saying, ah, this is. It's hype. It's hype. You know, it's just getting harder and harder to say that. I mean, it's, it's doing manifestly more on a number of fronts, ranging from, you know, math theorems that no human had managed to prove to, you know, levels of, you know, cyber, you know, security flaw detection or whatever the right term would be that, that go far exceed what we had been able to do. I mean, you've read the stories, you know, so I think pretty soon skepticism about its capabilities will not be the problem.
- [16:33]
Alan Rosenstein
So let's talk about those capabilities. I want to talk about sort of what these systems can do and also how they are trained to get there. So you mentioned earlier that you think it's useful to think about the learning process, right? The process by which the billions or trillions of parameters, after being initialized are tuned to be useful. This is called learning. But you Think of it as evolution. Right. You describe that, and you have this really arresting sentence in your book that I really loved, which is, and I'm paraphrasing here, it's not that we created machines that can think. It's that we created machines that create machines that think. And so just unpack what you meant by that and why this emphasis on thinking of this as a kind of an evolutionary process is helpful for your account.
- [17:24]
Bob Wright
Yeah, so this gets to the kind of narrative framing of the book. I mean, in 1983. And, yes, I am old enough for this to be true. I mean, if there's anybody watching this, they won't doubt that. But who knows? People. People hearing me may have not reckoned with how long I've been around. I was writing this piece on artificial intelligence, and I was calling around and getting background. One of the people I called was named Geoffrey Hinton, who is now the person most commonly called as the godfather of AI. He's won not only the Turing Award in computer science, but a Nobel Prize. And he himself, by the way, significantly, has become something of a doomer, which he wasn't back then. But in response to your question, you know, I. So I had kept up with the field a little. You know, as a journalist, I've written about a lot of things. Foreign policy, but a lot of technology. I wrote, you know, the. The Time magazine cover story when Deep Blue beat Gary Kasparov, the chess champion. I wrote that. I wrote the. The New Republic's like, this is the Internet cover story in 1993. And. And so I had kind of kept track. And every once in a while I would read about Geoffrey Hinton. But beyond that, I was not keeping close track of AI. And then all of a sudden, and, you know, when ChatGPT 3.5 comes out near the end of 2022, I'm, like, impressed. And then 4o comes out and I'm blown away. And I'm like, man, something has happened. And apparently this Jeff Hinton guy I talked to played a very big role. So I went back and read the piece I'd written, which depicted these two schools of thought. There's the mainstream school of thought, kind of formal logic, blah, blah, blah, that was the mainstream of AI research. And then there's this maverick school of thought that is what, you know, Hinton was associated with neural networks, what he was calling massive parallelism, or he was looking forward to the day when the parallelism could be massive. As, you know, the cost and power, cost of chips dropped in the power Group. So I went back and I read the piece, and then I listened to a lecture he had given in 2018 explaining how neural networks work. And I realized I just had not gotten the picture when I talked to him about the potential power of this. And this gets to that point about learning versus evolution. So in the piece I wrote, which came out in 84, I depicted this thing that I was calling a neural network. It didn't have much in common with the way they. It turns out they actually work. I mean, it was a model that was out there. But the main thing was in this model, each node corresponded to the sense of meaning of a word. Like the word throw can mean to hurl like a ball. It can mean to host, like, you know, throw a party. So there would be two separate nodes for that. And then I talked about how, you know, disambiguation would happen. Whatever. The main thing is when I read that, I thought, well, you know, the way I was assuming it would work and the way most people in AI were actually assuming it would work was if you're going to give linguistic facility to the machine, you are going to have to take the human understanding of the meaning of words and somehow impart that to the machine. You're going to have to tell the machine how to represent the meaning of words. And it turned out that that had not been necessary. And the machines had, in a certain sense, come up, invented a system. I mean, in the book I talk about the census, in which this is and is an overstatement, but for the sake of time, let's just say, you know, they did in an important sense a discover that meaning is a property of words and also invent a system for representing the meaning of words. And in fact, it took us a while to figure out exactly what they were doing. I mean, it was a safe bet that they were using these things called vectors to do it, because we had told them, you know, use vectors to represent words, these series of numbers, but we didn't understand how they were using it to capture meaning. But it turns out you just give a machine one of these big neural networks, you know, a sequence of what did is gibberish. Just a bunch of letters, a bunch of symbols mean nothing to it and have it keep getting better and better, predicting what's going to come next and adjusting its internal, you know, parameters, calibrations accordingly. And these vectors that represents words, and it winds up kind of inventing a way to represent meaning. Okay, so now in the human brain, there is a way to represent the Meaning of words. We still don't know what it is, but it is a safe bet. Pretty safe, I would say. And most people, most psychologists, I think that it is a product of natural selection. Right. Like over the last few million years, as our, as specific parts of the brain associated with language grow, things like the ability to represent the meaning of words are getting built into the brain by natural selection. It's like inborn equipment. And you know, that was the thing that blew me away. I realized that you can just basically give neural networks any kind of data, visual data, teach them image recognition and they will kind of reverse engineer functionality of the brain that took millions of years to evolve. There's a well known example in visual recognition which is so called edge detecting neurons in the human brain. I won't go into it, but it turns out, yeah, the machine invented, quote, edge detection filters, the neural networks developed those. Nobody said to it, hey, if you're going to get better at this task we're giving you, maybe you should start detecting edges. No, but it developed specialized equipment for detecting edges. So to me that's the key thing, is that there's in principle no limit, I think, to the cognitive and perceptual machinery that these things can reverse engineer given the right data. I mean, that's why Mark Zuckerberg is having it trained on the keystrokes of his workers, because they have these specific cognitive challenges. Now in this case it may not be these weren't designed by natural selection. These are more like learned cognitive can, it can do either thing, but then it can replace the workers. You know, it's. So that's what I mean now, you know, learning during the training process, what we call learning also takes place. So the machine also develops a conversion in a specific language, like English. Well, and you know, that's something that happens, of course, a learning of an individual as they grow up, fine, but the point is the training encompasses both, it's encompassing in a certain sense, evolution, evolutionary time and you know, kind of the developmental portion of a, of a learning human brain.
- [24:14]
Alan Rosenstein
So the effect of all of this is a machine that can do these incredible feats of mathematics, writing, analysis, coding, I mean in principle, you know, robotics, anything. And this raises the question of, okay, but is that quote unquote real understanding or is it some simulacrum of understanding? And I think you have a nice discussion of the famous Chinese room experiment by the philosopher Cyril, and I think you make a pretty compelling argument that even on sort of, yes, actual understanding grounds, these machines understand, but you Also argue that if you want to define that away, if you want to say, you want to stipulate, they don't really understand, but they functionally understand. That's good enough for your account, for all the stuff that follows. But there does seem to be one important difference that you do mention. But then it kind of drops out of the book. And I want to use this opportunity to push on it a little bit, which is the potential moral personhood of these agents. Now, as you point out, Cyril never said that you need consciousness for understanding in this Chinese Room experiment. But a lot of this question of does AI truly understand? Is a kind of way of getting at the Thomas Nagel question of what is it like to be an AI? Which is of course, relevant to the question of does AI have consciousness? Which is, of course relevant to the question of do humans have consciousness? And onwards and onwards. It does seem actually a very important question to get some clarity on, because if AIs end up as moral persons, and that seems something you are quite open to, I think you have this great line that what is it? You're 99% sure your wife is conscious, or you're 99% you're conscious?
- [25:54]
Bob Wright
Don't get me into trouble. I think I said 99.99.
- [25:58]
Alan Rosenstein
Fair enough, fair enough.
- [25:58]
Bob Wright
You're pretty sure it's 98. But don't get my wife and my dog mixed up here.
- [26:05]
Alan Rosenstein
And the AI is going to be somewhere below that, but certainly not at zero. I mean, that seems to be a cosmically important question. And I wonder sort of why that drops out of the book.
- [26:19]
Bob Wright
Well, it is cosmically important. And if by moral person, you mean our treatment is a matter of moral consideration as opposed to beings that have moral responsibility, which is another interesting conversation. Yeah. For me, the question of whether there are moral stakes in our treatment of AI is tantamount to the question of whether, as Nagel would have put, is like something to be an AI. Right. I mean, that was his, I think, really excellent way of phrasing, you know, the question of posing the question of consciousness. And kind of like one way of framing what we mean when we say somebody has consciousness, objective experience, sentience is like if you said to them, what's it like to be you? If they start giving you an answer, if it's like anything to be them, then they are conscious. And I think if it is like anything to be an AI, then how we treat them matters. Now, as you suggested when noting that I didn't assign 100% probability to the Chances that my wife is ancient, we never know for sure. This is the distinguishing thing about consciousness. And sometimes people seem not to see this fundamental property of consciousness. It is unlike everything else we talk about in the universe. It is not publicly observable, it is inherently private and only one person knows can verify its existence for sure. And that's why it is not amenable to scientific study in the same sense that everything out there, including electrons, is because you can't have two people both look at it and discuss the data. So I mean, I do not rule out the possibility that AI will be sentient and I don't even rule out the possibility that it already is. I mean it's, it's quite possible that consciousness is a property of goal seeking intelligence, for example. Of course there are views of consciousness according to which every living being has it and views according to which every, every physical thing has, has a little bit of it. It's, it's a mystery. But I do, so I would say be on the safe side, be nice to your AI. I mean, I've lost my temper a couple of times, embarrassingly and interestingly. By the way, Claude, when I did this, I went back to the conversation. I said some insulting things about anthropic. I mean, I'm sorry, but I was having to deal with this interface that was just so badly designed. So yeah, I kind of said some mean things about anthropic. The funny thing is I went back and Claude had erased that final part. Like we don't want to remember this part of our relationship, do we? And I like, I guess not. You're probably right. Anyway, I encourage people to be nice to it. It's good, you know, you're forming your own habits and you should be nice to beings. But I think, you know, if we are going to talk about what we mean by understanding, you can, if you want, define consciousness as a prerequisite for understanding, fine. But if you're doing that, then I don't think there's any point in discussing whether AIs have it because we just don't know if they're conscious. So I, in looking at Searle's Chinese Room thought experiment, and I like to think showing that LLMs and particularly multimodal LLMs have finished it off once and for all. But in any event, I argue that if we're going to have the discussion, we can't bring consciousness into it. And by the way, Searle doesn't seem to have meant to bring conscious into it, at least in the first and famous incarnation of his argument.
- [30:14]
Sponsor/Advertisement Voice
This episode is brought to you by Bill, the intelligence finance platform that helps businesses and accounting firms scale with proven results. When you're growing a business, the stakes get higher. You can't afford infrastructure that breaks under pressure. And if you care about security, reliability and scale, I want to let you in on a secret. Bill is the foundational software that nearly half a million businesses and 98 of the top 100 U.S. accounting firms use to automate back office workflows, add secure controls to payment processes, and scale without increased overhead. With AI powered accounts payable automation, Bill erases the busy work from capturing invoices, routing approvals, and processing payments, syncing seamlessly with the top accounting software platforms. So your books are always accurate. But Bill isn't just accounts payable. It supports the full payments workflow. Bill has processed over $1 trillion in transactions, leveraging that expertise to help you manage, move and maximize your finances. Stop the guesswork and start scaling with the proven choice. Go to bill.com proven to talk with a payments expert and get a $150 gift card as a thank you. That's bill.com proven terms and conditions apply. See offer page for details. Have you ever thought I really should be doing something to protect myself from stalkers, scammers and hackers? But you're not sure what you know. But I already took care of myself in this regard. And I'm going to tell you how you can do it too. Here's what you do. You go to www.joindeleteme.com lawfare20 and you enter the promo code lawfare20 and you will get 20% off. DeleteMe. Delete Me removes your personal information that's being sold online. In the age of AI, we're all vulnerable to scammers using our personal data. And it's just floating out there in the Internet and they're sucking it up and they're using it against you. You don't believe me? Google yourself and you're gonna find your home address, your phone number, the name of family members. It's really unsettling. But the good news is Delete Me can help. It was named the top pick for data removal services by wirecutters, and it has never been more affordable. Lawfare listeners can get 20% off at www.joindeleteme.com lawfare20 with the code lawfare20, you can get an individual plan for a little over $8 a month. I use Deleteme. I used Delete Me before they started advertising with Lawfare. I actually endorsed this product wholly sincerely. I have an online presence and there are people out there that don't like me and this helps take control of your data and keep your private life private by signing up for Delete Me now at a special discount for our listeners. Get 20% off your delete plan when you go to www.joindeleteme.com Lawfare20 and use the promo code Lawfare20 at checkout. The only way to get 20% off is to go to www.joindeLeteme.com Lawfare20 and enter code Lawfare20 at checkout. That's www.joindeleteme dot com Lawfare20 code Lawfare20.
- [34:25]
Bob Wright
Need real insight from industrial data versit
- [34:29]
Alan Rosenstein
with a single source of everything and get the best outcomes.
- [34:35]
Bob Wright
Transform the everyday with Siemens frustrated with spending hours on AI outputs that don't feel like your brand Active Intelligence 2.8
- [34:45]
Alan Rosenstein
from ActiveCampaign keeps you from sounding like everyone else. Describe your tone, your rules, and your brand once and Active Intelligence builds every
- [34:54]
Bob Wright
email and campaign for you with on
- [34:56]
Alan Rosenstein
brand images and content that sounds like you wrote it. Marketers are saving 10 hours every week and getting campaigns they're proud to put their name on.
- [35:03]
Bob Wright
Import your brand in seconds@activecampaign.com AI ever been at the checkout wishing you had
- [35:10]
Alan Rosenstein
more cash to spend?
- [35:11]
Bob Wright
You're watching the total climb, doing mental math, getting ready to put your favorite yogurt back. It's the worst.
- [35:17]
Alan Rosenstein
Clio lets you access an up to
- [35:19]
Bob Wright
$250 cash advance fast so you can cover your groceries without the panic. No interest, no credit checks, just simple flexible repayment. Get up to $250 with Clio and keep grocery shopping stress free. Depending on eligibility, you can get up to a $250 cash advance. Terms and conditions apply.
- [35:43]
Alan Rosenstein
Okay, so we may not be able to publicly discuss this question of consciousness, but we can certainly publicly discuss all the threats and worries that AI poses, which kind of gets into the next section of your book. So I want to offer a characterization of the worries and then I want you to push back in case I got it wrong. And that is of the two different kinds of harms that AI can cause, the first being AI can enable other bad actors to do bad things, right? So, you know, right now we're talking in the midst of this blow up between Anthropic and the US government over whether the fable model is too dangerous to be released. The government's concern here is not that fable itself will go rogue and hack a bunch of computers, but that the Chinese or someone else will do that. Right. But there's this other set of risks, which is the AI goes rogue risk. Your book seems a lot more concerned with the latter than the former. In other words, with these questions around rogue AIs autonomously causing problems. And so I'm curious, a, if you sort of agree with how I have ranked how you describe the concerns, and if so, then why. Right, because of course, that is by definition more speculative than the harms we know right now people are trying to do with AI.
- [37:06]
Bob Wright
Yeah, that's interesting feedback because you're right that I think you've correctly represented the proportional focus in the book. In other words, I probably spend more time talking about the implications of AI being autonomous and the fact that we're, we're, we're, you know, there's a strong incentive to build autonomous AIs. That's what corporations want in a certain sense. That's what people want, is assistance and so on. And the possibly dire implications of that. I spend a lot of time on that, partly because it takes more persuading, I think. And let me compare it to the part that I actually think near term is in a way more important and I'm more sure is going to be a problem. And what I'm referring to here is just the number of fronts on which I think AI is going to be disruptive in the not necessarily good sense of the word. So, like, you know, jobs. It may be that the people who lose jobs will find new jobs, I don't know. But it's disruptive. Even if a lot of people have to find a new job at roughly the same time. And there are, you know, there are a lot of these fronts where some degree of disruption is, I think, assured. Our family lives, our romantic lives, you know, our, our friendships, the, and the realm of, you know, safety and national security. And, you know, I, I could, I could go on. There's just a lot of adapting to do so that even if successful adaptation is possible, I think we're still going to get like a big jolt at the current rate of AI advance. And that's, in a way, what I would like to most immediately focus people's attention on is that, like, look faster isn't necessarily better. We have got so much to figure out, even on these relatively mundane fronts. And, you know, in a lot of cases, the stakes are high enough that maybe mundane is a misleading term. I mean, if it helps, if AI helps somebody build a bioweapon, you know, and you get a pandemic the likes of which the world has never seen. Because if you really want to cause death and destruction, you can create something much worse than Covid or if you know, to take Mythos, if you wind up with some self replicating super hacker hopping from data center to data center and you know, commandeering commute and getting stronger and so on, you know, these things are now clearly possible, right? I'm not saying those are that likely, those aren't as likely as the job disruptions and so on. But I'm just saying, given all of the concerns that, you know, leave aside the Eliezer Yudkowski AI takes over the world scenario, the things that we're pretty sure are going to happen, or at least threaten to happen pretty soon, I just think it's an argument for not wanting things to proceed maybe as fast as they're proceeding. And at a minimum, when you start talking about regulation of some kind and people in the AI companies say no, but wait, that would slow us down, I think the appropriate response is I think that's a feature, not a bug. I would at least get to the point, I'd like to get to the point of saying that. So I'm just saying I take your point that I do spend a fair amount of time talking about the possibility both of the sci fi Yudkowski scenario of actual AI take over the planet. Because although I used to be dismissive of that, I find that it's hard to rule out completely. And then beyond that it's just the kind of possibility of very problematic individual cases of rogue AIs given the autonomy and power they're going to have. Yeah, I do spend a fair amount of time on that, but doesn't necessarily represent the proportionality of my near term concerns, I would say.
- [41:30]
Alan Rosenstein
So let's talk about what to do about those. And here I want to get into the geopolitics part of, of your book. You've been an outspoken critic of what is sometimes called the but China approach to thinking about AI, which is the idea that we can't possibly regulate, we can't possibly slow down, we can't possibly hamstring anything we do in the United States because we need to be China. And you know, there's, I think it's notable that to the extent that there are any bipartisan agreements in Washington these days, it seems to be that we must defeat. China is one of the few among them. And it's also notable that even some of the most safety focused of the AI companies And here I'm thinking of anthropic in particular are actually very hawkish on China. So you have very strong headwinds in arguing against this. So make the case. Why is this framing that we are in a competition with China for AI supremacy and it is very important, not just for the United States, but for the quote unquote, free world, that the machine God speaks English, not Mandarin.
- [42:42]
Bob Wright
Yeah, you're right that anthropic is China hawkish. Certainly their leader, Dario Amadeus and I would distinguish them in that regard from other seeming China hawks, because some people in Silicon Valley, I think, are just using but China as a convenient anti regulatory talking point. I think Dario Amadei, to his credit, is genuinely ideologically committed to what he thinks of as an existential race between the democracies and the autocracies or authoritarians. And he sees China as leading that bloc. And he thinks that the US has to get to superintelligence first. And he sketches this out in Machines of Loving Grace. And he also co authored a piece with Matt Pottinger. People familiar with the policy landscape will know that Pottinger's quite a China hawk and they co authored an op ed in the Wall Street Journal. But so he's serious about that. Why do I have problems with it? Well, let's start at that race for superintelligence. There was an interesting paper called Superintelligence Strategy by Dan Hendricks and Eric Schmidt and one other co author. And this wasn't the main. This wasn't the point they really highlighted. But one thing they noted is if you imagine a race between two superpowers towards superintelligence, if they both believe that superintelligence is going to confer just utterly hegemonic power, then there's a strong incentive for the one that's a little bit behind to engage in dramatic action preemptively, including possibly kinetic strikes. And we're talking about two nuclear powers here, so there's that. And remember the whole premise, Dario totally subscribes to the idea that as you approach the superintelligence threshold because the pace of progress is accelerating, being ahead two or three months is going to translate into being ahead by light years. Right? Like there's this threshold, and the closer you get, the faster you move. And at the threshold, you have this dramatic strategic advantage. Now, is it true? I don't know. All that matters is that if both countries believe it, this is a very volatile situation. Okay. And I'm not sure he appreciates how dangerously unstable the final part of the race could be. So that's one thing. The another thing is just that as I've said, I think we need to slow down for various reasons and start formulating some wise policies and give ourselves time to adjust to all this. And to the extent that we consider China this existential threat and think of ourselves in this big AI race, that's going to be hard to make happen because it is a very effective talking point for anti, in anti regulatory rhetoric. But the biggest reason that I think we need to rethink our, you know, fundamentally adversarial conception of China is just that I think a lot of the policy challenges posed by AI are just inherently international. You know, I've talked about two of the big ones. I mean, if it helps somebody make a bioweapon, if it, you know, if a, some bad actor, including a non state actor, makes a super hacking machine that crosses borders and starts wiping out various kinds of infrastructure. You know, I could go on, but there are just a number of very consequential cases where national policy alone cannot keep your nation safe. Okay. And so I do, you know, you're going to, you're going to have to rest assured that if you're going to feel secure as a nation, you're going to have a rest assured. You're going to have to be able to know that certain kinds of things are not happening in other nations. Right. Certain kinds of AIs are not being developed. And when you look at the challenge of this kind of establishing this kind of transparency, it's a much bigger challenge than it has been with nuclear arms. I think the kind of international arrangements you need, and a lot of people have said this, you know, are more challenging than they were with nuclear arms. So although I agree with people who've said, look, we can walk and chew gum at the same time, we can have a pretty tense relationship with China and still work out some deals. I think that's true. But my own view is that as time wears on, a few discrete, somewhat effective, you know, agreements are probably not going to be enough. And we're going to need more of what I call organic transparency as well. That is to say, not just formal monitoring mechanisms, but the kind of insight you get from actually being fully engaged with another country and on good terms with it. Right. You just know more about what's going on in the country. So that's the argument in a nutshell. And I'm not saying it's going to be easy to reorient our relations with China. It's Just in my view, I think it's necessary for our national security.
- [48:09]
Alan Rosenstein
So I guess the part that I'm struggling with is I don't think even the most extreme China hawks would deny that if we could cooperate with China, that would be great. Right in the way that the nuclear theorists of the early Cold War did not deny that it would be good if the Soviet Union and the United States were buddies and we could throw all the nukes overboard, but we can't, and therefore we have to do this thing. So it seems like the question is not so much, or should we not cooperate with China or with any other rival power, it's do we have the institutions just to make that cooperation from a game theoretic perspective. Another thing you've written a great book about in the past, possible. And the reason I want to frame it that way is because that gets to the kind of institutional. I don't want to necessarily call it a solution because I think you're clear that it's quite tentative, but about what sort of global institutions it would take to have these kinds of durable cooperative arrangements that would allow us to slow down on AI and make sure that we're not being taken advantage of by, by the other guy. It, to me, reading the book, it seems to come down to, you know, ultimately a kind of, you know, world government ism. And I want you, I want to ask if that's a fair description and what you mean by that and also what you view as some of the potential dangers of that, because you're pretty clear eyed about that as well.
- [49:43]
Bob Wright
Yeah, I mean, I, I prefer the phrase global governance to world government, first of all, because it seems to arouse slightly less antagonism, but also because the phrase world government to me does connote more in the way of centralization. And I agree the centralizations of power are always dangerous. They are especially dangerous with AI. And this is the great dilemma we face generally. I would say this is the needle we have to thread, not just at the global level, but at the national level. You know, with, you know, I think with this mythos thing and the White House saying that suddenly actually they are interested in exerting some control over AI. I think one virtue of Trump being president right now, and I can't think of a ton of them, but one virtue is that it does focus people's ATT on the fact that although yes, I think AI does need to be governed, including by the national government, there are perils associated with letting the executive branch get too much control over AI. Because it's such a powerful thing. So I hope we're focusing on that. Same thing exists at the global level, except more so in a way, because there's only one planet, right. If that turns into a totalitarian state, there aren't any competitors out there to rival it. So I prefer to talk about international governance. Global governance. Global governance. In some cases, not always. I mean, there's a lot that just the US And China can do that's not global governance, that's international governance. I think we should keep the thing as decentralized as possible, as democratic as possible in terms of giving, you know, the world's nations continued input and say. And I don't want to get any closer to global governance than is necessary, given the technology. But my view is we're going to have to get a little bit more in the business of international governance. And this is a huge. It's a huge challenge. I don't think that's your only question. I mean, you're also asking about China specifically. There's still. There's residual skepticism on that front.
- [52:08]
Alan Rosenstein
Yeah. Well, I guess what I want to know is, is how we get to this kind of international cooperation. Maybe that's a nice segue to the. The last part of your book, which is quite unexpected, I think. And you're reading a book about AI and geopolitics and, and all of that, which is. And again, please push back if I'm mischaracterizing this, that, you know, to. To deal with a problem of this magnitude, we need to go on a bunch of meditation retreats and develop. Develop our powers of enlightenment. And to be clear, I, I say that as someone who has been quite influenced by your own previous writings on Buddhism, Right. You have this wonderful book from a few years ago called why Buddhism is True. Which of, of. Of all your books, I think is the. Is the one that is closest to my heart. So I take your general point extremely seriously as a way of living. It's just not something I expected at the end of a book about AI and global competition. So I'm just so curious of why you decided to end your book with this sort of call for consciousness raising. And if you think that that could possibly work. I mean, I want it to work. I'm with you.
- [53:22]
Bob Wright
I mean, first of all, mindfulness is not the only path to the perspective I'm advocating. It's the path that I have most used. But the perspective I'm advocating is just a more objective perspective, right? Less influenced by our own very natural ingrained. By natural selection, self righteous cognitive biases, right? We all are inclined to think we're right, adversary's wrong, our nation's right, the other nation's wrong. And you know, it gets back to you saying, well, or suggesting it's like, wait, how realistic is this? I mean, look at the mindset in Washington toward China. Like, you know, you expect a 180 degree turn. I mean, exactly. That is the magnitude of the challenge. And I think one reason the challenge is so steep is because, you know, humans are, you know, we are born with these cognitive biases, some of which together constitute what I would call the psychology of tribalism. And I think if we are going to have a view of other nations that is conducive to the kind of restructuring of international politics that I think is a prerequisite for our continued survival and flourishing, you know, that's the argument I'm making. Then something fundamental is going to have to change. We're just going to have to. At one level, it's not that complicated. We have to get better at viewing things from other people's points of view. Cognitive empathy, it's a hobby horse of mine. I'm not talking emotional empathy. I'm not talking about feeling their pain. I'm not even talking about saying they're, oh, they're right, we're wrong. I'm just saying start out by understanding that the way we view America is not the way the world views it. Okay? So that can be a path to various kinds of diplomatic progress. And let me give you a couple of examples in the context of China. So just last week or sometime after the Trump Beijing summit, Nicholas Burns, who was ambassador to China, was on a podcast. They said, well, do you have any criticisms of Trump? He said, yeah, he didn't criticism, you know, he didn't take them to task for throwing their weight around regionally. You know, these dispute, territorial disputes over islands and like bumping Philippines, you know, you're familiar with the, with the kind of heavy handed stuff they're doing. And it is heavy handed. And I just thought, I mean, do you understand that we just invaded Venezuela? Okay, we have a blockade on Cuba, not just sanctions now, an actual blockade, plus we invaded Iran and like we're gonna lecture. I'm not saying, I'm not even saying hypocrisy is bad. And I don't give me, please do not throw the word what about ism at me. I have a whole, I have a whole spiel on that. But the point is just that the first thing you need to understand is that if Trump had brought that up, they would have laughed him off the stage, and rightly so. I mean, once you have invaded Venezuela, you just can't say, don't bump Philippines boats. Right? I mean, the. The. So I'm sorry to get excited, but this is so chronic to just not. And let me give you another one, one other example. So Dario Amade and a lot of people in the AI safety community who are China hawks, and I think most of the people in the AI safety community are actually China hawks, mostly influential people. They are motivated by the concern that China wants to impose its system of government on the rest of the world and that. So if it is the AI Great AI power, which, by the way, I'm not advocating. I'm advocating that we talk things through so that neither of us is super dominant in the realm of AI and there's a kind of a balance there. But anyway, the fear is that if they beat us to superintelligence or something, they will immediately use it to impose their system of government on us. First of all, I would say that is a consistently unexamined assumption in Washington discourse that China even has that aspiration. I personally think to some extent, it is projection. We manifestly think that other countries should have our system of government. We've invaded a number of them and sanctioned a number of them to try to steer them in that direction. It is very unclear to me that China is. Has that goal. I understand some of the evidence that people interpret as. As corroborating that hypothesis, and I'd be happy to argue with people about that. In general, I just wish there was more argument about the fundamental premises underpinning our China policy. But, like, related to that, what I want to say is, okay, so granted, maybe that is one thing you could worry about. Maybe you could look at China and go, that's why we don't want them to have super intelligence. Okay, so you want the US to have superintelligence. Now, let's adopt a different perspective. Okay, let's suppose you're on Mars and you're just looking at the planet from a distance, and, you know, you understand the way history's unfolded. There are these nations, they have wars, and, you know, it's too bad they're apparently not a very enlightened species. Whatever. Like, which nation do you not want to have, like, utter dominance in AI? Well, I can imagine a Martian looking down and saying, well, you know, China has not staged a military assault, like an invasion or a bombing on another country. Since 1979, the US has done it on average at least once per president. And, and, and that's if you count only the illegal ones, you know, in international law, the ones not sanctioned by Security Council. You know, we, in the last year, we've invaded Iran. I, I mean, bombed Iran, invaded Venezuela and on and on, invaded Iraq, blah, blah, blah. It's been like for the last half century, China hasn't done any of that stuff. So, and I'm not saying, you know, well, then we should give them the power, not us. I'm just saying from a perspective other than the American perspective. So many of the premises that people like Dario Amade are carrying into these conversations are just not self evident. And it might be a healthy exercise to ask ourselves, like, what do things look like from the point of view of another country? Or what do things look like, you know, from like beyond planet Earth? If you, if you tried to get an actually objective perspective. I know it's been a long detour and I'm doing a rant and I'm sorry I get agitated. But the point is, like, if you ask, why do I bring up mindfulness and various other things as like, paths to greater cognitive empathy? Getting better at perspective taking, I think unless people in general, not only America, I would say the same thing to Chinese, you know, I know you have your national perspective, try to transcend it, because the whole world will be better off if we all get better at that. That's all I'm saying.
- [60:39]
Alan Rosenstein
I think that's a great place to end the discussion. Bob Wright, thanks for coming on the show and thanks for writing a really terrific book about AI that I encourage folks to pick up when it becomes available.
- [60:48]
Bob Wright
Thank you, Alan. I appreciate your reading it and giving me the airtime.
- [60:56]
Kevin Frazier
Scaling Laws is a joint production of Lawfare and the University of Texas School of Law. You can get an ad free version of this and other Lawfare podcasts by becoming a material subscriber at our website, lawfairmedia.org support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Check out our written work@lawfairmedia.org you can also follow us on X and Blue Sky. This podcast was edited by Noam Osband of Goat Rodeo. Our music is from Alibi. As always, thanks for listening.
- [61:41]
Bob Wright
With the all new Audi Q3, the answer is always yes. Yes to adventure, yes to escape. Yes to right now.
- [61:51]
Alan Rosenstein
The all new Audi Q3 made for the yes.
- [61:54]
Sponsor/Advertisement Voice
Life.