← Sinica Podcast
Loading summary
Transcript139 lines
- [00:01]
Progressive Insurance Announcer
You're listening to this podcast, so I know you've got a curious mind. Here's a helpful fact you might not know yet. Drivers who switch and save with Progressive save over $900 on average. Pop over to progressive.com, answer some questions and you'll get a quick quote with discounts that are easy to come by. In fact, 99% of their auto customers earn at least one discount. Visit progressive.com and see if you can enjoy a little cash back. Progressive Casualty Insurance Company and national average 12 month savings of $946 by new customers surveyed who saved with Progressive between June 2024 and May 2025. Potential savings will vary
- [00:40]
Kaiser Kuo
ever notice how life's best stories don't happen in your living room. They happen on the open road, out on the water or parked under the stars. At Progressive they get that you want to focus on the experience, not worry about the what ifs. And that's why they offer quality insurance designed for your ride, whether That's a boat, RV or motorcycle adventure with confidence. Visit progressive.com and see how easy it is to protect your favorite way to get away. Progressive Casualty Insurance Company and affiliates not available in dc. Prices vary based on how you buy. Welcome to the Cynical Podcast, a weekly discussion of Curtis affairs in China. In this program we look at books, ideas, new research, intellectual currents and cultural trends that can help us better understand what's happening in China's politics, foreign relations, economics and society. Join me each week for in depth conversations that shed more light and bring less heat to how we think and talk about China. I'm Kaiser Guo, coming to you this week for the last time from my soon to be former home in Chapel Hill, North Carolina. All packed up. The movers are coming soon and it's official we're putting the house on the market. Sinica is supported this year by the center for East Asian Studies at the University of Wisconsin Madison, a national resource center for the study of East Asia. Listeners, you can support my work by becoming paying subscribers@senecapodcast.com I do need your help to get keep this work going, so please subscribe so I can continue to bring you these conversations this week on Seneca, a conversation I've been looking forward to for a long time and one that takes us a little outside our usual beat. Though as you'll hear, China sits much closer to the center of it than you might expect. My guest is Robert Wright and if you don't know his work, you are in for a treat. Bob is one of the most original and consequential thinkers that we have on the big questions of today, where our species came from, where it might be headed, and how the technologies we build are bound up with both. He's the author of a string of books that have shaped how a generation thinks about human nature and history, certainly me the Moral why We Are the Way We Are the New Science of Evolutionary Psychology, which was his landmark 1994 book on evolutionary psychology. I will confess I read it when it came out with enormous relish. I think I had just read, either before or right afterward, Consilience by Edward O. Wilson. I was really on a kick, and this was one of those landmark books for me. It stuck with me, really, ever since. There was also the Logic of Human Destiny, which argues that the long arc of biological and cultural evolution has a direction driven by the logic of non zero sum games. Highly relevant to today's discussion, as you will see the Evolution of God, a finalist for the Pulitzer Prize, and why Buddhism Is True, a bestseller on meditation and the modern mind. Bob writes the widely read NON zero newsletter and hosts the NON zero podcast. He's taught at Princeton at the University of Pennsylvania, and he's been a New America Fellow. But more than any of the titles, what comes through in his work is a rare combination of intellectual range, genuine humility, and a stubborn insistence on cognitive empathy as the thing we most need and most lack. And there's the personal connection, which I'm happy to finally get to repay. We are both big fans of cognitive empathy. I've had the honor of being a guest on Bob's podcast twice now, and I'm delighted to at last be able to reciprocate and have him on Seneca. But the truth is, I became reacquainted with his work after those moral animal years. At a very particular moment, right around the time I was starting Seneca, I heard Bob articulate this idea that he was calling cognitive empathy, this discipline of genuinely modeling what's going on in another person's mind as distinct from feeling basic emotional empathy for them. And I had a small jolt of recognition because I'd been going on about exactly the same thing for ages, except that I'd been clumsily calling it informed empathy, my own coinage. He had the better name for it, and the better account of why it really matters, a more sort of expansive view of it. And it's become super central to how I think about China ever since. And it turns out to be really central to his new book too. Indeed, that book is a marvelous melding of many themes from Bob's work spanning three decades. It is called the God test, and it's the reason we're talking today. It's an ambitious, genuinely cosmic reckoning with what artificial intelligence means. Not just for jobs or geopolitics, though that's important in it, but for the deep story of life on this planet. It's a truly terrific read. Bob's voice is at once really erudite and playful and his ability to think about all the implications, to really ask all the right questions, to come up with these brilliant analogies and keep the reader just cruising along through the text, it makes it just one of the best books I've read yet on AI and the human condition. Seriously. Bob argues that large language models that we're all super familiar with now are best understood not as things we built, but as things we evolved. That the AI moment is forcing a kind of test on our species. Whether we pass that test depends on our capacity to transcend the kind of tribal psychology that's really bedeviled us since the dawn of history. And running right through the heart of it, more than I think many readers will expect, is China the race framing the foot race between China and the United States. The AI race, the export control wars or advanced semiconductors. The question of whether fear of Beijing is itself the thing most likely to lead us astray. It's a book that rewards a China minded reader, and I've got a lot I want to put to him about it. He's also, I should say, just an all around great guy. Just warm and funny and intellectually fearless and exactly the kind of person you want to think out loud with. About the largest questions that we now face. Bob Wright, a very warm and long overdue welcome to Seneca.
- [07:14]
Bob Wright
Well, thank you, Kaiser. I don't know if I'm going to be able to live up to that billing, but it felt really good to hear it. And I know it partly is rooted in your generosity to be so kind in your introduction. In any event, it's an honor to be here. You know, I'm a longtime listener and admirer and I guess I should be even more honored if this is the final production from this particular location. Your home, right?
- [07:39]
Kaiser Kuo
Yeah. Yeah. Leaving Chapel Hill, where, I mean, it's been something in process for a while. I've spent most of this year already living in Beijing. We. I just came back because I had. I had some talks to give in my daughter's graduation and we sort of put off the finishing touches on getting the house ready for the market until this trip. But no, I'm headed back to Beijing in the middle of the month and we say goodbye to Chapel Hill.
- [08:04]
Bob Wright
Well, cool. I hope you'll be willing to appear in my podcast from Beijing.
- [08:09]
Kaiser Kuo
Happily, happily, happily. So, Bob, let's jump right in. I guess that's the place to begin. Is you place Pierre Teilhard de Chardin, near the center of this book's imaginative architecture. Teilhard was a French Jesuit priest and a paleontologist who died in 1955. He was a man who spent his life trying to reconcile evolution with his faith, and his more mystical writings were actually suppressed by the church in his lifetime. Teilhard, gentle listeners, is a name you're going to hear a lot in this conversation, so you should probably take note. T E I L H A R D Teilhard. Bob Teilhard gives you this idea of the noosphere, the thinking envelope of the Earth, a kind of planetary mind which you illustrate beautifully with what you call sort of Martian's eye view. If you can imagine sort of a time lapse of life on Earth viewed from afar, evolving from the biosphere that we have, you know, our atmosphere and then life on Earth into networks of human communication, you know, just speech writing, trade networks, radio, television, and then eventually the Internet. And now finally, AI Again, listeners, I'm going to flag this word noosphere, N O O S P H E R E because it's going to come up again. So noosphere, we have Teilhard and the idea of the noosphere. So Bob, could you walk listeners through Teilhard's idea of this noosphere and explain why you think AI makes that idea newly urgent rather than just, I don't know, just poetically suggestive? Because you actually make a really strong claim here, not merely that a global brain is now possible, but that it's become necessary, that there's no realistic alternative that can safeguard human interests in the age to come. So why is it necessary and why is it kind of inevitable?
- [10:12]
Bob Wright
Yeah, so he coined that term in 1923, comes from the Greek word for mind, noos, and I believe in intentional reference to the term biosphere. Yeah, certainly one of the people who embraced his term, Vernadsky, this Russian thinker, was the person who had done more than anyone to popularize the term biosphere already by that time. And the Martians eye view you alluded to captures how sweeping Teilhard's vision was. He got the big picture. He was a paleontologist as well as being a Jesuit priest. And he Saw that life had, had kind of climbed this hierarchy of organization. In some sense you get cells, you get multi celled life, you get societies of multi celled organisms. Then one society of multi celled organisms, I.e. human society, generates a second evolutionary process, sometimes called cultural evolution. It encompasses technological evolution.
- [11:15]
Kaiser Kuo
Sure.
- [11:15]
Bob Wright
And that allows the social organization of the species to expand, you know, ancient city states, so on. Finally you get to this global level. And he was prescient. He saw even in 1923 how information technologies were coming along that were increasingly binding humans to, into a global web. And I think the Internet winds up being a kind of vindication of him and an affirmation of his prescience. And he saw this as having great theological significance. He saw the culmination of this whole process as being something called the Omega Point, which would be characterized by a kind of profusion of brotherly love. At long last, people would, would overcome the tendency toward conflict. You know, he had been in World War I and he knew it all too well. And the one place I would differ with him is in the view, my view of biological evolution is a pretty straightforward Darwinian one in terms of the mechanics. You know, I just focus on natural selection. It's not impossible. There's more at work. But in any event, he was convinced, I think, that there was more at work. Something like Henri Bergson's Alon Battal, some sort of immaterial force pushing it along. But in any event, I think he got the big picture right and I agree with him on a couple of critical points. First of all, I do think there's a kind of moral directionality in the sense that if we are going to cross the threshold and form a true global community, we're going to have to in some sense become better people. I mean, we're going to have to deploy the cognitive empathy you alluded to more widely, more effectively. It's hard for all of us to do that, especially with rivals and adversaries, to really understand how things look from their point of view. How, for example, things that we do that we think of as defensive may be seen by them as offensive, as threatening and so on. And I, I, I think that's kind of moral progress when you get better at looking at, at, at things that way, at people's perspectives that way. Because for one thing, it may, it may lead you to treat them better. And so I agree with him. There's a, there's a moral directionality. And I think we're approaching a kind of moral test because I think we really have to cross the threshold to a true global community if we're going to handle the coming technologies, including, and especially including artificial intelligence, as, as we can get to. You know, another sense in which I agree with Teilhard, even if we don't agree exactly on the mechanics, is that I agree that there is a powerful force pushing us toward this global brain. He called it a brain of brains, a global brain, a global mind. Because, as you suggested, I think that to govern these technologies in ways that are compatible with our ongoing safety and security, we're going to have to see more international governance, more global coordination. And, you know, coordination is what brains do. So it will, you know, I will say I'm leery of an overly centralized, like global government. Government, but there is. We are going to need more, I think, in the way of global governance. And I think the impetus pushing us toward that, namely our own self interest, is powerful. The last thing I kind of agree, well, I partly agree with him on, is I'm technically agnostic on the question of higher purpose. Is this evolutionary process, the unfolding of some larger purpose in some sense, in any sense? And if so, does is it a divine purpose in any sense? I don't know. I'm agnostic, but I don't. I don't rule it out. And I see the process as kind of suggestive of that, again, in part because it has this moral dimension. And I think we're approaching this kind of test. I mean, that's one reason the book is called the God Test, is because it's the kind of test you would expect a God to set up. Right. It's like we're going to have to upgrade our species morally. We're going to have to, I think, really, in some significant sense, more closely approach enlightenment in the Eastern sense of the term, more than the Western, probably, but in a way, in both senses, if we're going to cross this threshold. So that's kind of Teilhard in a nutshell. His worldview has a lot in common with mine, if not quite everything.
- [16:01]
Kaiser Kuo
Yeah, yeah. Let's dig in a little bit about his view of evolution. It's not just complexity happens, but there's this directionality to this whole story, what he calls complexification. And it culminates so far in consciousness, in cultural evolution. And you know what they call mimetic now, right? Mimetic evolution and the noosphere. But modern evolutionary biology is generally allergic to teleology. Right, Bob? And listeners to the show will know that I am too. Evolution has no Foresight, it has no intention. It doesn't have a built in moral arc. The eye didn't develop so that we could see. It was a whole lot of happy accidents. There's a lot of contingency in it. So how do contemporary biologists respond to this kind of argument? And how much of Teilhard can still be salvaged if we strip away what you were talking about, the elan vital, the mysticism, and simply say that natural selection can under some conditions generate increasing complexity and can generate intelligence and consciousness. So there's that. What I really want to press you on is his directionality comes with this destination, with this meaning, what you've been talking about. And once you've thrown out the Ilhan vital and the divine purpose, what is there left to underwrite this claim that the moral arc of history points in any direction at all? I mean, that it's somewhere where we'd actually want to go? I mean, couldn't it just be more complex? Couldn't a maximally complex, maximally interconnected planetary system also be a maximally efficient tyranny?
- [17:51]
Bob Wright
Well, tyranny is a problem that we can get to. But I would first say that you can separate the Alaan vital question from the divine purpose question. He didn't. Teilhard saw the elan vital as a manifestation of the purpose, as one of the key drivers of the process in the direction that would realize the purpose. But you can in principle have what I have, which is a conventional Darwinian mechanistic view of how biological evolution works, a mechanistic view of how the subsequent technological evolution works, and still say that maybe the whole process was set in motion in some sense with a purpose in mind. I mean, for example, a conventional deistic view where God sets the thing in motion and then keeps hands off, lets the machine go of itself. And of course, part of that is the idea that the machine does naturally tend in the direction that would realize this purpose, whatever it is. And I do think that you can argue that the evolution of higher and higher levels of biological complexity and higher and higher levels of intelligence, up to and including a point where a whole new kind of evolution would be born, this cultural evolution was very likely just by virtue of the mechanics of the Darwinian process. And I'm not alone here. I mean, some of the great biologists of the 20th century, William Hamilton, agreed and took seriously the possibility, again, like me, technically agnostic on the purpose question. But he recognized that it was certainly possible. Darwin himself did. Of course, he needed to accommodate himself to a Very religious era. But. But he recognized that it was not. There was no logical incompatibility between his saying that, yes, maybe it was realizing some divine purpose, although I doubt he honestly believed it was. Was. But. But he pointed out, yeah, it could be and still be driven by this mechanistic process. So there's that.
- [19:57]
Kaiser Kuo
I mean, there's obvious instances in which increased complexity does seem to confer an evolutionary advantage. Right?
- [20:05]
Bob Wright
Yeah, absolutely.
- [20:06]
Kaiser Kuo
Gathering, you know, resources or in. In mating and all the things that. That matter in. In evolution. But you actually do kind of recognize the potential for this kind of darker turn. In that opening chapter or in the second chapter, you shift from that kind of Martian's eye view to what you call that cell's eye view. From far away, the emergence of the planetary brain may look like this magnificent thing, Isolated minds knitting together into a global cognitive system. But, yeah, from the standpoint of one human being, the metaphor turns pretty ominous. I mean, who wants to be just a cell in a larger organism? I guess some of us do. I don't know, Bob. Have you seen Pluribus, that new Vince Gilligan show on Apple tv?
- [20:51]
Bob Wright
I have seen much of it, yes.
- [20:55]
Kaiser Kuo
Okay. Yeah. I mean, that's what I'm reminded of. Right, yeah.
- [21:01]
Bob Wright
No, that's a good reference, and it is a great concern. And it was a concern of Teilhard's. He was very kind of anti fascist, anti authoritarian, Even though he was clearly talking about the planet becoming a kind of giant superorganism, humankind becoming, in some sense, a giant superorganism. He saw that as compatible with what he called, I think, individual, Asian or something, anyway, with personal autonomy and agency. But I would say this is one of the great questions that is underscored by artificial intelligence, because on the one hand, it poses grave risks that lead you to want it to be under some kind of control. And as I suggested, I think to some extent that control needs to be international. And yet there are also grave risks posed by the power it confers on whoever wields it. You know, if it is too centralized, it obviously has tremendous potential. AI as a tool of surveillance and oppression. And yet, if you just let it run wild, the equally bad things can happen. I just think this is one of the great dilemmas of our time and was a dilemma posed by Teilhard's view of where we're heading, I think. Now, he didn't know about AI. He died in 1955. He knew about computers, and he talked about their role in the global brain, but he didn't he imagined the neurons of this global brain being human brains only. And now we face the serious prospect that there will be some silicon brains that are neurons. And some people worry about that they'll wind up running the show. I think that's, that's not a crazy concern. So it's a. We are in a situation fraught with peril. And I, I hope we recognize that.
- [23:09]
Kaiser Kuo
I mean, to get out of that peril, the stuff that you recommend, I mean, I'm going to ask the question that a lot of people listening certainly are thinking right now. It's like if the way that we're going to save ourselves requires that we all kind of, you know, transform the species psychologically and morally, that we need to all collectively take this gigantic step toward enlightenment. If that's the prerequisite for governing AI, then we're kind of screwed. I mean, it's a tall order on a very, very short clock. I mean, isn't there something almost self defeating about a plan whose first step is that humanity just gets wiser? I mean, what makes you so sanguine as to think this AI moment forces us to make that transformation rather than just punishing us for failing to achieve it?
- [23:56]
Bob Wright
I'm not sanguine, I'm not necessarily optimistic. I'm just trying to explain what I think the prerequisites for the ongoing flourishing and freedom of our species are. And I do think it's going to require kind of moral progress. I don't think you have to have, you know, true, full on enlightenment dawn on the planet before anything else can happen. And I don't think you ever have to get to that point. Enlightenment in the deepest kind of Buddhist sense or something. Although, you know, I wouldn't mind getting there myself should that ever be bestowed on me.
- [24:36]
Kaiser Kuo
But surely you would refuse Nirvana and become a Bodhisattva among us.
- [24:42]
Bob Wright
I guess I would go the Bodhisattva route, yeah. Yeah. In fact, candidly, you know, I could, I could have done the enlightened thing, Bobby Sotomayor thing, you know, but I thought, no, let's, let's go beyond Kaiser's podcast instead and save the world. So, yeah, the. But I want to emphasize a couple things. First of all, in terms of what's possible, leave aside enlightenment for a second. Just what, just how ambitiously people can think about new arrangements of global politics. You know, in the wake of traumas we have seen, people think big League of Nations, United Nations. Did they succeed? Well, not exactly, I would say, but the second attempt was better than the first and we've seen this time and again. And I will repeat what I've heard a lot of people who take, who have a fairly dramatic view of the kinds of threats that could be posed via AI, which is that I hope what we get is more like a near miss than a full on catastrophe because for one thing. Well, for a lot of reasons. But let me just say for now, you know, people's, people's minds are pretty malleable. People are capable of tremendous cooperation when they are galvanized in certain ways. The one other thing I want to say that I know is kind of jumping ahead. We'll probably get to this later, but you know, there are people, a lot of people in America and a lot of people in the quote, AI safety community whom I largely agree with, but on this one point of authoritarianism, they think the greatest threat of authoritarianism would come from not beating China in the AI race. They have this idea, and I know you probably want to save this for later, but they have this idea that China would, if they were ahead in AI, they would impose their system of government on the world or something. What I want to say is in addition to taking issue with that assumption, I want to say that I think nothing is more likely to lead to an authority authoritarian AI mediated world than the race itself. And I can get into that later. But you can imagine a couple of scenarios where that very plausibly happens. And I just have a lot of trouble imagining a happy ending here if we don't start talking to China about this right away. And I was very cheered to see that that's one thing that came out of the recent Beijing summit is they're at least going to have a conversation about AI.
- [27:25]
Kaiser Kuo
Yeah, yeah, yeah. Let's break this down a little bit. The reason the mechanism by which you say this, this bad outcome would happen is that, you know, if we think we can't slow down because China won't, that is a recipe for deploying exactly the sorts of systems that your own framework says we have to be most careful about, right?
- [27:44]
Bob Wright
Well, yes. Plus I think an AI race, if this technology is as disruptive as I think it's going to be, then if we let it proceed full speed ahead, the way a racing dynamic ensures, I believe American society will get so destabilized that authoritarianism will be much more likely to show up just domestically, you won't need China. I mean, we've already seen over the last four years there is potential for this kind of thing. Right? Like Our norms aren't as strong as, I mean, in a way they're holding up, you know, better than I had feared, but in a way not. And I think nothing is more likely to bring in authoritarianism than chaos. And I just think this technology, you know, it isn't that I subscribe to a specific sci fi doomer scenario, although as people who read the book will see, I find myself unable to rule out those scenarios after careful consideration. It's just that this technology is going to be so disruptive on so many fronts, you know, jobs, home life, in lots of ways that I think it stands, not to mention geopolitically, that I think it stands a good chance of being deeply and regrettably destabilizing unless we make a decision to kind of not only back off of a racing mindset, but maybe slow it down just a little bit, which again can only be done via international cooperation. Because as a practical matter, so long as you can point to China as this, as this terrifying adversary, then the AI companies can say, no, no, you can't do, you can't even like tax data centers. That would slow us down. You can't bring up copyright. That would slow us down. And China's out there. You have to, you have to quell the race dynamic before you have any hope of proceeding at the kind of moderate pace that I think would be good for all of us.
- [29:46]
Kaiser Kuo
Yeah, no, I completely agree and I think that you've articulated this so, so, so very well. And we'll come back to this in more detail later, but I want to tie this to your evolutionary model because in a very real sense, large language models aren't simply engineered or even trained in the everyday sense, but they in fact evolve. They're shaped by what you call selection, by consequences. They go through countless rounds of prediction, error, feedback, adjustment. If these systems are evolving, then the selection environment that we create is everything. It's the thing that determines what survives and what doesn't. Right now that environment is being set by this race dynamic, by market competition, by national security anxiety. So my worry is like, that we are not selecting for the right things. Systems that are truthful, that are aligned, that are cooperative, that are transparent, that are truly beneficial. And instead we're selecting for the sort of harder things only for capability, for persuasiveness, for strategic utility. And these are things that are less governable. Right. Ultimately. So that's what I think is like the center of your argument there. And that's what I thought was one of the most interesting connections in the book, between this evolutionary environment that we create and the bad outcomes that you worry about under the arms racing dynamics.
- [31:29]
Bob Wright
Yeah, there's evolution happening at two levels with artificial intelligence. I think they're both kind of underappreciated. The first one, and this isn't what you were talking about, which is the second level. But the training of a large language model is often referred to as learning. And it kind of is because for example, you know, you wind up with the thing basically speaking English. Well, learning the English language is something that does happen within the lifetime, you know, within a, of a person during childhood. That, that is learning when you learn a specific language. But the other thing that takes shape during training are some of the structures that seem to be products in humans of biological evolution. For example, a system for representing the meaning of words that actually takes shape during the same, well, technically pre training phase that learns, that leads to the mastery of particular languages. And the reason I think it's important to emphasize that is because it drives home how limitless in principle the ability of AI engineers to replicate pretty much everything about the human mind in silicon is. So what I mean is you basically you, you show the machine the output of a human being language and you give it a test predict the next word and you keep changing the, basically the strength of the connections among the neurons inside a neural network so that it gets better and better through trial and error at kind of predicting language. And in the course of that, it kind of reverse engineers certain functionalities that biological evolution built into us. And we've seen that in other realms like perception, like the way visual perception works in these things as well. And you know, the exact mechanism of like edge recognition that evolved biologically in us evolves in these machines. And, and the upshot of this is that you can like you may have, you may have heard recently that Mark Zuckerberg in his infinite warmth, you know, after firing, he at the same time fired 8,000 people and said, oh, and also the rest of you, we're going to start monitoring your keystrokes. Now the reason that matters is because the upshot of what I just said about how these things reverse engineer kind of parts of the human mind is that if you give it the input data and output data that these workers create, it can reverse engineer the parts of their minds that do their job and then you can fire them. And it's the same with self driving cars, with robots, with everything, you know, visual perception, auditory, whatever you give it to data. It reverse engineers the functional equivalent of components of the human mind, I would say. I mean, this is a little controversial. Not everyone in the field would agree with that, but a lot of people would. And so that's critical, I think, evolution at that level. And then there's the level you alluded to, which is that, you know, these Companies build these AIs and they say, well, you know, did this one sell? Do people like this one? Did this one optimize engagement? If not, how do we optimize engagement? Right. And these are, this is an evolutionary process as, as well, you know, a company comes out and fine tunes the thing so that it's an expert at law, or it's a particular kind of therapist, or it's a particular kind of agent that does things for you. And then they see does it sell? Do people use it? That, that is also a kind of evolution. And it is, it isn't always just the market that decides. I mean there are powerful players in this realm such as the government that are exerting influence and billionaires who own companies and have their own ideological preferences. But still all of this is in a sense a second level of evolution. And the good news is when we all make choices about what kinds of AIs we want, we're in a sense being part of the selective force, right? Like we're, we're sending a signal to the market. On the other hand, there's just one of us. And sometimes it's hard to make the choice that's most constructive. Right? It's, it's tempting to say, yeah, I want the AI to tell me I'm great and that my worldview is great and that my group is right and the other group is wrong. That all feels good. And yet that may not be exactly what we need. That is kind of sycophantic AI, probably
- [36:29]
Kaiser Kuo
exactly what we don't need.
- [36:30]
Bob Wright
Arguably, yeah.
- [36:32]
Kaiser Kuo
Bob, you have this striking image of model weights as the crown jewels. These are guarded by some companies that are released openly by others. The so called open source, they're actually open weight, not really open source models. That usually gets framed as a safety question. At least here in America. Open weights can be fine tuned by bad actors, but it misses what's strategically happening. I think these Chinese labs have really leaned very hard into open weight models. Deep Seq and Kuwin, because under the constraint of chips that we've talked about, openness is a good way to build a global developer base and to win market share without having to match Frontier compute capability. You have a Chinese open model running on developers, machines in Lagos, Nigeria, or in Jakarta, Indonesia or in Sao Paulo. And it shapes this global ecosystem in a way that no locked down American model necessarily can, even though those remain very, very popular. I wonder when you have American policymakers arguing for restricting open weights on safety grounds. I wonder whether they're making a safety argument in fact, or it's really a competitiveness argument that's dressed up as safety.
- [37:57]
Bob Wright
Yeah.
- [37:58]
Kaiser Kuo
Do you have a.
- [37:59]
Bob Wright
Well, I think it depends on who is saying it in a way. I mean, you're right. At the moment China is going pretty heavily with open source. I gather you'd know more about this than I do. That to some extent draws on a tradition that predates AI, that in the Chinese software development community there had long been a kind of tradition of contributing to open source projects internationally. Perhaps partly in a bid to get the kind of recognition they thought they deserved, even though they weren't part of the Western software world. But in any event it's a tradition that is has indigenous roots and they were I guess expressed to some extent in this, in the current trajectory. I do think this is a place where we, we see actual tension between the considerations I alluded to before, which is on the one hand you worry about concentration of power. Well, nothing, nothing fights that better probably than open source. On the other hand, you worry about things that people could do if there's no degree of control. And open source kind of also represents that. Now is it a convenient talking point for some Western companies to talk down open source? Absolutely. At the same time, Geoffrey Hinton, who, you know, he's sometimes called the godfather of AI as it happens, I interviewed him in 1983. That's how I begin the book, with a piece I wrote about on AI for something called the Wilson Quarter, which like pretty much everywhere I've worked except for the New Republic, no longer exists. But you know, I, I wrote a piece back then about these two contending views of AI. There was the mainstream view and then this kind of marginal maverick view having to do with neural networks. Well, in the end the maverick view prevailed and Hinton himself is very concerned about the risks, including the AI sci fi doomer risks. And he is worried about open source. And I quote him to that effect in the book. You know, he says, you know, it's, you might as well sell nuclear bombs at Radio Shack or something is the way he puts it. So I think there are legitimate concerns in both directions. And, but again, this is something where what's best, it seems to me is, is to sit down and Talk to China about it. Because if it's the case that, that some degree of governance needs to be imposed on open source, you're not going to do that at the national level. I mean, I mean, just in general, the thing about risk posed by AI is that national policy alone is rarely going to fully address the issue you're concerned about. So I think open source, you know, I'm like agnostic. I see the pros, I see the cons, and I'm kind of glad that there's an alternative to the three particular models that dominate in America or increasingly even two models, I would say OpenAI and Anthropics.
- [41:15]
Kaiser Kuo
Yeah, well, if they really are so worried about open source, we've identified the reason why Chinese companies have come, had to go there, and that's because of compute restraints. Why are the Chinese companies facing compute restraints? Let's ask bis. We come at last to cognitive empathy. We started talking about how now we need to talk to one another. And this is of course, the key. And this is where, of course, you and I are kindred spirits. Bob, I preach cognitive empathy from this microphone as often as you do. And I suspect, I think I do it for the same reason that you do, you know, because so much human catastrophe comes down to a failure to model what's actually going on in the other guy's mind. You, you reach often for the run up to World War I, you know, Theodore Roosevelt watching the Kaiser and the other Kaiser, not me and the British, each, you know, convinced that the other was, you know, coming to destroy them. There's this kind of mutual terror walking these two peoples to the edge of a war that neither wanted. Not just two, you know, there's everyone, the Russians and the Austrians and Serbs. And you, you float this genuinely hopeful idea that our AI models have a theory of mind, right? They're capable of something like cognitive empathy. And, you know, there's, you cite in your book this, this sort of, these experiments that you do, asking a person, how does this student whose boyfriend was humiliated by the teacher feel? How does somebody who is the romantic rival of the humiliated person feel? The AIs are pretty good at figuring out. They clearly have something approximating a theory of mind. They understand that there is stuff that we think about in our little human noggins. But your hopeful idea is that an AI that blessedly lacks our adversarial biases might read the other person's intentions more clearly than we can and help us to avoid the kind of lose, lose outcomes that we all fear. But the worry I can't share. And I want to put it to you early because it really haunts the rest of the book, is the very same capacity, in your own examples, the engine of deception, the chatbot that it's going to lie about in order to pass a captcha, because it correctly models what humans would do in that situation. Say, oh, I have a visual impairment, therefore I can't complete this captcha. And we've had examples of them doing this. Cognitive empathy is just as useful, unfortunately, for manipulation as for peacemaking. So what determines which way it's going to cut? Is there something about how these systems are built or who's building them that tilts an LLM that can read us toward helping us understand each other rather than exploiting what it understands about one another?
- [44:27]
Bob Wright
Yeah, I basically agree that you should enter this conversation with concern and skepticism about a happy ending for the reasons you laid out. I mean, first of all, things like.
- [44:42]
Kaiser Kuo
Well, they're the reasons you laid out, just to be clear.
- [44:44]
Bob Wright
Well, okay, anyway, I wasn't the first either to emphasize, for example, the fact that these things do have tendencies toward deception. You know, to what extent? That's because they're modeling human motivation and to what extent it's because they basically discover that deception is useful in the pursuit of a goal. They've been designed. You know, these things aren't entirely settled, but they clearly do exhibit that. Moreover, to get back to us being the selective force in the revolution, people are going to want that. If you've got an AI agent that is like negotiating on your behalf, for example, like, you don't want it to say, you know, frankly, I'm on a level with you. I mean, Kaiser, I'm his agent, but I got to tell you, you're his only alternative. If he doesn't do his deal with you, there's nothing. He has nowhere to go. You don't want your agent to say that even if it's true. Right. Good agents don't. So people are going to select agents that deceive. And it's related to the sycophancy thing. A lot of us don't want an agent that's always. Or a companion that's always giving us the brutal truth about ourselves. In fact, true friends know when that's most needed, but don't always do it. So anyway, there. There's, you know, and the same with seeking power. It's the. They've illustrated that they'll do it, these AIs, they recognize, apparently, that it's a way to to realize some goals, they
- [46:15]
Kaiser Kuo
have a subordinate goal, usually just, you know, in order to do this, to accomplish X, I might first have to do Y, and Y might be taking over the world.
- [46:26]
Bob Wright
And, you know, as you suggested, the. To get back to cognitive empathy, they, you know, people like, you know, we have this natural bias against cognitive empathy in certain situations, and we understand these cognitive biases better than we used to. So, for example, if we see somebody as an enemy, we are reluctant to acknowledge that they've done something good out of any motivation other than temporary expediency. For example, there's all kinds of attribution bias, attribution area. And so that's the way we naturally feel. It, like, feels good to view our adversaries that way. So the natural thing is to choose AIs that agree with you, that affirm your cognitive biases. So when we act as an evolutionary force in choosing our AIs, there is absolutely a danger that we will just choose AIs that reinforce the very biases that keep getting us into wars and so on. My only point is that in principle, you can have an AI that does exactly the opposite. You can have an AI that doesn't say, hey, in this marital dispute, you were right, and of course your spouse was wrong. No, you can have one that says, well, you got to understand that from her point of view, when you do this thing, it just seems like this. It seems threatening in this way or seems annoying in this way. Like, it is trivially easy in a technical sense to create AIs that will give people a clearer view of the world than they naturally have when obeying the cognitive biases built into them. I'm just saying that's doable. These things could make us better people. But to get back to your skepticism about enlightenment dawning on the planet overnight, I acknowledge that something a little implausible has to happen, which is that we need people to say, hey, wait, you know, there's virtue. Like, I don't know, we need to start a movement, or we need to. Or at least show people how in the long run, this kind of clarity vision can help. It can help your marriage, it can help this, it can help that. We, we need to do things. It can keep you out of wars. We need to sell people on the virtues of having AIs that may give us feedback that feels uncomfortable in the short run sometimes because it runs against our natural grain, but are good for us in the long run. And my main point is, just like this is possible, and we are the ones who decide. And now is the time to decide. This is the fluid moment.
- [49:18]
Kaiser Kuo
But, Bob, I want to get back to evolutionary biology and maybe what evolution can teach us about the way that AIs evolve. You've argued that they're not really built so much as they evolve. So I think it makes sense that we would turn to evolutionary biology to understand how this works. And so in the natural world, you've got the cheetahs and the gazelles, the cuckoos and the warblers, these arms races, these biological arms races, right? You know, faster cheetahs, faster gazelles, deceptive cuckoos and warblers that learn to discern, you know, learn to tell which is the fake cuckoo egg that they don't want to, you know, be cuckolded by these arms races. Ratchet and ratchet. And no one's able to really call a halt in the natural world. I do want to call some attention to the limits of the analogy, because this really matters. A gazelle can't negotiate with a cheetah. A warbler can't sign a treaty with a cuckoo. The defining feature of a biological arms race is there is no exit. Both sides just run flat out forever because neither can afford to stop. But in the us, China, AI competition, this is actually run by human beings who theoretically could pick up a phone. When we call an arms race, are we describing it, or are we importing a kind of fatalism that doesn't actually apply, a sense that this is a law of nature, that we're somehow helpless before? And isn't that fatalism awfully convenient for the people who don't want to see it slowed down? You've got the perfect illustration. In one of the chapters I can't remember, you have the O3 model where deep seq really, really rattled OpenAI, right? Deep. Seat drops are for Nvidia. Prices plummet. What does ChatGPT? What does OpenAI do? They drop the O3 model ahead of schedule. It's more dangerous. It's less tested. And this is, as you say, what extreme competition does. This is you quoting the center for AI Safety? He told you I can't remember his name now.
- [51:42]
Bob Wright
Dan Hendricks. Yeah.
- [51:43]
Kaiser Kuo
Oh, yeah, yeah. Hendricks. Is that arms race as an iron law? Is it a choice we keep pretending we can't make? Or shouldn't we think a little bit more about the agency we actually possess here?
- [51:57]
Bob Wright
Yeah, I don't mean to use the term fatalistically at all. I mean, after all, there are actual examples of arms races being slowed or halted by communication. You know, during the Cold War there were nuclear agreements. You know, we, we, I mean remarkably, we've done some backsliding in some very stupid ways. But you know, I thought one of the most impressive things was the Anti Ballistic Missile Treaty, which of course, you know, George W. Bush abandoned. But, but because that was a subtle kind of piece of logic which is that if, if you worry that your superpower, nuclear superpower rival could actually intercept your missiles, that's actually destabilizing, you're better off being assured that neither of you has a defense against a nuclear attack. So let's just ban the dam missiles that would intercept the missiles. And we did, we did in the 70s and it lasted quite a while. And, and that was a real achievement. And we're capable of these things. And so yeah, when I use the term arms race, I use it to convey how bad things can get if we don't intervene. And you're absolutely right. You know, in nature, I mean, I suppose in a certain sense genetic evolution could, you know, catch up to the non zero sum logic of constraining armaments and you could point to examples of that. But look, we can do it in real time. We can communicate with each other and say, look, this is nuts. And I do, again, to get back to the pessimism optimism question, I do think that we are just beginning to wake up to the magnitude of the AI revolution. There have been some real kind of landmarks in that, like the mythos thing and the Pope's encyclical. I mean, how often do Popes given encyclical devoted to a single technology? The correct answer is this is the second time in three centuries. And, and that last one was about motion pictures. And the Pope spent no time arguing, arguing that they're not sentient. Okay. There's something special about what's going on when the Pope does have to argue that a technology is not sentient. And so anyway, we're waking up and, and I don't. I think we should give short shrift to the possibility that great things will happen. After all, it was the magnitude of the nuclear threat that led to unprecedented levels of cooperation on arms control. Even if we didn't in the end have the brains to hang on to some of those gains.
- [54:40]
Kaiser Kuo
We should do more research into why eusocial behaviors actually evolve, why symbiosis between species evolves and things like that. That happens too, right?
- [54:52]
Bob Wright
Oh, absolutely.
- [54:53]
Kaiser Kuo
Let's say a little bit more about what actually played out with Deepseek and OpenAI. Because I mean, what Strikes me is the direction of wasn't that competition made the model worse? Right. OpenAI had already built O3. It's that this Chinese release, Deepseek, changed what Americans felt they could afford to do carefully and what they would just sort of let slide in the interest of competition. And it's the safety practice that was the thing that got cut that feels like will inevitably be the thing that gets cut. And inevitably China is the trigger for that. That seems to me the mechanism that you worry about that's sort of at the center of it. Not that China builds something dangerous, but that the fear of China makes us treat our own caution as a kind of dispensable luxury.
- [55:48]
Bob Wright
Absolutely. And it's a strange thing. I mean, you, I know, have thought a lot about the kind of changes in the American mindset toward China. Seems like only 10 years ago things were very different. But you know, then almost overnight, a kind of unreflective China hawkism was just predominant in Washington. And the thing that surprised me again is not just in Silicon Valley where you'd expect it because it's convenient for say venture capitalists whoever to say, oh, China's terrifying, so you can't regulate us. But even in the AI safety community, I mean, that has been the case. I do think there's a little of a shift back toward reason. But this is, this is the linchpin of, of the arms race is this one talking point about how scary China is. And you know, it's it again, it's a reason you can't have significant regulation of the industry. You can't do this, you can't do that. And it will be the reason if they ever start talking about serious degrees of international governance, formal agreements between the US and China, it will be the talking point brought up by people who don't want to see that China will find a way to cheat. Even though it seems like this monitoring is, you know, fairly, you know, robust, but still, you know.
- [57:33]
Kaiser Kuo
So Bob, it's too bad your book went to press of course, before this, but just last week it happened, you know, at the presidential level. Trump had this executive order. He was ready. Right. And it's a fairly mild thing, a pre release review process for Frontier models. Right. And of course he sells it after a phone call from David Sack. His, his AIs are former AIs are. He argues that it would, you know, slow the US down against China. And you know, it's, it's like what Max Tegmark, the MIT professor who heads the Future of Life Institute said, do you. On your own show. It's like, it's just two words. But China, right? Yeah, it's always but China.
- [58:14]
Bob Wright
I mean, the irony is Sachs is not a super China hawk. But. But reportedly he did place a call. I mean, he's a hardcore libertarian and he's basically a spokesperson for Silicon Valley capital. But. But yeah, it's. It's tough. I do see signs of progress. I think you do too, right? I mean, in terms of the mindset in Washington, and I just heard a very reasonable guy today. Is his name Brad Carson? Have you had him on your.
- [58:46]
Kaiser Kuo
No, no, I don't know him.
- [58:47]
Bob Wright
I think that's. I'll check to make sure it's his name. But he used to. He was like counsel for the army or something, the Obama administration. And he was talking about this very, you know, kind of toxic talking point, and he was referencing a conversation between Jack Clark of Anthropic and Tyler Cowan, in which, according to him, they just kind of agreed. Oh, yeah, there's no point in talking to China. I don't know how far, far back this was, but I will say on the subject of Anthropic, it's important to understand that I believe for Anthropic, this is not just a convenient talking point. I believe for Anthropic, there's an actual ideological commitment to China Hawkism, and you see it in a lot of ways.
- [59:38]
Kaiser Kuo
Yeah, I don't get where that comes from. I mean, because Dario Amade and I actually overlapped at Baidu, right?
- [59:43]
Bob Wright
Oh, did you, when he was there?
- [59:46]
Kaiser Kuo
I didn't know him. I may have met him. It's possible. He worked in the Silicon Valley office. He was there for what, like 10 or 11 months. I don't know what happened to him that turned him so. I mean, he's never talked about it. I've scoured the Internet looking for any connection to his time at Baidu, but I mean, let me be fair to him, because Dari Amadei, I don't think he's being. Like you said, I don't think he's being cynical. And this week and last few months, he actually sort of cut the other way on as far as hawkishness goes. Back in February, he stared down the Pentagon, right? I mean, Hegseth demanded Anthropic, stripped the guardrails off of Claude and allow, quote, unquote, all lawful use. And Amade refused. Right. Good for him. And that was great. He wouldn't permit it for autonomous weapons, for mass domestic surveillance.
- [60:31]
Bob Wright
He.
- [60:32]
Kaiser Kuo
He you know, he took a threatened cancellation of a couple hundred million dollars. Right. So good, good.
- [60:39]
Bob Wright
Yeah. I think he's genuinely concerned about the, the mass surveillance thing. And I think, yeah, it's related to his China Hawkism. I think he's among the people who think it's an aspiration of the Chinese government to turn the whole world into a surveillance state. I don't see the evidence for that. But, but that is, that is, I think that's his sincere belief and I've talked to other people at Anthropic about their views and I think from the get go, it's a company, they kind of had that in its DNA.
- [61:09]
Kaiser Kuo
Yeah, yeah, yeah. I mean, whatever. I mean, but this the same guy, right? He co writes this, this Wall Street Journal op ed with Matt Pottinger. I don't know if you saw that. You know, man, tighter chip controls. He tells the Trump administration that America has to stay dominant over China and our other adversaries. So, you know, he says it's got, he's got like this three plank argument. AI is existentially dangerous, so therefore we need guardrails. We deny China the chips and we win the race. Right. He plainly thinks that these arguments cohere. I think that he's not arguing in bad faith necessarily. The controls, he says, buy you the lead. That lets you then afford to be careful. Right. You've got time on your hands, you've got a lead over China. Your book argues that these ideas do not cohere, that the race framing and the sprint that it demands of these frontier model companies is exactly what makes careful evaluation impossible. I think listeners know that. I completely agree with you. But it's not just that this doesn't cohere internally. This guy, Jeff Ding, who is a political scientist at George Washington, he runs the China AI newsletter, really one of the most careful empirical scholars of Chinese AI that we have. I've had him on the show before. I don't know if you heard that episode talking basically about diffusion of technologies. That's a book that he wrote. I think anyone who follows China and AI is familiar with his work. Anyway, just last week he published, I'll put a link to it, this devastating critique of Anthropic's recent Two Scenarios paper, which is sort of their follow up to the Machines of Loving Grace essay that Dario wrote a few years back. Bob, have you had a chance to read that? I think I sent you a link.
- [63:06]
Bob Wright
I did, and I agree that the two scenarios thing put out by Anthropic, I Mean, first of all, it's just, to me, a little strange that they're that active. Like, they. They obviously timed that with Trump's Beijing summit to try to frame the whole thing and say, look, there's these two scenarios. You know, we win, we lose, we either beat China in this race or we don't. And I want to. I want to add a particular peril to the. To the race that goes beyond the fact that it's just this headlong, chaotic thing that strips us of our ability to govern the technology. And this gets at a paper that was co authored by the aforementioned Dan Hendricks and some others, including, ironically, Eric Schmidt. I mean, I think Schmidt signed on at the last minute as a cause or something. I don't know how exactly.
- [64:02]
Kaiser Kuo
He really blows hot and cold sometimes. He's so sensible and sometimes, yeah, see,
- [64:07]
Bob Wright
he was a seed funder of anthropic. And at that point, I believe he was a more unequivocal China hawk or something. And I will say this paper I'm about to describe, my takeaway from it was, though not quite subtext, was not the main thesis, in a way. But anyway, it did call attention to the following dynamic, which is that if you believe that there is this superintelligence threshold, and Dario does, and he's been very clear that the goal should be the US and these Western democracies together, you know, that we. We maintain these ship controls and we beat China to this point, and then. And he uses the phrase military dominance, and then basically, this is in Machines of Loving Grace. It gets a little vague, but the idea seems to be to bring China to its knees and get it to. I think he. The way he puts it is like, quit competing with democracy. I don't know what he means. Does he mean change their form of government, or does he mean doing something that.
- [65:15]
Kaiser Kuo
I don't even think. Very vague.
- [65:16]
Bob Wright
I don't even think they're doing anything. Anyway, I want to say that he is embracing this race through superintelligence. And this paper, co authored by Dan Hendricks and a couple of other people, including Eric Schmidt, does point out that in a situation like that, if you buy the premises, and I don't think China as much as some people in the US Currently buy the premise that there's a superintelligence threshold that confers utter dominance, that lies ahead. But anyway, if. If you accept those assumptions, it is in the rational interest of whichever country is behind, whichever super power is behind, to resort to dramatic action as the other country approaches that threshold. You might bomb data centers, you might do this, you might do that. And I would just point out that the factories that produce the most advanced chips are pretty close to China. Right?
- [66:12]
Kaiser Kuo
Yeah.
- [66:12]
Bob Wright
And so anyway, the point is, the race that Dario is embracing and advocating is dangerously destabilizing at the level of international politics. It's a level of international superpower rivalry, leaving aside the issues of regulation and so on, so. And yet it's gone almost entirely unexamined. I've never heard anybody bring it up in the interviews that he grants people, at least. And it's just a real failing of the whole ecosystem of American discourse that it hasn't really gotten prominently vetted, this whole scenario that he clings to.
- [66:58]
Kaiser Kuo
Well, I mean, Jeff Ding goes after it. I mean, some of his arguments aren't necessarily ones that I would completely agree with. You know, he argues from this. This idea that China actually has a serious diffusion deficit, that it actually lags the US on civil military integration. But I think I'm completely in agreement with him when he says that this. There's this fantasy that Amadei and others subscribe to that you have like a. This full stack American AI versus a full stack Chinese AI. The actual stacks are super interpenetrated, as Ding argues, anyway, so you make the case that the chip controls backfired on their own terms, that they actually lit a fire under China's domestic chip industry that drove the efficiency innovations that gave us deep seq. So you don't even slow them down, you end up speeding them up. That's the capability argument. I'm down with that. I think I've made that argument innumerable times as well. But I want to push you on a second kind of backfire that you don't quite get to. It's a cognitive empathy point. Put yourself in Beijing's shoes for real, not the projection version that we often indulge in. But you just live through this deliberate declared campaign to kneecap your most strategically important industry, to deny you the inputs to a technology that everyone agrees is like civilizationally decisive. Right. Advanced semiconductors. What does that do to how you read the other side's intentions?
- [68:32]
Bob Wright
Right.
- [68:32]
Kaiser Kuo
I mean, my instinct is certainly it doesn't incline you toward benevolence. Right. It doesn't chasten you either. It actually convinces you that the hostility is permanent, that it's structural, that engagement was a mask all along, and that the only safe path is really just to harden and to assume the worst. So the containment policy just doesn't just fail to stop China it may actually be, perversely manufacturing exactly the kind of implacable and belligerent adversary that the hawks say they already found. Yeah, I don't know why that is absent from the debate. Why don't people don't bring up that argument? Because that seems to me.
- [69:14]
Bob Wright
I have tried to. I mean, I have said, I've drawn comparisons between, you know, I've said, look, if. If indeed, you know, AI is the. The key technology of the future, which, you know, probably it is, and you're trying to cut off their supply of chips, like, how different is that from trying to cut off Japan's supply of oil before Pearl Harbor? And we saw what happened there. And, you know, it's like, not a crazy response. Right. When you're trying to perform something that is plausibly perceived as economic strangulation. And again, you know, it doesn't matter whether we think of it that way, as we've, as we've established the great starter of wars, in a way, if there's a single misperception, it is the perception on both sides that things the other side does that are defensively motivated are offensively intended. And we need, as you suggested, we need to account for that possibility. When we do these chip export controls that we see as defensive, it's like, why would China not see them as threatening? You know, quick anecdote on how bad the cognitive empathy is in this realm. I just heard an interview with Nick Burns, who was ambassador to China, and this is not about AI.
- [70:31]
Kaiser Kuo
Started on him, man.
- [70:33]
Bob Wright
This is not about AI. But he was criticizing Trump's performance in the summit, and he said he didn't bring up China trying to throw around its weight regionally, intimidate the Philippines, blah, blah, blah. And I thought, do you not understand that once you have invaded Venezuela and are in the process of blockading Cuba, you just can't bring that subject up? China would laugh you off the stage. I mean, I understand Nick Burns. You may not, you may not have supported those policies, but that was the situation going in. Trump obviously can't bring that up. Trump has obviously signaled that, yeah, we both get to do whatever the hell we want. Right? I mean, it's just crazy. He was the ambassador to China and he can't do that simple kind of exercise of cognitive empathy. It is discouraging.
- [71:32]
Kaiser Kuo
Yeah. Bob, your podcast and your newsletter have both been called non zero for, well, as long as I've known your work, and I bet a lot of listeners have never stopped to ask what's actually behind that name. I mean, it's not a throwaway. It's the core of a theory you've been developing since the book non zero in 2000. It's based on game theory. It's central to this chapter where, like you and Bostrom, you converge on this idea of this singleton. Right. So lay it out for us. What is non zero sumness? And why do you think it's the engine behind, as I hinted at the beginning, behind this whole long arc of rising complexity of what Teilhard calls complexification, from single cells to nation states to whatever's next and where ultimately you think that non zero sumness ultimately is taking us.
- [72:25]
Bob Wright
Yeah, let's get back to a theme early in our conversation about why I think something more like a global drain, at least some system of global governance has powerful, at least logical impetus behind it, in the sense of being in our interest, in the interests of other nations. So yeah, it starts with this book I wrote called non0, which was, among other things, an attempt to explain the, the growth in the kind of scope of, in depth of human social complexity that is, you know, from kind of hunter gather band, nation state all the way to the verge of globalization and so on, to explain that in game theoretical terms. And both zero sum and non zero sum dynamics are important because competition among political entities, sometimes in the form of war, did encourage them to increase their kind of internal non zero sumness, their cooperation and so on. But at the same time there was increasingly dense webs of cooperation and collaboration, if sometimes implicit among polities. Global trade, you know, obviously has intertwined really all the nations in the world in a game that is, you know, it has fallout of various kinds that are not good. But it is technically a non zero sum game. The two parties engage in trade because they both believe they both would rather exchange what they have for the other thing. They think they're better off having the other thing that's non zero sum. And then there's this whole second category of non zero sum dynamics that I contend to become increasingly relevant as we approach the global level of social organization, which is common threats faced by multiple nations, climate change, ozone layer, which by the way is something we actually did the rational, smart thing about one of our great triumphs, the treaty on chlorofluorocarbons, or not a treaty, it was an international agreement though, and I don't think it was a treaty. And things like genetic engineering that can increasingly be done in relative obscurity and could in principle start a global pandemic. That's another thing that that means that you can't have security in your nation unless you have, unless things are kind of under control in other nations as well. And.
- [75:10]
Kaiser Kuo
Right.
- [75:10]
Bob Wright
And I argue that AI just really deepens this kind of dynamic. I mean specifically in the area of genetically engineered bioweapons for starters, there's a lot of talk about that. But then there are a lot of threats it poses that are kind of analogous to bioweapons in the sense that they cross borders, maybe even self replicate. This mythos model by anthropic raises the possibility of a self replicating hacking AI that jumps from data center to data center and commandeers the compute and get stronger as it goes and learn stuff. You can come up with plenty of examples of how as AI advances, it's going to be hard to have security if you don't collaborate. That's a non zero sum situation. There's a lose, lose outcome you want to avoid. If you avoid it, that's when, when relatively speaking, you know, it's like nuclear war. If you have a treaty that reduces the chances of nuclear war, which would surely be lose, lose between superpowers, that's win, win. So I trace, I mean actually in the book Non zero, I even trace the rise of biological complexity in the same terms because natural selection can quote, recognize and act on game theoretic logic. And I mean that's why you have multi celled organisms in the first place and various other forms of collaboration.
- [76:44]
Kaiser Kuo
Yeah, and, and social behavior.
- [76:46]
Bob Wright
Yeah, yeah. And so anyway, I don't, you know, I, I'm not painting a rosy picture of relentless cooperation by any means. In fact, as I said, there's been a lot of bitter and brutal competition along the way between political entities. And that has galvanized a lot of cooperation within political entities. And one threat, one problem we face now is that increasingly the nations of the planet do face common threats, but they're not as like intuitively instinctively galvanizing, because.
- [77:23]
Kaiser Kuo
That's right.
- [77:24]
Bob Wright
They're not like invaders from another planet. And you know, we most naturally collaborate in the face of those kinds of threats, like animate threats. Now interestingly, AI is a little more like that. Right. Like it'll be interesting to see exactly how its psychological impact plays out. But in any event, logically speaking, I maintain, you know, we're in the non zero sum logic is getting stronger and stronger. And if we abide by it, we will find a way to put aside the kinds of conflicts that have impeded international cooperation so often. I agree with you. You know, there aren't necessarily grounds for tremendous optimism. I'm just making the argument that we need to recognize that the alternative is, Is much worse.
- [78:22]
Kaiser Kuo
Yeah, for sure, for sure. It's funny because I feel like we're most often now in another kind of a game, theoretical model, which is. Is the prisoner's dilemma. I mean, I feel like there's just too much incentive for both sides to, you know, act in bad faith, to defect. And we, we can't get to that. Even though, you know, we're not prisoners, we're not, in fact, unable to communicate with one another. We can coordinate.
- [78:48]
Bob Wright
Right.
- [78:48]
Kaiser Kuo
Actually, yeah, we can collude.
- [78:50]
Bob Wright
It's. I mean, one thing the prisoner's dilemma illustrates. You know, it's a non zero sum game, but it illustrates the importance of communication. Communication. And that's why people who say, literally, there's no point in talking to China are almost ruling out intelligent cooperation.
- [79:07]
Kaiser Kuo
That's right, that's right. I mean, to your point about, you know, our need for an enemy at the gate. Yeah, that's, that's, that's true. It's external threats that have congealed tribes, you know, into chiefdoms, into, you know, nations and alliance systems. But, yeah, and these threats that we face right now, I remember hearing a speaker some years ago, ago saying, it doesn't have a mustache. Right. All the bad guys of our imaginings, there are all these evil men with mustaches.
- [79:32]
Bob Wright
Climate change. Climate change needs a mustache.
- [79:35]
Kaiser Kuo
Yeah, exactly. I mean, Trump can maybe take a Sharpie and draw one on. I'm sure he doesn't.
- [79:42]
Bob Wright
But that's not what he wants to draw on.
- [79:44]
Kaiser Kuo
Yeah, unfortunately, yeah, he wants to draw his name back on the Kennedy center, but I guess it's darker territory. The scenarios you'd promised earlier in the book. Book. The logic of takeoff is that whoever crosses the superintelligence line first gets a decisive, maybe permanent advantage. Walk us through what that does to two rivals who are watching each other. Because the part that strikes me as most dangerous isn't even the sci fi version where one AI takes over the planet. It's the much more mundane version where the US and China each fear the other is six months from the finish line. And that fear alone is just enough to justify something truly drastic. Sabotage. A cyber attack on arrivals, data centers. You point out that bombing has a way of starting wars. You don't actually need super intelligence to arrive in order for the race itself to be catastrophic. You just need both sides to believe it's coming. And that's the scenario that worries you more, I think.
- [80:46]
Bob Wright
Right, right. And I will say in that paper, the Dan Hendricks and, yes, Eric Schmidt paper, they pointed out the stabilizing value of transparency in a situation like that of, of having a clear view on what is actually happening on the other side so that you don't start imagining that more is happening than is. And that I think they even underscored the. The rational incentive for a nation that's way ahead to start slowing down short of the finish line and let the other nation see that it's slowing down. I think they. They said that. But. But that is. That is the kind of logic you have in a situation like that. And transparency. Oh, that reminds me, I want to get this phrase, organic transparency out there.
- [81:33]
Kaiser Kuo
Okay?
- [81:34]
Bob Wright
That's a phrase I. I trot out in the book.
- [81:37]
Kaiser Kuo
Yeah, yeah.
- [81:37]
Bob Wright
And the point is, look, transparency is often stabilizing in situations like this. You want to, you know, you don't want your imagination to get out of control. Moreover, if you know that there is transparency into what you're doing, you're less likely to try to get away, you know, to try to go too far. So that's great. And it just has a generally reassuring effect that calms people down and enables, like, norms to flourish and be stable because there's a little more trust. And my point is, I mean, look, this technology is not as easy to kind of regulate internationally as nuclear weapons in some ways. And so you'd like transparency to come wherever you can get it. And one point I make is that, you know, a kind of informal transparency comes from rich economic and cultural engagement. Right. It's like if. If scientists from the two countries are sitting down at conferences commonly and going out and having drinks with each other, if business people are doing the same thing and entertainers are doing the same thing, both countries just have a better idea of what's going on, even inside of, quote, you know, AI labs that don't make a habit of broadcasting what they're doing. You just have a better sense. And then these people, of course, talk to government officials informally. And I just, I'm a big believer in economic engagement. I recognize that, you know, it can have a downside if it happens rapidly in a way that leads to rapid job loss in a country. You know, I think there's a tendency to blame more of American job loss on engagement with China than is the case and not enough on technology. But still, it was not nothing, and you need to guard against that and do things gradually sometimes. But I think the fundamentally engagement among nations is just a good thing. It just increases the non zero sum dynamic.
- [83:53]
Kaiser Kuo
Amen.
- [83:54]
Bob Wright
And increases the costs of conflict when
- [83:57]
Kaiser Kuo
it comes to AI too. Look, let's just be frank, hero. So many of the US and Chinese AI researchers share not just one language, but actually two. So many of them are Mandarin speaking as well.
- [84:09]
Bob Wright
Oh yeah, the number of, you know, Chinese names on AI research papers, I don't know how many of them are Chinese, American, how many, I don't know what their various, you know, citizenship statuses are, but it's obviously a rich talent pool and I think it's in everybody's interest for that kind of collaboration to continue.
- [84:35]
Kaiser Kuo
So, Bob, you know, cognitive empathy, you and me, we agree, it's the master key. You nominate, and we mentioned this briefly, a specific cognitive bias as the big obstacle to, to cognitive empathy. It's one that's badly underrated, which is attribution error. You argue the textbook version is wrong or at least incomplete. It's not a uniform tendency, it's actually tribal. You lay out that argument for people, if you would. The asymmetry where an enemy's good deeds get explained away and a friend's bad deeds get excused. How does this whole thing work? And then I want to add a wrinkle of my own to this.
- [85:17]
Bob Wright
Okay. So yeah, the concept has evolved. In its original version, attribution error referred to what was thought to be just a general tendency to, in explaining people's behavior, over emphasize their kind of nature and under emphasized circumstance.
- [85:39]
Progressive Insurance Announcer
Right.
- [85:40]
Bob Wright
So you would, you would, you know, the person in front of you in the checkout line is rude to the checkout clerk and you say, that guy's a jerk. I mean, that's, that's a reaction. I think we tend to have, we make a pretty definitive judgment when for all you know, that day the person got laid off or the person found out that his wife has cancer or something. You just don't know. And yet, and yet it's true. We, we very often tend to lean toward disposition as opposed to situation. Those were the two terms used. Now subsequently we learned it's a little more subtle than that, actually. And there's a difference in patterns of judgment depending on whether somebody is a friend, ally, relative, or yourself maybe, and whether they're like a rival adversary, Emini, enemy. So if they're on your side and they do something bad, you attribute that to circumstance. Like, yeah, my daughter was mean on the playground, but she didn't get her nap. Now the other kid, mean to my daughter. That's A bad kid, right? Like that's just a sign that it's a bad kid where. And if they do something, whereas you know, with your enemies, you know, it's flipped, they do something bad and you go, yep, it's, they're bad people, of course you would expect that they do something good and you say, oh well, they were showing off, he was trying to impress so and so, blah, blah, blah, you dismiss it as something circumstantial and you know, this gets us into all kinds of trouble. I mean, if you as a social scientist, I quote in the book pointed out, you know, once you have an enemy like framed as an enemy, it's hard for them to get out of the box. Like anything bad they do, you will dismiss is not reflective of the way they are. I mean, no, the opposite. Anything good they, they do, you will dismiss. And of course, this is why such pains are taken by people who favor invading a country to just start painting the leaders as evil, right? And, and because if you can get them in that psychological box, there's no way out. So that's the more nuanced modern understanding of attribution error. And it impedes cognitive empathy, right? Because you know, if you're biased toward any type of explanation of why somebody did something bad or something good, then you're just not looking at things clearly to the extent that there's any kind of bias with friends or enemies. Right? It works both ways.
- [88:20]
Kaiser Kuo
So I want to add a thing I've noticed. I think there's this counterfeit of cognitive empathy that's actually powered by the same kind of attribution error bias. You know, it's projection, right? The hawk genuinely believes he's doing the empathetic thing some AGI pilled Valley founder is thinking. If I were Xi Jinping, of course I would grab AGI and then crush my rival. Right? But you know, he's filled Xi Jinping's shoes with his own mind, his own ruthlessness, his own like, you know, 15 year old adolescent brain, and he's never noticed that he's doing this. It feels somehow like to him like a form of empathy. It's actually the opposite. So I always am on the lookout for what is real cognitive empathy toward another country, toward Beijing, for example, apart from, and to separate that from just mere projection. And here's the part I want you to be straight about. Does it, when it's done honestly, actually make China look more tractable than the hawks claim? Or could it be just as easily confirming to them in their pre Existing beliefs. I mean, I always have this idea that if you practice cognitive empathy, you will end up with a much more positive picture that somehow. But I think most people use a bit of attribution and a little bit of projection.
- [89:52]
Bob Wright
Yeah, I mean, I don't think you'll always wind up with a positive picture. I mean, I just finished listening to a biography of Hitler, you know, and I think on the one hand the Munich analogy is overused to, you know, to discourage compromise, even when it makes sense. But look, he had some very unfortunate intentions and there's just, you know, no doubt about it. It can happen that you, you know, have that an accurate view of what's going on in someone's mind, someone's mind is an unflattering view that should lead to tremendous wariness. That can happen. Now as for people as pseudocognitive empathy people pretty much always, it seems to me, purport to be engaging in cognitive empathy. Right. They usually have a theory of what's going on inside the person's mind. But yeah, projection happens. I mean in the case of China and look, I don't, I don't necessarily have a, a view of their intentions as benign or anything. I try to be agnostic until I figure stuff out. But on, on the issue of their alleged desire to impose their system of governance on the world, I think that may be projected because obviously we, the United States is very explicit in thinking that the entire world should be governed the way the United States is. Look, I like democracy. I'm not, I'm not like necessarily against that aspiration. I'm against some manifestations. It has had various ill advised regime change operations. But the point is that that could well be an example of projection. There's, I mean this isn't exactly what you asked, but there are also self fulfilling prophecies like look at this authoritarian axis. That's more evidence that they want to get together and impose authoritarianism. It's like an alternative explanation is that because we sanction authoritarian countries, like they have that in common and they get together and collaborate economically as a reaction to that and so on. But I think I've taken this too far from your original. Remind me, what is your actual. What thread have I lost?
- [92:17]
Kaiser Kuo
You got there in the end. I was just sort of, you know, how do you discern the sort of honest, good faith use of cognitive empathy from mere projection?
- [92:25]
Bob Wright
It's, look, it's hard and like I just saw a piece, I think it was in the New York Times, I got to track it down about how. And it was very. It was almost an editorial, but it was a reported piece about how China, you know, whereas U.S. assistance to nation assumes the form of strengthening their defense with our military aid, China's aid assumes the form of bringing repressive surveillance or something. Now, it was a. It was a. I mean, my first thought was, well, wait a second. We have often helped client states be more repressive. You know, the Cold War is full of this stuff, right? And we use the finest technology available at the time. But that said, I have not done the deep dive I need to do on that piece to, you know, expand my understanding of what China's actual motivations seem to be. I just. What's clear to me now is that a lot of China hawks are making assumptions that they haven't thought through. And I would like to challenge more of them to think them through and show me the evidence. But sometimes it's hard to get them to show up for debate.
- [93:49]
Kaiser Kuo
I mean, you gave a really good example of this sort of thing. China funds the who, the World Health Organization, and American China hawks read that as a cynical influence play, and you say, fine, maybe that's partly true, who knows? But it's also kind of exactly why the US funds the World Health Organization. When America does it, we read it as basic decency and then it's the same act. You got two readings sorted, of course, by tribe. Yeah, yeah.
- [94:17]
Bob Wright
No, I think this is where we want the struggle for influence to take place in international institutions. Ideally, yeah, that will be fine. There will be a struggle for influence. I recognize that. I'm not. I'm not naive.
- [94:29]
Kaiser Kuo
Yeah, yeah. So I want to be conscious of your time here. We've gone on a nice long time, and I knew that we would. But you end the book with Ed Fredkin, this MIT digital physics pioneer who you profiled way back in the 80s, this man who told you that our mission was to create artificial intelligence, that sort of, like, why humans are here, that it was the next step in evolution. But the part that gave me pause wasn't his optimism, it was his worry. Because back in the early 70s, Fredkin tried to build an international AI lab, right? He wanted Americans, Soviets working together. And his explicit reasoning was that it had to happen before, in his words, certain countries would decide that AI was of strategic importance, at which point it would be too late. And when he told you that decades ago, this is in the 80s, his verdict was already past tense. This has now happened. So a founding figure of this field of AI saw The exact trap that we are now finding ourselves in. Saw that once AI became a strategic prize coveted by both the superpowers, that cooperation would be foreclosed. Look, this guy tried to head it off 50 years ago, and he concluded that even then the window had basically shut. I want to end where the book ends, on the constructive turn, but I want you to be concrete here, because this is the part that I think everyone waves at and nobody really specifies. You say that the path is to begin forming a true global community. Wonderful. I'm all for it. I love your idea of organic transparency to slow the race just enough to buy time to let the dialogue dissolve. All these fearful nationalisms that imprison us. We want to build toward monitoring and maybe even a pause in our headlong rush. Fine, great. I'm all for it. But if Fredkin already thought it was too late in 75, why isn't it far too late now? If it isn't, if there's a real first move available, what is that first move? What is it specifically between Washington and Beijing? Not the destination, but the actual first step. The thing that a US President and a Chinese chairman of the Chinese Communist Party could actually do in the next year that bends this even slightly toward the easy way rather than the hard way.
- [97:10]
Bob Wright
So, yeah, first of all, Fredkin, fascinating figure, never went to college, wound up a tenured professor at mit.
- [97:18]
Kaiser Kuo
Wow.
- [97:18]
Bob Wright
Had a theory of digital physics that I wrote about in the book, whose current version is kind of Stephen Wolfram's. I would say they have something in common, if you're familiar with that. Anyway, Fredkin, also self made millionaire, he had an island in the Caribbean where I interviewed him. He owned it. And he is said to have been the inspiration for this professor in the early 1980s movie War Games with Matthew Broderick, a professor who was obsessed with the danger of nuclear war, which Redkin was. And yes, when he was head of the lab at mit. I forget the name of it then, but I think it was a lab where, you know, AI research was going on at the time. And he did try to undertake this initiative of creating an international laboratory of AI because he believed that, you know, an arms race would be bad. And, you know, he failed. And he was somewhat despondent about that. As you. As you say, he said, now it's too late. I would say if you ask, well, why can we still be hopeful if somebody as smart as that thought it was too late when he said that to me, which was the mid-1980s, first of all, we were still in the Cold War. And that's when he had tried this. I mean, you talk about your uphill battles. I mean, Kaiser, you and I are old enough to remember the Cold War mind, the Cold War mindset. And I mean, we were young. We, you know, we would both emphasize we were very young. But yeah, it's not beyond our memory. And it, geez, it was just two different worlds. Like, I couldn't. The idea of traveling to the Soviet Union, that was like nobody, you know, I once talked to somebody who had done that, but it just, it just, There was little awareness of what was going on on the other side and so on a sense of intense rivalry. So in a way, it's not shocking that he couldn't get critical mass for this idea. Now in the current milieu, I do think you have some things going for you. And one is the fact that the power of AI is becoming manifest. You know, more people are going, whoa, this is like weird. And I think it's going to intensify over the coming year or so as people start using these agents to do more and more things. You'll see it in the workplace, you'll see it in the home, and just weird stuff will happen. And I think, and you know, a point in my book that I'm not super explicit about, maybe I don't, I don't ever say that. Like, look, if you look at the whole evolutionary trajectory here, this is in some ways such a straightforward extension of evolution that maybe we should think seriously about considering it a form of life. I don't know, but it is that weird is my point. And I, and, and I don't know. Is it sentient? Could it be sentient? You know, in other words, could it have subjective experience? I don't know, but I've never had a tech. Seen a technology where you. It was so hard to rule out the possibility. I mean, this is just weird. And you know, I think as that dawns on people and maybe you'll have like a significant mishap. I hope that the lethal effect of it is minimal, maybe even non existent. And in a way, you know, the mythos moment had a little of this. It was like nothing bad happened, but people went, whoa. You know, if you ask, why are we talking to China now? I suspect it may be because of Mythos, because, you know, Tom Friedman wrote this column about how, look, bad actors could do this. We need to talk to China. And it happened. So, you know, I think you have a couple of things going for you. First of all, a Cold War like the old one is not even possible in the modern technological world. To have that little visibility into what's going on on the other side is not going to happen. And you do have inevitably contacts among a number of people, even aside from formal diplomatic relations in the US and China. So you got that and then you've got this technology that I think is going to have more and more striking implications. And it could, I suspect that in a year we will be talking about more serious collaborations with China in an attempt to avoid the downsides of the technology than we have even now, I'm guessing. But in any event, it's not crazy. But I would add, I'm not predicting success, I'm just saying, first of all, there are several different ways this technology could work out badly up to and including the sci fi doomer scenarios which I examined closely in the book and wish I could rule out, but can't. And certainly I think there are various more mundane kinds of disruptions that would be very, very bad. Yeah, and I'm.
- [102:45]
Kaiser Kuo
Things that fall short of turning us all into paperclips.
- [102:48]
Bob Wright
Yeah, a little short of that. Although in some cases we might say paperclip would be no worse. But, but you know, I'm. I'm just saying if we want to minimize the chances of some very bad outcomes, we have to think in innovative ways about the degree of global community that is possible, that we could build a true global community where we actually see, you know, we're all in some. We all are in this together, this very important mission we have as this species on this planet. I don't think that's impossible.
- [103:25]
Kaiser Kuo
Well, Bob, there are so many other questions I had prepared to ask you, but the time will not allow it. But as you said, you know, during the old Cold War, you never even thought about going to Moscow. But you know, going to Beijing is something people routinely do now and which I heartily invite you to insist that you do come out. We'll have drinks together in Beijing and I can ask you all the other questions I'm there about to yar and about, you know, Steven Pinker and teleology
- [103:53]
Bob Wright
and look, Chinese translation rights to this book have been sold. So. And my. Tell me if this is a stereotype I shouldn't indulge in, but I think of China as a very. A country capable of great efficiency. So who knows, the Chinese version of the book may be out very soon and I will take you up on that offer for sure. I've been to Beijing once about 15 years ago, very different situation obviously in terms of US Chinese relations. But I would love to go again and have somebody like you to show me around.
- [104:33]
Kaiser Kuo
Well, I will make myself freely available to you. I can make all the time in the world to show you around.
- [104:38]
Bob Wright
I'm show up. Don't be too. You might not make this offer too open ended. I could become a very long term guest if you're not careful.
- [104:47]
Kaiser Kuo
Okay. I will be sure to annoy you just enough so that you won't overstay your welcome. Bob, what a fantastic time talking to you and thank you for spending so much time. But we've just got one more little segment. It's called Paying it forward. Oh, and then recommendations of course. So the paying it forward section was something I devised some time ago to, you know, put a little spotlight on a younger person in the field. Somebody who you work with ideally who you know, you want to just sort of name check.
- [105:19]
Bob Wright
Yeah. In the, in the realm of kind of AI, there's a guy with the lovely name of Garrison, lovely spelled like it sounds, who's got a book coming out in a few months called Obsolete. And what I like about him, I mean there are things we disagree on for sure, but he's on the left. And on the left there is more of a tendency than I'd like to kind of minimize the impact of AI or to say, yeah, you have to worry about the environmental impact of the data centers, but these companies are hyping this and it's not going to be nearly as powerful as people are saying. And then further, some people on the left, you know, kind of have a policy of just not using the technology. And I think two things. First of all, I think it is going to be very powerful. And if you have, whatever your ideological values, it's in your interest to try to engage in the discourse and instantiate them in, in policy. I also think that you don't, you know, it's like to not use AI, given the role I think it's going to play for better or worse. And look, I'd love the world to, to be the world I grew up in. I'm not, I'm not wild about all technological change, but the reality is AI is going to be, you know, part of it's going to be a tool, maybe a weapon, you could say, in the discourse. And I think it's kind of like unilateral disarmament to not be using it. Anyway, Garrison Lovely is an exception. This book takes seriously the possibility that AI will, for example, have Huge impacts on the job market and lead to job loss and so on.
- [107:07]
Kaiser Kuo
That's the very least. Yeah.
- [107:08]
Bob Wright
Yeah. And he grapples with all these things, so I would look forward to that. It's published by, I believe, you know, the Nation magazines kind of book. Great imprint. Yeah.
- [107:19]
Kaiser Kuo
Garrison. Lovely. All right. What about for a recommendation? Do you have a book that you've read recently that or a film or anything at all that you want to call it?
- [107:27]
Bob Wright
Well, my wife and I watched this series, this. This animated series called Pantheon. It's kind of. I think anime is probably the term for the. For the aesthetic vibe. And it's about super intelligence. It's not exactly AI as we know it. I mean, the premise is they're uploading human brains, that and so on, but it's. A lot of the issues are the same. I think it's very well done and it's. You know, a number of people in the AI community have seen it, and it's available on streaming. It also reintroduced me to this song, Don't Dream It's Over. I mean, you're a rock guy. I don't know if you know that song by Crowded House.
- [108:17]
Kaiser Kuo
I don't. I don't.
- [108:18]
Bob Wright
It's a. It was a. It was a big song in maybe the 90s, and it became very important to us because when our dog died last summer, it just became for Reasonable Good, kind of his theme song in our mind. But also, there's a line in the song about, they try to build walls between us, but they won't win. It's. There's this line, there's a battle ahead. It's. They bring the song on at the end of the first season. You hear it for the first time. It's like, there's a battle ahead. And then there's this line about they're trying to build walls between us. And, like, that's what they're doing. There are people in whose interest it is to make us fear other groups of people. And some of them make money off of it. Some of them become politically powerful off of it. But, like, they are the enemy. And of course, there are. There are real threats out there. There are things to be concerned about that other groups do and other nations, fine. But there are just people who are in the threat inflation business. They're trying to build walls between us at a time when we can't afford that. And I hope that, as the song says, they won't win.
- [109:40]
Kaiser Kuo
Amen. That sounds great. I'm going to check it out. Pantheon All Right. So I have one highly relevant recommendation for this. It's the high capacity podcast by Kyle Chan, who is banned from ever being name checked ever again on Paying It Forward because he was name checked about six times just in the last two years. But he's fantastic. He writes on industrial policy and is a fellow at Brookings now, but particularly the episode that he recently did with Matt Sheehan. Matt is at the Carnegie Endowment for International Peace and he works on AI regulation in China, AI governance, and probably knows more about it than anyone else in the English speaking world that I know. Matt is just a fantastic guest and the episode is called Is China Getting Worried about AI? And it sort of challenges this assumption that China has been extremely, extremely cavalier about AI and that its only efforts to govern or regulate it have been, you know, sort of content, moderation, that kind of thing, you know, political sensitivities. That's not the case and Matt and Kyle get into it in terrific detail. So that's my a very show adjacent recommendation. Something more fun. I have been I started about to finish the first novel of Patrick o' Brien's Aubrey Maturin novels on the set in Nelson's Navy during the Napoleonic Wars. Everyone's read these things, but I'm rereading them. I read a whole bunch of them years and years ago, back in the 90s, I think when I was in China, often like touring with a band, I would bring a stack of them and just sort of read them while we were. You know. There's a lot of hurry up and wait when you're actually like a touring band. But the two central characters of this series just make for one of the dearest pairings I've ever come across in literature. You have Captain Jack Aubrey and Dr. Stephen Maturin, and their intelligences are so totally asymmetric. And it's the source of a lot of the comic relief in this series of novels. But I've just been marveling at the prose, which o' Brien makes you feel like he was writing in the time when this takes place. He never breaks character. He's able to be really deeply funny and appeal to all sorts of, I guess, modern sensibilities. But he never feels like he's like sitting in the present and kind of winking at you or just cosplaying at it. He just never. It's crazy, but if I end up using a ton of nautical metaphors in upcoming shows, you will know why. Because I'm all about like tagallants and topsails and I'M starting to know the rigging on these damn 18th century sailing ships better than I should.
- [112:36]
Bob Wright
That's a warning sign.
- [112:37]
Kaiser Kuo
Yes, it is. But have you read these?
- [112:40]
Bob Wright
I have not. I am actually halfway through that episode you mentioned of high capacity and, and it's a really good podcast, but I have not. I mainly listen to books these days and I don't listen to enough novels as my wife would.
- [112:55]
Kaiser Kuo
Yeah, I love to listen to books and that's what I'm doing. When I say reading half the time, I mean listening, but I'm listening. And yeah, the narrator, I mean, they're recorded a long time ago, so the audio quality is not great, but the acting, the voice acting is terrific. There's a lot of accents and stuff that he does extremely well.
- [113:17]
Bob Wright
Well, Bob, thank you, thank you. This is, I, I, I, I've been, I've been waiting for my star turn on your podcast for years and years. I listen to it every week to see if I'm on it, but it turns out I'm not.
- [113:29]
Kaiser Kuo
But here, you've gotten a lot of shout outs over the years.
- [113:32]
Bob Wright
I have. You've mentioned my name in connection with cognitive empathy and, and I appreciate it every time.
- [113:38]
Kaiser Kuo
What else would I want to mention your name in connection to, after all? Right. Well, Bob, thank you for spending, my God, a couple of hours of your afternoon with me and I hope to take you up on that offer. That you take me up on the offer.
- [113:53]
Bob Wright
I guarantee you you will see me in China.
- [113:56]
Kaiser Kuo
I love it. All right. You've been listening to the cynical podcast. The show is produced, recorded, engineered, edited and mastered by me. Kaiser Goa. Support the show through substack@cinecopodcast.com where you will find a growing offering of terrific original China related writing and audio. Or email me@cinecopodmail.com if you've got ideas on how you can help out with the show. Don't forget to leave a review on Apple Podcasts. Enormous gratitude to the University of Wisconsin Madison's center for East Asian Studies for supporting the show again this year. Huge, huge thanks to my guests. My love, long awaited guest, good friend Bob Dry. Thanks for listening. See you next week. Take care.
- [114:58]
Progressive Insurance Announcer
Insurance isn't one size fits all. That's why customers have enjoyed Progressive's name your price tool for years now. With the name your price tool, you tell them what you want to pay and they'll show you options that fit your budget. So whether you're picking out your first policy or just looking for something that works better for you and your family. They make it easy to see your options. Visit progressive.com find a rate that works for you with the name your price tool Progressive Casualty Insurance Company and affiliates price and coverage match limited by state law.
- [115:28]
Kaiser Kuo
I started Ornod in 2013 and we make bike apparel.
- [115:32]
Bob Wright
The best part of Shopify for me
- [115:34]
Kaiser Kuo
is our ability to run the business as essentially non technical people. We're able to admin everything on the back end, front end and sell things online easily. If Shopify were a bike access accessory, I think it would actually be the bicycle. It's the thing that you do the thing on. We run the business on Shopify.
- [115:54]
Bob Wright
Start your free trial on shopify.com.