Loading summary
Transcript39 lines
- [00:00]
Arvind Narayanan
We should be prepared for radical shifts, like people like, you know, professors having a lot of status, because we can produce something that is unique today, might not be unique in the future. So there's going to be no need for people, you know, doing podcasts like this. This sort of stuff is all things that you can get from it. Maybe. I think we should be ready for that.
- [00:17]
Yascha Mounk
And now the good fight with Yasha Monk. A lot of the debate about artificial intelligence has a very straightforward assumption that this is an extraordinary piece of technology that is going to transform the world in fundamental ways and that any attempt to grapple with what it'll do to the world and any attempt to try and regulate it, to try and respond to it from the perspective of individuals or workers or corporations or the state, needs to recognize, though, in these extraordinary times. Well, my guest today is the most prominent voice in, in the debate arguing that that basic assumption is wrong. Avind Narayanan is a professor of computer science at Princeton University, and he is the author, with his graduate student Syesh Kapoor, of AI as Normal Technology, one of the most influential texts about AI written in the last years, which argues that AI is a powerful technology, it's going to change the world in all kinds of important ways, but that we systematically get wrong. We what it'll do, how fast it'll do those things, and how we can possibly respond to it if we think of it as abnormal. In this conversation, we talked about everything from how many jobs are likely to be lost because of AI to the kinds of broader military and existential risks that AI might pose to how we should respond to its rise, particularly from a broadly philosophically liberal framework. In the last part of this conversation, we talk about how AI is going to transform warfare, and I push Arvind a little bit on explaining to me what his framework means for how we should and shouldn't respond to the rise of AI. Is understanding AI as normal technology ultimately not just a fig leaf for not doing very much at all in response to it? Or is that actually what will allow us to really deal with it in an intelligent way? To listen to that part of a conversation, to support what we do here on the podcast? To get access to two full episodes without ads every week on your premium podcast feed, go to writing.yashammuk.com listen and become a paying subscriber. That's writing.yashamonk.com listen and be sure to set up your private podcast feed. Avind Narayanan, welcome to podcast.
- [02:59]
Arvind Narayanan
Thank you for having me, Yascha.
- [03:00]
Yascha Mounk
I'M excited for this conversation. You have coined a phrase that really is central to some of the debates about artificial intelligence. But what you mean by it goes a little bit beyond, you know, what you would intuit just by looking at the phrase. The phrase is AI is normal technology. What does it mean to think about AI as a normal technology?
- [03:20]
Arvind Narayanan
Sure, AI is normal as technology doesn't mean this is mundane, boring, nothing to see here. Move along. We're not AI skeptics. The paper starts out by acknowledging that this is and is going to be a transformative technology, perhaps on the scale of electricity, the various industrial revolutions, et cetera. We do think that it's a transformation for cognitive work in the same way that in the past we had transformations for mechanical work. But I think that historical analogy gives us a lot. We do think that there are deep lessons from history to think that just because this technology is so powerful, it's not going to transform the world over a period of a year or two. There are many technologists, tech leaders, business leaders predicting that just because numbers are going up on the charts, they're looking at AI capability charts, that it's going to make human labor unnecessary, you know, put people out of work, even end the concept of money, or pose existential threats to humanity. You know, we think these are all valid things to worry about. We're glad people are doing research on them, but we think there are so many bottlenecks between AI capability increasing and. And it's having these impacts, both good and bad impacts, especially good impacts, especially if you want to really integrate it into the economy and derive something useful out of it. There are so many things that have to go right, and we think those are going to happen over a period of decades, not months or years. And we have a lot of collective agency in shaping how AI is going to transform various professions, transform the economy, democracy, et cetera.
- [04:57]
Yascha Mounk
Yeah. So I find this to be a really useful way to think about this. And again, you're very careful to say you're not an AI skeptic. I guess I'm a little unsure about what exactly we mean by the term normal in this context. When you look at electricity, even when you look at the Internet, these were technologies that did transform the world in just a very, very fundamental way. Now, when you're comparing that against the most naive kind of pronunciations that some people in Silicon Valley are making, where in 18 months everything is going to have changed, I remember sitting in a conference room with somebody who's now very famous in Silicon Valley in 2017, and I'm pointing out of a window at a green field and saying in 18 months robots are going to be building skyscrapers over there. And it's 2026 and the housing crisis in California is as bad as it ever was. So I'm very, very sympathetic to that point. But even if this is going to take decades rather than years, and the way in which the technology shapes the world is obviously going to be inflected by all kinds of human choices and institutions and regulatory obstacles and so on, something on the scale of electricity still feels in some senses pretty abnormal, right?
- [06:12]
Arvind Narayanan
Well, it is a technology to be taken seriously. I think individuals, businesses, policymakers, all of them should be taking it seriously. And that's why we have been working in AI and we have my co author Syas Kapoor and I, and we have a long history of advising policymakers on how to respond to these changes. Yeah, I mean, if we weren't taking it seriously, we wouldn't be spending our own careers doing all this work. No question there. Just like electricity was a transformative technology. Absolutely. But to put the finger on some of the disagreements, exactly like you pointed out, there are so many claims that this transformation is going to happen over a year or two. And if that's the case, then all the more normal policy levers that we might have help people find new skills, given this new technology, so they can find new jobs. All of that goes out the window because nothing can happen fast enough on this timescale. And the only thing we can prepare for is mass joblessness. And so we should be talking about anything less than the scale of universal basic income and is completely disproportionate to the scale of the problem. So that's exactly what tech leaders are advocating, and we're strongly against that. We do think the normal tools of policymaking and business and human adaptation can work with AI as well. We do need to fix some problems, I mean, even quote, unquote, normal policymaking, even when you have a technology that's not advancing as fast as AI, often takes a very long time to acknowledge problems. Let's say with social media, it took well over a decade after it started having these massive impacts on society for the research to actually get there, for policymakers to wake up, et cetera. So those are problems we've had and we're going to have those problems with AI as well. We do need to improve the speed at which we respond to these changes. But it's not as if it's a one or two year thing. And we have to throw out our entire existing playbook and prepare for either some utopia or catastrophe.
- [08:09]
Yascha Mounk
Perhaps let's distinguish between the speed with which the transformation is going to happen and the scale of the transformation that will ultimately come. I think that we strongly agree about the speed. And having read a lot of your work, I'm not yet sure whether or not we agree about the scale. But let's start where it's easy, which is the speed. Why, in precise terms, do you think that the impact of this technology is, is going to take much longer to materialize in the real world than a lot of these people in Silicon Valley seem to believe? Why is this a question of decades rather than years or even months?
- [08:45]
Arvind Narayanan
Let's take a concrete example. Software engineering. And that's the area where AI adoption has been most rapid. AI capabilities are furthest along. And there has been this prediction that AI is going to automate software engineering once AI can write all the code. We already have enough evidence now, and, and we just put out an essay on this to reject this model of AI sympaths. AI is writing most of the code now already. It has not made software engineers obsolete. Software engineer hiring the number of employed software engineers is still growing, maybe slightly slower than before, but even that is disputable because it's also allowing software engineers to more easily become entrepreneurs, for instance, and that's not quite captured by the data. So we looked into why is this, and we kind of have a model to explain this. It's what we call the decide, execute, deliver sandwich. So these are three layers that we think applies to most kinds of cognitive work, and again, cognitive work, thinking for a living. Those kinds of work are what people are saying is going to get automated with AI. But our analysis is that most of those jobs, whether you're a software engineer or a lawyer or a researcher, many other kinds of cognitive work, like a good third of it is just figuring out what the problem is and making decisions about is this problem worth solving, how are we going to solve this problem, how are we going to build the software, design the system, etc. Then maybe another third of it, again in the software engineering context, is writing the code, fixing the code, et cetera. Then another third of it is delivering the code, and that means verifying it, making sure it passes quality checks, being accountable for it, integrating it into the customer systems, maintaining it over time. Especially when you're thinking about enterprise software rather than consumer software, this is a big chunk of it. What we're finding is that AI is compressing the middle of the sandwich. It can do the writing of the code, but the other parts of it that require a human to be accountable for what is ultimately delivered, it has not taken the human out of that process. If anything, those layers of the sandwich have expanded to, to kind of fill the gap, if you will, of the time savings that you get from having AI write all the code. There's more to say, but I'll stop here and curious for your thoughts.
- [10:56]
Yascha Mounk
Yeah. To what extent is that a function of a stage of a technology ChatGPT 3.5 was ruled out, what, three and a half years ago? The progress in what these systems are able to do has been astonishingly rapid. Clearly, on the core task of writing the code, they now rival the very best human engineers. But of course, in terms of putting in place the structure for AI agents to be able to reflect on what actually is a worthwhile problem to solve within an organization, or thinking about how it is that you set up AI agents that are actually able to reliably test the quality of this code in the context of, of the company's needs and so on. We're not there yet. And that certainly speaks to your point that it's going to take a long time for these things to roll out. But what is to say that they won't be rolled out over the next three or five or ten years? More broadly, when you're thinking about a lot of these kind of essentially political demands within a corporate context, some needs to be responsible if this leads to some terrible outage. You got to have somebody you can fire, right? You got to have somebody you can blame. That's certainly true in current organizations, and it is very hard for those organizations to reform themselves. But surely when we're thinking about a longer timescale, we might imagine AI native corporations that have a totally different setup, which is much better able to use AI technology to replace some of that human judgment.
- [12:40]
Arvind Narayanan
Yeah, that's a great question. And I think this is a key point of disagreement. This is where we disagree with a lot of our critics who might agree about the timeframe, but disagree about the scale of transformation. So, two points. Yeah, like you said, political demands. I don't think these will change simply because the capabilities improve. One of our key points is that we have agency to insist that humans ultimately remain accountable for what is delivered. That might be true even if in some narrow sense of capability, AI could do a better job. Precisely because AI is not something you can punish, if you will keep accountable, does not have some of those limitations that humans do. I think the right thing to do is to insist that humans remain accountable. There is something inherent about human accountability that you cannot get with AI, and that's not up to the AI companies, that's up to the downstream organizations who are deploying AI. And if it becomes necessary regulation to insist that this be the case. And this is what we've done historically with many technologies that are very powerful but dangerous. I like to give an analogy of a crane operator. The crane can do the heavy lifting and construction. We don't need human physical labor. But even though we could, we don't let the crane operate autonomously. We put an operator in it, and the crane becomes a massive amplifier of human ability here. And that's the model that we think is the right one. And we can choose to keep it that way. We can't guarantee that it will be that way. But at the same time, I don't think it's a matter of the tech companies themselves deciding. And the second point, and this is another key area of disagreement. So we do think it's true that gradually, over time, the set of tasks that you can delegate to AI is going to expand. But what that means is that once AI can do something, it no longer is a source of competitive advantage in the enterprise. So it's not a fixed amount of things we're trying to build. And then once you can build those things with AI, you're done. There's no need for work. Companies can automate it. That's not how the economy works. Once you can automate certain things, delegate those things to AI, that becomes a common capability that every firm has. And then now what firms are competing on is. Is what is scarce. And human labor will always be what is scarce, because it's not. You can't make infinite copies of it. And so those remaining pockets of judgment that AI is not yet able to perform, that is what will be in the domain of human judgment and expertise. And that's what companies will compete on again in the distant future. This might change several decades down the line, but for a long time, we're going to be in this period of a constantly upward adjusting equilibrium.
- [15:24]
Yascha Mounk
So again, I take these arguments very seriously. Just two observations about this. The first is that you slipped in this race several decades. And I'm trying to think about this at the timescale of history and the timescale of the challenges that our society is going to face. Are people who are today in the 50s or 60s likely going to be fine? I think so. Are people in their 30s and 40s going to be fine. I think it's more open question, what is the world going to look like for people in their 20s today who are about to go to college today? I think that is much, much more challenging. So insofar as the answer is it'll take several decades, that is reassuring in some kind of sense, and it certainly is totally in line with the evidence from the Industrial Revolution, from how long it took for the invention of a printing press to have a major impact on European culture or society. I absolutely buy that. It's just not clear to me that several decades is as sort of reassuring as it might be. Right.
- [16:21]
Arvind Narayanan
Yeah. And look, I mean, the AI as normal technology framework is reassuring to some people, it's not reassuring to others. Our goal is to give this framework for people to do with it as they will and not necessarily to provide a reassurance. The reason it's reassuring to me is because over a several decade timescale, I do think people can adapt. Almost everyone can adapt. If it's one or two years, then most people can't adapt. If it's several decades, then we can kind of use game theory, if you will, to think about what that equilibrium is going to look like whenever that happens and try to see what we can start doing now to prepare for that. So one way in which things might turn out, and the equilibrium part is something we're only now starting to think about. It's not in the essay itself, is that it could be that, for instance, any task in the economy that you can specify precisely enough for it to even be legible as a task that's going to be done by AI. So most of the things now that, okay, my job consists of these 10 things, all those 10 things are going to be done by AI, and what jobs will mean in the future is what economists sometimes call interstitial tasks. The things that lack a precise specification, where maybe part of the task is even to figure out what should be done next. Things that maybe have never been done before. So those are the kinds of things that we think will be in the domain of humans. And the reason I think we will be very unlikely to leave those things to AI is because of unknown unknowns. It's because you can't be sure that when you task AI with doing something that's so different from anything that's been done before, it's going to do a halfway reasonable job. So that's one way to think about what a new equilibrium might look like. Another, perhaps even More radical shift is that maybe everything that's kind of a commodity and a commodity, not just a physical good, but even cognitive labor, a commodity is one where you don't care which particular human or AI produces it. So a lot of what we produce now are commodities. So I produce research, I write these articles. It doesn't matter that it came from me. It's really mostly the ideas that matter. And actually maybe that's mostly going to be done by AI in the future. So we should be prepared for radical shifts, like people, like professors, having a lot of status, because we can produce something that is unique today, might not be unique in the future. So there's going to be no need for people doing podcasts like this. This sort of stuff is all things that you can get from it. Maybe I think we should be ready for that.
- [18:50]
Yascha Mounk
But I think the three P's are always going to survive. Professors, podcasters and prostitutes. But we'll see.
- [18:57]
Arvind Narayanan
Nice. Yeah, maybe. I'll bet against that. So those things are very different, right? I think some of those services are very different than others. So sex work, for instance, is inherently relational. And the point, it's about a particular person. And there are many kinds of jobs in the economy, even some low status jobs, like a barista, for instance. And Alex Amos, economist, has nicely written about why some of those jobs have been very resistant to automation is because the human interaction actually matters there. And what the economy in the future will look like maybe is basically people paying each other for our time, for the unique relational value that they can give us. And the value of going to a coffee shop will be the customer service. If you have good coffee and shitty customer service, that's totally useless because robots can make the good coffee, but it's the good customer service that they can't replace.
- [19:53]
Yascha Mounk
I'm currently at my beloved place in Italy working on the next book, which I'm excited to share with you. And while it is less oppressively hot here than it is in most of Europe at the moment, it's still pretty damn warm. That's why I'm very lucky to be wearing my linen shirts and pants from Quince. Quince's 100% European linen pants and shirts are breathable, but really easy to throw on. Whether you're just sitting at home writing or going out to meet friends with a summer upgrade your rotation needs and they start at just $34. Make your summer wardrobe easier. Go to quince.com good fight for free shipping on your order and 365 day returns. Now available in Canada too. That's Q U I n c e.com Good fight for free shipping and 365 day returns. Queens.com Good fight. So let's take a step back here because there's a lot of really interesting ideas in this. So a few thoughts, right? The first is it's not clear to me why an AI system that at this point is very good at writing short stories. We've recently learned that an AI generated taxes, won a major literary prize, with judges being unaware of this fact at the time that increasingly does have good judgment about medical conditions, for example, looking at a lot of contradictory data and coming up with the best kind of adaptation of what's happening and so on, won't also be able to learn these kind of interstitial tasks. So there is some kind of future where AI systems actually prove to be as capable as the most intelligent humans. Even a task that looks slightly ragged or irregular. The second kind of possibility surely is that there's some human beings that have exceptional skills that will still be required. There may in fact be some human beings who are able to command tremendous wages because they are so good at orchestrating these AI swarms to accomplish tasks relative to others. But it may be that the number of people who are capable of those kind of cognitive skills remains very limited, leading to a kind of class or caste or super gainers from AI and then a large portion of a population whose skills are much less valuable. A third kind of possibility, and to some extent you can mix and match these, is to think that perhaps there's even need for relatively ordinary human skill. Obviously being a great barista or a good bartender is a genuine skill, and I admire the people who do it. But it's a less rare skill than somebody being able to code at the very highest levels was a few years ago. But perhaps the number of positions we need for that are just much less than the extent of available human labor. It may be that 9 out of 10 times I'm very happy to take my excellent robot made or coffee machine made coffee for 10 cents. And yeah, once a week I want to go and meet somebody for coffee in a really pleasant environment. And I'll definitely want a super cheery nice barista to make my coffee on those occasions. But that just may not create enough work for people to keep going. And of course in that kind of world there'll be an army of people who are out of work who are very happy to take that job as a barista, depending on the kind of socioeconomic Arrangements and background conditions that hopefully would cushion poverty. And in that case, even the people whose work is still in need, whose skills they still demand, may not make a very good salary. Right. So I think here we need to sort of be careful not to go into a world of either no human being ever is going to have a job anymore, which I think the two of us are going to agree on, is a kind of weird Silicon Valley. I'd call it a dystopia. Some of them would probably call it a utopia. But I think it's important then not to say, well if that's a 1, then the 0 is some jobs are still going to be needed. And so therefore the problem is solvable. There's all kinds of worlds in which certain kinds of jobs are needed, but they're not the ones that create the most economic value. They're not the skills that are actually scarce. And therefore they fall very far short from creating the structural preconditions for an affluent middle class society.
- [24:26]
Arvind Narayanan
Yeah, look, all of those are possible and I'm glad that we're talking about them. People are looking at leading indicators for which way things are trending. I do think these are solvable problems over a multi decade timeframe. Over that kind of timeframe, even if the answer turns out to be universal basic income, we can plan for that over the long term. That's not something we can make happen in 18 months, but it is something we can make happen over a several decade period. I genuinely don't think something that radical will be required. I agree with most of what you said. Maybe the key disagreement is that I don't think the set of jobs that are necessary in the economy versus not necessary. Is that helpful to me, at least in my view. The vast majority of jobs today are already doing things that are not near the bottom of the Maslow's hierarchy. The things that we need like agriculture and shelter, that's, I don't know, maybe something like 5% of jobs. All of the other stuff, if you just took yourself away from the everyday intensity of life and the economy and really thought about it from an alien's perspective, it's all just stuff that we do to amuse ourselves. I really like David Grady.
- [25:44]
Yascha Mounk
Sometimes I walk through the streets of Brooklyn, I say there's so much wealth here. What does everybody do? What makes this perpetual mobility work? It's just people selling each other goofy services and somehow living well off of it.
- [25:58]
Arvind Narayanan
Yeah, I mean, I do, you know, in many parts of the world there's obviously genuine crushing Poverty in affluent societies, 95% of jobs are things that we do because we need things for people to do, not things that we do for people to survive. And I'm saying that in the future that 95% will go to 99%, maybe even 100%.
- [26:16]
Yascha Mounk
Yeah, I mean, I'm torn on this. I do have a feeling walking through the streets of Brooklyn, But I guess when I think about this example a little bit more carefully, one of the huge reasons why New York is so affluent is that finance is headquartered in New York. And of course, many other industries, from fashion to media and so on, have huge presence in New York as well. Now we can have a big debate about whether finance really is a productive enterprise or whether it's not. But the point is that that is where a lot of the value in the society is created at the moment. And in order to run Goldman and JP Morgan and all of those companies, the owners of that capital currently require a lot of scarce and very high skill labor. And one of the reasons why the streets of Brooklyn are full of expensive yoga studios is that a lot of the people who go to those yoga studios work for those banks or they work for people who work for those banks. And so this is where the kind of tremendous wealth of the city diffuses. In an economy in which AI systems produce the huge majority of value and human beings then are kind of left in the role of we can educate each other and look after each other when we're old and put on concerts for each other, but all of that is kind of peripheral to the core activity. Can that sustain itself? Right. And I think perhaps in some simplistic way, that does depend on whether the first image I have of Brooklyn is right or the second image I paint of Brooklyn is right. And much as I like to hate on Brooklyn, which I love to be in, I think perhaps the second image is a little bit closer to the truth than the first one, which is somewhat troubling in its implications for what the world would actually look like.
- [28:11]
Arvind Narayanan
Yeah, again, look, it's a possibility. I think the reason I'm betting against that is when you look at what actually has value, it's just very contingent on what happens to be scarce at any particular point in history. Things like light and clothing. I think these things at the bottom of the mass laws hierarchy, they used to be extremely scarce because it was hard to produce them. And they employed a very large fraction of the workforce. And their costs have fallen by a factor of well over 1000 over a period of a Couple of centuries. And that's because of automation and that's because of larger scale energy production. Cheaper cost of energy, again, well over a thousand fold. Decrease in the cost of lighting. It could be closer to a million. I forget the numbers, but it's many, many orders of magnitude. I think a lot of the things that people do today are scarce and valuable precisely because human labor is scarce. I disagree with the premise that in the future AI systems will be creating most of the value. That's not consistent with my understanding of economics. I agree that AI systems will be doing most of the work, but any work that AI can do, again is infinitely reproducible. When it's infinitely reproducible, it doesn't have a lot of value because its cost is going to come down in the market because anybody can provide it at very low cost. The value is going to shift to the kinds of things that are scarce today. The labor of taking care for the elderly, for instance, very low status, it's very low paid. But in the future, the argument is that when a lot of these things that are higher status and higher paid today, maybe like the things we're doing, those are going to become very, very easily reproducible. And so what is considered valuable in the economy is going to shift.
- [30:03]
Yascha Mounk
Well, how does that work mechanically? So you're saying taking care of the elderly is going to be the thing that is really valued and presumably that'll command a salary that is more generous than what Professor Princeton gets today, hopefully. But where does the elderly person get the money to pay the person taking care of them such a handsome salary? And presumably the answer to that is that they themselves must have done something in the economy that accumulated that money for them. Or that we have something like system of universal basic income, which I take it you're mostly opposed to. And that'll be interesting to go into that provides them with the money from a third party. Right. And again, even though looking at Brooklyn today, the intuition is that it's a kind of weird perpetual mobile where everybody's just selling stuff to each other. I think when you look at it a little bit more closely, you can say no. There is actually a way in which the banks are financing manufacturing companies and agriculture companies and all kinds of other stuff that produce things. And there's a lot of very well paid employees of those places. And they're the ones that explain why people can pay $10,000 to go see the Knicks in the NBA Finals. You take that link to the underlying economy away because so few humans are required to run that underlying economy because whatever the most productive firms of a future are just requires so many fewer human beings working in them. And it's unclear how that money is going to diffuse into the economy such that even if socially we say, what a wonderful task to look after elderly people, that elderly person actually has the money to pay a good salary to the person that is taking care of them. I mean, a very different kind of way of putting this point is that you compare Walmart, which has a valuation of about a trillion dollars, a little less, I think, depending on exactly when you look and has 2 million employees and then you look at Entropic, which probably is going to be valued at about a trillion dollars when it IPOs and it has at present about 5,000 employees. That is a really striking difference. And that surely is going to have a huge impact on who can actually pay for these kinds of human centered skills to be valued in the labor market.
- [32:38]
Arvind Narayanan
Yes, it is. Yeah, it will be a challenge. I don't want to minimize the scale of the challenge. It is a challenge we'll have to solve politically and economically. It won't automatically happen. I'm just saying that we will have the means to solve it, even though it will require a lot of work to solve it. And the reason I'm saying that is that we're kind of making kind of fantastical assumptions in one dimension, which is that AI is going to be able to automate nearly all work and we're assuming that the world is going to stay the same in all the other dimensions. And I think that is a bit of a pitfall. This future might not happen. It might be that even though I'm kind of on the more slower transformation side, maybe even I'm being way too optimistic. And maybe there are really fundamental barriers to AI being able to take over a lot of this work over even a multi decade timescale. But let's assume that all of that happens right then clearly it's going to be creating an incredible amount of benefits to humanity because so many of our material needs are going to be taken care of and, and there is going to be, I think you agreed, some accumulation of wealth. It's the distributional question that is unclear. And so if greater and greater amounts of wealth are going to be accumulated, we can start putting policies in place Now. I'm not opposed to ubi. Maybe UBI is what we should be talking about again over a multi decade timescale. I just don't think it's the sort of thing that can happen if we're talking about a one to two year timeframe and so on the point about anthropic being a small company that might have a trillion dollars in value, yeah, maybe we're headed to a dystopian future where only three companies are going to be able to do that. But actually maybe what's going to happen is that entrepreneurship is going to become so much easier that there are going to be many millions of millionaires, maybe not anthropic level wealthy, but far more millionaires than they are today. Because you can have one or two people owning firms that use mostly AI to, to accomplish a lot of valuable things. And then it becomes. It's still challenging, but at least it's possible to think about politically and economically, how do you distribute that wealth throughout the rest of society?
- [34:55]
Yascha Mounk
By the way, this is a side note, but there is something strange in these discussions where we want to assign this deep value to the kind of human tasks that on one future may still be required. And of course I do think that relating to humans is something incredibly meaningful and that often people who are in jobs where they relate to others actually have a high job satisfaction and so on. But there is at least something melancholy about imagining a future in which all of the kind of tasks by which humanity has historically distinguished itself from other animals that we thought was kind of distinctive to us, right? Our ability for higher order reasoning, or even just in a social sense, the way in which for the last century or two for a huge number of human beings, where pride and the sense of self development came from acquiring an education, acquiring skills, becoming expert at some kind of mental work sort of go out of a window. And we're saying the most primal human skills and abilities to take care of babies and to take care of old people are now the thing that's left. And that's what we really value. Again, I mean, I really do not mean by this to devalue the people doing incredible work as nurses, or as people who work in old people's homes, or mothers and fathers who are taking care of little kids. But it is perhaps less than a holy fulsome endorsement of the future to say that that's the one kind of thing that's going to be left for humans.
- [36:38]
Arvind Narayanan
Yeah, that's totally fair. Again, I'm not saying that this framework for thinking about the future should necessarily be reassuring to everybody. I think there's lots that we will lose that we like about our current society. There's lots that we will mourn. And I think that's a very valid reaction to all of this. But again, it is parallel to what has happened in the past with valorization of physical strength. If you think about a Roman general, they moved up the ranks economically and politically because they started from their physical strength being very valuable in the army, and that's how they rose up the ranks. And we've been talking about professors in academia, and you probably know this better than I do, but if I recall correctly, the word academy, the original academy took its name from garden, where the educational component happened in parallel with the development of physical strength. And those two things were really valued in very much the same way. But over time, we've lost that. We still have athletes, for instance, having high status positions in society, but not because they directly contribute to the economy using their physical strength. It's just something we like to appreciate in a competition setting precisely because it's lost its direct economic value. Now it's become sports. And maybe what will happen with intellectual ability, just as has become with chess, for instance, is that it's a sport that we enjoy. And similarly, people will, in the future present their manually handcrafted ideas without AI in a format that we can all enjoy to appreciate the limits of the strength and limits of human cognitive ability. But, yeah, it'll be kind of disconnected from the economy. And there's genuinely a reason to be sad about that.
- [38:26]
Yascha Mounk
Yeah. I wonder to what extent the Roman general was prized for the physical prowess or for the mental skills in a very different context. About 20 years ago, I was dragged to a bullfight, and I was quite reluctant to go because I thought it would be quite an ugly, skeptical, in part because I thought it would be kind of macho spectacle about how strong the bullfighter is and how big his muscles are and so on. And it turned out that at its best, to my surprise, I rather enjoyed the spectacle. And to my surprise, what I enjoyed about it specifically was that I came to understand that it was actually a medieval form of celebration, perhaps an early modern form of celebration of mind over matter, which is to say that the bull is 50 times stronger than the strongest man in the world. There's no way that any human being, no matter how often they go to the gym or how many protein shakes they take, can take on a bull in a competition of strength. What a good bullfighter does is to demonstrate that they can use their human skills, their ability to understand how the bull operates, which is that he goes after the red thing, the ability to wave the flag in such a way as the direct, the Bull's movement to do a kind of dance with Bull, in which they demonstrate, look, what I have is intelligence. And using that intelligence, I can prevail against this beast that is vastly stronger than me. The cruelty of it is nevertheless something that troubled me. But I found that to be actually very interesting. And so you are right that we have historically valued physical strength a lot more than we do now. But that argument, of course, cuts both ways, because it is now quite hard to make your living from just having physical strength. And the people who do command much lower salaries. And when we look at, and when we look at past transformations, like the transformation after the Industrial Revolution, a lot of people were screwed over by them. And this happened over decades and over centuries. But the thing that they had, the thing that they resorted to, is what I sometimes call it sort of mental reservoir. There's a reservoir for demand of human skill which was to carry out mental jobs, mental tasks, cognitive tasks. And so if you were 45 years old and you lost your job in a factory, or in weaving cotton, or in cultivating a field, your life might be screwed, might be too late for you to have a good income, but your kids were probably fine because they acquired those cognitive skills and that's what kept them in the mud and ultimately gave them a much better standard of living than you likely had. But if we're now tapping the mental reservoir, is it clear that there is some higher order skill that humans can go to in order to compensate for that? And we just go back to the things we've kind of always done all along, like taking care of kids and old people? It's unclear to me if that's going to cut it.
- [41:39]
Arvind Narayanan
Yeah, again, I think you're right to worry. I think there's a lot that we will lose. I guess if there's one point of disagreement, it's that I don't think that in the future we will distinguish ourselves by our skill. We will distinguish ourselves just definitionally by the fact that human time is scarce. And AI's biggest weakness is that anything one AI can do is something a billion copies of AI can do. And scarcity always commands value. That's how it's always been. And we can create value out of even completely artificial types of scarcity, like wedding rings and diamonds, even when there's actually no intrinsic value. And yeah, in one sense that's very depressing. But again, I think from an economic perspective, I think we'll be okay. There's going to be a lot of demand for human time for us to have just various forms of companionship. We're going to call it various things. We're going to call it coaching therapy, we're going to call it care. And a lot of professors jobs, I think in the future will just be about managing students emotional journey as they undertake their path through the education system. The content we don't have to provide. That's a good thing.
- [43:01]
Yascha Mounk
That's already more administrators than professors at most universities. Because really what we need is not professors. These administrators maybe.
- [43:09]
Arvind Narayanan
Look, if this all sounds incredibly depressing, I'm not here to say we should, we should embrace this future. I'm just saying that it's so. I'm saying a few things. It's not a catastrophe of everyone being out of a job will have jobs simply because human time is scarce. And we'll just find artificial ways to turn that scarcity of human time into jobs and call it various things, whether it's professor or a therapist or caregiver. And lots of wealth will be created. Maybe we're going to have a problem with a few companies trying to capture most of that wealth, but that is something we can and should change with policy. And all of this is going to unfold over a period of decades, not one or two years. So we have time to prepare both in terms of new things we'll need to do, but also just getting mentally ready for this future. Whether you find it empowering or depressing or anything in between. Those are my core points.
- [44:07]
Yascha Mounk
When I started Persuasion, one of the hardest things was not figuring out what I wanted this project to be or who I wanted to be involved. It was setting up a lot of the straightforward stuff that you need in order to run a business. And I'm sure that many of you are sitting on business ideas that you would really love to do, but perhaps you don't know how to overcome those obstacles. That is where Shopify comes in to help you. Shopify makes it really easy for customers to check out whatever it is they may be purchasing from your business. And because it simplifies this whole process, it also allows you to really focus on growing your business instead of dealing with terrible bureaucracy and minutiae. With Shopify, nothing stands between your idea and a real business. So go make it one. Start your free trial at shopify.com start your free trial at shopify dot com good fight. Yeah, and I'm pushing on them, but I think that they're very interesting points. Clearly one of the things that you've implied a few times in the conversation that you've written about more explicitly is that there's real dangers to thinking of artificial intelligence as an abnormal technology. If we assume everything to change the next two years, and we assume that we have to kind of throw out our usual toolbox of policymaking, you think that has really bad consequences for our ability to govern this technology and for the world? Why? How would we be pushed in the wrong direction if we don't recognize that AI is actually a normal technology in the sense you outline?
- [46:02]
Arvind Narayanan
Yeah, it's very simple. There's this narrative coming out of the AI safety community, and they're doing great research. I respect all the work they're doing. Where I disagree is on the policy implications and this overarching narrative. And that narrative is that AI is getting more and more capable, and it's going to increasingly be able to do things like hacking into critical infrastructure or creating biological weapons. And so if this technology diffuses throughout society, it's extremely dangerous in two ways. One, AI systems itself might turn against their creators, or bad guys might get a hold of them and do dangerous things. And the way to prevent this is to stop really powerful AI systems from being released to the public without guardrails. So there are, I think, a lot of problems with this. So the first one is the idea that increasing AI capabilities are automatically dangerous. That's not what we've seen historically when it comes to cybersecurity, for instance, superhuman ability to find and exploit vulnerabilities in software. That started a couple of decades ago, and it's been going up since then, even before modern AI systems. Actually, that's made software more safe, not less safe. And the reason is that defenders, the companies that create software, can use those same capabilities to fix those bugs before they even put the software out there. So generally speaking, these powerful technologies have actually helped us improve resilience rather than worsen it. So that should be our default presumption. Maybe there are exceptions in some cases, but we should insist on evidence that increasing capability is actually making the world more dangerous. So that's the first big area of disagreement.
- [47:43]
Yascha Mounk
So presumably, one of the things that you might conclude on the most dire version of this claim that you're arguing against is we're about to have these incredible cyber warfare capabilities. Rogue actors are going to be able to break into your bank account and probably break into the database of US Military and possibly commandeer nuclear weapons. If that happens, then we need to sideline any kind of historical protections we've had for private businesses, but also for individuals in order to make absolutely sure that that doesn't happen. Right. And so perhaps we need to empower our governments with all kinds of capabilities that it doesn't currently have in order to sort of avoid that kind of worst case scenario. Is that one of the kinds of fears that you have?
- [48:33]
Arvind Narayanan
Yeah, that's one of the claims that's often being made. And we have a different position. I do think the government has to be nimble, adaptable. Some of the things that have happened in the US policy context recently, like having voluntary agreements to have governments test the capabilities of new technology and maybe hold off for a month or so before releasing them, those are pretty low on the ladder of kind of extraordinary interventions. And, you know, if we do them carefully, I think that can be okay. But where the problem comes is saying that the way to prevent rogue actors getting access to this technology is to control it so tightly that we make sure they never get their hands on it. Our view is that the only way you're going to do this is by having authoritarian governments. And not just one authoritarian government, it's a kind of world authoritarian government, if you will, because it's getting so much cheaper each year to build these incredibly powerful capabilities. And when you look at those exponential charts, they're amazing. It's just kind of Moore's Law all over again. And so at most you're buying a couple of years and eventually this technology is going to diffuse into the hands of everybody. The thing we can do is to shape it.
- [49:45]
Yascha Mounk
And one side point here, by the way, that I'm sure you know more better technologically than I do, so I'd love to hear your thoughts on it, is that our frame of reference often is nuclear weapons, right? So that is an incredibly dangerous, powerful technology that can destroy the world. And we've been able to to some extent prevent its diffusion through international treaties and regimes of inspection control under certain circumstances, et cetera. My understanding is though, that there is principal differences between nuclear technology and the capability to build cutting edge AI models. And one of them is that nuclear technologies requires much bigger physical machinery. So you can, you know, the United States is quite good at figuring out through satellite images and other things whether the centrifuges in Ishvahan or wherever they are in Iran are spinning or not. It would be much harder to do that for data centers. And the other point is that there's always some dual use issues in nuclear power in terms of civilian use of nuclear power. But broadly speaking, you know what it is being used for. So much of our civilization already depends on chips and data centers. I mean, literally some of the most powerful chips used to develop artificial intelligence were originally designed to power graphics for video games that were actually controlling whether or not if the United States and China had a big treaty agreeing that they're not going to develop cutting edge AI models, controlling from the American point of view that China is sticking to this, or from the Chinese point of view that America is sticking to this would be incredibly hard because of this challenge of dual use.
- [51:27]
Arvind Narayanan
You've exactly captured it. There's dual use. There's the powerful economic logic of diffusion of AI capabilities and chips. There's the observability and the physical bottleneck. Let me add one more point. There's no particular threshold at which AI becomes dangerous. Every level of AI capability that we've had has always brought some dangers with it. And you can go back to very recent history and there is just striking moments of panic around GPT2. For instance, OpenAI didn't initially release the model because they thought it was so dangerous. And it's a model that today my grad students build over a weekend on a single GPU in order to teach themselves how to build these AI systems. So that's how rapidly the technology has diffused. And we're so much more powerful than GPT2 now with our recent models. And yeah, if you stood up and said today that GPT2 is too dangerous, that would sound so ludicrous. And my prediction is that whatever we're saying with Mythos or whatever is going to look exactly like that a decade from now. And the last point I'll say is that the good thing about the fact that we have been releasing these systems is that it's also forced us to improve our defenses. Because if some of these models can be used for hacking, it's forced companies to work with various kinds of software makers to give them better access to these models again so that they can find and fix these bugs before the software even ships out there. If we instead follow a policy approach of nobody having access to the technology in the first place for these dual use purposes, then we will lose the opportunity to build that immune system and the dam will break eventually. And when the dam breaks, it's going to enter a world that doesn't have those defenses in place.
- [53:07]
Yascha Mounk
So I'm very worried about giving the government powers over individuals and to some extent over corporations as well. I'm a philosophical liberal. I give great importance to individual freedom. And I think that actually we're going to face certain kinds of dilemmas in this context that are quite as similar to those that previous technologies posed. So going back to nuclear weapons, for example, we obviously need to govern how states use nuclear weapons. And one of the scary things is that from Kim Jong Un to not to compare them entirely, Donald Trump, some rather irresponsible people currently have their finger on the button, and that's deeply discomforting. One thing about nuclear weapons though, is that while they can destroy and blow up the world, they aren't very useful for domestic political control. You can't really threaten to win in control of the next election by nuking some city you don't like. I mean, I guess you could, but it's a very blunt tool. Right. The scary things about some of the most cutting edge AI models from a political point of view is that they can be used for domestic forms of repression. And that means that if you want to avoid the government having all of its powers, you might say it's better for OpenAI and Entropic and so on to keep control over most cutting edge models. But of course, if those companies have control of those cutting edge models and they can potentially break into nuclear weapons, or they can potentially have military applications which are absolutely central to the ability of the government to keep its citizens safe, then we can't really do that either. So how are you thinking about how to solve these real trade offs and perhaps these genuine dilemmas without allowing the government to have powers that could very quickly become tyrannical?
- [55:15]
Arvind Narayanan
Yeah, simply that it's not all or nothing. I think we can give the government limited power. So for instance, pre release access to these models so the government has access before the general public in order to again, for instance, find and fix vulnerabilities in critical systems. But that's very different from giving the government the power to try to carry out a mass persuasion campaign. And we've written about this, and in our view, the critical bottleneck to the government being able to do that is not the capability to generate text or images or whatever, but rather the distribution channels. So in other words, if the government were to control, for instance, social media, that's a much more dangerous thing than governments controlling AI systems when it comes to the ability for persuasion and propaganda. Because when you go back to media theory naively, back in the 50s, people used to have this hypodermic syringe model. The idea that there can be messages coming from companies or governments or any other adversarial actor that are so powerful that if people come into contact with those messages, even subliminally they will change their view. Now, we know that that's not true, but yet in the AI conversation, a lot of the discourse proceeds as if that is the case. That's not how people get persuaded. People get persuaded because they're embedded in a social community where those beliefs take root for various kinds of tribal reasons. So when we look at how AI is being used for disinformation, for instance, it's being used by those in power, including politicians, to, if you will, persuade or trick their own supporters as opposed to their adversaries, because it's in that context that it becomes very powerful. Right? So really, the problem of controlling AI disinformation and propaganda reduces to the existing problems that we've had with the quality of our epistemic channels and discourse. What do we do about the level of polarization and tribalism and those types of things? And those are hard problems. They're not easy. And yeah, on the margins, AI is going to amplify them. I don't think AI is newly creating those problems, though.
- [57:25]
Yascha Mounk
Thanks so much for listening to this episode of a good fight. In the rest of this conversation, Arvind and I talk about warfare and how artificial intelligence is already transforming that in Ukraine and in the Iran war, whether warfare will make it much harder, even for superpowers like the United States to actually keep their citizens safe. And we talk at length about public policy. What should individuals and workers fight for? How can governments respond in a sensible way to the age of artificial intelligence? Is the framework of artificial intelligence as normal technology what we need in order to avoid giving up on basic individual freedoms out of panic? Or is it ultimately soothing slogan that will stop us from taking the necessary steps for us to stay safe and have a good standard of living? To hear us debate these issues. To support the podcast, to get access to two full episodes every week, go to writing.schmonk.com listen and become a paying subscriber. That's writing.as schmonk.com listen and become A paying subscriber.
- [58:43]
Arvind Narayanan
Sa.