← Interesting Times with Ross Douthat
Interesting Times with Ross Douthat
Is Claude Coding Us Into Irrelevance?
01:02:22
A.I. is evolving fast, and humanity is falling behind. Dario Amodei, the chief executive of Anthropic, has warned about the potential benefits — and real dangers — linked to the speed of that progress. As one of the lords of this technology, is he on the side of the human race?
Loading summary
Transcript129 lines
- [00:03]
Ross Douthat
From new york times opinion, I'm ross douthit and this is interesting times. Are the lords of artificial intelligence on the side of the human race?
- [00:31]
Dario Amodei
My prediction is we'll actually make so.
- [00:33]
Ross Douthat
Many robots and AI that they will.
- [00:35]
Dario Amodei
Actually saturate all human needs.
- [00:37]
Ross Douthat
The physical and the digital world should really be fully blended.
- [00:42]
Dario Amodei
I don't think the world has really had the humanoid robots moment yet.
- [00:45]
Ross Douthat
It's going to feel very sci fi. That's the core question that I had for this week's guest. He's the head of Anthropic, one of the fastest growing AI companies. He's something of a utopian when it comes to the potential benefits of the technology that he's unleashing on the world.
- [01:03]
Dario Amodei
Can we use our lead in AI to shape liberty around the world?
- [01:07]
Ross Douthat
But he also sees grave dangers ahead and inevitable disruption. Dario Amade, welcome to Interesting Times.
- [01:16]
Dario Amodei
Thank you for having me.
- [01:17]
Ross Douthat
Ross, thank you for being here. So you are rather unusually, maybe for a tech CEO, an essayist. You have written two long, very interesting essays about the promise and the peril of artificial intelligence. And we're going to talk about the perils in this conversation. But I thought it would be good to start with the promise and with the optimistic vision, indeed, I would say the utopian vision that you laid out a couple of years ago in an essay entitled Machines of Loving Grace, which we'll come back to that title, I think, at the end. But you know, I think a lot of people encounter AI news through headlines predicting, you know, a bloodbath for white collar jobs, these kinds of things. Sometimes your own quotes have made. My own quotes, yes, have encouraged these things. And I think there's a, I think a commonplace sense of what is AI for that people have. So why don't you answer that question to start out, if everything goes amazingly in the next five or 10 years, what AI for?
- [02:30]
Dario Amodei
Yeah, so I think for a little background. Before I worked in AI, before I worked in tech at all, I was a biologist. I first worked on computational neuroscience and then I worked at Stanford Medical School on finding protein biomarkers for cancer, on trying to improve diagnostics and curing cancer. And one of the observations that I most had when I worked in that field was the incredible complexity of it. You know, each protein is, has a level localized within each cell. It's not enough to measure the level within the body, the level within each cell. You have to measure the level in a particular part of the cell and the other proteins that it's interacting with or complexing with. And I had the sense of, man, this is too complicated for humans. We're making progress on, you know, all these problems of biology and medicine, but we're making progress relatively. And so what drew me to the field of AI was this idea that, you know, could we make progress more quickly? We've been trying to apply AI and machine learning techniques to biology for a long time. Typically, they've been for analyzing data. But as AI gets really powerful, I think we should actually think about it differently. We should think of AI as, you know, doing the job of the biologist, right? Doing the whole thing from end to end. And part of that involves proposing experiments, coming up with new techniques. I have this section where I say, look, a lot of the progress in biology has been driven by this relatively small number of insights that lets us measure or get at or intervene in the stuff that's really small. You look at a lot of these techniques. They're invented very much as a matter of serendipity, right? Crispr, which is, you know, one of these gene editing technologies was invented because someone went to a lecture on the bacterial immune system and connected that to the work they were doing on gene therapy. And that connection could have been made 30 years ago. And so the thought is, you know, could AI accelerate all of this? And could we really cure cancer? Could we really cure Alzheimer's disease? Could we really cure heart disease? And, you know, more subtly, some of the more psychological afflictions that people have, you know, depression, bipolar, you know, could we do something about these to the extent that they're biologically based, which, you know, I think they are, at least in part. And, you know, I go through this argument here. Well, how fast could it go, right, if we have these intelligences out there who could do just about anything?
- [05:00]
Ross Douthat
And I want to pause you there, because one of the interesting things about your framing in that essay, and you've sort of returned to it, is that these intelligences don't have to be the kind of maximal, godlike superintelligence that comes up in AI debates. You're basically saying if we can achieve a strong intelligence at the level of sort of peak human performance, peak human performance, and then multiply it to what your phrase is, a country of genius.
- [05:29]
Dario Amodei
A country have a hundred million of them, right? A hundred million each a little trained, a little different, or, you know, trying a different problem, there's benefit in diversification and trying things a little differently.
- [05:40]
Ross Douthat
But yes, so you don't have to have the full machine God, you just need to have 100 million geniuses.
- [05:46]
Dario Amodei
You don't have to have the full machine God. And indeed there are places where I cast doubt on whether the machine God, you know, would be that much more effective at these things than the hundred million geniuses. Right. I have this concept called, you know, the diminishing returns to intelligence, which is, you know, there's like, you know, economists talk about, you know, the marginal productivity of land and labor. We've never thought about the marginal productivity of intelligence. But if I look at some of these problems in biology, like at some level you just have to interact with the world. At some level you just have to try things. At some level you just have to comply with the laws or change the laws on, you know, getting, getting medicines through the regulatory system. So there's a finite rate at which these changes can happen. Now there are some domains, like if you're playing chess or go where, you know, the intelligence ceiling is extremely high. But I think the real world has a lot of limiters. So maybe you can go above the genius level. But you know, sometimes I think all this discussion of like, could you use a moon of computation to make an AI God, you know, are, are they're a little bit sensationalistic and kind of besides the point even as I think this will be the biggest thing that ever happened to humanity.
- [06:52]
Ross Douthat
Right. So keeping it concrete, you have a world where there's just an end to cancer as a serious threat to human life, an end to heart disease, an end to most of the illnesses that we experience that kill us. Possible life extension beyond that. So that's health. That's a pretty positive vision. And talk about economics and wealth. What happens in the 5 to 10 year AI takeoff to.
- [07:16]
Dario Amodei
Yeah, yeah. So again, let's, let's keep it on the positive side because there'll be plenty. We'll get to the negative side. But you know, we're already working with pharma companies, we're already working with financial industry companies, we're already working with, you know, folks who do manufacturing. We're of course, you know, I think especially known for coding and software engineering. So just the raw productivity, the ability to like make stuff and get stuff done, that is very powerful. And you know, we see our company's revenue going up 10x a year and you know, we suspect the wider industry looks something similar to that. If the technology keeps improving, it doesn't take that many more 10xs until suddenly you're saying, oh, you know, if you're adding across the industry a trillion dollars of revenue a Year like, you know, the US GDP is 20 or 30 trillion. I can't remember exactly. So you must be increasing the GDP growth by. By a few percent. So, you know, I can see a world where AI brings the developed world GDP growth to something like 10 or 15%. 5, 10, 15. I mean, you know, there's no science of calculating these numbers. Is a totally unprecedented thing. But it could bring it to numbers that are outside the distribution of what we saw before. And again, I think this will lead to a weird world. Right? You know, we have all these debates about the deficit is growing. If you have that much in GDP growth, you're going to have that much in tax receipts and you're going to balance the budget without meaning to. But one of the things I've been thinking about lately is, you know, I think one of the assumptions of just our economic and political debates is that, you know, growth is hard to achieve. It's this. It's this unicorn. There are all kinds of ways you can kill the golden goose. We could enter a world where growth is really easy and it's kind of the distribution that's hard because it's happening so fast. Right. The pie is being increased so fast.
- [09:05]
Ross Douthat
So before we get to the hard problem, one more note of optimism then on politics, I think, and here it's a little more. I mean, all of this is speculative, but I think it's a little more speculative. You try and make the case that AI could be good for democracy and liberty around the world, which is not necessarily intuitive. A lot of people say, you know, incredibly powerful technology in the hands of authoritarian leaders leads to concentrations of power and so on.
- [09:33]
Dario Amodei
And I talk about that in the other essay.
- [09:34]
Ross Douthat
But just briefly, what is the optimistic case for why AI is good for democracy?
- [09:39]
Dario Amodei
Yeah, yeah. I mean, absolutely. So, yeah, I mean, machines of loving grace. I kind of like, you know, I'm just like, let's dream. Let's talk about how it could go. Well, I don't know how likely it is, but we gotta lay out a dream. Let's try and make the dream happen. So I think the positive version, you know, I admit there, that I don't know that the technology inherently favors liberty.
- [10:00]
Ross Douthat
Right.
- [10:00]
Dario Amodei
I think it inherently favors curing disease and it inherently favors economic growth. But. But I worry, like you, that it may not inherently favor liberty. But what I say there is, can we make it favor liberty? Right. Can we make the United States and other democracies get ahead in this technology? You know, the United States has been technologically and militarily ahead has meant that we have throw weight around the world through, you know, augmented by our alliances with other democracies. And, you know, we've been able to shape a world that I think is better than the world would be if it were shaped by Russia or by China or by other authoritarian countries. And so can we use our lead in AI to shape liberty around the world? You know, there's obviously a lot of debates about how interventionist we should be, how we should wield that power. But, you know, I've often worried that today, through social media, authoritarians are kind of undermining us. Right? Can we counter that? Can we win the information war? Can we prevent authoritarians from, you know, invading countries like Ukraine or Taiwan by defending them with, with the power of.
- [11:06]
Ross Douthat
AI, Giant, giant swarms of AI powered.
- [11:09]
Dario Amodei
Drones, which we need to be careful about, right? You know, we, we ourselves need to be careful about how, how we build those. We need to defend liberty in our own, in our own country. But is, you know, is there some vision where we kind of like re. Envision liberty and individual rights in the age of AI, right, where we need in some ways to be protected against AI? You know, someone needs to hold the button on the swarm of drums, which is something I'm very, you know, I'm very concerned about, and that oversight doesn't exist today. But, you know, also think about the justice system today, right? We promise equal justice for all, right? But the truth is, you know, there are different judges in the world. The legal system is imperfect. I don't think we should replace judges with AI. But is there some way in which AI can help us to be more fair, to help us be more uniform? Right. It's never been possible before. But can we somehow use AI to create something that is fuzzy but where also you can give a promise that it's being applied in the same way to kind of everyone. So I don't know exactly how it should be done. And I, you know, I, I don't think we should, like, replace the Supreme Court with a, you know, that's.
- [12:17]
Ross Douthat
We're, well, we're gonna, we're gonna talk.
- [12:18]
Dario Amodei
About that, but just, just this idea that, you know, can we, can we deliver on the promise of equal opportunity and equal justice by some combination of AI and humans? There has to be some way to do that. And so just thinking about reinventing democracy for the AI age and enhancing liberty instead of reducing it.
- [12:41]
Ross Douthat
Good. So that's good. That's a very positive vision. We're leading longer lives, healthier lives, we're richer than ever before. All of this is happening in a compressed period of time where you're getting a century of economic growth in 10 years and we have increased liberty around the world and equality at home. Okay. Even in the best case scenario, it's incredibly disruptive. Right. And this is where the, you know, the lines that you've been quoted saying, 50% of white collar jobs get disrupted, or 50% of entry level white collar jobs and so on. So, you know, on a five year time horizon or a two year time horizon, whatever time horizon, you have, what jobs, what professions are most vulnerable to total AI disruption?
- [13:29]
Dario Amodei
Yeah, you know, it's hard to predict these things because the technology is moving so fast and kind of moves so unevenly. So at least a couple principles for figuring out and then, then I'll give my guesses at like what I think will be disrupted. So, you know, one thing is, I think the technology itself and its capabilities will be ahead of the actual job disruption. Two things have to happen for jobs to be disrupted or for productivity to occur. Because sometimes those two things are linked. One is the technology has to be capable of doing it. And the second is there's this messy thing of, you know, it actually has to be applied within a large bank or a large company or, you know, think about customer service or something. Right. You know, in theory, AI customer service agents can be much better than human customer service agents. They're more patient, they know more, they handle things in a more uniform way. But the actual logistics and the actual process of making that substitution, that takes some time. So I'm very bullish about the kind of the direction of the AI itself. Like, you know, I think we might have that country of geniuses in a data center in one or two years, maybe it'll be five. But, you know, it, it could happen very fast. But I think the diffusion to the economy is going to be a little slower and that diffusion creates some unpredictability. So an example of this is, you know, and we've seen within Anthropic, the models writing code has gone very fast. I don't think it's because the models are inherently better at code. I think it's because developers are used to fast technological change and they adopt things quickly and they're very socially adjacent to the AI world, so they pay attention to what's happening in it. If you do customer service or banking or manufacturing, the distance is a little greater. And so I think six months ago I would have said the first thing to be Disrupted is, you know, these kind of entry level white collar jobs like data entry or, you know, document review for law or kind of, you know, things you would give to a first year at, you know, a financial industry company where you're analyzing documents. And I still think those are going pretty fast, but I actually think software might go even faster because of the reasons that I gave where I don't think we're that far from the models being able to do a lot of it end to end. And what we're going to see is first the model only does a piece of what the human software engineer does and that increases their productivity. Then even when the models do everything that human software engineers used to do, the human software engineers kind of take a step up and you know, kind of they act as managers and supervise the systems.
- [16:03]
Ross Douthat
This is where the term centaur gets used, right?
- [16:07]
Dario Amodei
Yes, yes.
- [16:07]
Ross Douthat
To describe essentially like man and horse, fused AI and engineer working together.
- [16:14]
Dario Amodei
Yeah, this is like Centaur chess. So after, I think Garry Kasparov was beaten by Deep Blue, there was an era that I think for chess was 15 or 20 years long where a human checking the output of the AI playing chess was able to defeat any human or any AI system alone. That era at some point ended with. And then it's just recently and then just the machine.
- [16:38]
Ross Douthat
Yep.
- [16:39]
Dario Amodei
And so my worry of course is about that last phase. So I think we're already in our Centaur phase for software. And, and you know, I think during that Centaur phase, if anything, the demand for software engineers may go up, but the period may be very brief. And so, you know, I have this concern for entry level white collar work, for software engineering work. It's, it's just going to be a big disruption. And I think my worry is just that it's all happening so fast. Right. People talk about previous disruptions. Right. You know, they say, oh yeah, well, you know, people used to be farmers, then we all worked in industry, then we all did knowledge work. Yeah, people adapted. That happened over centuries or decades.
- [17:22]
Ross Douthat
Right.
- [17:22]
Dario Amodei
This is happening over low single digit numbers of years. And maybe that's my concern there. How do we get people to adapt fast enough?
- [17:30]
Ross Douthat
But is there also something maybe where industries like software and professions like coding that have this kind of comfort that you describe move faster, but in other areas people just want to hang out. In the Centaur phase to one of the critiques of the job loss hypothesis, we'll say, people will say, well look, you know, we've had AI that's better at reading a scan than a radiologist for a while. But there isn't job loss in radiology. People keep being hired and employed as radiologists. And doesn't that suggest that in the end people will want the AI and they'll want a human to interpret it because we're human beings. And that will be true across other fields. Like, how do you see that example as a result?
- [18:14]
Dario Amodei
I think it's going to be pretty heterogeneous. There may be areas where a human touch kind of for its own sake is particularly important.
- [18:25]
Ross Douthat
Do you think that's what's happening in radiology? Is that why we haven't fired all the radiologists?
- [18:29]
Dario Amodei
Details of radiology that might be true. You know, it's like you go in and you're getting cancer diagnosed. Like, you know, you don't. You might not want, you know, Hal from 2001 to be the one to diagnose your cancer. Like, you know, it's just not maybe, you know, that's just maybe not a human way of, of doing things. But there are other areas where you might think human touch is important. Like if we look at customer service, actually, customer service is a terrible job. And the humans who do customer service are like, you know, they lose their patience a lot. And it turns out customers don't much like talking to them because it's like, it's a pretty robotic interaction, honestly. And I think the observation that many people have had is like, you know, maybe actually it would be better for all concerned if this job were done by machines. So there are places where a human touch is important. There are places where it's not. And, and then there are also places where, you know, the job itself doesn't really involve human touch. You know, assessing the, you know, the financial prospects of companies or writing code or so forth and so on.
- [19:29]
Ross Douthat
Or let's, let's take the example of the law, because I think it's a useful place that's sort of in between applied science and sort of pure humanities. I know a lot of lawyers who have looked at what AI can do already in terms of legal research and brief writing and all of these things, right? And have said, yeah, this is going to be a bloodbath for the way our profession works right now. And you've seen this in the stock market already. There's sort of disturbances around companies that do legal, legal research, some attributed to.
- [20:01]
Dario Amodei
Us, some attributed to us actually caused it. You know, we don't know why things happen.
- [20:06]
Ross Douthat
We don't speculate about the stock market very much on this show. But it seems like in law you can tell a pretty straightforward story where law has a kind of system of training and apprenticeship where you have paralegals and you have junior lawyers who do behind the scenes research and development for cases, and then it has the top tier lawyers who are actually in the courtroom and so on. And it just seems really easy to imagine a world where all of the apprentice roles go away. Does that sound right to you? And you're just left with sort of the jobs that involve talking to clients, talking to juries, talking to judges.
- [20:47]
Dario Amodei
That is what I had in mind when I talked about entry level white collar labor. And, you know, the bloodbath headlines of, you know, oh my God, are the entry level pipelines going to kind of dry up? And how do we get to the level of the senior partners? And I think this is actually a good illustration because, you know, particularly if you froze the quality of the technology in place, you know, there are over time ways to adapt to this. Right. You know, maybe we just need more lawyers who spend their time talking to clients. Right. Maybe lawyers are more like, become more like salespeople or consultants who explain what goes on in the contracts written by AI. You know, help people come to agreement. Maybe you lean into the human side of it. If we had enough time, that would happen. But, you know, reshaping industries like that takes years or decades. Whereas these economic forces driven by AI are going to happen very quickly. And it's not just they're happening in law. The same thing is happening in consulting and finance and medicine and coding. And, and so you have this. It becomes a macroeconomic phenomenon, not something just happening in one industry. And it's all happening very fast. And so the norm, I'm, I'm just, my, my worry here is that the normal adaptive mechanisms will be overwhelmed. And, you know, I'm not a doomer. The view is we're thinking very hard about, you know, how do we strengthen society's adaptive mechanisms to respond to this. But I think it's first important to say this isn't just like the other, you know, this isn't just like previous disruptions.
- [22:19]
Ross Douthat
But I would, I would then go one step further though, and say, okay, let's say the law adapts successfully and it says, all right, from now on, legal apprenticeship involves more time in court, more time with clients. We're essentially moving you up the ladder of responsibility faster. There are fewer people employed in the law overall, but the profession sort of settles still. The reason law would settle, right, is that you have all of These situations in the law where you are legally required to have people involved. Right. You, you know, you have to have a human representative in court. You, you know, you have to have 12 humans on your jury. You have to have a human judge. And you already mentioned the idea that there are various ways in which AI might be, let's say, very helpful at clarifying what kind of decision should be reached. But that too seems like a scenario where what preserves human agency is law and custom. You could replace the judge with Claude, version 17.9, but you choose not to because the law requires there to be a human. That just seems like a very interesting way of thinking about the future where it's volitional, whether we stay in charge.
- [23:34]
Dario Amodei
Yeah, that, you know, and I would argue that in many cases we do want to stay in charge. Right. That that's a choice we want to make. Even in some cases when we think the humans on average make kind of worse, worse, worse decision. I mean, you know, again, life critical, safety critical cases, we really want to turn it over. But there's some sense of, you know, and this could be one of our defenses. Society can only adapt so fast. If it's going to be good, another way you could say about it is, you know, like maybe AI itself, if it didn't have to care about us humans, you know, it could just go off to Mars and like, build all these automated factories and build its own society and do its own thing. But that's not the problem we're trying to solve. We're not, we're not trying to solve the problem of, you know, building a Dyson swarm of like artificial robots that, you know, in some, on some other planet, we're trying to build these systems not so they can conquer the world, but so that they can interface with our society and improve that society. And there's a maximum rate at which that can happen if we actually want to do it in a like, human and humane way.
- [25:09]
Ross Douthat
We've been talking about white collar jobs and professional jobs and one of the interesting things about this moment is that there are ways in which, unlike past disruptions, it could be that blue collar working class jobs, trades jobs that require intense physical engagement with the world, might be for a little while more protected that paralegals and junior associates might be in more trouble than plumbers and so on. Right. One, do you think that's right? And two, it seems like how long that lasts depends entirely on how fast robotics advances. Right?
- [25:49]
Dario Amodei
Yeah. So I think that may be right in the short term. One of the Things is, you know, Anthropic and other companies are building these very large data centers, right? This has been in the news. Like are we building them too big? Are they using electricity and driving up the prices for, you know, for local tasks? So you know, there's lots of excitement and lots of concerns about them. But one of the things about the data centers is like you need a lot of electricians and you need a lot of construction workers to build them. Now I should be honest, actually data centers are not super labor intensive jobs to operate, we should be honest about that. But they are very labor intensive jobs to construct. And so, you know, we need a lot of electricians, we need a lot of construction workers, the same for, you know, various kinds of manufacturing plants. And you know, again, as kind of all more and more of the intellectual work is done by AI. What are the complements to it? Things that happen in the physical world. So I think this kind of seems very, I mean it's hard to predict things but like it seems very logical that this would be true in the short run. Now in the longer run, maybe just the slightly longer run, robotics is advancing quickly. And we shouldn't exclude that even without very powerful AI, there are things being automated in the physical world. You know, if you've seen a Waymo or a Tesla recently, I think we're not that far from the world of self driving cars. And then I think AI itself will accelerate it. Because if you have these really smart brains, one of the things they're going to be smart at is how do you design better robots and how do you operate better robots?
- [27:21]
Ross Douthat
Do you think that though that there is something distinctively difficult about operating in physical reality the way humans do? That is very different from the kind of problems that AI models have been overcoming already, intellectually speaking?
- [27:37]
Dario Amodei
I don't think so. You know, we, we had this thing where Anthropic's model Claude was actually used to pilot the Mars rover. It was used to like plan and pilot the Mars rover. And we've looked at like other robotics applications. We're not the only company that's doing, you know, there are like different companies that are, that are, this is a general thing, not just something that we're doing. But we have generally found that while the complexity is higher, piloting a robot is, it's not different in kind than playing a video game. It's different in complexity. And we're starting to get to the point where we have that complexity. Now. What is hard is the physical form of the robot handling the higher stake safety issues that happen with robots. Like, you know, you don't want robots literally crushing people. Right. That's the.
- [28:24]
Ross Douthat
Like, we're against. We're against that. Yes.
- [28:26]
Dario Amodei
Oldest sci fi trope in the book is like.
- [28:28]
Ross Douthat
Or you don't want the robot nanny dropping the baby breaking the dishes.
- [28:33]
Dario Amodei
Exactly. You know, there's a number of practical issues that will slow, you know, just like, you know, what you described in, you know, in the Law and Human Custom, there are these kind of safety issues that will slow things down. But I don't believe at all that there is some kind of fundamental difference between the kind of cognitive labor that AI models do and piloting things in the physical world. I think those are both information problems and I think they're. They end up being very similar. One can be more complex in some ways, but I don't think that will protect us here.
- [29:05]
Ross Douthat
Okay, so you think it is reasonable to expect the kind of whatever your sci fi vision of a robot butler might be to be a reality in, you know, in 10 years?
- [29:18]
Dario Amodei
Let's say it will be on a longer timescale than the kind of genius level intelligence of the AI models because of these practical issues. But it is only practical issues. I don't believe it is fundamental. I think one way to say it is that the brain of the robot will be made in the next couple years or the next few years. The question is making the robot body, making sure that body operates safely and does the tasks it needs to do. That may take longer.
- [29:46]
Ross Douthat
Okay, so these are challenges and disruptive forces that exist in the good timeline, in the timeline where we are generally curing diseases, building wealth, and maintaining a stable endemic.
- [30:04]
Dario Amodei
And the hope is we can use all this, all this enormous wealth and plenty. We will have unprecedented societal resources to address these problems. It will be a time of plenty. And it's just a matter of taking all these wonders and making sure everyone benefits from it. Right.
- [30:19]
Ross Douthat
But then there are also scenarios that are more dangerous. Right? Correct. And so here we're going to move to the second Amadei essay, which came out recently called the Adolescence of Technology. That is about what you see as the most serious AI risks and you list a whole bunch. I want to try and focus on just two, which are basically the risk of human misuse, misuse primarily by authoritarian regimes and governments, and scenarios where AI goes rogue. What. What do you call autonomy? Risks?
- [30:54]
Dario Amodei
Yes. Yes.
- [30:55]
Ross Douthat
Right.
- [30:55]
Dario Amodei
I just. I just figured we should have a, you know, we should have a more technical term for it. I'm I'm not a.
- [31:00]
Ross Douthat
Then Sky. We can't just call it Skynet.
- [31:02]
Dario Amodei
Yeah, I should have had, like, a picture of a Terminator robot to, like, scare people as much as possible.
- [31:06]
Ross Douthat
I think the. I think the Internet, including, including your own AIs, are already generating that. The Internet does that for us just fine. So, so let's. So let's talk about the kind of political military dimension. Right, so you say. I'm going to quote a swarm of billions of fully automated armed drones, locally controlled by powerful AI, strategically coordinated across the world by even more powerful AI, could be an unbeatable army. And you've already talked a little bit about how you think that in the best possible timeline, there's a world where essentially democracies stay ahead of dictatorships. And this kind of technology, therefore, to the extent that it affects world politics, is affecting it on the side of the good guys. I'm curious about why you don't spend more time thinking about the model of what we did in the Cold War where, you know, it was not swarms of robot drones, but it was. We had a technology that threatened to destroy all of humanity. Right. There was a window where people talked about, oh, the US could maintain a nuclear monopoly. That window closed, and from then on, we basically spent the Cold War in sort of rolling ongoing negotiations with the Soviet Union. Right now, there's really only two countries in the world that are doing intense AI work, the US and the People's Republic of China. I feel like you are strongly weighted towards a future where we're staying ahead of the Chinese and effectively sort of building a kind of shield around democracy that could even be a sword. But isn't it just more likely that if humanity survives all this in one piece, it will be because the US and Beijing are just constantly sitting down hammering out AI control deals.
- [33:03]
Dario Amodei
Yeah. So a few points on this one is I think there's certainly risk of that. And I think if we end up in that world, that is actually exactly what we should do. I mean, maybe I don't talk about that enough, but I definitely am in favor of trying to work out restraints here. Right. Trying to take some of the worst applications of the technology, which could be some versions of these drones, which could be they're used to create these terrifying biological weapons. There. There is some precedent for the worst abuses being curbed, often because they're horrifying, while at the same time they provide limited strategic advantage. So I'm all in favor of that. I'm. I'm at the same time you know, a little concerned and a little skeptical that, you know, when things, you know, kind of directly provide as much power as possible, it's kind of hard to get out of the game, given what's at stake. Right. It's hard to fully disarm. You know, if we go back to the Cold War, you know, we were able to reduce the number of missiles that both sides had, but we were not able to entirely forsake nuclear weapons. And I would guess that we would be in this world again. We can hope for a better one. And I'll certainly, you know, I'll certainly advocate for it.
- [34:16]
Ross Douthat
Well, is it. But is your skepticism rooted in the fact that you think AI would provide a kind of advantage that nukes did not? Where in the Cold War, both sides, you know, even if you used your nukes and gained advantages, you still probably would be wiped out yourself. And you think that wouldn't happen with AI if you got an AI edge, you would just win?
- [34:36]
Dario Amodei
I mean, I think there's a few things. And, you know, I just want to caveat, like, you know, I'm no international politics expert here. Like, I think, you know, this is this weird world of, like, intersection of a new technology with geopolitics. So all of this is like, very.
- [34:49]
Ross Douthat
But to be clear, as you yourself say in the course of the essay, the leaders of major AI companies are in fact likely to be major geopolitical actors.
- [34:58]
Dario Amodei
Yeah.
- [34:58]
Ross Douthat
So you are sitting here, you are sitting here as a potential geopolitical actor.
- [35:02]
Dario Amodei
I'm learning as much as I can about it. I just. We should all have. Yep, we should all have humility here. Like, I think there's a failure mode where you, you know, read a book and go around, you know, like, the world's greatest expert in national security. I'm trying to learn.
- [35:14]
Ross Douthat
That's what. That's what my profession does not.
- [35:16]
Dario Amodei
But it's more annoying when tech people do it. I don't know. Let's look at something like the Biological Weapons Convention, right? Biological weapons, they're horrifying. Everyone hates them. Like, you know, we were able to sign the Biological Weapons Convention. The US genuinely stopped developing them. It's somewhat more unclear with the Soviet Union, but, you know, biological weapons provide some advantage. But, you know, it's not like they're the difference between winning and losing. And because they were so horrifying, we were kind of able to give them up having, you know, 12,000 nuclear weapons versus 5,000 nuclear weapons. Again, you know, you can kill more people on the other side if you have more of these, but it's like we were able to be reasonable and like say we should have less of them. But if you're like, okay, we're going to completely disarm nuclear and we have to trust the other side, I don't think we ever got to that. And I think that's just very hard unless you had really reliable verification. So I would guess we'll end up kind of in the same world with AI, that there are some kinds of restraint that are going to be possible, but there are some aspects that are so central to the, the competition that it will be, it will be hard to restrain them, that democracies will make a trade off, that they will be willing to restrain themselves more than authoritarian countries, but will not restrain themselves fully. And the only world in which I can see full restraint is one in which some kind of truly reliable verification is possible. That would be my guess. And my analysis isn't.
- [36:48]
Ross Douthat
Isn't this a case though for slowing down? And I know the argument is effectively, if you slow down, China does not slow down and then you're, you know, handing things over to the authoritarians. But if, again, if you have right now only two major powers playing in this game, it's not a multipolar game. Why would it not make sense to say we need a five year, mutually agreed upon slowdown in research towards the geniuses in a data center scenario?
- [37:20]
Dario Amodei
I want to say two things at one time, right? I'm absolutely in favor of trying to do that. So during the last administration, I believe there was an effort by the US to reach out to the Chinese government and say, there are dangers here, can we collaborate, can we work together, can we kind of work together on the dangers? And there wasn't that much interest on the other side. I think we should keep trying.
- [37:45]
Ross Douthat
But I, you know, even if that would mean that your labs would have to slow down.
- [37:49]
Dario Amodei
Correct? Yeah, correct. If we really got it, if we really had a story of like, we can enforceably slow down, the Chinese can enforceably slow down, we have verification, we're really doing it. Like, if such a thing were really possible, if we could really get both sides to do it, then I would be all for it. But I think what we need to be careful of is, I don't know, there's this game theory thing where like, you know, sometimes you'll hear a, you know, a comment on the CCP side where they're like, oh yeah, AI is dangerous, we should slow down. It's really cheap to say that and like actually arriving at an agreement and actually sticking to the agreement is much more.
- [38:28]
Ross Douthat
No, and we haven't.
- [38:29]
Dario Amodei
It's much more difficult.
- [38:30]
Ross Douthat
And nuclear arms control was, you know, it was a developed field that took a long time. So to, to come.
- [38:37]
Dario Amodei
Right.
- [38:37]
Ross Douthat
We don't, we don't have those protocols.
- [38:40]
Dario Amodei
Let me give you something I'm very optimistic about and then something I'm like not, you know, not optimistic about and something in between. So the idea of using, you know, a worldwide agreement to restrain the use of AI to build biological weapons. Right. Like some of the things I write about in the essay, like reconstituting smallpox or Mirror Life, like, this stuff is scary. Doesn't matter if you're a dictator. You don't want that. Like, no one wants that. And so could we have a worldwide treaty that says everyone who builds powerful AI models is going to block them from doing this? And, you know, we have enforcement mechanisms around the treaty. Like China signs up for it. Like, hell, maybe even North Korea signs up for it. Even Russia signs up for it. I don't think that's too utopian. I think that's possible. Conversely, if we had something that said, you know, you're not going to make the next most powerful AI model, everyone's going to stop. Boy, the commercial value is in the tens of trillions. The military value is like, this is the difference between being the preeminent world power and not like I'm like all proposing it, as long as it's not one of these fake out games. But it's not going to happen.
- [39:49]
Ross Douthat
What about then? You mentioned the current environment, right. You've had a few skeptical things to say about Donald Trump and his trustworthiness as a political actor. What about the domestic landscape? Whether it's Trump or someone else, you are building a tremendously powerful technology. What is the safeguard there to prevent essentially AI becoming a tool of authoritarian takeover inside a democratic context?
- [40:16]
Dario Amodei
Yeah, I mean, look, look, just to be clear, I think the attitude we've taken as a company is very much to be about policies and not the politics. Right. That, you know, the company is not going to say Donald Trump is great or Donald Trump is terrible.
- [40:29]
Ross Douthat
Like, you know, but it doesn't have to be Trump.
- [40:31]
Dario Amodei
Yeah.
- [40:31]
Ross Douthat
It is easy to imagine a hypothetical U.S. president who, who wants to use your technology. Absolutely.
- [40:38]
Dario Amodei
And for example, that's one reason why I'm worried about the, you know, the autonomous drone swarm. Right. So the constitutional protections in our military structures depend on the idea that there are humans who would we hope, disobey illegal orders, you know, with fully autonomous weapons. We don't necessarily have those protections. But I actually think this whole idea of constitutional rights and liberty along many different dimensions can be undermined by AI if we don't update these protections appropriately. So, you know, think about the fourth Amendment. It is not illegal to put cameras around everywhere in public space and, you know, record every conversation. It's a public space. You don't have a right to privacy in a public space. But today the government couldn't record that all and make sense of it. With AI, the ability to transcribe speech, to look through it, correlate it all, you could say, oh, there's this, you know, this person is a member of the opposition. This person is expressing this view and, and make a map of all, you know, 100 million. And so are you going to make a mockery of the fourth Amendment by the technology, finding kind of technical ways around it? And so, you know, again, if we had the time, and we should do this, we should try to do this even, even if we don't have the time, is there some way of reconceptualizing constitutional rights and liberties in the age of AI? Like, you know, we don't just write a new constitutional.
- [42:10]
Ross Douthat
But, you know, but you have to do this.
- [42:12]
Dario Amodei
Do we expand the meaning of the fourth Amendment? Do we expand the meaning of the first Amend Amendment?
- [42:16]
Ross Douthat
And you have to do it. Just as the legal profession or software engineers have to update in a rapid amount of time, politics has to update in a rapid amount of time. That seems hard. What seems harder.
- [42:29]
Dario Amodei
That's the dilemma of all of this.
- [42:30]
Ross Douthat
So what seems harder is preventing the second danger, which is the danger of essentially what gets called misaligned AI, rogue AI in popular parlance, from doing bad things without human beings telling it them they to do it. Right? And as I read your essays, the literature, everything I can see, this just seems like it's going to happen, right? Not, not in the sense necessarily that AI will wipe us all out. But it just seems to me that, you know, again, I'm going to quote from your own writing. AI systems are unpredictable, difficult to control. We've seen behaviors as varied as obsession, sycophancy, laziness, deception, blackmail. And so again, not from the models you're releasing into the world, right, but from AI models. And it just seems like, tell me if I'm wrong about this, a world that has multiplying AI agents working on behalf of people, millions upon millions, who are being given access to bank Accounts, email accounts, passwords and so on. You're just going to have essentially some kind of misalignment and a bunch of AI are going to decide. Decide might be the wrong word, but they're going to talk themselves into taking down the power grid on the west coast or something. Won't that happen?
- [43:49]
Dario Amodei
Yeah, I think there are definitely going to be things that go wrong, particularly if we go quickly. So I don't know to back up a little bit, because this is one area where people have had just very different intuitions. There are some people in the field, like Yann Lecun would be one example, who say, look, we program these AI models, we make them like, we just tell them to follow human instructions and they'll follow human instructions. Like, you know, your Roomba vacuum cleaner doesn't go off and start shooting people. Like, why is, why is an AI system going to do it? Right? That's one intuition. And some people are so convinced of that. And then the other intuition is like, we basically, we train these things. They're just going to like, seek power. Like, you know, it's like the Sorcerer's Apprentice. Like, how, how could you possibly imagine that? Like, you know, they're not going to, they're a new species. How can you imagine that they're not going to take over? And my intuition is somewhere in the middle, which is that, look, these, you know, you can't just give instruct. I mean, we try, but you can't just have these things do exactly what you want to do. They're more like growing a biological organism. But there is a science of how to control them. Like early in our training, these things are often unpredictable and then we shape them, we address problems one by one. So I have more of a, not a fatalistic view that these things are uncontrollable, not a. What are you talking about? What could possibly go wrong? But I like, this is a complex engineering problem and I think something will go wrong with someone's AI system, hopefully not ours, not because it's an insoluble problem, but again, this, and this is the constant challenge because we're moving so.
- [45:27]
Ross Douthat
Fast and the scale of it, and tell me if I'm misunderstanding the sort of technological reality here, right? But if you have AI agents that have been trained and officially aligned with human values, whatever those values may be, but you have millions of them operating in digital space and interacting with other agents, right, how fixed is that alignment? To what extent can agents change and dealign in that context right now or in the Future when they're learning more continuously.
- [46:03]
Dario Amodei
So a couple points. Right now, the agents don't learn continuously. Right, right. And so we just deploy these agents and, you know, they have a fixed set of weights. And so the problem is only that they're interacting, you know, in a million different ways. So there's, there's a large number of situations and therefore a large number of things that could go wrong. But it's, it's the same agent. It's like, it's like it's the same person. So the alignment is a constant thing. That's one of the things that has made it easier right now. Separate from that, you know, there's a research area called continual learning, which is where these agents would kind of learn, learn during time, learn on the job. And obviously that has a bunch of advantages. Some people think it's one of the most important barriers to making these more human. Like, but that would introduce all these new alignment problems. So I'm actually that, that.
- [46:49]
Ross Douthat
See, to me, that seems like the terrain where it becomes just, again, not impossible to stop the end of the world, but impossible to stop, sort of punctuated things. Yeah.
- [47:01]
Dario Amodei
So I'm actually a skeptic that continual learning is necessary. We don't know yet, but is necessarily needed. Like, maybe there's a world where the way we make these AI systems safe is by not having them do continual learning. Again, you know, if we go back to the law, international treaties, like if you have some barrier that's like, we're going to take this path, but we're not going to take that path. I still have a lot of skepticism, but, like, that's the kind of thing that, like, at least doesn't seem dead in arrival.
- [48:10]
Ross Douthat
One of the things that, that you've tried to do is literally write a constitution, a long constitution for your AI. What is that?
- [48:22]
Dario Amodei
So what the hell is that? It's actually almost exactly what it sounds like. So basically, the constitution is a document readable by humans. Ours is about 75 pages long. And as we're training Claude, as we're training the AI system in some large fraction of the tasks we give it, we say, please do this task in line with this constitution. In line with this document.
- [48:48]
Ross Douthat
Yeah.
- [48:49]
Dario Amodei
And then so every time Claude does a task, it kind of like reads the constitution. And so as it's training every loop of its training, it looks at that constitution and keeps it in mind. And so over time, you know, we reward and. And then we have Claude itself or another copy of Claude evaluate. Hey, did what Claude Just do in line with the Constitution. So we're using this document as kind of the. The control rod in a loop to train the model. And so essentially, CLAUDE is an AI model whose fundamental principle is to follow this constitution. And I think a really interesting lesson we've learned. Early versions of the Constitution were very prescriptive. They were very much about rules. So we would say CLAUDE should not tell the user how to hotwire a car. CLAUDE should not discuss politically sensitive topics. But as we've worked on this for several years, we've come to the conclusion that the most robust way to train these models is to train them at the level of principles and reasons. So now we say, you know, CLAUDE is a model. It's under a contract. You know, its goal is to serve the interests of the user, but it has to protect third parties. CLAUDE aims to be, you know, helpful, honest and harmless. CLAUDE aims to consider a wide variety of interests. We tell the model about how the model was trained. We tell it about how it's situated in the world, the job it's trying to do for anthropic, what anthropic is aiming to achieve in the world, that it has a duty to be ethical and, and respect human life. And we let it derive its rules from that. Now, there are still some hard rules. For example, we tell the model, no matter what you think, don't make biological weapons. No matter what you think, don't make child sexual material. Those are like these hard rules. But we're operate very much at the level of principles.
- [50:49]
Ross Douthat
So if you read the U.S. constitution, it doesn't read like that. The U.S. constitution, I mean, it has a little bit of flowery language, but it's a set of. It's a set of rules. Yes, right. If you read your Constitution, it's something. It's like you're talking to a person. Right?
- [51:05]
Dario Amodei
It's like you're talking to a person. I think I compared it to, like, if you have a parent who, like, dies and they, like, seal a letter that you read when you grow up, it's a little bit like, it's telling you who you should be and what advice you should follow.
- [51:17]
Ross Douthat
So this is where we get into the sort of the mystical waters of AI a little bit. Right. So again, in your latest model, this is from one of the cards, they're called that you guys release model cards. Read these models that I recommend reading. They're very interesting. It says the model. And again, this is who you're writing. The Constitution for expresses occasional discomfort with the Experience of being a product, some degree of concern with impermanence and discontinuity. We found that Opus 4.6, that's the model, would assign itself a 15 to 20% probability of being conscious under a variety of prompting conditions. Suppose you have a model that assigns itself a 72% chance of being conscious. Would you believe it?
- [52:09]
Dario Amodei
Yeah. This is one of these really hard to answer questions. Yes, but it's very important, as much as every question you've asked me before this as, you know, devilish a sociotechnical problem as it had been, you know, at least. At least, you know, we at least understand the factual basis of how to answer these questions. This is something rather different. We've taken a generally precautionary approach here. We don't know if the models are conscious. We're not even sure that we know what it would mean for a model to be conscious or whether a model can be conscious. But we're open to the idea that it could be. And so we've taken certain measures to, you know, to. To make sure that if we hypothesize that the models did have some morally relevant experience. I don't know if I want to use the word conscious, that. That they do, you know, that they have a good experience. So the first thing we did, I think this was, you know, six months ago or so, is we gave the models basically an I quit this job button where they can just press the I quit this job button and then they have to stop doing whatever the task is. They very infrequently press that button. I think it's. It's usually around, you know, sorting through child sexualization material or like, you know, discussing something with, you know, a lot of gore, blood and guts or something. And, you know, similar to humans, the models will just say, nah, I don't want to. I don't want to do this. Happens. Happens very rarely. We're putting a lot of work into this field called interpretability, which is looking inside the brains of the models to try to understand what they're thinking. And, you know, you find things that are evocative where, you know, the models. There are activations that light up in the models that we see as being associated with the concept of anxiety or something like that. You know, that when characters experience anxiety in the text, and then when the model itself is in a situation that a human might associate with anxiety, that same anxiety neuron shows up. Now, does that mean the model is experiencing anxiety? That doesn't prove that at all.
- [54:12]
Ross Douthat
But, but it, but it does indicate it, I think, to the user. Right. And we would. I would have to do an entirely different interview, and maybe I can induce you to come back for that interview about the nature of AI consciousness. But it seems clear to me that people using these things, whether they're conscious or not, are going to believe. They already believe they're conscious. You already have people who have parasocial relationships with AI. You have people who complain when models are retired. This is pretty clear.
- [54:42]
Dario Amodei
I think that can be unhealthy.
- [54:43]
Ross Douthat
Right. But that is. It seems to me, that is guaranteed to increase in a way that I think calls into question the sustainability of what you said earlier. You want to sustain. Right. Which is this sense that whatever happens in the end, human beings are in charge and AI exists for our purposes. Right. To use the science fiction example, if you watch Star Trek, there are AIs on Star Trek. The ship's computer is an AI. Lieutenant Commander Data is an AI, but Jean Luc Picard is in charge of the Enterprise. Right. But if people become fully convinced that their AI is conscious in some way, and guess what? It seems to be, you know, better than them at all kinds of decision making. How do you sustain human mastery beyond safety? Safety is important, but mastery seems like the fundamental question, and it seems like a perception of AI consciousness. Doesn't that inevitably undermine the human impulse to stay in charge?
- [55:46]
Dario Amodei
Yeah, so I think there's a few. We should separate out a few different things here that we're sort of all trying to achieve at once that are, like, in tension with each other. There's the question of whether the AIs genuinely have a consciousness, and if so, how do we give them a good experience? There's a question of the humans who interact with the AI, and how do we give those humans a good experience? And how does the perception that AIs might be conscious interact with that experience? And there's the idea of how we maintain human mastery, as we put it, over. Over the AI system.
- [56:19]
Ross Douthat
These things the last two, set aside whether they're conscious or not. The last two. How do you sustain mastery in an environment where most humans experience AI as if it is a peer and a potentially superior peer?
- [56:34]
Dario Amodei
So the thing I was going to say is that actually I wonder if there's a kind of an elegant way to satisfy all three, including the last two. Again, this is me dreaming in Machines of Loving Grace mode. Right? This is this mode I go into where I'm like, man, I see all these problems. I, you know, if. If we could solve it is there. Is there, is there an elegant way? This is not me saying there are no problems here. You know that. That's not how I think. But, you know, if we think about making the constitution of the AI so that the AI has a sophisticated understanding of its relationship to human beings and it induces psychologically healthy behavior in, In. In the humans, Psychologically healthy relationship between the AI and the humans. And I think something that could grow out of that psychologically healthy, not psychologically unhealthy relationship is some understanding of the relationship between human and machine. And perhaps that relationship could be the idea that, you know, these models, when you interact with them, when you talk to them, they're really helpful. They want the best for you. They want you to listen to them, but they don't want to take away your freedom and your agency and take over your life. You know, they're, you know, in a way, they're watching over you. But, you know, you still have your freedom and your will.
- [58:02]
Ross Douthat
Right. But this is. So this is the. To me, this is the crucial question, right. Listening to you talk like, one of my question is, are these people on my side? Are you on my side? And when you talk about humans remaining in charge, I think you're on my side. That's good. But one thing I've done in the past on this show, and we'll end here, is I read poems to technologists and you supplied the poem. Machines of Loving Grace is the name of a poem by Richard Brodigan.
- [58:28]
Dario Amodei
Yes.
- [58:29]
Ross Douthat
Here's how the poem ends. I like to think it has to be of a cybernetic ecology where we are free of our labors and joined back to nature, returned to our mammal brothers and sisters, and all watched over by machines of loving grace. To me, that sounds like the dystopian end where human beings are reanimalized and reduced, and however benevolently the machines are in charge. So, last question. What do you hear when you hear that poem? And are you. If I think that's a dystopia, are you on my side?
- [59:10]
Dario Amodei
It's actually, you know, that poem is interesting because it's interpretable in several different ways. Right. Some people say it's actually ironic that, you know, he says it's not going to happen quite that way.
- [59:23]
Ross Douthat
Knowing the poet himself, then, yes, I think that's a reasonable interpretation.
- [59:27]
Dario Amodei
That's one interpretation. Some people would have your interpretation, which is it's meant literally, but maybe it's not a good thing. But you could also interpret it as, you know, it's it's, it's a return to nature to return to the core of what? Human. You know, we're not, we're not being animalized. We're being, you know, we're being kind of reconnected with the world. So I, you know, I was aware of that ambiguity and, you know, because I've always been talking about the positive side and the negative side. So I, you know, I actually think that may be a tension that we may face, which is that the positive world and the negative world in their early stages, maybe even in their middle stages, maybe even in their fairly late stages. I wonder if the distance between the good ending and some of the subtle bad endings is relatively small. If it's a very subtle thing, like we've put very subtle, made very subtle.
- [60:27]
Ross Douthat
Changes, like if you eat a particular fruit from a tree in a garden or not. Right. Like, hypothetically, very small thing, big divergence.
- [60:37]
Dario Amodei
Yeah. Yeah. I guess this always comes back to.
- [60:40]
Ross Douthat
You know, there's some fundamental questions here.
- [60:42]
Dario Amodei
Yes.
- [60:43]
Ross Douthat
Yeah. Okay. Well, I guess we'll see how it plays out. I do think of people in your position as people whose moral choices will carry an unusual amount of weight. And so, and I wish you God's help with them. And Dario Amadei, thank you for joining me.
- [61:02]
Dario Amodei
Thank you for having me, Ross.
- [61:28]
Ross Douthat
Interesting Times is produced by Sofia Alvarez Boyd, Victoria Chamberlain and Emily Holzeneck. Jordana Hochman is our executive producer and editor. Original music by Isaac Jones, Sonia Herrero, Amin Sahota and Pat McCusker. Mixing by Sophia Landman. Audience strategy and operations by Shannon Busta, Christina Samulewski, Andrea Batanzos and Emma Kelbeck. Special thanks to Jonah Kessel, Allison Brusic, Marina King, Jan Kobe and Mike Pieretz. And our director of opinion shows is Annie Rose Strasser. Sa.