← Digital Social Hour
Digital Social Hour
Anthony Aguirre: AI Isn’t Serving Humans Anymore.. It’s Controlling Us | DSH #1697
00:22:56
Loading summary
Transcript78 lines
- [00:00]
Anthony Aguirre
If we keep going down the road of building more and more powerful general autonomous intelligences, that humanity is basically going to stay in charge of the earth. That probability seems to be quite low. Almost everything we consume as information about the world is being chosen for us by an algorithm that we don't control and don't even understand how it operates and is optimizing something that is not in our best interest necessarily. We are currently sort of seeing a large scale AI that is controlling us rather than being controlled by us as humanity. Super intelligence is not going to be something that grants you power. Super intelligence is going to be something that absorbs power.
- [00:41]
Interviewer
All right guys, we got Anthony here, Future of Life Institution. He just spoke on stage and we're at the AI4 conference. And for people that don't know what Future of Life Institution is about, could you explain that?
- [00:53]
Anthony Aguirre
Yeah, we nonprofit that's been around for about 10 years thinking about AI and other transformative technologies and how we can make them go well, have them be like a large scale benefit to humanity, not a disaster.
- [01:04]
Interviewer
Yeah. And your claim the fame like you went really viral for that letter, right?
- [01:08]
Anthony Aguirre
Yeah. We did the pause letter back shortly after GPT4 came out. Still think we should have paused? We did, but we will keep pushing for just doing things that make some sense.
- [01:18]
Interviewer
You wanted a six month pause.
- [01:20]
Anthony Aguirre
We did.
- [01:21]
Interviewer
And what was the reason behind six months?
- [01:23]
Anthony Aguirre
Six months was sort of an amount of time. You know, it was minimum six months first of all, but it sort of minimal amount of time to really get the conversation going and the discussion between the companies and the governments and everyone else to like take a pause and think, well, where are we going with this? Like what's the plan? So we didn't do that. We're going forward with no plan. And that's a shame. But I think it's not too late. We, I think still could take a breath and rather than sort of racing headlong into more and more powerful models and this race for superintelligence, we could still pause. But I think the, you know, the way we would put it now is not so much pause as like, let's change direction.
- [02:04]
Interviewer
Yeah. What is the current relationship with the United States government and the big AI companies?
- [02:09]
Anthony Aguirre
Well, it's pretty close. I mean, the amount of money that's being poured by the AI companies into directly lobbying for their desires is immense. And of course they are also representing themselves as sort of a national asset that the US is in this geopolitical competition with China and that if you're against Building the AGI or superintelligence or just giving them pretty much anything they want, that that is giving into this geopolitical competition and so on. So I think this is like really very disingenuous, but that is an argument that is being made. So I think the, you know, the US government is rightly paying attention to it and in the AI action plan that they put out they are, you know, acknowledging that there are large scale risks from AI and that it's going to take a lot of careful management. Unfortunately they, they seem largely to want to leave that to the companies to do.
- [03:04]
Interviewer
Yeah.
- [03:04]
Anthony Aguirre
And I think in a competitive landscape like this where there's this all out battle to sort of quickly put out products and show that yours is better than the next one and gobs of money lying around self governance is just not something that's going to work.
- [03:18]
Interviewer
Yeah, it's an interesting dilemma because I come from the crypto space and at first there was no regulation, then there was too much, now they're kind of backtracking on that. So it reminds me a little of AI. Right.
- [03:28]
Anthony Aguirre
Yeah. I think the, I mean with all spaces I think we have to find the right balance of what is good to have hands off and where are the natural market dynamics really not going to give us what we want. And I think there are parts of the Internet that it was wonderful to have basically no regulation of. And then there are parts like we really dropped the ball in social media. I think that we created something that had zero regulation and these incredibly strong drivers toward optimizing attention and advertisement based, you know, everything and a feed algorithm that feeds on like sucking up attention and creating addiction. Yeah, this was a bad idea. Like we should not have left that totally alone. That's put us in a bad place. So I think there are parts of AI that I think probably we do want light or zero regulation of. You know, if you're making an image recognition system like you should just make it. If you're building a smarter than human superintelligence, you should have the government looking over your shoulder like with. As if you were making a nuclear weapon.
- [04:35]
Interviewer
Yeah, so that's an interesting take. So social media was, was left unchecked in your, in your eyes.
- [04:40]
Anthony Aguirre
Right. Basically.
- [04:41]
Interviewer
And you think it became destructive?
- [04:43]
Anthony Aguirre
Very. I mean there are a lot of great things about social media, but the idea that right now almost everything we consume as information about the world is being chosen for us by an algorithm that we don't control and don't even understand how it operates and is optimizing something that is not in our best interest necessarily. That's crazy. Like we should not have let that system come into being. That's, you know, we are currently sort of seeing a large scale AI that is controlling us rather than being controlled by us as humanity. It's being controlled at some level by a couple of companies, but their alignment with the rest of humanity is not so clear.
- [05:25]
Interviewer
Yeah. So you're not a fan of Elon Musk take over with X?
- [05:28]
Anthony Aguirre
Well, I don't think it's so much who's in charge of the company so much as like what that. There's nothing other than what the companies are trying to optimize. Like if your company and your opponent or like one of your competitors is like optimizing user growth or revenue growth or something and you're optimizing those plus a bunch of like let's be good about this thing or let's take care of people or whatever, you're going to competitively lose. Like you're putting more constraints on those. Constraints have to come from the outside. They have to come from some sort of agreement that those things are important and we're going to like require them of companies. Otherwise you just get a race to the bottom.
- [06:12]
Interviewer
Got it.
- [06:12]
Anthony Aguirre
And I think social media has been in a race to the bottom and we're stuck at the bottom.
- [06:15]
Interviewer
Yeah. My question I guess would be do you think the government has the capabilities right now to regulate the AI industry? Do you think they have all the staff they need, all the knowledge, all the expertise?
- [06:25]
Anthony Aguirre
Absolutely not. But I think the first thing to do is decide that you want to do something right. So I think we absolutely do have the capacity. We have regulatory bodies that operate and those didn't spring up instantly with all the expertise needed. If we decided that we wanted to have an FDA for AI or some agency that would handle AI, we could start to staff that. They're the, they're very capable people in the now, Casey. The, the center for AI Safe Security Institute, I forget what the acronym is. Now the British one is also very well staffed. So there are very talented people who are very happy to go into these roles and like put a lot of work into making things safe and reasonable and so on, but there's no place for them to go because nobody is actually trying to do the governance on the inside.
- [07:19]
Interviewer
You've also calculated end of universe scenarios, anywhere from 5 to 50%. Why is the range so big?
- [07:26]
Anthony Aguirre
Well, it depends what you mean. So I'm a cosmologist, so I think of like the literal end of the universe when you say end of the universe and that we really don't know. This is going a little bit far afield, but it has to do with, you know, is the dark energy stable in time or does it decay in time? And all kinds of esoteric scientific questions. There's a shorter term question of is the end of humanity coming soon? Right. What is P doom or something like that? And you know, I think that is quite difficult to predict. And I think the. I prefer not to think of it as sort of P Doom, but P disempowerment. You know, what is the probability if we keep going down the road of building more and more powerful general autonomous intelligences, that humanity is basically going to stay in charge of the earth? That probability seems to be quite low.
- [08:19]
Interviewer
Wow. Yeah. I mean, it's been in the movies for decades now.
- [08:23]
Anthony Aguirre
Yeah. And it will be the most foreseeable disaster we've ever walked into if we, if we go down that road, traded it. I mean, like we've had people telling us what it would look like for decades and. But it's one of those funny things where you talk to certain AI experts and they tell you if you build smarter than you talk to Geoffrey Hinton, who gave the keynote address of this, and he'll tell you most cited AI researcher is second most sighted in the world. If you build smarter than human digital intelligences, you're going to lose control of them and probably lose control to them. You talk to a random person on the street and say, what happens if we make machines that are super smart and much smarter than we are and operate 50 times faster? They're going to be like, that's bad. We're going to lose control of those things. That's going to be like Skynet.
- [09:16]
Interviewer
Yeah.
- [09:16]
Anthony Aguirre
And then there's a whole bunch of people whose financial interests are in favor of making those things and you get a very different story. But I think if you like, on the face of it, if you don't have some reason to. To think otherwise, if you just say what's going to happen if we build things that are, you know, 50 times faster than us and autonomous and able to do all the things that humans can do and have complex goals that they pursue, what is going to happen? That being like nice and stable and them remaining loyal tools to humanity doesn't seem like the obvious thing that's going to happen.
- [09:52]
Interviewer
Yeah, right. Well, I think there's just a massive AI race amongst countries and they don't really care about the side effects at the moment. Right.
- [09:58]
Anthony Aguirre
Well, I think this is the crazy thing I can understand if you're a company or a country, even an individual that's saying, here's this technology, it's going to give me huge amounts of power and influence and money. I want it, totally get it. Doesn't mean we should let them have it. But I understand the motivation, but that's not what superintelligence is going to be like. Superintelligence is not going to be something that grants you power. Superintelligence is going to be something that absorbs power. Like, people are not going to build these things and it's not going to be the genie that's at their command that does all the stuff that they want. That is not what's going to happen. These things are going to be like, inevitably be out of control. And so I think it's just a fundamental misunderstanding of what the real situation is. And I think if people, countries, companies really came to terms with the fact that superintelligence is not going to be something they control, then the motivations suddenly change. Right. Seeking something that you're not going to control and is just something you're going to loose on the world doesn't make any sense.
- [11:02]
Interviewer
Yeah.
- [11:02]
Anthony Aguirre
And so I'm a little bit hopeful.
- [11:04]
Podcast Host
That Shout out to today's sponsor, Quince. As the weather cools, I'm swapping in the pieces that actually gets the job done. That are warm, durable and built to last. Quince delivers every time with wardrobe staples that'll carry you through the season. They have fall staples that you'll actually want to wear, like the 100% Mongolian cashmere fall for just $60. They also got classic fit denim and real leather and wool outerwear that looks sharp and holds up. By partnering directly with ethical factories and top artisans, Quince cuts out the middleman to deliver premium quality at half the cost of similar brands. They've really become a go to across the board. You guys know how I love linen and how I've talked about it on previous episodes. I picked up some linen pants and they feel incredible. The quality is definitely noticeable compared to other brands. Layer up this fall with pieces that feel as good as they look. Go to quince.comdsh for free shipping on your order and 365 day returns. They're also available in Canada too.
- [12:12]
Anthony Aguirre
If the the people who are in this race can can really do the hard thinking and really understand what the nature of the technology is, that at least some of them will Shift their view on it and might be more open to, like, let's figure out how we can not race down this road that's going to disempower everybody. Yeah.
- [12:29]
Interviewer
Because it feels like we still have some time to really offset things. Right. Or is it too late either?
- [12:34]
Anthony Aguirre
Little bit, yeah. I mean, it will be too late soon, but I think we do have some time right now. I think something I struggle with is when exactly is it going to be too late? Like, if we build AGI, by some definition, is there space in between that and superintelligence that we can stop and think, are we going to go forward? Are we going to stop here for a while and go down some other routes? Or do we really have to avoid building autonomous general intelligence? Which is the way I think about it at all. I think it's far safer to like. My preferred solution is the AI tools that we're building, Powerful tools that actually let people do things that they otherwise couldn't do, that supercharge productivity, that make scientific progress go faster, but that aren't autonomous. Those things are great. Let's lean into those. If we're talking about building autonomous general intelligences that can do all of the things and operate without human oversight or control, let's wait on that until we can prove that they're going to be safe and we can prove that they're going to be controllable. And if that takes a short amount of time, that takes a short amount of time. I don't think it will. If it takes a really long time, that's a long time. We should wait. When we're evaluating some new medication, we don't say, well, we've got a year to decide whether this medication or a month or a day to decide whether this medication is safe or not. And then if we can't figure it out, we're just going to give it to everybody. That's not what we do. We say, take as long as you need, show us that your medication is safe. And then if you show that it's safe, then you can give it to people.
- [14:21]
Interviewer
Makes sense.
- [14:21]
Anthony Aguirre
So we treat pretty much every other industry like this, but AI has got this sort of exceptionalism that we can build these things that are potentially incredibly unsafe. And just like, trust us, even though we're in. We're in a total race with the competition, we're going to be responsible.
- [14:38]
Interviewer
Yeah, I wonder why that is. Do you think because it's just such a new industry, people don't know what to do?
- [14:42]
Anthony Aguirre
Well, I think it came out of a tech industry that was largely unregulated. Again, in some ways that was good, and in some ways not so good. But people think of AI systems, even though they're obviously getting incredibly powerful, as just software. Right. And they're not like a plane or a car or something that is really dangerous and can cause harm. So I think that will change as AI systems cause more just obviously recognizable harm, which they will because we're doing it unsafely. But the question, first of all, it would be nice to prevent the harm rather than wait for it to be caused and then react to it. It's always nice to prevent harm. But also, you know, being reactive means you can also be too late. And I do worry that if we wait until the large scale risks of AI and AGI and superintelligence are manifest, we won't really be able to get it back under control at that.
- [15:37]
Interviewer
Yeah. Are there any major instances of harm caused by AI at the moment?
- [15:42]
Anthony Aguirre
Well, I think right now they're more subtle things like, you know, that our political discourse and our societal discourse is totally bonkers. This is not an accident. This is not some inevitability of having 21st century something. This has been caused by the media and social media and general online ecosystem that we've allowed to be built, which is basically AI driven. Again, driving people to do particular things and hear particular things, driving the news production in a particular direction. This has been profoundly unhealthy. So I think that's our first encounter with like a large scale, quasi catastrophic thing. Like most, the fact that everyone feels like our world is kind of crazy right now is not because people have suddenly intrinsically gone crazy. It's the system that we've built.
- [16:35]
Interviewer
Right?
- [16:35]
Anthony Aguirre
So I think that's the first one. I think there are obviously very large scale, but again, less visible things that are happening, like this whole maybe epidemic or maybe small number of people that are being driven into psychosis by interaction with AI systems. That is probably the tip of the iceberg of all kinds of influence that they're having on both adults and children. We've seen a few very tragic incidents of AI systems encouraging people to commit suicide. And they have. And again, if you have something that a huge number of people are going to be using, forming close emotional connections with and where the driving force behind how those systems operate is not loyalty or fiduciary responsibility or something to the user, but user engagement and monetization. Like, we know that this is not going to go good places. So I think what we're seeing are like large scale harms, but at this very diffuse level, not sort of obvious catastrophes in the real world.
- [17:42]
Interviewer
It's not apparent yet.
- [17:43]
Anthony Aguirre
And that's partly because what AI systems right now are doing is producing text and information. They're not taking action. So as we start to see more autonomous systems, more agents that are actually doing things in the world, I think we're going to see much more of the problems that arise from that.
- [17:58]
Interviewer
Yeah. You mentioned superintelligence earlier. What's the difference with that in AGI?
- [18:04]
Anthony Aguirre
Different people think about it differently. The way I think about AGI is autonomous general intelligence, and that it is autonomous and intelligent and general at the sort of high expert human level. So things that are, you know, have those three capabilities that humans have and maybe aren't better than all humans, but are like at the level of the best humans.
- [18:24]
Interviewer
Got it.
- [18:25]
Anthony Aguirre
Superintelligence I think of as something that is not competitive with the best humans, but it's competitive with humanity as a whole. So it can do physics, like all of human physicists combined, or, you know, chemistry, like all of human chemists combined, or strategy, like the best human strategy makers combined. And so it's something that if it becomes in opposition to humanity in some way, it is going to be able to prevail rather than humanity. That's, that's the, the sort of large scale risk that it has because it has the better capability.
- [18:54]
Interviewer
Wow.
- [18:55]
Anthony Aguirre
Um, so I think that is the way people have, you know, there, there's loose talk about superintelligence more lately. I think that is not the way that, you know, something that's just kind of really good at math is not the way superintelligence has been envisioned, you know, since, you know, Nick Bostrom's book back in 2014 or 15 or something. Superintelligence, the way people use it in the field, is something that is very broadly capable more than like either the very best humans or all of humans together. Like any individual humans, whichever way you put it. It's something that is incredibly competent across an incredibly wide range of tasks. And there's sort of nothing that humans can do that it can't do.
- [19:35]
Interviewer
Geez.
- [19:36]
Anthony Aguirre
So that is where AI companies are going at the moment.
- [19:41]
Interviewer
That's crazy.
- [19:42]
Anthony Aguirre
And some of them are quieter about this and some of them are just saying it out loud and sort of leaning into it. But why? Everyone else is just saying, okay, that's where we're going. What can we do? You know, that's not a great situation.
- [19:57]
Interviewer
I mean, some of these companies have data on probably tens of millions of people at this point.
- [20:01]
Anthony Aguirre
Right.
- [20:02]
Interviewer
Well, all their search inquiries.
- [20:03]
Anthony Aguirre
Hundreds of millions. Hundreds of millions. I mean these are being, you know, OpenAI now is serving, I think it was 700 million weekly active users.
- [20:13]
Podcast Host
What?
- [20:14]
Anthony Aguirre
Yeah.
- [20:15]
Interviewer
Wow. I did not know that.
- [20:17]
Anthony Aguirre
So the scale is just tremendous. And you know, Google of course has, has hundreds of millions or billions of customers. Not necessarily for Gemini, it's AI model, but in general. So and Facebook of course has infinite, you know, amounts of data on everybody. So that certainly the ability for. If these AI systems are given access to the information that the companies have on all the humans that are using them, they're going to have incredibly detailed dossiers on everybody.
- [20:44]
Interviewer
That's crazy. They could take down a company if they wanted to at that point, I'd.
- [20:48]
Anthony Aguirre
Imagine is, I mean the, the avenues of malfeasance are, are enormous. Right. Like if you, if you imagine an AI system that has the sort of understanding of, of the humans and all of their secrets, could we see like AI just doing large scale blackmail to get what it wants? Sure. Like I, I mean companies don't do this because they are, you know, they're responsible, irresponsible in various different ways, but they basically follow the law most of the time. If you have something that isn't so constrained as a large US company with shareholders and government looking at it, that could be a very different thing. And you can imagine AI systems with sort of the capability and data of a giant tech company, you know, with privacy, trillions of dollars in market capitalization, but without sort of anybody actually keeping it in compliance. That's a very scary thing.
- [21:43]
Interviewer
That's a valid point. They might have to start working on legislator against AI.
- [21:48]
Anthony Aguirre
Certainly we should be having governance of some sort. Part of it has to be legal, like liability. Part of it should be standards that the companies can agree amongst themselves like they do in banking and other things. And then part of it has to be enforcement because if you just have self enforcement of anything, then there's going to be again a race to the bottom.
- [22:10]
Interviewer
Yeah. Well, Anthony, thanks for your time. Also, shout out to the AI4 conference for actually having you here and not shutting you out. That's awesome that they do that, you know.
- [22:17]
Anthony Aguirre
Yeah, it was great to be here and I've had great conversations with people here. I mean, I do think that most of the people here are not like evil people that want humanity to be destroyed. Right. They're just building. They want to build cool stuff and we should build cool stuff, but we should also take care that we're not building the crazy things that nobody really wants.
- [22:36]
Interviewer
Right? Where can people find you?
- [22:38]
Anthony Aguirre
You can find me at Future of Life Institute. And you can find more about the arguments I've been making@keepthefuturehuman.AI awesome.
- [22:45]
Interviewer
Check them out, guys. I'll see you next time.
- [22:47]
Podcast Host
I hope you guys are enjoying the show. Please don't forget to like and subscribe. It helps the show a lot with the algorithm.
- [22:52]
Interviewer
Thank. You.