Loading summary
Transcript38 lines
- [00:00]
Kelly Bowen
It's being adopted quickly and then it's in a technology that's being adopted on top of a society that is just as, you know, really unstable. Just a lot of, you know, challenges around trust in institutions, polarization, economic inequality, that's pretty, you know, significant. And so introducing a technology this quickly, that's general purpose, I think, is causing
- [00:22]
Yasha Monk
the concern and now the good fight with Yasha Monk. As you know, I have been getting really interested in the topic of artificial intelligence, which, like it or not, is going to have huge impacts on the world. Now, in some of the past episodes we've looked at just how AI systems actually work. We've had Geoffrey Hinton and David Bau talk us through how to understand the technology behind artificial intelligence. We have talked about some of the really high stakes implications of AI. Nick Sorry's told us about why he's really worried about the existential risk from AI. But we haven't really had a broad conversation about the different kind of ways in which artificial intelligence is likely to transform the political world and how it is that we should think about governing AI. What kind of public policy response we should mount to this important new all purpose technology. Well, a little while ago I was at a conference where somebody I have known for a while gave a presentation about those topics that I found to be really helpful in making me think through just the different kind of buckets of impact that AI is likely to have, from straightforward stuff like the administration of elections to much bigger things like the way it'll transform our basic socioeconomic context or might change warfare in the 21st century. And so I decided to to invite Kelly Bowen on to have a conversation about all of these things together on the podcast. Kelly is the inaugural director of a Democracy Rights and Governance initiative at the Packard foundation and she has been thinking a lot about the intersection of AI and democracy. In the final part of this conversation, we think about what to do in response to the rise of AI, AI and in particular two questions. The first is if the tendency of a technology really is to become super intelligent in a way that we're unlikely to be able to control, is it realistic that laws, regulations, UN covenants are going to stop humanity from developing that technology? Or is our fate just in the hands of the gods of technology and wherever they take ChatGPT 8.0 and I also ask Kelly about how she thinks as a philanthropist about where she can be useful. How is she thinking about how the Packard foundation should be investing its money in order to help us hopefully come to good solutions on this topic. That part of the conversation is behind the paywall. So if you want to listen to that part of a conversation, if you want to stop hearing those annoying pre recorded jingle ads from people, if you want access to all full episodes of this podcast, and most importantly, if you want this intellectual enterprise to work to be able to pay its bills to bring you these two episodes a week, please go to yashamonk.substack.com and become a paying subscriber. That's yashamon.substack.com. Kelly Bourne welcome to podcast.
- [03:46]
Kelly Bowen
Thanks Yasha. It's great to be here.
- [03:48]
Yasha Monk
So you know, Kelly, we've known each other for a long time, we run into each other at all kinds of conferences. But you gave a really interesting presentation at a conference I was at recently. Really helping us think through the different kind of impacts that AI is going to have on the political world. Before we sort of dive in a little bit more detail into some of those. How should we think about this? From the kind of obvious, straightforward ways in which AI is already influencing the world to the more remote but potentially more important impactful ways in which it might transform the world 5 or 10 or 20 years from now?
- [04:25]
Kelly Bowen
Yeah, I mean, my sense is that AI is going to impact just about everything, right? I mean, this is a, it's a general purpose technology. So with any kind of technological change in the past, if it's the printing press or radio or TV or Internet, you know, you see these big social, political, economic changes as a result. But I think AI is a little bit different for a few reasons. It's a, it's a general purpose technology. So it's going to not just impact isn't like social media just impacting communications for the most part. This is going to impact health and science and national security and banking and transportation. And not only is it going to have this broad impact, but it's being deployed faster than pretty much any technology we've ever seen. So we have much less time as a society and a democracy to adapt. And then on top of that, it's being introduced.
- [05:19]
Yasha Monk
Explain that claim to me, because I think it's something that people who are thinking about AI sort of have heard a lot about, but which may not be as obvious to others. You know, there's this weird residual skepticism about AI and a lot of the population saying, oh, it's kind of useless and it hallucinates a lot and it doesn't actually work. You know, when you're saying it's being adopted faster than any other technology. What's the evidence for that?
- [05:40]
Kelly Bowen
Yeah, you're, I mean granted it's a lot of company disclosures about uptake, but you're seeing, you know, usage in the hundreds of millions. And I think what you're also seeing is that everyone is talking about it and using it. So I think you don't just need to take the company's word for it. I think it's very much in the zeitgeist and, and very clearly being adopted quickly.
- [05:59]
Yasha Monk
I'm trying to remember off the top of my head the kind of particular stat for it, but when you just look at how many millions of users even are paying for OpenAI every month now, as well as for other AI platforms, there's a question about whether or not the huge investments that these companies are putting into the technology is going to be sustained by those expenditures. But when you're just looking at, you know, the share of consumers in the United States who now actually proactively pay for some AI platform, it's just a very large number of people. And so, you know, for technology that's only really been out there in a commercialized way for what, three years now, you know, that is just actually very rapid, very rapid take up, which seems to indicate that at least those people seem to think, rightly or wrongly, that it's delivering some value for them.
- [06:50]
Kelly Bowen
Yeah, I think that's right. And I guess the last thing that I'd add though, Yasha, is it's being adopted quickly and then it's in a technology that's being adopted on top of a society that is just as, you know, really unstable. Just a lot of, you know, challenges around trust in institutions, polarization, economic inequality, that's pretty significant. And so introducing a technology this quickly, that's general purpose, I think is cause for concern.
- [07:17]
Yasha Monk
Yeah. And then one thing that I find really interesting is that when people talk about AI, I sometimes feel like it's very polarized between both the people who think it's completely useless and just hallucinate and the people who think like in three years it's going to be so intelligent that we're all going to be unemployed. But it's also polarized, I think, between people who worry about these very kind of small ball concrete things and these vast intangible things. So on the one hand, a lot of the literature on ethics and AI is about algorithmic discrimination. And when some AI system is helping to determine what the price of your influence is, might it end up discriminating against vulnerable groups in all kinds of ways. Which of course is a bad outcome, but feels like a relatively small element. It might influence the world. And on the other side of this, you have people like Nick Sores, who was on the podcast recently, who's co authored with Alieza Yudkowski this book, if anybody builds it, everybody dies. Right? So there it's sort of like, I think that the end of humanity is around the corner if we don't put a moratorium on air tomorrow. Tell us a little bit about sort of what's in between those two things. What are all the different kind of impacts that you're trying to think through that are sort of a little bit beyond these just very straightforward immediate impacts, but perhaps not quite as far as the super intelligent AI is going to enslave you and all of your children.
- [08:45]
Kelly Bowen
Yeah, yeah. Well, Yash, as you know, so I've been working in democracy for a while now. So I think about it in terms of the implications for democracy. And the way I tend to hold it in my head, because there are so many different ways that AI is going to impact democracy is to think about it as a series of, if you can imagine in your mind, like a series of concentric circles. And you start with the most sort of obvious, the sort of bullseye that everyone thinks about AI in elections. And you kind of think about the pros and cons in each of these domains. So, you know, for example, in the elections context, on the positive side, we know that election administrators in the US are really under resourced, and so they're starting to use it to, you know, locate polling places, verify mailing addresses, this kind of thing. And then, of course, there are all kinds of risks. The disinformation side of things is talked about a lot, but also phishing attacks that it enables, right? So there's like the really obvious stuff of the machinery of democracy, elections. And then you move one circle out to like government use of AI, executive, legislative, judicial, military. You know, you're seeing cities deploying it to improve bus routes, or the State Department says they, you know, they reduced time spent on FOIA requests by like 60%. Right? So there's the. The implications for the machinery of democracy, sort of elections and governance. And then you get out to what I think of as sort of the prerequisites for democracy, right? Political engagement and culture. And you move one circle out from that and you're looking at like the information ecosystem. And then you move out to like a fifth rung around the socioeconomic conditions, you know, concentrations of wealth and power labor market implications. And then you go one level out from that to the geopolitical conditions. What is the relative balance of power between authoritarian and democratic nations? And how are we engaging vis a vis China with this? And then you move to that furthest circle out, and that's the sort of systemic or existential risk circle where a lot of the biorisk and cyber risks and things like that, when you talk about the AI is going to kill us all, that's where that circle is. Right. And so I think what's confusing in this field right now is particularly when they talk about, when folks talk about AI and democracy, is that people are jumping across all of these levels, from the very sort of nitty gritty of the machinery out to the preconditions for the world to survive or democratic governments to survive within it.
- [11:22]
Yasha Monk
Yeah. So I think that's a really helpful way of just sort of organizing the different kind of spheres in which AI is going to have an impact. And obviously those spheres are interrelated. The point of this is not to say that these are somehow completely separate spheres to each other, but I think it just helps sort of with conceptual clarity to think about it in those terms. Why don't we start with a kind of narrower of those concentric circles and then move out a little bit? So when it comes to government, I guess that is one of the areas where you might think the potential positive impact is relatively higher than in some of those other areas. If AI systems really do make us more productive and they somehow are able to improve the delivery of government services and so on and so forth in these ways, that would obviously be a good thing. The skeptic in me then wants to say, number one. Well, whether or not it's able to do that depends just in general on whether it's possible to adopt this technology as seamlessly as people predict. And, you know, there's some studies which seem to claim that 95% of companies that have tried to implement AI into the processes actually end up not using it that much after a little while. That might be a question of a stage of a technology. Right. That's just a question about, you know, if the MTA uses AI in order to analyze which routes are most sensible and whether it should change where it runs some bus. Is the new roadmap actually going to be better than the old roadmap, or is it going to fail at that? And then there's these ethical concerns about things like if some government decisions are being made by this algorithm that we don't fully understand and we don't really know what goes into that decision. Should we be comfortable with that? Does that potentially lead to certain forms of discrimination? Or simply does that sort of belie the idea of government by the people and for the people? If the machine is making those decisions for us, does that somehow erode democratic agency? So how are you thinking about the kind of pros and cons of how AI is going to transform the way in which our government actually does its job on an average Wednesday?
- [13:45]
Kelly Bowen
Yeah, yeah, It's a complicated space. I mean, I think that we are clearly seeing benefits already. There was a great study by a group at Stanford that had worked with the city of San Francisco to review thousands of pages of reporting requirements that I think had come that were required by the federal government and found that something like 35% of them just weren't used, weren't reviewed. And so they used that data to then go back and basically advocate to no longer be required to submit these reports. Right. So there are clearly places where people are already finding value with the technology. I'm seeing ideas even for things like policy sandboxes in the legislative branch. So could you create an AI system that basically would allow policymakers to think through, okay, here's a environmental policy I want to consider. Let me run that through four different potential future scenarios and sort of theorize, well, how might this policy play out in the real world under each of these scenarios and how might I want to sort of tweak it or future proof it? Right. So you're seeing all kinds of interesting ideas, some of which are already showing, you know, actual results, and others are much more in nascent stages. So I do think that this is an area where there's probably more potential benefit than cost. And at a time when we really need to improve trust in the ability of democratic institutions to deliver, there are tons of risks. I mean, there are horror stories like the Michigan Midas one is always brought up where it was deployed for, like, unemployment to screen unemployment benefits for thousands of people and had, you know, an unbelievable. I want to say it was like in the 90% inaccuracy rate and people were bankrupted as a result. I mean, there are these horror stories that was in 2013. We've moved a pretty long way from there. And I think a lot of the challenges, and I'm not saying that I don't think there are problems here because I think there are very real risks, but a lot of folks are clearer eyed now about trying to build these systems Like RAG systems, these retrieval augmented generation systems where you have a very narrowly defined corpus that you're pulling from. And so the opportunity for bias is at least less than that. And then they're figuring out these questions around how do we keep humans in the loop? So that to your point about a government buy in for the people, I always love when people do sort of the Democrat like the Economist does, the democracy ratings and they talk about the threat of autocracy and of technocracy. Right. As a sort of equal and parallel threat. And so I think this question of like how do you keep humans in the loop? Is a really important one. But there are ways to design systems that are at least less biased than what we've seen so far.
- [16:42]
Yasha Monk
Yeah. And obviously I think there's a risk in trying to think if there's an AI system involved and some form of bias might result that is completely unacceptable. Where you're sort of tacitly assuming that the status quo or the hypothetical alternative is no bias at all. One obvious example of this in a different context is debates about Waymo, which according to the best studies we have so far has a much better road safety record than human drivers. And it seems likely that if we adopted much more self driving cars at the current level of technology, let alone the level we might get to in five or 10 years, we would save thousands and thousands of lives across the United States every year. Tens of thousands, hundreds of thousands of lives across the world. But the fact fact that there's been some instances where Waymo ran over a cat in San Francisco, then it becomes unacceptable. Now obviously we would want to be very, very careful that we don't increase how much discrimination results in our society from incorporating AI systems. But a good goal is zero discrimination. But that can't be sort of the precondition for when we start to adopt some AI systems if it reduces the amount of bias and discrimination in decision making about whatever state benefits we want to give out, that seems like an improvement over the status quo. It seems weird to make the standard that it has to be zero from day one. The other thing I was thinking about,
- [18:15]
Kelly Bowen
don't compare me to the Almighty, compare me to the alternative continues to have legs. Yeah. I do want to caveat though that no one is pretending or saying that these systems don't have bias. Right. Everyone at this point clearly agrees that these systems are being trained on the corpus of human knowledge, which itself sort of has bias built into it. But I think your point about the alternative is not a perfect decision making system by humans. It's, you know, years and years of wait times on decisions that then themselves might also be biased.
- [18:47]
Yasha Monk
Yeah. And that brings me to the other point. I thought it was very interesting that he said that, you know, these FOIA Freedom of Information Access requests can now be dealt with more easily. Right. The idea here, presumably is that some part of that is just going through a ton of documents and figuring out which are relevant, which are not, and some of that can be automated. And so you're able to respond to these important requests in a more timely manner, which is obviously what the law making the government transparent was supposed to achieve. But of course, it's an arms race. I'm struck by the fact that I spoke to someone who works with a large department store and they have a real problem in the legal department because suddenly a lot of the customers that used to write letters, which were obviously coming from a customer who doesn't have a lot of legal expertise and making demands that didn't seem very plausible. And so they could very easily say, all right, this is not a reasonable request. Now they're getting a glut of letters that are much more sophisticated sounding because they were written by ChatGPT or some other AI bot. And so the legal department is having huge trouble keeping up with all of those requests. Right. And presumably the same thing is going to happen in the case of the government. So on the one hand, the government is going to be better able to respond to citizen requests, which is a good thing. On the other hand, it's also going to be much easier to bring spurious lawsuits to gum up the bureaucracy for any building project with plausible sounding objections to some environmental review. And so I guess it's a little bit hard to know where this arms race is going to end up. Is the better ability to deliver certain services going to be a major factor here, or is the better ability of some well meaning citizens and some perhaps also not so well meaning special interest groups or people who just want to gum up the system going to be more powerful? Right. Which side of this arms raid is going to prevail? That's very hard to predict, I think.
- [21:02]
Kelly Bowen
Yeah. And I think what you're. I mean, we're already seeing that the notice and comment is already broken. And so there's one world in which you imagine that AI is part of the problem in breaking it, and then it's part of the solution and screening it. And you hear lot of folks in conversations about, you know, when AI agents become more dominant in the scene and then you just have a bunch of AI agents in dialogue with each other and you know, the government agents are screening the inbound and the, you know, the citizens agents are, you know, flooding them with information. There's another area, I don't know if you and I have talked about this, but of work that I do that is not so. I do work on AI and democracy. We also do work around improving the effectiveness of, of government. And I think this question about who wins the arm race is sort of predicated on the idea that the system is going to stay more or less stable as it is right now, that we're going to have notice and comment section sessions or things like that. We are also in conversations with groups to think about like how do we reimagine the future of governing institutions that might have a completely different alternative to notice and comment altogether. And so I think there's an interesting question about where are we going to see see an arms race within the existing systems and where are we going to see sort of new systems invented that will then I imagine have an arms race of their own to contend with?
- [22:25]
Yasha Monk
Yeah, that's a really good point. Yeah, the arms race dimension is going to be present in any system. But the question is also sort of like how is the system going to change to try and accommodate this technology? If we move up a little bit on this kind of ladder of concerns to the points around political culture and engagement and a little bit more broadly kind of information ecosystems, I guess this is the area where we start to worry about things like are we still going to be able to have a real conversation as citizens or is so much of a content that is produced on the Internet? Perhaps so many of your op ed articles published in newspapers, but certainly so much of what you see in social media going itself to be produced by AI systems, but it degrades our ability to speak to each other more broadly. What I worry a little bit about is a kind of age of cynicism. What happens if there's fake videos? It's not even that people I think are likely to believe these really fake videos, but the impact might be the opposite. But because you never quite know for sure what is authentic and what is not, it can just breed a kind of generally cynical attitude to the world. We say, well, I just can't really know what's true or false and what's actually what. And that I think may be quite corrosive of the basic factfulness we need for a democratic system to work in a meaningful way.
- [23:57]
Kelly Bowen
Yeah, I think that's right. I tend to think of the information ecosystem impacts, at least the negative ones and kind of three buckets. The one that I think people worry about the most. But I, I am actually not quite as concerned about is the, the persuasion piece of, you know, someone's going to flip their vote or, you know, you're going to, that this content is going to change people's minds. And, and my sense is that unfortunately we are at a place where we are so polarized that at least within the existing sort of political system, I'm sure if a new kind of pizza gate comes up, you might, you know, convince someone of something. But I'm a little bit less concerned like you, I think about the persuasion piece versus the sort of learned nihilism, the kind of liar's dividend of like nothing is true and anything is possible. The kind of Russia model of propaganda where there's just so much noise out there, no one can make sense of things. Which I think is sort of the second bucket. The third that I think we talk about less is are these questions that come up around quality erosion, just the corpus of human knowledge and what is going to happen there? There's, you know, the, obviously the copyright conversation of, you know, what are incentives going to look like to create new content in the future. But then there are other pieces that I worry about too. You know, model collapse comes up a lot. The idea of like, at what point are the models just starting to ingest their own outputs and no longer have new content to build off of? But you also get into these questions around just the modal, like how we're interacting and as agents come online, you know, Google, you, you ask a question and you get 10 responses and you get to kind of think through it. You ask Siri, or I'm afraid to even say the other name or it might start talking to me in the background. But you know, you, you say you ask a question of these systems and they have to somehow get to a single answer which really vastly oversimplifies the world and truth. So I think it's not just the persuasion, it's not just the liar's dividend and sort of nihilism. It's also just how is this going to change the overall corpus of information that we're working with and how we're interacting with knowledge.
- [26:04]
Yasha Monk
And I wonder whether the concerns about echo chambers and filter bubbles, which are quite old at this time and around which empirical evidence is, I think, quite mixed, may finally come true. So I think this idea that we end up in an echo chamber online is intuitively appealing. I think there's some evidence that this may be true to a certain extent. Perhaps when we encounter content from people we disagree with, it's often nut picked. It's often the kind of most extreme and enraging content on the other side. But it's been less clear that this is actually the case than people think. And of course, the pre existing media landscape was already one in which most likely, if you're a progressive, you might have read the Nation Native, you might have read National Review, so it's not as though everybody was just consuming this kind of politically neutral content all of the time, but you could imagine that there's very obvious tendency, at least for now, in these chatbots to want to please you, right? I mean, they are literally programmed to get positive feedback, right? They are trained to put out text that the user is likely to enjoy. And probably as these models get more mature and as competition between these different model companies intensifies, each is going to have a really big incentive to figure out ways in which these models really speak to your taste. And I already find that. But when friends of mine show me some question they put to ChatGPT on their ChatGPT or whatever, it sort of has a different tone than it does on mine because it's sort of adapted over time to what they seem to like hearing, right? And how they speak themselves in ChatGPT. And so you could imagine that if you have a very progressive worldview, these chatbots are going to say, well, of course, you're right, of course this is objectively how things are. And they're going to draw on all of the smartest writing that has exactly your point of view and denigrate that of everybody else. And you might have, on the other side, the opposite. If you're conservative, it's going to say, well, of course all these progressives are completely nuts, and I completely agree with you. Now, perhaps we could figure out a way to not do that. Perhaps the people who design these chatbots recognize the danger and say, no, we should have some common reality and it should actually go with what we think is best in some kind of way. But that seems just as problematic, that seems just as concerning, because then suddenly somebody in Silicon Valley is deciding on the political values that these AI chatbots are representing as the obvious true state of the world. And perhaps they happen to have values that I agree with, and perhaps they actually are quite close to some things that are true on contested issues. Or perhaps they have values that I find deeply troubling. Or perhaps they actually end up just on the wrong side of some factual questions. Right. So it's not even clear what we would want the right state of affairs to be here. Either way, it's something that seems quite fraught with dangers.
- [29:08]
Kelly Bowen
Yeah. And I think I would add two things to the sycophancy and the like. Who is the determiner of truth? Sort of related to the sycophancy is this ability to handle conflict. You know, a successful democracy requires the ability to engage in a sort of spirited debate. And I think the sycophancy that you're seeing with these models is a challenge to that. And then there's also these studies, and some of them, I think, have been debunked. I'm still trying to get my arms around. I think, at least theoretically it makes sense, this idea of this sort of cognitive decline associated with model use. And so they've done these sort of brain mappings where they look at, you know, this is your brain in conversation with you, Yasha, and then this is your brain on Google, and then this is your brain in interacting with the GPT. Some of these studies, I think there have been real questions about, but I think people are still poking at these questions of how does one's ability to engage in democracy develop? If you're a young person engaging with a sycophantic, agentic AI model and have, you know, very little conflict from that and aren't kind of doing your own desk research and, you know, what kind of capacity does that yield for engaging in democratic debate?
- [30:24]
Yasha Monk
Yeah, and I mean, there's also a question about sort of how will the technology develop? I'm a little skeptical about these claims for now, and perhaps that's just how I use them, but I actually find that the ability to, you know, you could go down Wikipedia rabbit holes, but you kind of always stay at a certain level of generality, whereas you can sort of dig down in a back and forth of ChatGPT. I had realized recently that I hadn't kept up as much as I would like with the impact that Giorgio Meloni's government has had on Italy itself. There's been a lot of writing on her role in the kind of international system where she's ended up being more Atlanticist than was expected. But I realized that I wasn't really up in, like, the details of a constitutional reform that she's trying to put forward. And I feel that on that kind of thing you can really be like, okay, so this proposal. But I Don't quite understand really the nature of this proposal, giving more detail. It's like, okay, so what are the pros and cons? And it feels to me in that that actually my brain is just as active as if I was trying to find relevant articles in Italian newspapers, on Wikipedia or whatever. And of course it is still something where you're building literacy, right? I mean, one of the striking things is that the main mode of interaction we have so far is that we're typing and we're reading, which is a form of literacy. But of course, just as social media started in a relatively text based form and eventually ended up with TikTok, which is mostly videos, and then every other social media platform sort of emulating that, and Instagram today is much more video based, where at the beginning it was photograph based, we're likely going to end up with something similar. With AI already it's possible to speak by voice to an AI system. You see some of your big AI companies investing a ton of money into feeds where you can just have video content fed to you over time, where you're much more passive. So I guess I'm a little skeptical about some of the studies that have been out so far about how we so far engage with AI in this research respect, but they may end up kind of being directionally right in terms of where AI technology is likely to go and what the mode of engagement of AI is likely to be for most people in most hours of the day in the future.
- [32:42]
Kelly Bowen
I think that's right. I think it really depends on how you're using it, what you're using it to do. When I'm doing research, I had dinner, was it last week with one of the guys who was one of the first 20 people at Anthropic and we were talking about, you know, how we were using AI. And he said that he takes one week a month and tries to do everything using, like he doesn't do anything without first trying to do it using AI. And, and I think you find that sometimes I'm thinking much more clearly. Like you said, I'm doing research and I'm back and forth as if I'm talking to the world's expert on some topic, going deep into sort of understanding the nuance. And then I try and get AI to write something for me and it's just a disaster. And I feel like I'm getting dumber the more that I read it. So I think it really depends. I don't think there's one answer about how it's going to impact cognitive competency.
- [33:37]
Yasha Monk
Well, and certainly you're right that I mean, one of the concerns I have, motivated in part by teaching and the way that universities are currently dealing with, with AI is that I think a lot of academics and probably teachers in high school and so on are deeply naive about what AI is able to do. I've talked to a number of people who say, well, of course, some standards assignments, perhaps AI might be able to get a B minus, but on my clever assignment, it just wouldn't be able to do it. And number one, on the standard kind of essay based assignments in the humanities, that social sciences, AI is today able to get an A or an A in just about any class at universities I've taught at in my life, which is Harvard and Johns Hopkins and other places. And number two, all of those creative assignments, of course, AI would be able to do just as well. And so I do think that there's a whole generation of students being educated at the moment that have probably never written an essay in high school because it's just too tempting to have ChatGPT do it. And they may never have written an essay in college. And since in my mind at least, the act of thinking does require writing, a lot of the time when people say they're bad at writing, harsh as that sounds, it's actually that they haven't learned how to think because it's once you try to commit a thought to paper that you realize where you've made logical leaps and where you haven't quite thought it through. And in that area, I am very worried about cognitive loss unless academics get smart about how to address it. So I, for the first time in my life, this coming term at Hopkins, I'm going to have students do a pen and paper exam. And I also allow them to use AI as much as they want to write the final paper. So on the one hand, I want them to explore the use of AI in order to produce the best product they can. I think that still requires some of their own writing and some of their own thinking, but use AI tools as much as you want. On the other hand, I'm going to sit them down in a classroom and have them answer some general questions about the themes of a course to make sure that we're still able to actually think through the material and formulate clear, coherent thoughts about it in their own voice. But I think at the moment you can probably get through college with having to do that. Very, very little.
- [35:52]
Kelly Bowen
Yeah, I'm going to be fascinated to see how schools figure this one out.
- [35:55]
Yasha Monk
Yeah. And I just think that to a remarkable degree that's sort of in denial so far. Tell me about the sort of socioeconomic conditions. You know, this is one where can
- [36:05]
Kelly Bowen
we step back really quickly because there's a piece that we missed which I think is the sort of political engagement and political culture. I mean, we talked about it a little bit in terms of like the broken. What are the mechanisms through which people engage and voice their opinions in democracy? We talked about the information which is, you know, often a one way street of people are collecting facts and sort of making decisions based on that. But the engagement piece I think is interesting here as well in that you're seeing at least four challenges. The broken system, the flooded notice and comment and feedback loops that we talked about. You hear a fair amount of concern about sort of just active silencing, doxxing, trolling, state surveillance, what that might do to the democratic sort of conversation and then some conversation around. So I think about kind of these four buckets, the broken systems, the active silencing, the sort of passive silencing, the, the kind of thin engagement that we're seeing with some of these, you know, online deliberative democracy processes where people sort of feel like they've checked their box or they're using, they're engaging through AI versus really engaging with others in the real world or they're just opting out altogether or this concern that they're going to start deferring to AI agents as those come online for more of their, their civic engagement. So it's just been interesting to think through on the challenges side, what might you be grappling with? But then also to see, just really interesting, a lot of what Google at Jigsaw now has really pivoted to focusing a lot of their work on these AI powered citizen assemblies. You're seeing different kinds of polling and sentiment analysis being done and then you're even seeing in the movement building space people using these tools to really understand in authoritarian regimes who are the sort of pillars of power or financial backing behind, you know, something happening in Georgia or who came out to, you know, protest ICE deployments in Los Angeles, and to really go deeper in understanding the, the power and the, the funding behind a lot of these movements. And so you're, you're. I think there are going to be some interesting changes in how people engage with democratic institutions and each other as we see these technologies playing out. So I didn't want to miss that because there is a world of nonprofits and civil society actors Working to kind of figure this out.
- [38:36]
Yasha Monk
That's really fascinating. And I agree it's important. What's more important is if 80% of us lose our jobs. So I'm joking. But to move up sort of one further step in this ladder, this bucket of socioeconomic conditions that seems to be the one where it's just hardest to. I think, as you've established, it's hard to predict what's going to happen in any of those buckets. But this is one where I just feel the uncertainty is so radical. On the one hand, there's these people who are saying that past technological disruptions have always led to significant job loss in some category, but people just retooled. There's the example of radiologists. A bunch of things that radiologists used to do has been automated. People thought that as a result radiologists wages would go down, that a lot of radiologists would get fired. Instead what's happened is that we just use a lot more radiology and that radiologists now actually spend a lot more of a time on high value tasks rather than relatively simple repetitive tasks that they had to do before. And this seems like on the whole a happy story. On the other hand, it's just we've never had a system that has a general intelligence that matches that of most and perhaps soon all human beings. And so the fact that in the past when you could print books, and so the people who had painstakingly copy off books line by line were no longer needed. Okay, those are skilled people. And the economy had a need for skilled people. There was a huge lack of skilled people in all kinds of areas of the economy. So we went and did something else, perhaps not really. People who were 50 years old and for them it was tragic, but the next generation was fine. Is that still true when the machine can do anything that a human can do intellectually at the same level, it's really unclear to me. So I feel like to think through these impacts on socioeconomic conditions. Are we going to live in a world of plenty where also miraculously we still need skilled human beings for all kinds of things, or are we going to have on the one hand an ability to do a lot of things through these machines, but nobody making wages and the entire kind of socioeconomic base of our economic system and our political system just sort of give way? How are you thinking about this?
- [40:52]
Kelly Bowen
Yeah, I agree. I think this is the biggest question and it's the one where there's just a huge range in terms of uncertainty. I talked to some people at the tech companies, and they say we're going to see 20% unemployment in the next couple of years as a result of these technologies. The last stats I heard from the IMF, it was like 60% of jobs in advanced economies could be affected. McKinsey. I think one of their stats was 14% of the global workforce. So the estimates are kind of all over the map. And so it's hard to plan, I think, as a result of that. But making it even harder, I think, is that no one seems to have a plan. And I just had a conversation recently with the former head of the future of work for the Newsom administration and the conversation was very much so what's the plan? And there wasn't one. I think there are a lot of scholars working on this. There are some interesting ideas on the table. I think most have moved away from universal basic income for a couple of reasons. It seems hard to pull off financially. There are these questions about the sort of dignity of work and whether people want a universal basic income, at least across the board, politically. So then folks started moving to this conversation about universal basic capital. The idea or the distinction being that you are entitled to capital if you are an owner of an asset. So if you have universal basic compute, you reap dividends from the ownership of that asset. And you can make the argument that because all of humanity for, you know, many hundreds of years has contributed to the corpus of human knowledge upon which these models are trained, that we all then should have a stake in that. And, you know, and there are precedents for this. Alaska, I don't know if you're familiar with their dividend, their oil dividend model. They pay out whatever, a few thousand dollars a year and to their citizens. And no one seems to be angry about it. Like that seems to work. Okay, so there are these ideas. Then I've seen others move to this idea of, okay, maybe redistribution isn't the way to go. And the argument is around pre distribution and we need to increase worker rights at the table in these conversations about how technologies are deployed in their domains so that they have greater bargaining power about it. I've seen other arguments coming out around sort of guaranteed employment, which seemed to have support on both sides of the aisle, where in particular communities that are hardest hit, the community would come together to agree upon the jobs that were most needed and then there would be paid reskilling versus unpaid reskilling. There are a lot of ideas being kicked around, but the basic point is nobody has a good plan and the potential impacts are enormous. I mean, every major economic disruption in history is, I'm sure you are more than familiar, has some massive political upheaval soon following. So it's concerning that no one has a plan here.
- [44:06]
Yasha Monk
Yeah. Two thoughts on this. The first is that I agree with you that there's a lot of ideas in the space and none of them seem to be particularly convincing for a host of reasons. I mean, we could go through each of them. I think that might be overdoing it. But I think there's both very significant logistical and financial problems with all of them and just a problem of meaning if people don't have jobs that give them some kind of meaning in the world. I do think that that is a big personal challenge and it can become a big political challenge. And I think sort of inventing jobs that an AI system might do just as well, but because of rules and regulations, humans are still doing it is as best a very short term solution. And in one way or another, all of these solutions seem to fall into one of those buckets. The other thing is that I think this is a really helpful concept of an AI resource curse in political science. There's a long literature on why Saudi Arabia and other countries like that are not democracies. And one of the obvious answers seems to be that you get democracies when monarchs and other people who are in charge really need an educated middle class as a revenue base. And this both gives them an incentive to invest in education. And it makes it easier for citizens to make demands because they can say, hey, you're living with our taxes. In return, we want political representation, we want a say in what happens with that money. If you have a lot of resources for a monarch or a dictator at the top, just from selling oil, you never get the kind of socioeconomic mechanisms which empower a middle class to make those political demands. And that seems to be really bad for long term economic development. Now, one way of thinking about AI, if it does end up replacing a lot of middle class jobs and leading to a much more polarized income distribution, is as a form of resource curse. It just means that people at the top of society are going to be less needful of ordinary citizens for their tax income, going to be less needful of an educated citizenry, and by the way, less needful of a military that is recruited from citizens. You could imagine that at one point this still feels a little bit sci fi, but. But it could be around the historical corner that a lot of these security needs are outsourced to drones, to robots, to Other ways in which you don't actually have to have a loyal citizenry staffing the key elements of your security forces anymore. And all of that would just be sort of a structural boon to people at the top of society. I get so many headaches every month.
- [47:03]
Kelly Bowen
It could be chronic migraine, 15 or more headache days a month, each lasting four hours or more.
- [47:08]
Botox Advertisement Voice
Botox Onobotulinum toxin a prevents headaches in adults with chronic migraine. It's not for Those who have 14 or fewer headache days a month. Prescription Botox is injected by your doctor. Effects of Botox may spread hours to weeks after injection causing serious symptoms. Alert your doctor right away as difficulty swallowing, speaking, breathing, eye problems or muscle weakness can be signs of a life threatening condition. Patients with these conditions before injection are at highest risk. Side effects may include allergic reactions, neck and injection site pain, fatigue and headache. Allergic reactions can include rash, welts, asthma symptoms and dizziness. Don't receive Botox if there's a skin infection. Tell your doctor your medical history, muscle or nerve conditions including als, Lou Gehrig's disease, Myasthenia gravis or Lambert Eaton syndrome and medications including botulinum toxins as these may increase the risk of serious side effects.
- [47:53]
Kelly Bowen
Why wait? Ask your doctor. Visit botoxchronicmigraine.com or call 1-844botox to learn more. Yeah, I think you're touching on two different pieces here and so I'd love to unpack them. I think, you know, the first was this question about dignity and even if we had, you know, universal basic capital, compute, income, whatever you want to call it, that question I feel like is more solvable. And I used to do scenario planning years ago, mostly for the intelligence community. I was in the private sector before coming into philanthropy and we would do sort of featuring and forecasting work to think about, you know, what's the world going to look like in 2050 and what might the national security implications be if, you know, Sino Russian relations evolve in some particular way. And so we did this recently for the democracy field and had to really work hard to think about what are scenarios in 2050 that could actually go well given all the different trends. AI being one amongst many, alongside declining birth rates and climate change and growing gender gaps and you know, we, we explored a lot of terrain and the way that we were able to arrive at dignity in a AI fueled world was really getting back to service in one's community and, and that was kind of the only path through that included both heavy, heavy reliance on AI and some level of dignity. So that that felt somehow, you know, you could at least theoretically get to an idea of what that would look like. The challenge on the power side that you're talking about, the sort of resource curse and the fact that we might not need a middle class anymore is I think, a much harder one at a time when we have, I think, you know, Bezos earning whatever it is, 8 million per hour. And so I think we don't quite know what to do about the floor of how do we ensure everyone is taken care of. We do have ideas about how to handle a ceiling and how to make sure that people aren't accruing so much wealth and power that is impossible for average citizens to have economic and thus political agency. But again, it's a question of political will. So I sometimes like to unpack these questions of where do we actually not know what to do and where do we know what would need to be done and need to figure out the political will to get there?
- [50:23]
Yasha Monk
That's a really helpful distinction. We could spend a lot more time talking about this subject, but I want to make sure that we at least cover one more that you teased earlier, which is the kind of geopolitical context, once again, there's so many facets and aspects to this. One is about great power competition. Already we see that the prospect of ever more powerful AI and the kind of power this was give states is leading to a competition between China and the US which might make cooperation and other dimensions more difficult. There's obviously the element of military technology. You know, how is the nature of the international system transformed? If you can mass produce drones that can attack another country and civilians in a very significant way, what happens to internal security? I mean, is it going to become much, much easier to assassinate people if you can send a tiny killer drone to kill a politician while they're giving a speech or something like that. And that's the question of the geopolitical implications of AI itself. I mean, one of the things that's really striking when you read some of the. More I don't know if we call them optimists or pessimists. Some of the people who think that AI is really going to grow incredibly powerful very, very quickly, but you very much worry about whether or not AI is going to be aligned and what's going to happen is that they imagine that this era in which OpenAI and Entropic and Google have these cutting edge artificial intelligence models that they're just building in the proverbial garage. I Mean the private companies with relatively minimal security apparatus is going to be over very, very soon, that at some point the national security state is going to have to sweep in and say this is such implications for national security that this is going to look more like the Manhattan Project in Los Alamos behind barbed wire and security clearance, rather than a bunch of private labs tinkering around. How are you thinking about these different elements of the geopolitical context of AI?
- [52:38]
Kelly Bowen
Yeah, it's a complicated space, particularly given that this is the first really major technology that has been developed in the private sector and not by government. And because there's such a strong narrative because of this sort of AI arms race with China and this concern around losing economic or military dominance, that drives a narrative that many find very compelling, that we can't regulate these technologies lest we end up at a disadvantage to an authoritarian state. That puts us at sort of a permanent setback for democracy. And, and here, you know, you've seen a lot of Track two dialogues with China. It's mostly in the safety category. You know, nobody wants anyone to be able to, you know, develop a bioweapon or something like that. But there's much less collaboration, obviously, as people are trying to figure out, you know, who's going to come out dominant here. I think what troubles me most about the conversation is that the risk of being out competed by China is both a real risk and I think is a very convenient narrative that helps serve the company's interests because you're able to use it to justify absolutely no regulation at all. And I'm certainly naive here. I don't think we want bad regulation, but I do think that we need to be able to put some guardrails in place to ensure that these technologies benefit self government and democratic rule. And I really believe that we can balance, I have to believe we can balance those. And I think there's good data. I remember so much discussion about GDPR and how that was going to completely crush tech companies in Europe. And from what I've seen, European profits, at least for Google, basically doubled between GDPR and today. So I do think that there has to be a way to put in place sensible regulations that protect democratic values and still compete in this arms race with China.
- [54:47]
Yasha Monk
So this is a great segue to the next section of the conversation, which is you have all of these concerns we've talked about. Now we have these concerns about existential risk that I've talked about extensively in the podcast with other people like Nick. Sorry. So I think we can sort of skip that for a moment. That is a huge panoply of opportunities, a huge panoply of risks and concerns. How do you think about public policy response in that regard? Is this just 10 separate questions and you need 10 completely separate answers or is there some kind of emerging set of schools about the general approach we should take to channeling and regulating this overall space?
- [55:37]
Kelly Bowen
Yeah, I do think we will need some distinct or sort of bespoke interventions in each of those concentric circles. But I think more importantly there are some foundational kind of cross cutting interventions that would be really, really helpful. And I would say there's probably 15 or 20 that are being legitimately discussed amongst informed populations. Universal basic income, trade receivable jurisdictions, things on data center placements. I don't take in our work in philanthropy, we don't take policy positions. It would be sort of ironic to be very supportive of democracy and then to come in very heavy handed with the idea that we have all the answers. I very much believe in a sort of democratic conversation around these. But that said, I think there is a real consensus at least around three kind of cross cutting needs, transparency and privacy and restrictions on government use of these technologies. And so there's a lot of conversation about the kinds of transparency that we need to be able to have democratic accountability and visibility. And I think about the transparency piece sort of along the policy stack. So if you think about data and infrastructure and how that's being built out compute and then you move over to think about okay, well the data collection, what data is the model being trained on, what kind of bias and copyright protections are in there. And then you move over to model development and then you look at the actual deployments of the models and understand, you know, how they're being used, how they're impacting people. So there's sort of transparency that's needed at every layer of the stack. A great example that I've appreciated was just in infrastructure placement, folks in Oregon were trying to get a beat on how much water Google was using. I think if I recall correctly, they were a couple of years into a drought and they had to file suit against the government to get access to data to find out that Google was using something like 30% of the water in that county. And so there's a real need for some basic transparency laws on the books. We need that information if we're going to hold companies accountable. Privacy law, we could talk a lot more about that. I like to tell my friends that, you know, the average person, they are company there's like a thousand licensed data brokers in the US that have about 1500 data points on any of us. But my favorite fact is that every second something like 1.7 megabytes of data, like enough to fill out an 800 page book are being collected on us, which I had to do a lot more research into to really understand what that could even look like, which is like creepy and weird just in and of itself. But the mere fact that companies and governments are collecting this data isn't the problem. The problem is that they're using it to make decisions and inferences. And the risk for surveillance, as we've seen in China, is pretty significant. And then the government use case restrictions. I think there's a lot of consensus that, you know, there are some places where, as we talked about before, there might be bias. We want to be careful. But then there are other areas in a democracy where you just should not allow, you know, mass surveillance, predictive policing, social scoring, that kind of thing. So there are some areas where I think there is a policy consensus, at least amongst the public, which is not to say that we will actually see policy on that, but there, but there are areas where there's general agreement.
- [59:24]
Yasha Monk
Thanks so much for listening to this episode of the Good Fight. In the rest of this conversation, Kelly and I talk about two questions. First, if somebody builds super intelligence, are we going to be able to stop the disastrous impact from that through the right regulations, laws, covenants? Or is our fate just up to wherever the technology goes? And finally, how does Kelly, in her job as a program director at the Packard foundation, think about doing good in this space? What is it that she can do and what is it that ordinary citizens, politicians and others can do to really make a difference, to hopefully maximize some of the positive impacts of AI and minimize its risks? To listen to that part of a conversation, please become a paying subscriber. Please go to yashamunk.substack.com sa.