B (5:28)

It's cool that you were able to combine all of those different angles in one course because I think we as students have a tendency to compartmentalize things. So you take the ethics class and no professors necessarily going to force you to think about your other classes in terms of ethics or think about bring what you're learning into your other classes, into your ethics class. So by sort of forcing these topics together, that would be a more interesting course to take than any individual course. But I'm curious, did you take turns lecturing? How did three people teach one course?

A (6:01)

Yeah, well, we spent about a year working together before we started teaching to develop a bit of a common language to decide the architecture of the course. And we're all present in every single class session. One or the other of us typically has a lead responsibility for framing a discussion in class and leading the presentation. But frequently it's all three of us participating in class wide discussions. Despite the fact that the class is relatively large, about 300 plus people. We don't do it as just a one way lecture. We do small group discussions. We have case studies that were specially written for the course. We try to make the class as engaging as possible. I mean for any, any of the listeners who are have some took philosophy classes in college. My goal was to avoid a kind of cartoon like version of a philosophy and technology class in which the philosopher says you have to understand utilitarianism and you're going to read Bentham and Mill and you have to understand deontology or rights based moral theory and you're going to read a whole bunch of animal Kant and then we'll talk about the trolley problem with autonomous vehicles. And the philosopher never really engages where the technically minded students are. So this collaboration also signaled, I think to students. Oh my gosh, the guy who teaches the intro to computer science class is teaching this class. Oh, the guy who teaches the class on justice on campus is teaching this. Oh, the guy who teaches the class on public policy. All three of them in one class. That was the kind of the, the signal we wanted to send to students, avoiding, just as you said, learn as you will, tick a bunch of boxes as you go along the way. Oh yeah, it's good to get political science, good to get computer science, but they're not integrated, they're not speaking to each other necessarily.

B (7:43)

Yeah. And somewhere towards the beginning of the book you talk about how there's this culture in Silicon Valley and therefore in Stanford of people that are very tech savvy and ambitious, either starting companies or joining startups in sort of the Steve Jobs MO and caring a lot about quote unquote disruptive change, which becomes a buzzword after a while and inevitably just falling back onto making software that just caters to many of our lowest impulses and makes a lot of money because of its ability to be like what sugar is to our taste buds, but for our minds.

A (8:22)

Yeah, that's a good way to put it.

B (8:24)

Yeah. And it's not popular to put ethics first or to really think about, as you put it in the book. You talk about optimization, Right. This is a core go for anyone who cares about technology, but very rarely do people ask what's worth optimizing for to begin with. And I think that's sort of your impetus for writing this book, Nell.

A (8:42)

Yeah, absolutely. The optimization mindset is one of the core parts of the problem that we diagnose. Optimization has a place, but as a life outlook, I think it leads to some problems. Not everything in life can be described as an optimization problem, contrary to what some technologists or computer scientists might say. And in addition, I think, just as you described Coleman, the idea that optimization is itself the valuable thing is a misunderstanding, because optimization is only as good as the thing we're optimizing for. And unless we have an independent assessment about the goal or the end or the objective function, optimization for a bad outcome makes the world worse, not better. So I think computer scientists need to pay at least as much attention to the thing they're optimizing in addition to focusing on optimization as itself the good. And that's what I believe is a misunderstanding.

B (9:39)

You gave an interesting example at the beginning of the book, and I may be getting some of the details wrong, but there was someone who created a startup to help people automatically pay their parking tickets.

A (9:50)

Automatically get out of paying their parking tickets.

B (9:52)

Sorry. Yeah, that's right. Automatically protest their parking tickets in the easiest, quickest possible way, and which just seems like an awesome idea, as someone who's gotten fat parking tickets in New York once for sitting five minutes out of an apartment I was moving into, and others that were totally deserved. I suppose all were deserved in some sense, but it's a great idea on the surface. But as it progressed, you saw some problems with it. Can you describe what happened?

A (10:19)

Just as you say. We give an example of this company called Do Not Pay, which offers this service of optimizing the grievance procedure you can use to try to get out of paying a parking ticket. And I can imagine a whole bunch of stories a founder or a technologist could tell about why this would be a valuable undertaking. But the kinds of things we heard from the particular person who started the company amounted to basically an idea of sticking it to the man. It's just somehow unfair that parking tickets exist, and if you can get out of paying them, and it better if you can do it in an automated way, why wouldn't you do that? The origin story he tells of the company is that he got a whole bunch of parking ticks as a high school kid and he was sick and tired of having to pay them. And so he found a way to just make it far easier for him to get out of paying parking tickets. And then why not bring it to other people, as we say, than in the opening pages of the book? Well, parking tickets serve a legitimate function. They source of revenue for municipalities. They avoid alternate side of the street parking problems so that there can be street cleaning, parking in a disabled spot, problem for other people who are disabled. If you take the spot, it's a way of then trying to also provide a modest incentive for people to take public transportation. So I can imagine that there are stories to tell about why this would be a good thing, but none of those stories were on offer in our experience from the person. And the idea was basically if you can get out of paying a parking ticket, you should, because who wants to deal with the hassle of parking tickets? And that just seems to us like the prototypical way of disrupting for disruption's sake, with no account of the other values that might be at stake in optimizing a grievance procedure for getting out of parking tickets.

B (12:02)

Yeah, and it's also a classic tragedy, the commons problem, because the moment parking becomes effectively free is the moment you can no longer park anywhere. Definitely in a city like New York, you will never find a single spot ever. So. And that's certainly not optimal. But I thought the even more interesting aspect of this was I can imagine this idea being sold as an effort to fight income inequality and to help poor people who don't have time and really can't not think about a hundred dollar traffic ticket. Fight it easily. And I can imagine all of that being compelling until I think I recall the person who came up with this technology said, well, deep in the future I'm looking towards a where we can just sue each other with the press of a button.

A (12:46)

Yeah, wants to disrupt lawyers, we'll have lawsuits filed at the click of a button as well. Yeah, it's the thirst for disruption run amok. Disrupting is a positive valence in Silicon Valley and disruption of old industrial goliaths that are inefficient in various ways is one thing. And developing new efficiencies, new business models, all good, count me in favor. But you also should have an account story to tell about if you succeed at disrupting what Are the other potentially good values, good social systems that exist that you might want to have an account for? We give another example in the book of this thing, which, you know, is a kind of triviality at a certain level, but is a nice illustration, I think, of Soylent, this meal replacement powder also created by some Silicon Valley engineers. And the origin story of this, I find the following sentence, an incredible sentence for someone to have said. The founder of this company said, I discovered that food is an inefficient vehicle for the body's nutritional needs. You gotta shop for stuff, you have to cook, you have to clean up afterwards. You might have to socialize with people while you're eating. And it's just a pain in the ass to have to eat food. And so the idea was create a powder that allegedly is optimized for the body's nutritional needs. You add some water, you shake it, you drink, you're done for the day. Or have it three times and all right, let's grant the guys who created this the benefit of the doubt to say they've optimized for the body's nutritional needs in a super efficient, low cost way. Job well done. It turns out that food has other values attached to it. It's a source of cultural identity for some people, it's a source of pleasure for other people to eat the stuff that tastes good. And then it's also a source of social gathering and social connection. So a world in which Soylent, it was the only thing we had to feed ourselves with, would be a terrible world. Not because what they did was bad, they optimized for one thing and neglected a range of other values. And I think that's another story of disruption in Silicon Valley. You succeed at optimizing a particular thing that unsettles the broader ecosystem in which other values are also at stake. And so to give a concrete example come up later in the book that's more significant than say, Soylent is the end to end encrypted messaging platforms that you and I and all the listeners probably use, whether it's iMessage or WhatsApp or Signal or Telegram, those are optimized for privacy. And this technical accomplishment of encrypting messages so that the government can't inspect them and not the company either, so it, completely shielded from inspection, goes all in on one value of privacy. But there are other rival values in a broader social system of personal safety and national security. And I think when you create a technology and bring it to scale for millions of people to use, you need A story to tell about why privacy is the only value worth caring about. And when the government asks Apple to give a backdoor entry into the iPhone for the San Bernardino terrorist case, or Apple decides it wants to scan some of the photos that people upload to icloud to look for child pornography and people cry foul about privacy invasions. We need a broader story to tell about how to balance safety against privacy and allowing the technologists the only say and to optimize for only one value is to neglect these other values that we care about as well.

B (16:12)

Yeah, that's where a philosophy perspective becomes crucial, because you're dealing with two things that matter and you have to, I guess, think about where the trade off is between several important things in some kind of context of a notion of the good to begin with. So I'm curious, do you feel, as a philosopher that whatever your meta ethics are helps you decide these kinds of trade offs? Whether, I don't know, do you consider yourself a consequentialist or a deontologist or neither. So, yeah.

A (16:43)

Excellent. So one just preliminary comment, which is that I just agree 100% with you that we're entering into a realm of trade offs and rival good values, safety and security, objectively good things. Privacy also objectively good. Turns out you can't maximize all of them at the same time. So we have to face trade offs. Okay, from the standpoint of the technologist, when you tell them, as I do as a philosopher, there's no uniquely correct answer. There's no truth about the way to balance safety versus privacy. There are different ways of doing it. Some of them are better and some of them are worse. That sometimes makes the technologist uneasy or uncomfortable. Wait a minute, what do you mean there's no right and wrong answer? I'm like, everything that I do is computationally tractable in some way and either works or it doesn't work. So that's where the philosopher comes in. And as you say, I could give you my answer that sort of takes from my own background, moral theory, as a consequentialist or a deontologist or a virtue ethicist, or whatever it turns out to be. But since my training is as a political philosopher and in particular as a democratic theorist, the kind of answer I prefer to offer in this case is to say whatever I would want morally is only one part, one voice, one element in a larger social system. And democratic politics or democratic institutions are themselves a procedure to referee these value trade offs. So what I want to put into the equation is to Say any single person's answer has some value within a democratic procedure which allows us to engage in deliberation and contestation with diverse preferences and values themselves and deliver a temporary outcome in the form of a regulation or a policy that if we don't like in two years or five years or 10 years, we can update through the ordinary democratic product process to change the weights that we attach to them. And that seems to me an appropriate way to confront these value trade offs. Which is why part of the story about why Big Tech failed in the book is that the CEO of Signal shouldn't be the only person who decides whether or not end to end encrypted messaging for millions of people is the only way to go. There are other voices that matter. Voices of ordinary citizens, voices of people in the political system, voices of people who care about national security or privacy, excuse me, or safety. And what I want to say is it's not a meta ethical position I take here. It's a politically ethical position to say democracy is the technology we have to referee value trade offs at a social level and we should lean into that.

B (19:17)

So on this topic of optimization, the examples you've given are. I said they're different in important ways. So parking software example, that's a software that whether or not I use it could affect me negatively by the streets are becoming dirty because the street sweepers can't do their work. There's no parking anymore because it's free. The Soylent example is importantly different in that if I choose not to drink Soylent, which I don't, although I've tried Huel, it's. It's pretty gross. I'm pretty impressed that they can do it. But I'm done after one sip. It's pretty disgusting. If I don't use it, it doesn't really harm me. So it's optimized for a narrow slice of the population that has a unique set of preferences where they don't so much care about the social component of food and then that's fine for them. And so in those cases, it seems optimization is not bad or harmful. It's probably good. But in these other cases, such as end to end encryption, this is another thing that could affect me whether or not I choose to use WhatsApp or Signal exactly, in that government may not be able to stop a terrorist bombing that could affect me or all of these things. So it seems that's really what it turns on when optimization has externalities. And you talk a lot about tech externalities in the book And I actually thought of the word texternalities.

A (20:36)

Oh, that's pretty good.

B (20:37)

I doubt I'm the first to think of that, but when I was reading.

A (20:39)

We haven't thought about that one. That's good. Feel free to steal it. Yeah, free to steal it.

B (20:43)

So some of the textualities you discuss are misinformation and extremism from YouTube and social media. Algorithms that lead you deeper and deeper down rabbit holes. And this is an interesting topic because it has implications for how we see human beings. Are human beings easily manipulated by algorithms such that I can't help but believe the next video that is showing up in my feed. Or are we smart, capable individuals with agency that are not so easily manipulated by an algorithm? It seems on the one hand, if you're a person that believes in democracy and believes in the idea that we as a citizenry are capable of making these important decisions, you'd want to believe that we're not dumb animals that are just being manipulated by YouTube algorithms and fake news and so forth. On the other hand, if you want to argue that these things really are problems, you somewhat have to undermine your vision of how autonomous and intelligent we are. So how do you reconcile that?

A (21:46)

Yeah, exactly. So let's start with that last point, which I think is so important because it does sit at the heart of what a democratic society aspires toward. A society of independent, individually self governing or self determining people who then engage in an enterprise of collective self determination together. And if it turned out that we were, as you put it, nothing more than a collection of desires, kind of, you know, subject to, to our basest desires without any capacity to reflect upon them, update them through reason and deliberation. That would be a very depressing story to tell. We don't see democratic societies of animals because presumptively they don't possess the same type of higher order rational capacities that we do as humans. So let's go back to what you said earlier, which I thought was apt, namely that some of what AI or various types of algorithmic models do is they appeal to our taste for sugar rather than our higher order desire or our higher order values and preferences. And I think a lot of what happens in social media is a version of that, as I think you were alluding to, and I don't mean to make this any more complicated than the idea that when you walk by a piece of chocolate cake, many people have the reaction, I would like to have the chocolate cake. And they also then have a second reaction, I wish that I didn't want to have the chocolate cake. And we say the autonomous person is the one who's second order desire regulates their first order desire. But presented in a certain way. We eat the chocolate cake or think of the scientists who have been hired by the fast food companies to optimize for addictive taste of Cheetos or fast food stuff we buy. We get people who are trying to scientifically engineer our basest desires for our appetites rather than our higher order capacities that might regulate those base level desires. I think social media operates at that first level much more frequently and the business model certainly seems to operate in tune with that as well. So there's nothing inevitable that it needs to be that way. We could try to change the ways in which the social media ecosystem works so that it appealed to some of our higher order desires or preferences, or tried to help us to deliberate rather than to emote, which I think is a lot of what happens also online. And the more that our main source of information, which is the online universe these days, appeals to our base emotions, the more it's like feeding the population sugar. A certain number of people are going to take it, even though they have an interest, they might say themselves they have an interest in resisting the sugar. But people are weak willed. That's another feature of humanity. No genius insight there on our part to note it.

B (24:39)

So one of the most obvious potential solutions to misinformation and the ways in which social media warps our sense of reality is to censor misinformation, to simply ban it, to have people in charge of deciding what's true and what's not in important cases. And we've seen some of that happen. I mean we've seen the most examples I call to mind instantly are Alex Jones getting banned, New York Post article about Hunter Biden being banned from Twitter. Every once in a while we've had these sort of these episodes of censorship that are not yet totally normalized in our ecosystem. But it's definitely possible to imagine in 10 years that being the case. And there are many other places in the world that where it's just, it's completely normalized. So here it also does seem, on the one hand I'm a very pro free speech guy, so I really do think it tends to make sense to allow misinformation to exist and to combat it with more speech because inevitably I tend to not trust the people deciding what's true and what's not. But on the other hand, I think it would be dishonest to say that Censorship could never possibly work, ever.

A (25:47)

Yeah, agree with you. Well, so let me, if I can just give a quick reaction or.

B (25:51)

Shoji, do you want to finish? No, no, that's it.

A (25:53)

So in agreement with you about the basic diagnosis there. And then I want to add a couple elements that seem important to put on the table. So, number one, the United States and other democratic societies properly are committed to an ideal of freedom of expression that seems to me defensible important, and we should really glad we have that constitutionally in the United States. And the chief problem you're trying to avoid there is the government being a censor, not local companies or communities who aren't subject to the First Amendment. So there's no First Amendment violation when a company takes down someone's speech. So it's not a legal problem in that respect, but it is agreeing with you here, a problem from a society that should value freedom of expression. So then the question becomes, in my mind, I think we fixate too frequently on banning or deleting speech, which I'm with you, I think should be rare. And we ought to pay more attention to the algorithmic upranking and down ranking of speech, because I think that's where most of the action actually is. So in the book, we start this chapter about social media by observing that we live in an age of superabundance of content. There's so much content now because of the Internet that we need a search algorithm to surface what we look for online. We need an algorithm that up ranks and downranks information in our news feeds. And that feature of what happens online about how we are fed content, I think is the key thing to appeal to. So if there's a lot of evidence about a particular, say, I don't know, vaccine misinformation circulating online and companies decide to downrank that rather than banning it, it allows people who still want to search for it, allows people who still want to engage with it a chance to find it, but it won't be served up to you or it won't be up ranked in because it's been engaged with. That seems to me a sensible approach and relies upon the kind of internal policies of companies rather than a government imposition from the outside. That's the kind of approach that we favor in the book and seems to me to make sense.

B (27:59)

Yeah, so that's. I'm not sure if that's exactly what people mean by shadow banning on Twitter.

A (28:05)

Shadow banning, at least as I would be, it's neither up ranked or downranked. It will never show up in your feed unless you search for it, actually.

B (28:14)

Yeah. So it's basically just down regulating someone.

A (28:18)

That's right.

B (28:19)

Right. So that's the kind of thing that could only exist on the newer Twitter. Not that new anymore. The new Twitter that is no longer just a timeline where everything is not a reverse chronology. Yeah, right. It's leaning into the fact that we're now using an algorithm. Algorithm to decide what you see. And one of the many preferences we're going to optimize for is truth value as we see it, at least for important claims. I mean, that doesn't seem crazy if you're already having an algorithm. If I already accept that Twitter has an algorithm deciding what I'm going to see based on some values like whether I'm likely to click on it, engage with it, and so forth, and it wants to not show me what it. Or down regulate vaccine misinformation in a way that doesn't make it impossible for it to show up if I care enough.

A (29:03)

You got it.

B (29:04)

Yeah. So that's in theory, that seems like it could work. Of course, there is always the problem of who is determining what's true and what's not and biases inevitably of those people. And when it comes to big tech, there's a marked left wing bias on hot button culture war issues that is going to definitely frustrate the entire cultural right. And I've noticed on Twitter that the fact that shadow banning exists at all, and I'm not sure actually how often it happens, leads to an immense amount of paranoia in people who falsely think that they have been shadow banned. Or I get messages all the time asking if someone has been shadow. Like when I haven't been tweeting for a long time, people will be like, your tweets aren't showing up. I'm like, no, I actually just haven't tweeted in a week. And that happens often enough that I've noticed. I've really noticed the degree to which people become paranoid of persecution when there is a very small amount of shadow banning going on, which is kind of a consequence worth worrying about.

A (30:09)

Agree entirely. Let me try out on you another idea. I'll put it in the form of a question to you. So a colleague of ours at Stanford, a woman named Renee Diresta, has this nice line which is that no one should think that they're guaranteed to freedom of reach. You're only guaranteed freedom of speech. And what I think she means by that is if you get deplatformed on Twitter or YouTube or Facebook, you still can go say the identical thing out on the street corner. No one has blocked your freedom of expression. What they blocked you from is the algorithmic amplification of your speech. And in part because companies aren't subject to the First Amendment in the same way that it binds government. I'm sympathetic to that orientation in the following sense. Like the paranoia that you just described is this platform has been amplifying my content to my followers, and suddenly I don't get a whole lot of engagement. Well, I must have been shadow banned or I must have been downranged. And I can see how that would be frustrating, especially if someone was making a living that was their source of income, say as a creator on YouTube or something. But as a matter of freedom of expression, you haven't been blocked from expressing yourself. What you've been blocked from is, is algorithmic amplification to lots of people. And I don't know if I really am that worried about that at the end of the day, because I like this line. Freedom of speech is not freedom of reach. There's no guarantee to algorithmically promoting what you have to say.

B (31:34)

So the thing about that strikes me as not quite right is the variable of these tech platforms being virtual monopolies. Well, I guess we can talk about whether they are in fact monopolies, but. But I mean, there's parlor if you want to, if you're too conservative for Twitter. But the moment any of these big tech giants, Facebook, YouTube, Twitch, TikTok, once they enter a lane, the network effects make it such that it's unlike really the standard model of businesses, of private businesses we've known up till now.

A (32:09)

Absolutely.

B (32:09)

And so to be banned from one of them is not to have a buffet of other options one can. Can go to. It's sort of to be kicked out of the game.

A (32:17)

Yes, agree with you there. So if you're de platformed and you no longer have access to the very thing that everyone else is on because of the network effects, is this virtual monopoly? I think that is correct. Like, still, strictly speaking, you have freedom of speech. You can go bark on the street corner if you want, but if the online expression of speech is as important as it seems to me as the offline expression of speech, then you want access to the network. Network. So one solution, I think this is where you were heading, is antitrust. In order to ensure that we have multiple different networks with different content moderation approaches themselves, and we can sort ourselves into those content moderation networks, social networks that we wish the problem to confront there is that if the network effect is going to be that strong, people will naturally gravitate over time to the place where the greatest, largest number of people happen to be in order to reach the largest number of people. Even if we broke the companies up today, we should expect to see a return to a quasi monopoly over time. And that's when I think another technical feature comes into play. We need some type of data portability or data interoperability. So just as if you or I change our cell phone carrier, we can port our number over to the new carrier. If we can find a way, suitably privacy protected, to port our, our social graph over to a different provider, then we chip away at that network effect. And that's a technical problem.

B (33:45)

What would that look like? I'm not sure I totally understand what you're getting at.

A (33:48)

So let's say you get banned from Twitch or Twitter or whatever and you think, well, yeah, for criticizing Soylent too much, just hypothetically, well, if there's no meaningful other competitor, you've got nowhere else to go. And that's the core objection that I hear you giving. And I think it's right. So in a world in which you can could, you're banned now from this one platform, you could take your social connections with you to another platform. So if you could port the people you're currently connected to on Twitter over to Gab, say instead of having to have everyone be on Gab automatically in order then to make individual connections with them, that would be a form of interoperability across platforms. That would chip away at the network effect. As I say, it'd be really important to do this in a privacy protected manner. And that's what makes this a technical problem, not so much a policy problem. And I mean, I'm not the one as the philosopher to answer exactly how difficult the technical problem is, but conceptually it seems pretty straightforward. Oh, let's promote competition in the marketplace of social media companies by allowing people to shift their entire social graph across different providers where the providers vary in terms of their content moderation strategy. And then you could have multiple big companies with big social graphs. And it doesn't feel like you've got no other option if you get deplatformed from one.

B (35:08)

So yeah, I get deplatformed for talking about Soylent and then I transfer all my Twitter followers over to TikTok or.

A (35:14)

Something and off you go.

B (35:15)

That's interesting. I wonder how feasible that would be. But yeah, I'm also not the one to answer that question, I guess another interesting topic brought up by your book was artificial intelligence and the problems we will or won't face with those. So this is a topic where it's really hard to know what the world is going to look like in 50 or 100 years. On the one hand, there's some people are worried that we're going to build something that's going to turn us all into paperclips. And other people seem totally unconcerned. And it's very difficult to know actually how hard the problem of artificial intelligence is because we already have examples where machines have been better than the best chess player for 20 years now. Whereas if you asked a machine to write like a decent song, it would sound probably terrible or even something simpler than that. There are extremely simple tasks that were still so much better at machines then.

A (36:12)

And the gap is closing. The gap is closing like we talk at the end of the book about this new thing, GPT3, a language model, a kind of frontier technology which can write songs, it can produce poems and sonnets and rap lyrics and essays and journalism. And it's a middle school kid's dream because you can generate unique essays basically as responses to whatever teacher assignment you get. And it can't be detected by a plagiarism machine in addition to other kinds of problems. But you're right, I'm with you in general, that machines haven't achieved artificial general intelligence. They're not all things considered as intelligent as humans. They're task specific intelligence chess or whatever it happens to be.

B (36:58)

Yeah, and this is one of the thoughts I just often have is how the spectrum of difficulty and ease with regard to solving problems in my head is not what it is in reality. So like for instance, we can put on a man on the moon, but we can't. X is an expression that's used so often because to testify to this exact thing, it's like we don't have a cure for the cough. I went to the doctor with a cough and he said, do you want me to give you drugs or something? I said, I don't know. No, you're the doctor, tell me. He says, well, there's nothing. Just wait till it goes away. That's it. But we can put a man on the moon and we can do all this stuff. I'm coughing the same way cavemen coughed thousands of years ago. So this is the thing about AI is what is it going to be more like putting a man on the moon or curing the common cough. And it's Hard to know. But do you have a strong intuition about how much it's worth worrying about about artificial intelligence?

A (37:53)

Yeah. I'll give you really quick three different levels of worry, and two of them should seem familiar, the third maybe less so. On the one hand, machine intelligence has certainly gotten smart enough to displace humans from certain labor market tasks. Rather than humans doing factory fulfillment and boxing, you can get machines to do it. Autonomous vehicles, if they're successful, displace thousands and thousands of people who drive for a living, living truckers in particular. So the first kind of worry is just to think about the displacement of humans from the workforce and the transition costs that will be fall on society as a whole to find ways of reskilling, upskilling, transitioning to a different type of market where machines are doing more work than humans have. That's a pretty familiar thought, I think. The second kind of thing is not any one particular replacement automated technology, but rather the slow aggregation of many of them in which we humans outsource to machines individual decision making tasks that we previously did ourselves. And what that does is slowly chip away at our own agency or use the language we were using before our own autonomy in charting our own lives. So take for example, if you thought Mark Zuckerberg's announcement of the Metaverse and a kind of world of augmented or virtual reality in which many of us will spend a whole lot of time, if we could experience in virtual reality so many pleasures or things that we couldn't experience in ordinary life, do we have a reason then to go off into augmented reality and do it? Well, our agency gets severed from the outcomes we experience, and that seems a little troubling. So the more things that machines do for us and the fewer things that we do for ourselves, the less we are self determining beings and that threatens our own agency. I think that's a little bit familiar to people. Think of, say, autonomous vehicles again. I know a whole bunch of people who say, even if we can show that autonomous vehicles are just way safer, way cheaper, way more efficient than human piloted vehicles, I still love to drive the stick shift down the coastal highway. And God damn it, I'm not going to let any autonomous vehicle do that for me. It's like I don't care about the welfare effects, I don't care what other people say. It's my freedom of my agency. I want to feel the road under my feet, the wind in my hair, whatever it turns out to be. Think of that across lots of different machine replacement tasks. So finally, the thing that I think may be less familiar is that there's a kind of precautionary principle I'd want to take here. Maybe the day in which machines become as intelligent and more intelligent than humans as a general matter is far away or never will come. But if it is possible and some type of AI can outperform humans in Wall street, in the marketplace, and basically retune the economy to whatever artificial intelligence has been optimized to do, that should seem like a crisis that we ought to prepare for ahead of time in case it were to happen. And so whatever probability you assign to it actually coming about, if it's a non zero probability, and I think that it is non zero, we ought to take a precautionary approach to it. Because I do know some people, I'm part of this thing at Stanford called the Human Centered Artificial Intelligence Institute. And I talk to some people who do AI scientists and they say, well, why'd you call it Human Centered Centered? If the machines are going to be more intelligent than humans, then we should want to be subservient to the machines. We ought to be allowing machine overlords because they're better at intelligence than we are if we get there. And that's a future that I personally wouldn't sign up for. So as I say, I take this precautionary principle approach. Let's hedge our bets that the machines might become super intelligent and then they will organize us to fulfill their super intelligent values. Volumes. I want to avoid that outcome.

B (41:49)

Yeah. And then there's always a question of just because they're intelligent won't mean that they're sentient and won't mean that they therefore have any kind of there's any reason to care whether they suffer or are happy. But I want to go back to this virtual reality thought experiment. This is a version of this is the so called experience machine, where you could just go in and have any kind of experience that's not real, but as pleasurable as it could possibly, as anything possibly could be. I've always had the intuition that if everyone were able to enter an experience machine and have a life so pleasurable, and I don't just mean eating lots of sugar and having lots of sex, and I mean the deepest meaning of the word pleasure, which could include struggle and hardships and pain and what all the wise traditions would say happiness really is, and only bad experiences that have the perfect silver linings, only bad things that are worth doing to get to some better place. I actually don't have the intuition that that would be bad if everyone were in it. I just feel like if you think it's bad, you're just not actually imagining enough what it would be like for everyone. Good.

A (43:00)

So, to my mind, the reason it would be bad is that other than the choice to hook up to this experience machine, which you would be making through your own agency, all the other stuff that you get these deeper pleasures, because I'm with you all, don't assume it's just sugar and having sex. Take any of the most profound kinds of pleasures one can have as a human, and now you're guaranteed those in perpetuity. But your own exercise of your agency would be severed from those experiences or the subjective pleasures from those experiences. And the question is, is one of the things which we want or that we get deep pleasure from is knowing that somehow our will and our agency is causally connected to the production of what we experience? And if you get it on the cheap, are you losing something essential about what it means to be human? And I think that's the concern of the experience machine.

B (43:54)

One interesting answer to that is maybe that's true. Maybe our capacity for pleasure relies on there being a connection between our choices and outcomes. And if so, that would have to be somehow priced into the experience machine. So there could be some paradoxical solution where you enter it and the only way that it actually gives you pleasure is just by being another version of real life, actually just by being another very real world in which you can.

A (44:24)

An alternative universe in which your own agency determines your experiences.

B (44:28)

Right.

A (44:28)

Yeah. I thought you were going to go a slightly different direction, which was that the experiences you have give you the experience of your agency being connected to your.

B (44:38)

I was going to say that, too.

A (44:39)

I was going to.

B (44:40)

Yeah. Perhaps it just could be an illusion of agency so powerful.

A (44:43)

But then, you know, there's a lie at the heart of the experience machine, and that's the problem.

B (44:46)

But you don't know there's a lie. They're just.

A (44:48)

Well, that's true. That's true.

B (44:49)

You only know what you're choosing, Right.

A (44:51)

Yes. This is getting very Matrix, like, blue pill, red pill, but yeah, for sure.

B (44:55)

Right. All right. Well, I think that's about the end of our time.

A (44:59)

Yeah. This has been a real pleasure, Coleman. I really appreciate the engagement with the ideas in the book. And, you know, as I hope I communicate well enough here, I just get really enthusiastic and jazzed about these value questions at the heart of technology, and they just seem so essential to me for all of us to grapple with together and I really appreciate the chance to talk about it with you.

B (45:20)

Yeah, absolutely. It's been a really fun conversation and I want to point everyone in the direction of the book. The book is called System Error Subtitle Where Big Tech Went Wrong and How We Can Reboot. We did not There's a big book and we did not cover most of what's in there. So I very much recommend that you guys pick it up up and if you have anywhere else to point my listeners to see your work, a website or Twitter handle or.

A (45:44)

Yeah, Twitter handle is robre I C H robreich. And then the course website, if people want to get a look at the syllabus and all the readings is CS182 CS182 Stanford. Edu.

B (45:56)

All right, Rob, thanks so much for coming on my show.

A (45:59)

Thank you all.

B (46:04)

If you appreciate the work I do, the best ways to support me are to subscribe directly through my website, ColemanHughes.org and to subscribe to my YouTube channel so you'll never miss my new content. As always, thanks for your support.