← American Thought Leaders
Loading summary
Transcript82 lines
- [00:00]
A
If I tell you, Jan, tomorrow an AI is going to replace you and you quit your job because you know it's no longer worth it, then you're obeying. And if I'm a tech executive who's selling this product, well, then you're doing me a favor.
- [00:15]
B
We've been told that one in five jobs will be replaced by AI. But when we treat this prediction as fact, are we actually seeing the future or are we manifesting it?
- [00:26]
A
The future is doesn't get predicted, it gets produced many times through prediction. But when you're able to predict what somebody's going to do, that's bad news. That's not scientific advancement. That is a symptom of authoritarianism.
- [00:39]
B
Today I sit down with Carissa Valise, an associate professor at the Institute for Ethics and AI at Oxford University and a leading digital ethics expert. Her latest book, Prophecy, questions how prediction culture, from ancient oracles to modern AI forecasts, is actually determining our future. This is American Thought Leaders and I'm Jana Kelik. Carissa Veliz, Such a pleasure to have you on American Thought Leaders.
- [01:07]
A
Thank you so much for having me, Jan.
- [01:11]
B
I've been doing a lot of thinking about utilitarianism and this way of thinking, how it's on the rise in our society and also how AI is kind of fueling in a way, that kind of thinking. This is a centerpiece in some ways of what you write about in prophecy in your new book. Explain this to me in a way.
- [01:33]
A
Prophecy is about how we think about things, how we perceive things. And part of what I'm concerned about is how this very probabilistic, very instrumental way of thinking has been exported to all kinds of spheres of life and has colonized our way of thinking. So utilitarianism is roughly the moral theory that we should maximize utility. And utility usually gets cashed out as either well being or pleasure. And the idea is that the best thing to do is whatever maximizes the good for the majority.
- [02:08]
B
Well, so the greatest good is the greatest good.
- [02:12]
A
That's true. The greatest good for the greatest number. And this way of thinking has not only been dominant in policy making context, but it has also influenced culture in a way that even the way we speak reveals this mentality. So whenever you propose that we do something, oftentimes you will get a kind of reaction of people like, well, but is it worth it? And the question supposes that there is a calculation to be made of what are the chances of success, what's the effort involved? And you sort of weigh it. I'D like to push back against that culture because I think that there are many things that are wrong with it. One thing that is wrong with it is that it's kind of complacent because it encourages you only to go for the more likely than not successes. Whereas in life it's often very important to defy the odds. But the second thing that is wrong with it, and that is the narrative and the kind of the thread that weaves all of the book together, is that it heavily depends on prediction. So if you are very confident that you can predict the consequences of an action, then maybe utilitarianism seems more attractive. But the moment you start seeing the faults of prediction and realizing just how limited our predictive abilities are, then utilitarianism doesn't look half as attractive. Because you realize we can't actually predict the consequences of our actions.
- [03:44]
B
Because predictions are, in a way, portrayed as facts very often and are used as a tool that way. And that's really, again, it's kind of obvious the moment that you say it, as I was reading that. But it's not obvious actually, when we're just kind of dealing with it in everyday life. Tell me about that.
- [04:00]
A
I didn't start off realizing that I discovered it as I wrote. And you're right, in hindsight, it looks obvious. But it's not obvious in the way we engage with predictions, in the way we hear them and in the way we pronounce them. So a prediction just grammatically sounds so similar to a fact. If I say it is going to rain, it sounds so similar as it is raining or it has rained. And yet it's a completely different thing, because a prediction in the best case scenario is an educated guess. And sometimes educated guesses can be very important. But when you look at predictions in the public sphere, you realize more often than not, predictions are not even that they are marketing. They are wishful thinking, or they are power plays in disguise. If we hear them as if they were facts, and if we just assume that that future that is being predicted is the future and we act in accordance with that, what we're actually doing is, is obeying in advance. Because very often predictions have a veiled command. So if I tell you it's going to rain tomorrow, I might also be saying, take your umbrella or postpone that meeting or something to that effect. And if you take my prediction at face value and just act as if that was true, then you're obeying. And that gets trickier when there are social predictions. If I tell you, Jan, Tomorrow an AI is going to replace you and you quit your job because it's, it's no longer worth it, then you're obeying. And if I'm a tech executive who's selling this product, well, then you're doing me a favor.
- [05:35]
B
Talk about this relationship between tech executives and government which has developed and how this idea of prediction, this kind of oracle of Delphi nature of AI, and by extension the tech moguls, so to speak, or the technocratic class, if you will tell me about how that is impacting our society.
- [06:02]
A
The first thing to note is that these kinds of relationships are not new. When you look at the history of prediction, which, by the way, is a
- [06:08]
B
fascinating story, but your first few chapters, I mean, yeah, absolutely, I won't give it away, but please continue.
- [06:16]
A
Yeah, yeah. They're very historical because as I fell into this rabbit hole, I discovered all kinds of, of interesting facts and patterns. And one of the patterns you see is that when there are leaders, they usually attach themselves to some kind of prophet. In the past, it might been an oracle or a seer of different kinds. Then they became astrologers, and then that figure morphed. But there's always somebody who's talking about the future next to the leaders. Now it's computer scientists and tech executives who are mostly talking about the future and advising governments. And governments are asking for their advice. And so this is a common pattern. What is new and quite concerning is that it's not only that governments are taking advice from these people who have very high financial interest, but it's also that they are depending on them for things as important as getting access to the citizenry. If people don't read the news anymore, if they don't watch TV but they're on social media, then suddenly politicians need these tech executives to reach their audience. But even more concerning than that, governments are more and more depending on tech companies for infrastructure, for data servers and for even Internet cables, which many of which are now private. And so there is this relationship of cooperation but also dependence that is quite
- [07:44]
B
tricky for democracy, but also, you know, needing to compute vast amounts of data with the idea that you will be able to predict, that you'll be able to assess, come up with credible assertions. Right. This is, this is a fascinating intersection for me because I've been thinking about the totalitarian mind a lot over the last years. Right. And so there's this idea when I look at AI for the people that have that, let's say, predisposition to thinking I can control things into success, right? Like a society or whatever, they're like, well, we just didn't have enough information before. We weren't able to harness all that information. But now with AI, with the amount of compute, when we develop these huge data centers and all that, and finally we'll have the ability to figure everything out and control it just right and create that perfect society finally. What do you think?
- [08:42]
A
There's so much to say. I think we could spend the whole hour just talking, talking about this one question. One thing to bear in mind is that prediction is a business. It's always been a business. When I thought about the Oracle of Delphi, I imagined this very spiritual sanctuary, and it wasn't. It was actually a really profitable kind of setting in which the Oracle was put there to put Delphi on the map and to attract people who had to stay there. And suddenly the taverns and the restaurants and the inns were thriving. And there were not Olympic Games, games that were second in importance to the Olympic one. So there were Delphic Games and so on and so forth. And so this is a business. And part of how you create more business for yourself is you diagnose a problem and then you offer the solution. So ancient magicians would tell you what the future holds and then sell you the services to influence that future. And just like if you go into a pub in the corner today, you. You will be sold chips because that will make you thirsty and you'll buy more beer. And so one of the phenomenons we're seeing is that we are collecting all this data in the promise that the more data the better, which we'll come back to that, but it's not always the case. And then you have so much data that you don't know what to do with it as an institution. But don't worry, because there's a tech company that sells you products to analyze all that data that you don't know what to do with, and so they create their own business that way. Now, let's go back to this point about is more data always better? And the answer is absolutely not. Because often when we are searching for something in the data, it's like searching for a needle in the haystack and making the haystar bigger is not helpful. Data is only helpful if it's helpful data, if it's relevant data.
- [10:36]
B
Garbage in, garbage out.
- [10:37]
A
Yes, yes, but not only that, even when the data is accurate, if it's not relevant to you, it's not exactly garbage, but it's not going to help you. It's just going to make things harder, actually. But the most important thing is the situations in which you cannot collect the data that you would need to collect to get at the right conclusion, because it doesn't exist. Why doesn't it exist? Well, because we never collect the data of counterfactuals. So if I give a loan to someone, I can track how that person does, but if I don't give a loan to someone else, I will never be able to figure out how well would that person fared had I given the loan. I didn't give the loan.
- [11:18]
B
You're missing a whole big piece of the data set that would be essential, right?
- [11:21]
A
Always, no matter what. Because in reality you can only choose one thing. And we see this phenomenon in all kinds of ways. But in people who die, you never get the data of people who die because they're dead. And so data has this tendency to skew things even when it's precise.
- [11:41]
B
What do you think about the fact that now with these bigger data sets and more sensors and more cameras throughout society watching what people are doing, and more sensors on our fingers and our watches and so forth, that we actually aren't we getting a more complete picture of the, you know, the way our circadian rhythms work, the way our, you know, how much fat we have in our bodies, how we're in the interactions we have with others. We do seem to have more, better data, right?
- [12:17]
A
We have more data. It's not always better. So not long ago I was reading about a person who used different watches, smartwatches, to track their sleep and other metrics and how much diverge from one another. So something important is that in most of these cases, these sensors are produced, designed, fabricated, implemented by companies. And companies want profit and profit and truth or, or precision or science don't always align well. And so this is not a scientific environment in which we are trying to maximize accuracy. If it creates a profit, that's good enough. So, for example, I was listening to an incredible athlete not long ago, and one of the things that she said that struck me as interesting is that she thinks that part of her competitive advantage is that sure, she collects all this data like many athletes do today, but when her body feels different than the data, when those diverge, she trusts her body and it makes sense. Like your body has evolved through thousands of years of polishing a product like a smartwatch. A few years. And it's incredible how biased we are towards tech and data that we would follow the watch instead of your body. That is much more precise 100%.
- [13:46]
B
You know, I'm thinking of these examples where the navi chan I'm using, I have the Japanese word in my mind. No, that basically, like the Waze or Google Maps or one of these types of apps. And, you know, it ends up starting to take you somewhere where you don't want to be going, but you're like, hey, this is where I'm supposed to go, apparently, right? And this has happened and people have been robbed, and, you know, also all sorts of bad things have happened as these things have kind of cut through, you know, places that really. Where we weren't really supposed to be going.
- [14:19]
A
Exactly. But then, going back to your question, aren't we getting more data? Yes, we're getting more data. Is that a good thing? Well, it depends. It depends on the effects of that data and how we use the data and who holds the data. And at the moment, I think that no, it hasn't been a positive development. So a lot of the times we get sold the idea that if we have more data and more surveillance, we'll be safer. But I'm Spanish and Spain is one of the safest countries in the world with one of the lowest markers of homicide, for instance. And it's been that for quite a while. Is it thanks to surveillance? No, the UK has much more surveillance. Is it a safer country? No, it's not. And so there are all these assumptions that are questionable at best. Questionable, putting it very generously, if not outright wrong. And then let's suppose that, okay, we do have more data and we can better track people and better importantly, predict people. Predict people's behavior. Is that a good thing? Well, if you want an authoritarian regime, that's a great thing. If you want a liberal democracy that is healthy, in which people feel, for example, freedom to protest anonymously and peacefully, then, no, that's not a good thing at all. And part of what history teaches us about prediction is that the future doesn't get predicted. It gets produced many times through prediction. But when you're able to predict what somebody's going to do, that's bad news. That's not scientific advancement. That is a symptom of authoritarianism, because chances are, you're determining it. And just to go to the extreme, as an illustration, you want to know where somebody's going to be tomorrow? Jail them, you'll know. And is that what we want? I hope not.
- [16:11]
B
Now, I'm thinking of something that I. People that younger than me would call meta. Okay? Because we have this idea that, I mean, I'M certainly being programmed with this all the time. That the future absolutely involves. More sensors, more surveillance, more tracking, the end of privacy. It's inevitable. It's absolutely inevitable that for example, programmable currency. Inevitable, right? Is this a self fulfilling prophecy? I almost believe it, but I really don't want to. So help me here. Okay?
- [16:49]
A
One of the best news about self fulfilling prophecies is that they only work if we believe them. So in the paradigmatic myth of Oedipus, when he gets told that he's going to murder his father and marry his mother, he panics. And in his horror at that prospect, he tries to flee from his destiny. And it is in that trying to flee from it that he makes the prophecy come true. So there are many ways to interpret a myth. And the way that I got the lesson, or I got taught this myth was, well, there's destiny and you and you can't escape it. And if you try to escape it, you'll only make it come true. But for me, the actual lesson is no. If Oedipus had laughed at the prophet and chilled, he would have never made the prophecy come true. It's only because he believed it that he made it come true.
- [17:42]
B
Except that if I chill right now, then those people that are interested, these people that I was mentioning before that believe that with maximal data, with full sensor sweep of our bodies, of our minds, of everything, they're going to be able to create that perfect world and make life better for you and me. They're determined.
- [18:01]
A
You're absolutely right. And by chill, I don't mean not do anything, I mean not panic. I mean not panic and not act in a way that plays into the self fulfilling prophecy. So if Oedipus instead had said like, okay, I'm going to become a pacifist, I'm not going to murder anyone, then that's a pretty good method not to murder your father. So in a way, what we have to do is get busy building the future. Because the future is not written and it depends on us. And one very important red flag is when anyone says that any kind of future is inevitable, because that's just patently false. And it's a red flag because it means that that person is trying to get you to do something, is selling something. They have an interest that you think and you believe this. And so whenever somebody says this future is inevitable, red flag and start asking questions, who is this person? Why are they making this prediction? Who stands to benefit? Does it come from data? What kind of data? And what Are the data that we don't have that might falsify that?
- [19:06]
B
And how is he benefiting? You said that, right?
- [19:08]
A
Yeah, yeah. And who stands to benefit? And do I want this future and if not, what am I going to do to make sure that that is not my future?
- [19:17]
B
I was just texting with a friend of mine this morning. You know, something happens that goes against, you know, basically that doesn't go the way that we would like to see the world developing and you get this what people call black pilling. Sometimes it's just, you're okay, you know what, forget it. There's no hope. We're just stuck with it. The bad guys are going to do their thing. We can do nothing about it. And that's kind of the opposite of what the sort of mindset that you're describing. But it's very easy to fall into that, I think, right. When you see all these sort of machinations happening around you, moving in a direction that you don't see hope in or you don't see a future in.
- [20:01]
A
Yeah. And for people who are very powerful and who want to set the agenda, that's great news. Right? That's exactly how they want us to make. That's exactly how they want us to feel because that means that we'll just doom. Scroll on our phones on the couch and do nothing about it. And it's also a product of this kind of mentality of like, oh, these people are very powerful. I'm no one, I can't really do anything. I'm just going to disengage. And that seems on the surface the easiest way to go and the easiest option. And when I suggest that we should do something else, one question that I get is like, but is it fair to put that burden on individual shoulders? And the answer is no, it's not fair. We shouldn't have to fight for our rights to be respected. But the fact of the matter is our rights are not being respected. And if we don't fight for them now, it's just going to get harder. So one way to look at it is, look, life is full of problems. If you're out of problems, that's terrible news because it means you're out of life. And it's about solving better problems. A good life is having better problems than the alternative. And I think that making an effort to protect democracy is a much better problem to have than fighting authoritarianism once it's here. And the other thing is that even though on the surface it seems like the easiest option to Doom scroll on your couch. It's not going to make you feel better, it's not going to make you happier, and it's not going to make you feel satisfied or proud or hopeful or anything that makes life meaningful. And I wish we could invoke this sense of look. Part of the magic of being a human being is that we can defy the odds that we've seen it happen time and again. And when you encounter a situation that is very hard and in which you have to defy the odds, that the attitude is not like, okay, then I'm gonna back down. It's like, no, I like a good challenge. Let's take it on. Bring it on.
- [22:05]
B
I was just reminded yesterday of the Lord of the Rings, one of my favorite stories. But this is really the story, right? You kind of like really bad odds, right? Really bad odds for a very long time. But sometimes it works out and it's worth it, just giving it a shot. Right. That's kind of the message. I mean, I'm being very glib and oversimplified here, but that I think that's a compelling argument.
- [22:29]
A
Absolutely. And it's not only fiction, although I love fiction and fiction is very important. It's all the people that we tend to admire. The people we tend to admire are not the people who just stayed at home and just accepted their odds. They were the people who impressed us. I don't know, Rosa Parks, Martin Luther King, whoever you can think of as a hero probably had quite a few difficulties along the way and they were bucking the odds.
- [22:57]
B
It's very interesting that you say that because this idea of, you know, kind of probabilities and self fulfilling prophecies is again the centerpiece of what you're writing about. But really it's the people who say, you know what, I don't sure, but I'm still gonna do this.
- [23:12]
A
Yes. And the paradox is that the more we believe that we can do it, the better our chances. If you don't think you can do it and you stay home, like the chances of succeeding are zero. And sometimes in life, some battles are so important that I genuinely think that it's better to fight them and be on the right side of history, even if you're convinced that you're going to lose. So for example, if you have a child and that child has a difficulty, any kind of difficulty, social, physical, whatever, if it's your child and you love them, you'll do whatever you can to make their life better. Even if you think you're going to fail, because that's not the point, in a way. And so for the very important battles in life, when it's very important to be on the right side of history, it's kind of inappropriate to make a calculation about whether you're going to win or lose. It doesn't matter. It's that important.
- [24:01]
B
Let me tell you about something else I've been thinking about, which I think fits into a very profound thing I think you said, which is that we're analog beings, not digital beings. Right. And so I've been. Something I've been thinking about in terms of this utilitarianism that's sort of on the rise in our societies. I juxtapose these two concepts in the medical context. Right. So the traditional medicine is in the Hippocratic view, Right? The Hippocratic view is that as a medical practitioner, I would look at the totality of who you are and I have to figure out how to help you best. And the other far end of that spectrum is this utilitarian bioethics, which is that greatest good for the greatest number, sometimes called the greatest happiness for the greatest number. And this is the point where the challenge is that someone kind of gets to decide what that greatest good is and someone gets sacrificed. That is actually utilitarianism personified. Like, this is why you get atrocity in these communist societies, because someone says, well, the greatest good is the survive my survival or survival of the communist party. Right. It feels like the digital mentality lends itself to a kind of utilitarian, bioethical view of the human being. What do you think?
- [25:18]
A
Yes, I agree. Part of what I try to do in the book is give a little bit of a sense of who each character was as a person in the history of statistics, but also further on, when we come to utilitarianism, because the kind of people they wear tells you something about where they're coming from and. And why they might have thought what they did. And if you've met any utilitarians, you might notice that these are not particularly good people at the embodied part of life. For example, in their latest iteration in effective altruists, a lot of these people are making assumptions about what the future will look like. And part of the assumption is that we will upload ourselves into some kind of cloud. Of course, that has so many assumptions that I think are wrong precisely because we are not digital beings and we will never be digital beings. Part of what it means to be a human being is to enjoy each other's presence. We make Trips and all kinds of things, precisely because we're not digital and it's not the same to look something on a screen as to be there. When it comes to thinking about ethics, one way to make the point that you were making about how utilitarianism tends to think about the community as one organism. And John Rawls, a philosopher in the 20th century, famously argued against this idea that it is a kind of disanalogy. Because if you think of yourself as one organism and imagine that you're in the situation in which you have gangrene on your toe, and if you don't cut off your toe, you're going to die, you might well be willing to cut off your toe. But the analogy that we should do the same with society and the society is one organism is incorrect because your toe doesn't have objectives for itself, it doesn't have values for itself. And so the way John Rawls puts it is that utilitarianism denies the separateness of person, persons in the sense that every person is a project onto themselves with their own values and their own projects and their own life. And you can't sacrifice it as you would a toe, because a toe doesn't have that kind of separateness.
- [27:39]
B
I mean, fascinating. And its own problems, for that matter. Right. I want to jump back to this obedience idea, obedience to prediction, because I. You captured my imagination with this. Okay, I'm going to read something that actually, I think it was from Timothy. You quote Timothy Snyder, right?
- [28:02]
A
Yes, that's exactly right.
- [28:03]
B
Anticipatory obedience is a political tragedy. He says, by believing and acting in accordance with their predictions, we don't even notice we're obeying. If we take a second to recognize them as predictions, if we understand that predictions are prescriptive and not descriptive, we stand a better chance of controlling our fate. Now that I think about it, you've kind of been telling me this, right? But this really makes me want to remove all sorts of predictive decision making because we all have it in our lives, right? Even in our very simple lives, like deciding about the weather. But is there a way of going too far?
- [28:43]
A
I guess yes. I think there are all kinds of situations in which following predictions is a very good idea. I live in the UK and I check my weather app multiple times a day, and I will continue to do so. But what I'm arguing for is a more enlightened use of prediction to be clearer about the things that we can predict, the things that we can't predict. And if we have the illusion that we can predict it's going to lead us astray. And to be clear about the things that we shouldn't predict, even if we could, because we are partly determining it,
- [29:18]
B
what shouldn't we predict?
- [29:20]
A
I think we shouldn't predict the fate of individuals. I think that is a disrespectful way of treating people because it treats them as things and not as agents who have a say in that future, who can defy the odds. And there's a lot of arrogance in thinking that you know what the fate of an individual is.
- [29:38]
B
Isn't this the whole insurance industry based on this?
- [29:41]
A
I mean, yes and no. And of course there are situations in which it's tricky, it cannot be a blanket rule. More tricky, I think, is medicine because sometimes you need a sense of the diagnosis and a diagnosis carries some prognosis to treat someone. So it is more nuanced than that. But in the case of insurance, that is a great question because insurance is fascinating. It, it's the most important industry, I think, when it comes to prediction, and certainly one of the oldest ones. It's a very important industry for the whole of society because it provides a kind of safety net and in so doing it contributes to stability.
- [30:21]
B
But if I may. Right. As you mentioned before, there's a lot of profit motive in there as well.
- [30:27]
A
Absolutely.
- [30:28]
B
Yeah.
- [30:28]
A
And originally in insurance wasn't a kind of business, it was a kind of pooling of risk and solidarity of workers who knew that if the breadwinner died, their family would be in trouble and to kind of join forces to keep people safe. And that is quite a noble history of insurance.
- [30:48]
B
I completely agree, actually. And actually I use that kind of a system which they don't even call insurance. They call it a health share.
- [30:55]
A
Exactly. So when it comes to insurance, insurance is a good idea. Society benefits from insurance in a way that it doesn't from other kinds of industries. But what is very important is for insurance to make predictions at a population level because then it has much less of that force of self fulfilling prophecy. It's kind of diffused. And so if you predict that however many people in a million will get diabetes, that's something very different from thinking, oh, this person's going to get diabetes, and then charging them higher for their premiums. And we're seeing that trend towards more individualized kind of insurance. And individualized insurance isn't insurance at all. The whole point of insurance is pulling risk. If individuals are paying for their own way, then we might as well pay for our own way and not pay the middleman who is making a profit.
- [31:42]
B
There's a whole lot of insurance that smoking is one of these things. Very clear. You know, you're much more likely, you could smoke two packs a day and live to be 100. People have done that. But your likelihood of that goes down. It's in the probabilities. Right. And so you're going to get, you're going to pay higher premiums if you're a two pack a day smoker, if your insurance company knows that. Right. So this is. But you're kind of saying that you don't want any of that. Give it, pull the. Pull the risk equally.
- [32:12]
A
I'm saying we need to have a public debate and we need to make this decision in a conscious way. And I don't think we're doing that.
- [32:19]
B
By the way, I'm very attracted to your idea here in a lot of ways. Right. I can see pros and cons to it, but it's definitely heading in that direction. But explain again why we shouldn't do that even with something as obvious as smoking potentially.
- [32:36]
A
I don't know if we should do it with smoking because on the one hand I think it's a good incentive for people not to smoke, but it's a slippery slope because if we accept smoke, then we start tracking people for whether they are exercising. And you might think, well, that's fair. You know, if you exercise, you have lower chance of getting sick. And that's fair for everyone. Except it's not as simple as that because it's not only that, it's statistics. So you're right that somebody can smoke and then, you know, live to be 100. But when it comes to exercise, their bodies are very different. And there are situations in life and there are bodies that don't actually benefit from exercise in the same way as others. And then the way we track exercise is very rudimentary. So it's easy to track whether somebody runs. But then people also game the system. So you attach your garment to your dog and let them run. But also there are other kinds of exercise that are not that easy to track. If you do gardening, it's a kind of exercise. And so you start getting into situations that create unfairness. And in general, first there are conditions that are much more important for health that we have no control over, like our genes and we shouldn't be punished for those. And it's better not to know who's lucky and unlucky and just to pull risk altogether. And secondly, you have an effect on people. So if we start tracking people and if you start being too petty about things, that creates a kind of stress and a kind of competition and a kind of invasive imposure that will have effects on health. And so sometimes we undermine the things that we're trying to incentivize.
- [34:22]
B
Right. Well, I mean, 100%. And the extreme of that is the sort of the panopticon that communist China is developing with the social credit system. Every camera and your gesture and, you know, whatever it is, I say something, oh, your score just went down. It wasn't pro communist party. Right. On and on and on. It's a. Turns turning a human being into, I don't know, digitizing. Let's use your concept here, digitizing the human being. Maybe I don't know what it is, but this is, again, this is this idea with this, with just enough. We have. We have enough information, we can sort of shape and mold people just to live great, productive lives. Right. And everyone will be. It'll be wonderful. Yeah.
- [35:01]
A
And it assumes that there is a right and wrong thing to do or better and worse for everyone. And it's the same. Right. So one of the examples of the Chinese social credit system is that if you play video games, that's bad, and if you buy diapers, that's good. And you can see the reasoning behind it and you can see how roughly that might have some sense. But first of all, it's very judgmental to think if you have kids, that's a good thing. Well, it depends on the circumstances. And we should have freedom to have kids or not have kids. And then with video games, you know, there are situations in which actually playing a video game might be a very good idea. For example, there are studies that suggest that certain kinds of video games are actually quite good to process ptsd. So it creates a lack of nuance and a kind of totalitarianism. And part of what we mean when we use this word, totalitarian is a system that imposes upon itself on people's lives in. In all areas of life. So it's not only your job and it's not only the government, it's your neighbors and your personal life and everything in between. Whereas liberal democracies are supposed to have different spheres so that maybe you are a terrible neighbor and it's fair for your neighbor to hate you, but that shouldn't impact how much taxes you pay or how the government treats you.
- [36:23]
B
You raise all sorts of, you know, incredibly, incredibly important questions that the reason I mentioned earlier, like, I almost believe. Right. And I'M very skeptical and I don't want to believe this, but I almost believe that this, you know, sort of data panopticon is almost inevitable because things are moving so hard so fast in that direction. It's just where progress is going to. Now. I don't want to believe it, you understand, right? But I kind of do when I think about it, because it's so. It almost seems obvious that things have to go there. And I know we've talked about how you need to change your thinking. Right. And that's the only way that'll ever change. And that's exactly right. How do we get out of that mind frame? Have you, like, how do you get out of the mind frames that, that you're seeing being programmed onto you?
- [37:21]
A
So I have quite a long standing habit of being quite stubborn, which I wouldn't recommend for everything. But it has its silver linings or virtues. I think everyone has experienced this, but some people more than others and women and foreigners tend to experience this more than others. Of finding yourself in a situation in which, you know, people, people around you are predicting your failure. And instead of letting that bring you down, saying, I'm going to prove them wrong. And there's a pleasure in that and that kind of constructive, friendly defiance of like, no, that's not our world. And it partly depends on how we behave. Right. So are you going to buy surveillance cameras then? Yeah. Suddenly the surveillance state becomes a little bit more likely and a little bit harder to fight. So it really depends on how we act. I think people underestimate the power that they have. The power that they have not only as citizens, but as friends and as parents and as neighbors and as nodes in this social system. Another way to have a little bit of fun and defy the system is to study the predictions of the past. We are remarkably bad at predicting the future and futures that seemed inevitable not that long ago just didn't exist. One example is Paul earlier with his idea of the population bomb. There was a very, very strong belief that it was inevitable that there would be so much overpopulation, that there would be famine and it would be a disaster that never happened. And then a few decades later, it was inevitable that we were going to shrink to nothingness and that the population was going to be dwindling. And that didn't happen. And a few months before the Wright brothers managed to make an airplane fly, the New York Times published this scathing, scathing article saying that it would take from 1 million to 10 million years to make people fly in airplanes, they're completely wrong. So next time you listen to one of these predictions, just look to the past and read these predictions of the past and it gives you a lot of perspective to say no. And in fact, when you listen to our prediction, given the terrible track record of our predicting, if anything, rather than make you sad or panic, it should kind of give you a reason to be optimistic because we've been so bad at making predictions that if you listen to our prediction, that's probably going to be wrong as well.
- [40:05]
B
One of the areas that you've thought a lot about, in fact, you wrote your previous book about it, is privacy. And you're huge advocate for privacy and thank you for that, by the way. So I am also a deep believer in that. But when I look at this newest generation, Gen Z, right, that sort of, that's come of age, they have a very different view of privacy. It's a very different way of viewing their digital natives. They've grown up in the culture where, you know, these, the big companies have just been siphoning everything, selling it off. There's this whole information economy, right. And they don't even, in a sense, and many of them that I've talked to don't actually even grasp why you would even need it. The cult, the culture has shifted so much in that generation, right. They're just happy to give away all of it. I mean, as far as I can tell, my God, like, what are we giving an example was, right? It's a criminal offense to open someone else's mail, but it's like we forgot about all that. And just in today's digital economy, everything is basically up for grabs, whatever it is. Even like, you know, when I'm looking in the phone, they can, you can be picking up all the different things that are in the field of the camera and, you know, trying to sell to me based on that and so forth. So I'm concerned our culture has shifted certainly with this newest generation around this issue of privacy. And I'm worried that we're going to have a future where people don't even understand what that was like. And how can they make decisions if they don't even understand how that was? Like, how can they see the value of it?
- [41:49]
A
I'm quite torn about that. Sometimes when I look at trends on social media and if you focus on particular kinds of social media like Instagram or TikTok, I definitely get the same sense as you do. And I worry, I worry about, especially about us educating a generation of people who have Never had privacy. And I think that stunts people, that we are educating people to be subjects and not citizens. Because part of what it means to grow up is to make mistakes and break your arm and learn from that process. And if you're so afraid that anything you do will end up on social media, you are much more afraid to make mistakes and then you don't make them and you don't learn from them and you don't grow up. You don't in a sense. However, I think in history we have tended to underestimate the creativity and the wit and the ability to surprise us of young generations. More and more I encounter my youngest students that they don't want social media. And granted my students at Oxford are not particularly representative of the general population. However, there is something going on there that wasn't happening before. And often they have their first social media when they come to Oxford because they feel they need to belong to the Oxford societies, which is a shame. And I hope that we can work on that. And I ask them, what do you do with your time? Because at the same time I worry that their attention spans have become so short that they're not reading, which is a huge problem that maybe we can go back to later on in the conversation. But they are often playing role games and they are doing crafts things and
- [43:45]
B
they are like physical things, digital things,
- [43:48]
A
and they're going to the gym and they are meeting their friends. And so I wonder if we're starting to see a bit of a reaction of people tired of being online all the time, of being on front stage all the time. And there's an excellent documentary that I highly Recommend called Molly vs machines. And it's about the case of Molly Russell, who was a teenager in the UK who ended her own life after being exposed to around 2000 posts of self harm content on social media. And this was a very big case in the UK because her father, Ian Russell, took this case to court. And it was the first time that a judge ruled that social media companies were partly responsible for the death of that teenager. And part of what the documentary shows is the lives of her friends and how her friends process this experience. And of course these are, this is a very particular case because this is a teenager who ended her life. But these young women were incredibly reflected about social media. And even though they had social media, they were much more savvy about privacy and about limits than I think maybe the average teenager or what we imagine the average teenager to be. So privacy is something that we value. Many times when we learn the lesson of what happens when we don't have it. And I think many teenagers have suffered horrendously from exposure of different kinds, and they're much more wary than teenagers five years ago were.
- [45:36]
B
Right. I mean, we're kind of. We've gotten through some of that shift. Like it's that point where people are starting to wonder, wait a sec, is this really as important as good? I just interviewed a young woman, she's a bit older than a teenager, but she went through this earlier, who just decided one day I have to turn off my Instagram. Right? Because I just. It's just too much. It's too. And that was the moment. And she realized after she turned it off, I was like, I don't need this. But she was living in it, basically, as many young women are apparently, as I've been learning. Right. And that was what caused that self reflection. Wait a second. This has been a focus of my life and I actually don't need this. Maybe that's the thing you recommend. Try it out, get off the thing for a moment, see what happens.
- [46:28]
A
Yes. And something that I would like to instill in young people is how much reading paper books is the antidote to many of the ills that the digital era has brought. How instead of shortening your attention span, it will broaden it and it will not misinform you. It will keep you more informed and it will give you.
- [46:48]
B
Especially the great books, right?
- [46:49]
A
Yes, great books. And it will give you this resilience that social media tends to take away from you. It's just you and that author in the world of ideas. And for all the dazzling qualities of digital tech, the book is still the best technology we have by far of both conveying experience, another person's experience, what it is to be someone else. You only get to live once, but if you read, you get to live many lives in one and complex thinking. If you don't have the. If you have the attention span only of 14 seconds, there are arguments that you can't possibly comprehend people because their arguments are just more complex than 140 characters or whatever it is these days.
- [47:36]
B
And it takes you out of the digital world. It needs to be. I think it really needs to just be a part of our life because there's always this encroachment. More digital, more digital, more digital.
- [47:49]
A
It's a kind of bubble. Nobody's surveilling you. You don't get any notifications. The book doesn't ask anything of you. It's a really Special kind of activity. In fact, so special that when reading became more individualized. So at the beginning, reading was communal because it was so effortful to produce a book that you couldn't have books for everyone. You just had one book and so somebody had to read it out loud. But when we could produce more books, people were so afraid of the autonomy it conveyed that they didn't want to allow women to read alone because it's a dangerous thing. You strengthen your mind. You strengthen not only your mind but also your spirit in the sense of. It gives you so many more tools to think about the world, but also to face the world. It gives you this calmness. And so it's not only that for young people that you should read, because it's like eating vegetables and they're good for you, is that if you don't, you are truly missing out on one of life's greatest pleasures. And in my view, a lot of people who don't like to read, it's not that they don't like to read, but that they haven't trained their attention to make it pleasurable. Because books are so broad that surely there's something that you would enjoy reading, whether it's mystery novels or romance novels or, you know, or non fiction or whatever, there is something for you if only you can tap into it by training your attention.
- [49:12]
B
It's so interesting that you talk about this time where we went from, you know, the written word not being as available. Then the Gutenberg press comes into being, is built, right? And that first democratizes faith. People start thinking about, you know, what is this? What does the Bible really mean, right? And it causes wars and it causes the Reformation, it causes the Enlightenment. It probably, you know, library probably causes liberal democracy, you could argue. Anyway, this is a whole discussion, but a big, big change, a big shift as this media form of media is created. Now you can get books and a thousand people, can 10,000 people, whatever can be reading together. I'm wondering, and there's a number of books to this effect right now, the Information State by Jacob Siegel, for example, that I recently read and interviewed about that. We're in a situation like that now where we're shifting into a new kind of media with social media. This AI juiced up social media, the silos, it creates the democratization of information that creates the ability to control. It creates. And we don't even really know what it's going to look like on the other end. I'm curious if you've thought about this or whether you think we're in one of these times.
- [50:33]
A
Yeah, I don't know. I think it's tricky to make the comparison to books.
- [50:40]
B
I just mean like a shift in the way that. The way media is created and the way the. The. In a way. So the emotion that's conveyed through media because we can see each other, right? And we can sort of. It's like the. The transmission of body language now en masse. Right. I mean, that. That would be a piece of it.
- [50:58]
A
There's definitely a shift, I think that is undeniable. But I wouldn't assume that the shift is always positive or that it's always going to remain the way it is. So going back to predictions that were wrong when the electronic book came out, the belief, the majoritarian belief was that the paper book would disappear within a matter of years. And here we are decades later and the paper book is still stronger, I think partly because it's tactile. So anyway, in this shift, again, we have seen predictions about the end of books and that we'll only have this other kind of information in the shape of websites and podcasts and social media. I'm very suspicious of this term of democratizing information because it seems to me like in some ways, yes, the Internet has made access to certain kinds of information much more easy and easily available. So Wikipedia perhaps is the best example, in contrast to encyclopedias that were paper based and quite expensive and not everyone could access and even less precise than Wikipedia, or at least Wikipedia at a certain point in time. I'm not sure if that still holds. However. I had this unsettling experience that I wrote Privacy as Power six years ago. And researching that book was so much easier than researching prophecy in ways that were easy to compare in the sense that I was looking for a very particular claim by a very particular person. And I knew exactly what I was looking for in 2019 and 2020. I could find it in 20 seconds. Today it takes me six hours to find that claim that I was looking for. Partly because really, yeah, there's so much slop out there, there's so much more. And not all of it is information that is worthy or credible or precise.
- [52:57]
B
And some of it is just there to mix things up intentionally. Yeah.
- [53:02]
A
So I. My experience is that it's getting harder to get at the good quality information than it was a few years ago. And that is very concerning.
- [53:14]
B
I could talk to you for hours, but I think we're going to have to finish fairly soon. Do you have a final thought as
- [53:22]
A
we finish, perhaps going back to your question of how do we deal with this doom and gloom? And when you hear a prediction and you believe it, you don't want to believe it. And part of how we deal with it is humor. I think we're taking some people too seriously. And that humor is a very important tool in life to cope with the hardest parts of life, but also in democracy, because there is irreverence in humor and there is critical thinking in humor. And so turning to comedy is not as superficial as it might seem.
- [53:58]
B
Well, Carissa Veliz, it's such a pleasure to have had you on.
- [54:01]
A
Thank you so much. It was a pleasure to be here.
- [54:05]
B
Thank you all for joining Carissa Veliz and me on this episode of American Thought Leaders. I'm your host, Jan Kellek.