← New Books Network
New Books Network
José Marichal, "You Must Become an Algorithmic Problem: Renegotiating the Socio-Technical Contract" (Policy Press, 2025)
00:32:21
Loading summary
Transcript27 lines
- [00:00]
Jose Merchal
Limu Emu and Doug.
- [00:03]
Marshall Poe
Here we have the Limu Emu in its natural habitat, helping people customize their car insurance and save hundreds with Liberty Mutual. Fascinating. It's accompanied by his natural ally, Doug.
- [00:18]
Jake
Uh, Limu is that guy with the binoculars watching us.
- [00:21]
Jose Merchal
Cut the camera. They see us. Only pay for what you need@libertymutual.com Liberty Liberty Liberty. Liberty Savings vary unwritten by Liberty Mutual Insurance Company and affiliates. Excludes Massachusetts.
- [00:30]
Marshall Poe
Hello, everybody. This is Marshall Poe, the founder and editor of the New Books Network. And if you're listening to this, you know that the NBN is the largest academic podcast network in the world. We reach a worldwide audience of 2 million people. You may have a podcast or you may be thinking about starting a podcast. As you probably know, there are challenges basically of two kinds. One is technical. There are things you have to know in order to get your podcast produced and distributed. And the second is, and this is the biggest problem, you need to get an audience. Building an audience in podcasting is the hardest thing to do today. With this in mind, we at the NBM have started a service called NBN Productions. What we do is help you create a podcast, produce your podcast, distribute your podcast, and we host your podcast. Most importantly, what we do is we distribute your podcast to the NBN audience. We've done this many times with many academic podcasts, and we would like to help you. If you would be interested in talking to us about how we can help you with your podcast, please contact us. Just go to the front page of the New Books Network and you will see a link to NBN Productions. Click that, fill out the form, and we can talk. Welcome to the New Books Network.
- [01:37]
Jake
Hello, everybody, and welcome back to the New Books in Science, Technology and Society, a podcast channel on the New Books Network. I'm Jake, one of the hosts of the channel. Today we'll be talking with Jose Merchal, a professor in the Department of Political Science at California Lutheran University, about his new book, you Must Become an Algorithmic Problem. Jose, welcome to the show.
- [01:55]
Jose Merchal
Thanks, Jake. Appreciate it.
- [01:56]
Jake
I was wondering if you could begin this interview by telling us a bit about yourself and how you came to write this book.
- [02:01]
Jose Merchal
So I've been teaching Internet technology and politics for close to 20 years. Back in the early 2000 and tens, I wrote a book about Facebook and its impact on political discourse. So I've spent a lot of time trying to think about what impact technology has on political discourse, political identity, and for this particular book, I think after 2016, everybody suddenly learned about algorithms and everybody suddenly got wisened to the fact that maybe our discourse environment was being structured by them. Cambridge Analytica was a huge thing. I spent a good amount of time in my book talking about Cambridge Analytica in 2016, but a lot of the discourse that came out of that in 2016 didn't feel quite right. That a foreign actor is just manipulating us and we're all gullible, and that if algorithms are going to play some role on our political discourse, on our civic life, that is probably something more subtle than that. So I think that set me on a journ of kind of trying to understand exactly how I think algorithms change us. In 2017, I co organized a conference at one of our associations, the Western Political Science association, on algorithmic politics. And so we had a lot of panels kind of talking about what impact algorithms might have on political discourse. And so I gave a couple of other presentations and then the pandemic hit and then all of that just kind of got tabled for a little while. But it did give me time to think and reflect on what do I think is really happening here. And it gave me two insights. I think I had time for reflection. One of them is this idea of learning about machine learning. So the pandemic gave me the opportunity to study Python, to study data science, and to teach myself more about how machine learning works and understanding questions like the cost functions like gradient descent. And a light bulb kind of went off when I was reading about that because I thought gradient descent, in gradient descent you're marching towards a local minima. You're reducing the cost function by marching towards a local minima. That might be technical for some people, but generally the idea is you're trying to reduce the error in the model, in the predictive model. And so I thought, I feel like we're kind of acting like that. We're acting like people who are kind of marching towards a local minimum. We're like optimizing ourselves in that way. And the other insight was, gosh, you know, we are complicit or we have a role to play in this. When we go on TikTok, when we go on Instagram, we're part of it. It's not one way traffic. So we should think about this more as a relationship. And so what the theoretical perspectives out there lead us towards understanding our role, our relationship to technology in terms of a relationship. And there's obviously this great literature on socio technical systems and cybernetics, but there's also, there's not a lot of political scientists doing this work. And One of the areas in political sciences that thinks about the relationship, the relationship between the individual and, and the state is social contract theory. And I really hadn't seen anybody write about how you could apply social contract theory to technology or at least to social media algorithms. And so I thought, well, that would be an interesting frame to explore. What if we think about our relationship to social media, to algorithms, to engagement algorithms, to monitoring algorithms, as a socio technical contract, as a social contract. And so that's kind of in a nutshell where the motivation for the book came from.
- [05:40]
Jake
So I want to start this interview off very differently than I start other interviews. Usually I talk directly about the book and a specific passage or a chapter or theme, and we're going to get there. But I'd actually like to start with AI Slop. In preparing for this interview, I saw you wrote a really excellent article over the summer about AI Slop. And I think that's an interesting entry point into the algorithmic contract that you talk about in the book.
- [06:03]
Jose Merchal
Yeah, so, I mean, especially now that Sora, the Sora app came out last week, it's really been in the news. And what I wrote is this piece for Tech Policy Press, which is a really great publication and I'm a contributor, so I write a couple of articles a year for them. And what I think I was trying to get at in the book is that yes, absolutely, tech companies should be held accountable and the ethos of move fast and break things is not healthy for democracy, but that also we need to stop thinking about it just in terms of what they are doing to us. And so the gist of that article was that the reason AI Slop works is because of our desire for it. And I use the analogy there of junk food. And I say we know intuitively that if we eat cookies and crackers and a six pack of beer every night for every meal for 10 years, that's ultimately going to be really bad for our health. And as I'm getting older, it's even worse. I cannot eat pizza and beer and cookies and crackers, and it's not conducive to a good food diet. And so the argument I think I was trying to make is that engaging. You know, there's a great study that came out. The Washington Post did a study looking at TikTok use, and they found that there's a pretty good cluster of people that are what they call TikTok power users that spend more than four hours a day on the platform. And not to be moralizing, not to, but but to say four hours on TikTok, it's not good for your cognitive health, it's not good for your discursive health. Right. Those four hours could be used having conversations with friends about issues, spending time with family and loved ones. And I'm not the first person to say this, and I always worry that what I'm saying is going to make it sound like I'm exonerating tech companies, and I'm certainly not. I just want us to think about it as an interaction. We also should demand from tech companies things that contribute to our health. So we should ask the question, what.
- [08:05]
Marshall Poe
Do we, what do.
- [08:05]
Jose Merchal
What does AI owe us? What does Google owe us? What does OpenAI owe us? What does TikTok owe us? Not from a perspective of like, oh, you know, I must demand things, I'm a petulant. But more like if we're going to have an interaction with these, with these tools, if we're, if we are in a socio technical contract, what, what, what should they be doing for us that enhances our lives, that help us flourish as human beings. So, I mean, that was the point of the article.
- [08:34]
Jake
So is this what you term an algorithmic contract?
- [08:37]
Jose Merchal
Yeah, I think it's a useful framework in political science, a social contract. Everybody learns it in freshman political science. It's a metaphor, it's a framework. Right. Nobody actually signs a contract. We all know that. But it's a way of thinking about why do people pledge allegiance to a state, why do people allow them submit themselves to a state? And the counter is living in the state of nature, the world before government. Why do people in this hypothetical, leave the world before government, enter into a relationship with a government? And what philosophers throughout history, Hobbes, Locke, Rousseau, maybe Rawls, a little bit, have said is that because the state gives us something, we get something out of it. It's either security, if it's a Hobbesian framework, or it's helping the state preserves our right so that we can flourish as human beings. Which is more the locking perspective. Or it's we kind of submit to this greater general will, like Rousseau would say. So, and that gives us sort of meaning, a sense of meaning and a sense of solidarity. But all of those are frameworks that say the government, the state, is giving us something in return for our allegiance to it. We too often think about our relationship to tech companies as a market model. Right. Well, we can either take it or leave it. We can exit if we don't like, if we don't like TikTok if we don't like Instagram, we can exit, right? And we know that, like, there's a lot of. There's a very high exit cost to leaving Instagram, right? All your friends are on it. Everything that you're. That you might do that weekend or staying in touch with your friends and family. You know, back in the day when Facebook, you know, was something that everybody used and not just your grandparents, you know, there was a lot of transaction costs to leaving Facebook. And in the book I wrote way back then about Facebook, I talk about that. That, like, it sometimes posited itself or presented itself as a nation state, as akin to a nation state, way way back, early on with Facebook, you could vote on policies. Now they set a very high bar for which policies two thirds of all users had to vote. And that never happened. But they would, you know, do things like that to give users the impression that they were part of a big community. Right. And so they're not wrong that tech companies, especially engagement algorithms, that relationship is something different than a free market relationship because they are also structuring choice. They're also structuring reality in a way that is different than if I go to, I don't know, Buffalo Exchange or Macy's or whatever store I'm going to go to buy something, right? Macy's doesn't structure my ontology or my epistemology the way that. That. That TikTok might or the way that, you know, Instagram.
- [11:25]
Jake
I'm so glad you said that. There was a line in the book that I really appreciated, which was, algorithms are reification machines that calcify a particular ground truth. And, you know, situating that within the context of the fact that there's so much information out there is that the trade that we make in this contract is. I can't see every movie that Netflix or read every post that came out on Blue sky today. So the algorithm will do some of that for me. In exchange for what? Like what. What am I getting out of it?
- [11:55]
Jose Merchal
Yeah, I mean, I think that is the main question of the book, is what are we getting out of it? And I think we trade our autonomy for algorithmic curation and the security and comfort of relieving the anxiety of an uncertain world of fomo. Right. Of a world of so much information out there, so much content, that the algorithm relieves the anxiety by structuring choice for us. There's a great sociologist, Iran Fisher, who wrote a book, Algorithms and Subjectivity. And the term he uses that I really like is the toil of Choice that it's really stressful to have to navigate this world of endless possibility, endless choosing. So the algorithm kind of structures it for us, and it does it in a way where we feel like we have the agency we're driving. Right? It's my content, I train the algorithm. It's what I want, it's what I like. Right? And so that gives this false sense of, like, autonomy and control, when in reality, it's not just you choosing, it's also them injecting things that are steering you in one direction or another. It's reifying, right. It's locking you into a certain way of seeing the world. Kate Crawford, another great scholar of technology, talks about ground truth in her work and that the algorithm is presenting you some version of reality and it's leaving out another version of reality. But we misinterpret it, we mistake it for reality itself. People do it all the time. They think that their algorithmic feed is a real reflection of the world as it is, and it's not. It's a curated, abstracted reflection. And to the extent that we go down that road of the go. Go down the road with the algorithm, I think we reduce ourselves, we sort of truncate ourselves to that limited version of the ground truth that the algorithm gives us on the book.
- [13:56]
Jake
You touch on the fact that there are social harms of the algorithmic contract, and you've already begun to talk a little bit about it. But I'm wondering if you could kind of expand on that point.
- [14:04]
Jose Merchal
Algorithms primarily want engagement, and they're going to use your preferences as a starting point for that engagement. But they are also going to select out that content, which is salacious, controversial, affective. It's going to make you angry or excited or. Right. So the. In exchange for having the algorithm curate our world, they're going to curate it towards a little bit of our preferences, which is a problem in and of itself, because my preferences today may not be my preferences five years from now. So if I'm locking in to my sort of preference set today, yeah, I live in the world, I may take up new interests. Right. But you'll notice if you go on YouTube, YouTube will sometimes want to bring you back to your old preferences of a few years ago or of a few months ago. Why is it doing that? I don't know. All these algorithms are behind a black box, so we don't know exactly why they're doing it. But one suspicion I have is because a more predictable subject is a better consumer, is something that's easier to sell to advertisers. If we are by nature multitudes, like the poet Walt Whitman said, we contain multitudes. We are very varied and diverse people with a innate curiosity. Just think about who we are as children. We have an innate curiosity about the world. If the algorithm can't fully explain us, if we can't fully, and maybe this gets a little metaphysical, but if we can't fully become an engineering problem, we can't be understood from an engineering perspective, then the next best thing that tech companies can do is to try to make us more predictable, to try to make us act and behave more like algorithms do. And the harm in that is when we are being sort of trained to act more like algorithms and the algorithm preferences Extreme content A great political science colleague, Matthew Hinman did this study about Facebook use. He and his collaborators and he found that and I'm going to get the numbers wrong, but it's a very tiny fraction of Facebook users that produce 70 to 80% of the content that people see on Facebook. And that very small minuscule 2 to 3%, maybe the numbers are wrong, but that 2 to 3% of people that are producing a content are producing extreme content. They're either extremely right or extremely left. They are. And that, and within that context are the context of racist tropes, sexist tropes, homophobic tropes. The majority of people see that and then they see that that's the ground truth. The world is polarized. The world is inflamed. The world is extreme positions. And we both know that that's not the world. Right? The world is much more nuanced. People are much more nuanced. If you have a day to day conversation with most people, this is my experience. I mean, I can't, I can't speak for your experience, Jake. I can't speak for the experience of your listeners. But most people live in much more blended reality offline when they touch grass, quote, unquote. Right? Most people do not have consistent ideological positions. We know this from political science, right? Most people have some conservative views, some liberal views and are not particularly one way or the other. They can be moved, they can be persuaded in one direction or the other. But most people are not the very minuscule 2 to 3% that drive algorithmic engagement on social media platforms. But the more more time you spend on it, the more time you think that's the world as it is. What is most people's response to that? They either check out because it's too nasty, it's Too messy, or because they don't really have very consistent ideological positions, will pick up the extreme viewpoints, extreme opinions about what immigrants are doing, the crime level that immigrants are creating in society, or extreme positions about any number of issues. So I think that's the harm is that we have this contract that we think gives us agency and gives us control over a messy information environment. But what they really do is drive us towards the messier aspects of that environment and in doing so truncate, diminish, shrink our experience of the world.
- [18:18]
Jake
Given this harm, why does the algorithmic contract have such an allure over the United States?
- [18:24]
Jose Merchal
Great question. So I have two chapters that, where I try to, I try to lay that out. One of them is in the United States in particular, the sort of culture of individualism and this belief in the primacy of market capitalism. And this gets into all the great writing about liberalism, neoliberalism and in fact, the next book I'm writing is called Machine Liberalism and it's. University of Chicago is one of the people that's going to publish it. So I'm really excited about that. But we have this tradition of neol, of liberalism, of believing in the ability of the individual through the free market to construct one's life. That the market gives us the best opportunity to create ourselves. And that's connected to liberal like John Stuart Mill, experiments in living. That free market capitalism gives us the best opportunity to engage in experiments in living. And maybe that's true, right? But what the algorithm does is it interferes, it intervenes in that process of experiments of living and gives us this belief that we can completely curate our lives. We can construct ourselves in a way where we don't even have to engage with those discourses that we disagree with. That is very anathema to most liberal theorists. John Stuart Mill said the reason we should have free speech is because that allows us to experiment. We can engage in these experiments in living, a utilitarian perspective, and others can see our experiments in living and learn from them. We need to be engaged with each other in order for that process of self creation to be democratically useful, to be useful in a liberal democracy. There's a great philosopher, Karl Popper, who talked about conjecture and refutation as the main drivers of liberal democracy. So the way it would work is I have an opinion about something, I put it out there to others, and then others decide whether they agree with me or not. So they might refute parts of it. And then I go back and I reflect on what things did they Refute. And are those things true? Do I agree with them? And I adjust my priors, as you might say, right. You know, in game theory, right. I update my priors. You need that interaction with people that you disagree with. But the algorithmic contract gives us is rooted in that belief of we can create ourselves through the market, but doesn't give us that ability to then engage with others in a good faith effort in ways that would be enhancing, democratically enhancing. And then the other chapter talks about the Romantics and the appeal of Silicon Valley and Thoreau and sort of this drive that we have a divine spark that what is Thoreau has this everywhere society conspires against men, that basically the truth is within us and if we go take a nature walk, we'll have this divine inspiration. And it's very rooted in the American tradition. And, and so it also flatters us that algorithms kind of flatter us. And so that romantic tradition of the individual is like kind of having a divine spark is flattering. And algorithms can kind of, we can believe that about ourselves, that we have this divine spark and that we are the center of the universe. And the algorithms can give us that allure to say, yes, you have the divine spark, construct this ontological, epistemological world that you want because you are so valuable. Right. And I think part of what I argue is like, you know, who are we to be so arrogant that the truth that we construct around us should be the truth that we shouldn't, that we don't depend on others, that we don't live in relation in relationship with other people, and that their experience of the world might teach us something too. So I think those two traditions, especially, particularly in the United States, make us particularly, especially vulnerable to the harms of the algorithmic contract.
- [22:32]
Jake
In light of that, why must I become an algorithmic problem?
- [22:35]
Jose Merchal
Yeah, yeah. And I mean, I talk about that in the book, right? Because from one way of thinking about it is like, if we're all outliers, how do we ever engage with each other if we all think we're sort of an island unto ourselves? Right? There's a great political theorist and his name is Escaping Me, but he talks about liberalism as kind of an archipelago, right? But if we're all individual islands, can we ever sort of engage with each other? Right? So I kind of lay that out and I'm like, yes, true, if we are all sort of striving to be idiosyncratic and always looking for novelty, that's not good either. But it's a Balance, right. And I think the balance is tipped too far away from novelty seeking, self creating, authentic self creation. And so in the book I have three ways in which I think we can steer the algorithmic contract towards more beneficial outcomes. One of them is more engagement with serendipity. So there's a Dutch writer, Sebastian Olna, who wrote this great book about serendipity about a decade ago. And so I quote him a lot in that last chapter. And a lot about it is putting yourself in position to encounter surprise. A lot of political theorists, Diana Motz, Liliana Mason, wrote this great book about cross cutting cleavages, about how we used to, two or three decades ago, we used to have multiple organizational affiliations where we encountered difference. So maybe 20 years ago I might have been in a softball league and I might have liked classic cars. So I went to classic car shows on a weekend. And I might, maybe if I have children, maybe I coach their little league or I'm a member of their parent teacher association, or I was in the Rotary Club or the Kiwanis Club. This is the old classic Robert Putnam argument. But it makes sense that in every one of those different organizations I'm going to meet people that I disagree with and I'm going to be having counters with difference. I'm going to be having encounters with surprise that I might not otherwise have if I'm just engaged in my algorithmically curated life. Right. So by engaging with like just people that. And I completely understand that if you're part of a marginalized community, that might be scary, that might be dangerous. And I certainly, I wrestle with it because I don't want to put anybody in a position of danger. I don't want to put anybody in a position where they have to defend their humanity or defend their existence, because I think that's BS to have to do that. But at the same time we need to be put ourselves. It's a cost to not put yourself in situations where you may learn new things about yourself. Because we're all multitudes, marginalized or not marginalized and dominant. We are all blended, diverse people who benefit from engaging with others on safe terms. And so one of the things about the algorithmic concept is, yeah, how do you create moments of serendipity so that we can all learn and grow individually from each other? The other one is promoting fuzziness. And so I really like sort of this idea of fuzzy set. Charles Reagan was a professor at University of Arizona, political scientist who advocated for this idea of like fuzziness in methodology. And fuzziness is Basically, like, instead of seeing the world as a binary. Right. Is 1,0. You see the world as probabilistic. Right? And so it's very natural, you know, very, very natural for modelers and computer scientists and machine learning people to Y probabilities all the time. But instead of seeing ourselves as either members of group A or group B, we see ourselves as, well, maybe I'm 75% Group A, 25% Group B. Right. And this is all very theoretical and like, how you actually implement it is a whole different question. But algorithms have to begin to think about giving us things that are more, instead of saying, well, you like this, so we're going to keep giving you more of that, saying, well, what if you're like, you like you're 60% of this group and 4 and 30% of that group, so let's sort of give you something that's in between, right? And fuzziness is really another word for nuance. It's another word for seeing the world as in its blended reality, in its pluralistic reality. And so how we do that, how we move people out of like, very binary boxes, very binary, like I'm a member of this group into something that's a little bit more fuzzy and blended, is another thing that I care about a lot. And then the third one is I really like Henri Lefebvre, who's a critical theorist. He wrote a very influential essay in 1968 called A Right to the City. And he was critiquing urban design and geography. But his general, he was a neo Marxist. And so he's writing and he's saying, well, if we're never going to have the Marxist utopia in terms of economic relations, can we at least have it in terms of social relations, in terms of space and where people have full autonomy in their use of space, so that space is more flexible, you have more things are public, you have more access to the city. You see examples of this in Los Angeles. In Colombia, they had this thing called ciclavia, and on the weekends they would open up the freeways to bicycles. So the people had a right to. You know, the way this looks in real life is like in the Bogota, where you had a mayor who was in the 90s, the 2000s, who was a big proponent of this. And he made the sidewalks much bigger and the cars lanes for cars much smaller because poor people don't have access to cars. And so they could have access to the city by having more opportunities to walk. Right? So, you know, the idea of Lefebvre is The city should be a work of art. It should give out individuals an opportunity to become an oeuvre. Right? It's French, right? He's going to use French words. An oeuvre, a work of art. Nietzsche did this, right? Nietzsche talked about aesthetics as the Ubermensch. The uber mind should try to live an aesthetically beautiful life. Right? Certainly debatable. Certainly some issues with that. But the idea being that, well, can AI help us see possibilities, help us see how we can live differently? Right. One of the examples I use in the book is Google got into a lot of trouble a year ago because when they released one of their early vision AI models, it hallucinated too much. It hallucinated in a way that we might quote, unquote, call woke. Right? So if you asked it to give you pictures of a pope, it would give you a black pope and an Indian pope. Well, those have never existed, Right? If you asked it for World War II figures, it would give you. Or Nazis, people in Nazi uniforms. It would give you, you know, a rainbow of. Not of people in Nazi uniforms. And it's like, well, that. Or. Or in the Old west, right? It would give you all the. And I was like, well, that's not really. And. And people sort of said, well, this is broken because it's hallucinating. This is. This never happened. And it's like, well, yeah, on one level, if you're. If you're looking for the AI to be factual, then of course this is bad. But if you're looking for the AI to uncover possibilities for you, digital possibilities, potentialities. And those could be left or right. You know, maybe potentialities that bring you closer to God. If you're, like, more on the religious side, but. Or potentialities that encourage you to live. To live in a more utopian, elevated, idealistic way. Right. That we should maybe demand from these tools that they help us spark our imagination and our creativity in ways that unlock potentiality. And so those are the ways that I thought about. I think about, like, well, if we renegotiated a social contract, it could look something like that.
- [30:21]
Jake
Well, Jose, that's about all the time we have today. Thank you so much for coming on the show. If people want to learn more about you or your work or where to find this book, where can they go?
- [30:30]
Jose Merchal
So the book is published by Bristol University Press and you can go to Bristol University Press's website. It's also available on Amazon and everywhere else you might. You might want to. Want to purchase it. You can for me, if you just type my last name in. In California Lutheran University. You get my. My bio page with all everything I teach and write about and. And what have you.
- [30:52]
Jake
So wonderful. Well, thanks again. This has been really great.
- [30:55]
Jose Merchal
Yeah, thank you, Jake. I enjoy.