← The Lawfare Podcast
The Lawfare Podcast
Lawfare Daily: What’s Influencing Politics Online? X’s Algorithm, Creators, and the New Persuasion Machine
00:47:56
Loading summary
Transcript51 lines
- [00:01]
Bleacher Report App Advertiser
The Bleacher Report app is your destination for sports right now. The NBA is heating up, March Madness is here, and MLB is almost back. Every day there's a new headline, a new highlight, a new moment you've got to see for yourself. That's why I stay locked in with the Bleacher Report app. For me, it's about staying connected to my sports. I can follow the teams I care about, get real time, scores, breaking news and highlights all in one place. Download the Bleacher Report app today so you never miss a moment.
- [00:30]
Strawberry Me Career Coach Advertiser
Let's be completely honest. Are you happy with your job? The fact is, a huge number of people can't say yes to that. Too many of us are stuck in a job we've outgrown or one we never really wanted in the first place. But we stick it out and we give reasons. Like, what if the next move is worse? And I've put years into this place, and maybe the most common one. Isn't everyone miserable at work? But there's a difference between reasons for staying and excuses for not leaving. It's time to get unstuck. It's time for Strawberry Me. They match you with a certified career coach who helps you get from where you are to where you want to be, either at your existing job or by helping you find a new one. Your coach helps clarify your goals, creates a plan, and keeps you accountable along the way. Go to Strawberry Me Career and get 50% off your first coaching session. That's Strawberry Me Career.
- [01:28]
Filine - Researcher on Algorithmic Feed Effects
In fact, in our study, we show that if people start using the algorithmic feed on X as opposed to the chronological feed, they do change their political opinions, and they switch their political opinions to the right.
- [01:46]
Renee DiResta - Lawfare Podcast Host
Rene it's the Lawfare Podcast. I'm Renee DiResta, contributing editor at Lawfair, with Nathaniel Lubin, founder of Insight Studio, Survey 160 and the Better Internet Initiative, and Professor Helene Widmer of the Paris School of Economics.
- [01:59]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
There's lots of effects happening all the time, everywhere. And your exposure to these platforms is sort of a layering effect of lots and lots of different nudges. And any individual nudge might be very small or might even close to zero, but some of them are having effects, and in aggregate, they're having big effects. And so that point, I think, is the thing that is what's most important
- [02:19]
Renee DiResta - Lawfare Podcast Host
Today we are talking about social media and influence. The algorithm or the influencer. What's shaping political persuasion?
- [02:27]
Bernard - Podcast Host / Interviewer
One of the things that I really appreciated about your two different papers was that I thought that they Brought together and highlighted two different aspects of layers of influence online. Right. So one is the platform itself. What happens when the feed changes? When the feed changes, what people see. So Filine, yours, I think your paper in Nature really focuses on that aspect of it. And then Nate, yours, what happens when people follow particular types of creators, creators that they trust? What is the role that individuals play in influence and shaping political opinion? And so I was hoping that we could talk about the intersection of these two things because this question comes up a lot actually. Is it the algorithm or is it the influencer? Or is it both? Is it a combination of both? Nate, you were the co author of How Mass Creators Shape Mass Politics, a field experiment during the 2024 elections. And then Filine, your paper was the Political effects of X's feed algorithm. Maybe we can start by going into your respective papers, summarizing them and telling the listeners a little bit about them. We'd start with Nate.
- [03:32]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
Yeah, absolutely. So the paper is still a pre print on how social media creators shape mass politics set field experiment. So it was from the 2024 context. So for my part, we were working on the program side of this originally from a nonprofit called the Better Internet Initiative, which is A Fellowship for 501C3 nonprofit fellowship for content creators to integrate pro social material that is fact checked and accurate into their feeds. And so because that program is sort of longer term in nature, we were able to plan ahead to do this research. And so in the second half of 2024, we worked with some other colleagues on the paper to do a sort of longitudinal assessment of how a subset of those participants in that program, as well as some other participants outside the program, you know, what the effect was of exposure to following those combinations of creators during that period. So there's a very dense amount of information in that and a lot of different ways to cut it. But the kind of punchline is a couple of things. One is in the control people who spent more time online during that period had some shifts in their views and perspectives relative to people who were in the kind of basic control group. Second finding was that interventions that were done pretty much universally all had quite strong effects relative to what we would have expected. So they did shift their perspectives and knowledge of issues and some understanding of the world based on that. And then the main sort of comparison point of the paper was looking at what we called culturally or apolitical creators versus more politically oriented creators and people who ordinarily talk about politics writ large. Not necessarily partisan politics, but partisan politics generally. Versus people who are mostly focusing on other issues. And then this program was getting them to include more substantive material. And what we found was that on a sort of per video basis, the more cultural, less political people, those creators were more influential per video, so they had much larger treatment effects. When you did look at the data
- [05:45]
Bernard - Podcast Host / Interviewer
that way, can you talk a little bit about how this intersects with, you know, your understanding of political influence and influencers over time?
- [05:53]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
Yeah, I mean, this program or this research, you know, is based on sort of a larger project around thinking about how influence works. And algorithms also very much a part of that. So these are, as you pointed out at the beginning, mirror images of the same sort of question. So we think about trying to improve information landscapes. Understanding the algorithms and the sort of architecture of the platforms is one way to get at that. Another way is to change the lived experience of what humans on the platform are actually doing in different contexts. Those could have more or less effect. So this. This is not kind of a either or kind of way to think about it. It's another way to cut that. And so from a kind of program perspective, BI Initiative was saying it would be great if some of the incentives were a little bit different. But we're going to, in the meantime, try to help people who are interested in taking this challenge on do that more directly. For my part, we've thought about creators for a long time. I worked at the White House 10 years ago now, and so we were bringing in creators to the White House for the first time in 2014, 2015, to states, the Union and some other events. So that was sort of the genesis for me in thinking about other kinds of ways to get information into front of people. This sort of mental model of influence here is repeated exposure of any kind can have the kind of potential to persuade. It doesn't mean just because you're in someone's feed that it is effective, but it means you have the potential to do that and sort of thinking about the kind of linear sum of all the different influences that are happening. That's kind of the underlying theory of this.
- [07:28]
Bernard - Podcast Host / Interviewer
That's a perfect transition. I thank Feline into your work. I'd love to maybe summarize your paper.
- [07:33]
Filine - Researcher on Algorithmic Feed Effects
And yes, I'd love to give you an overview of our research. And so we start with the observation that algorithms decide what billions of people see every day. But at the same time, somewhat surprisingly, this question of whether feed algorithms shape political opinions hasn't really found a quantitative or a conclusive quantitative answer in the previous scientific literature. A Prior study found that turning off the algorithm on Facebook and Instagram had no detectable effect on political attitudes. Which is a bit puzzling because anecdotally we have suspected for, for, for quite a while now that feed algorithms matter for political opinions. And in fact, in our study, we show that if people start using the algorithmic feed on X as opposed to the chronological feed, they do change their political opinions and they switch their political opinions to the right outcomes that we measured were on policy priorities. So what do people think the government should address? Should the government prioritize, say, healthcare over immigration? We asked how people thought about the criminal investigations into Donald Trump that were ongoing at the time in summer 2023. And we also asked people about their opinions on the war in Ukraine. But what was most puzzling to us is that switching the algorithm off didn't reverse those effects. So for people that due to our study stopped using the algorithmic fee, we didn't find that they changed their political opinions. And this asymmetry between the two effects, it also explains this puzzle from the previous literature, because the study that I mentioned before, the previous study only looked at one direction. So they were only able to study what happens when people who had previously used the algorithmic feed. So what happens when these people turn off the algorithmic feed? And in our case, because we were studying X that already at the time had this choice between chronological and algorithmic feed, we could study the treatment in both ways. Right? People who came to the study and using the chronological feed, and we could randomize some of these people to using the algorithmic feed. And people who came to the study and using the algorithmic feed, we could randomize some of them into, into using the chronological feed. So we can really see what happens in, in both directions. And now a question you might have is, so why is there this asymmetry? And what our data suggests is that it's driven by the accounts that people follow. So if you are using the algorithm, you also see content from accounts that you don't follow yet. And then you start following these accounts, and when the algorithm is turned off, you don't unfollow these accounts, meaning that this like, list of followings, this, this network on, on X that you have built under algorithmic inference is going to stay there even if you turn the algorithm off. Yeah, and maybe one, one thing that I would like to add is we don't find effects on an political polarization. So this kind of feelings of like a warmth to the in group and out group, but we do Find these effects, as I mentioned before, on political opinions, on current events. And of course, this is speculative. We were only following people for seven weeks and we already found that they changed their political opinions on current issues if they started to use the algorithm. So, of course, this begs the question, because in reality, people use these algorithms for months or for years. This begs the question, what would happen in the long run to more deeply held opinions? And one thing that's worth noting is that we combine different types of data. The first type of data is surveys at the baseline and end line and where we ask people about their political opinions. But we also collect data on what people actually see under both settings. So what does a person see when they go on the algorithmic tab? What does a person see when they go on the chronological tab? Plus, we also collect the accounts that people follow, which is. And this combination of these different types of data allows us to explain this mechanism behind the asymmetry. Another finding I would like to emphasize is that in this analysis of what the algorithm promotes or demotes in that case is that we also looked at news outlets, and we find that the algorithm demotes news outlets. So those are much less likely to appear in, in the algorithmic feed and the chronological feed. On average, around one fourth of the posts there from news outlets, compared to around 12% in. In the algorithmic feed. So this is really a stark decrease in. In the presence of news outlets, which obviously typically follow different standards for their content, like fact and other types of things. And this might not apply to political activists that are upweighted a lot by the algorithm.
- [13:09]
Bernard - Podcast Host / Interviewer
The asymmetry is really the striking part. I wonder if we could maybe explain that a little bit. So switching the algorithm on, you see an attitudinal shift, but switching it off then does not lead people to shift back. And you talk a little bit about this, maybe you can explain why you believe that is.
- [13:26]
Filine - Researcher on Algorithmic Feed Effects
Yeah, exactly. So that's true. That's probably the most striking finding. And so I'm coming back to these different types of evidence that we gathered, and what we can show is that when you turn on the algorithm, you start following these conservative political activists. But if we turn off the algorithm for you, and you are still continuing on a list of followings that you build while on the algorithm. So, meaning that we switch. If we switch off the algorithm for these respondents, they still have the same type of accounts that they follow. And I think it's interesting from a policy perspective, because there's this discussion around the right of opting out of the algorithm. But if, if we take this asymmetry that we discover seriously, it casts some doubt whether this is sufficient given the stickiness of the effects. And of course, we can't rule out that eventually and people would, would also see effects in the other direction. But what we can clearly say is that if this happens, it's much slower than the change of opinions due to this turning on of the algorithm.
- [14:34]
Bernard - Podcast Host / Interviewer
Right. So you're seeing the feed shifts, the network essentially that people are following and then that network kind of outlasts the. You know, it persists even after they're. Even if they toggle to something, to something subsequently. What percentage of them significantly changed who they were following during that time? Because one of the things that's very interesting on platforms like Threads now and on platforms like TikTok is that who you follow doesn't actually matter very much on certain platforms in the sense that they have this concept of unconnected content, right. Where this is essentially the, for you algorithm where they're just going to push you stuff anyway. And so what I have noticed on platforms like Threads is that people don't bother to follow you. They just assume that they're going to see you. And you hear TikTok creators talk about this too, or that question of what. Actually, I see Nate kind of nodding along here also. What actually motivates the follow, right? What leads that sort of holy grail of the persistent connection to actually form. I'm curious what you see in your research.
- [15:39]
Filine - Researcher on Algorithmic Feed Effects
So I can't give you specific numbers of like a share of people. I can tell you that this switch, it leads to 0.1 and 0.2 and standard deviation change in the probability of
- [15:51]
Renee DiResta - Lawfare Podcast Host
following these certain type of accounts.
- [15:53]
Filine - Researcher on Algorithmic Feed Effects
To come back to your, I guess, like, broader question, obviously this asymmetry that we discussed, it only makes sense in a world where there is a chronological feat, right. And in a world where we only have an algorithmic feat and everything is just like we learn from your pattern, from your attention. This kind of deliberate choice of do I want to see more or less of this person? It's. Yeah, it's not the right question to ask. And I think it's actually very important consideration for, for like platform regulation in some sense because there's always this question about like, what would I like to see and what do I watch? Because there's also. So that's not our study, but there's a, There's a growing literature on how these algorithms exploit and some biases that we all have as, as humans like sensationalist content, perhaps very like emotional. Emotional content. And so there's this thing of being like hooked by things that if we were asked to deliberately choose, what type of experience would I like to have? What would I be like, nudge. Like to be nudged to on this platform? So yeah, this asymmetry is a bit specific to your question, where there is some form of like deliberate choice of Yes, I want to see more of this. And I think in the meantime on X, they've already changed. Like when you go to the, to the following tab now. And there's also some algorithmic curation in that now. So you have to like deliberately choose to like only see the ones that you follow. The broader takeaway. And here I'm obviously moving beyond what we show in this study, but something that I would find consistent with what we find is in some sense the fact that we find this stickiness. It could easily apply to other things too, because you just said, like, people know I'm going to see more of you if I watched your video, so I'm not following you. That's in some sense also a learned behavior. So whatever these algorithms do, I think this more general idea of these behaviors could be sticky. And then these kinds of the type of content you see or your opinions or how you go about the platform, we learn like the humans, we learn how to interact with the platform. So I think just generally cautions us a bit to look at these things very mechanically. Like we turn on algorithms, we turn it off, everything is reversible. I think it's something that we should be a bit more cautious with when thinking about policies that would improve the online experience on these platforms.
- [18:14]
Bernard - Podcast Host / Interviewer
Let me ask one more question that maybe ties into this question of feeds and removals you're probably familiar with. I believe you guys reference, in fact, some of the studies that were done, the big meta studies meta the platform, they were in science. But these studies that were also looking at the effects of turning off or shifting feeds and the what seemed to be non effects there. Do you want to talk maybe a little bit about the sort of intersection there? Because I think they had also found that there was, you know, essentially no depolarizing effect to ceasing to consume kind of algorithmically curated content. And the, you know, kind of public takeaway from that, the media takeaway and the coverage at the time was that that meant that there just wasn't an effect to the feedback. You showed something very different. And this led to some very interesting discussion in the social science community. About the difference in these findings, particularly as Twitter shows this, this very strong pull to the right in your findings. How did you, how did you think about your work relative to that, that prior work that had suggested that, you know, oh, it doesn't matter, turning it on, turning it off, it's all the same thing.
- [19:23]
Filine - Researcher on Algorithmic Feed Effects
Yeah, I think it's really, it's exactly as you, as you put it, Bernard, co contradicting that earlier study because what they looked at was turning off the algorithm and they showed that this doesn't impact your, your political opinions and they find exactly the same thing. And even though it's a different platform, a different time period, so it's really about this asymmetry and it ties back to what I said before. I don't think we can mechanically think that turning on an algorithm is the same as turning it off. And that's what, what our empirics suggest. And so in that sense we're, we're very consistent with what they found. It's just, I think for us there is a bit of a deeper question and going forward and also in future studies, what kind of research questions should we ask? Because we also have a bit of information asymmetry with respect to these platforms because as researchers it's typically very hard to know how these recommender and algorithms, these feed algorithms work. It's relatively hard for us to get access to data. So just, you know, like it requires a certain amount of imagination by the researcher to, to think about, okay, why could there be not an effect if you turn it off, even though we have all this anecdotal evidence. And so in that sense, I, I think it just highlights that it's, it's also very important the kind of like the, the details of the research and questions that we, that we asked. And another difference, like, very obvious and is that they look at the meta platforms and we look at X. And in that sense, from a scientific viewpoint, the effects are always specific to the given platform, to the given time. But of course, one does wonder in what we find, is there something general, would we expect to see a similar Right word pull on other platforms too? Maybe that's the right moment to cast on the word again. To Nate, I think you also do speak with this question in your paper.
- [21:15]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
Yeah, I think a couple of reactions to that one is I think we totally agree with the challenge in reconstructing feeds after the fact and people have already been influenced by prior exposures. I mean the larger architecture of these platforms includes a whole bunch of different inputs right and so I think, I think as an example that people often forget in these contexts is the algorithms often are recommending people to follow or accounts to follow when you create an account. Right. Or are introducing recommendations that are not the recommendations of the algorithm in the feed as part of the larger architectures more broadly. So if a platform like Twitter just to take, or X just to take a random one, happens to have Elon Musk be promoted all the time, that might have an outside effect outside of feed itself. In the same way that back in the day when Facebook, old school Facebook would recommend Barack Obama when you launched account almost all the time, that had an effect as well. So these things can work in different directions. I think that larger architecture around what people want is the thing that is missed in a lot of this. The Affiliance paper is amazing for showing a kind of contextual moment about a political effect. There are some other papers I've seen that are arguing that there are engagement versus chronological differences in people's satisfaction with the feeds and. Right. And so a few of us, in coordination with Georgetown, wrote a paper called Better Feeds, which was arguing a bunch of things again, but basically that people should have more control over choosing what they want and that the incentives should be around the effects of exposure to these architectures broadly. And I think that's very consistent with what Fleam is just saying, that if you just focus on the input side rather than the effect side, you're going to miss most of the effect that you're concerned with.
- [23:05]
Bernard - Podcast Host / Interviewer
So, Nate, let's. Let's focus in then on influencers for a couple of minutes. So your study suggests that predominantly apolitical creators may be especially potent messengers because they are seen as more informative and trustworthy. I'm curious, can you talk a little bit about that aspect of the cultural creator component?
- [23:22]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
Absolutely. So the distinction here is. So I mentioned before, the apolitical creators are predominantly talking about topics that are in no way considered political. And in fact, we did a classifier of the content feeds of the different groups and there's a table in the paper that describes the ratios. So these ones are in the range of 10 to 20%. This was during the height of a presidential election. That's actually a quite low number relative to the ones that were classified as political. More than three quarters of their content was considered that way. So that's the setup of this. We talk a bit in the paper about potential mechanisms, and that's a little bit more speculative. We don't have as much direct evidence for what the causal reasons for the difference in effectiveness are or persuasion amounts are. But we have some indirect indications here that are quite convincing. So one was people were following, continuing to follow the cultural creators, less political group, much more after the study period ended. And so you sort of see this kind of connection that was made outside of the incentives that we were introducing as part of the research. We also see sort of a pattern here where the more political groups seem to be having their effect based on frequency. So the number of videos that are actually shown during the intervention that are political is much, much greater for the political group than the apolitical group, which makes sense. Those are channels that are talking about this anyway all the time. And so you sort of see this frequency and much more like advertising, where that repetition is very likely the cause for the more political group. There are many other research directions we're interested in going in future work based on this. So diving deeper into the potential connections that people are making, kind of parasocial notions that are in the literature is something that we're interested in.
- [25:07]
Lawfare Founder / Gusto Advertiser
Hey, folks, I want to tell you a story about the founding of Lawfare. I started Lawfare, and it was just a blog, and then we realized we had to create an organization to support it. And all of a sudden I found myself doing paperwork, forms, logistics, personnel stuff. It just completely ate up my day. And I want to say I was bad at it. And it was repetitive, it was boring. And I thought to myself, there has to be an easier way to do this. I didn't know about Gusto at the time, and in retrospect, I wish we had. Small business life means hustling and figuring it all out a lot of times on your own. And I did it a lot of times on my own, and it was bad. But you don't have to make the same mistakes I did. You don't have to spend your evenings guessing at tax forms or tracking down onboarding documents. Gusto handles all of that so that you can spend your time on the parts of your business you actually love, like, in my case, running a magazine about national security and law. That part I love. Gusto is an online payroll and benefit software system built for small businesses. It's all in one, remote, friendly, and incredibly easy to use so that you can pay, hire, onboard, and support your team from anywhere. You save time with automated tools that are built right into the system. Offer letters, onboarding materials, direct deposit, and more. It's automatic payroll tax filing, simple direct deposits, health benefits, commuter Benefits, Workman's Comp 401k, you name it. Gusto makes it simple and has options for nearly every budget. It's quick and simple to switch to Gusto. Just transfer your existing data to get up and running fast. Plus don't pay a cent until you run your first payroll. So try Gusto today at gusto.com lawfare and get three months free when you run your first payroll. That's three months free payroll@gusto.com lawfair one more time gusto.com lawfair.
- [27:47]
Siemens Advertiser
Need real insight from industrial data Versit with a single source of everything and get the best outcomes. Transform the everyday with Siemens the Bleacher
- [28:02]
Bleacher Report App Advertiser
Report app is your destination for sports right now. The NBA is heating up, March Madness is here, and MLB is almost back. Every day there's a new headline, a new highlight, a new moment you've got to see for yourself. That's why I stay locked in with the Bleacher Report app. For me, it's about staying connected to my sports. I can follow the teams I care about, get real time scores, breaking news and highlights all in one place. Download the Bleacher Report app today so you never miss a moment.
- [28:31]
Strawberry Me Career Coach Advertiser
Let's be completely honest. Are you happy with your job? The fact is, a huge number of people can't say yes to that. Too many of us are stuck in a job we've outgrown, or one we never really wanted in the first place. But we stick it out and we give reasons like what if the next move is worse? And I've put years into this place and maybe the most common one Isn't everyone miserable at work? But there's a difference between reasons for staying and excuses for not leaving. It's time to get unstuck. It's time for Strawberry Me. They match you with a certified career coach who helps you get from where you are to where you want to be, either at your existing job or by helping you find a new one. Your coach helps clarify your goals, creates a plan, and keeps you accountable along the way. Go to Strawberry Me Career and get 50% off your first coaching session. That's Strawberry Me Career.
- [29:28]
Bernard - Podcast Host / Interviewer
There's these studies that I've read that have come out arguing that polling, I think maybe is a better way to put this. Not studies, but that people are just disinterested in news, that they're just disinterested in politics, that they're kind of burned out when they're on social media. They're there for for entertainment. And this is one of the reasons why apolitical? Supposedly content performs so well. Do you hear anything like this either from the creators that you work with directly in your prior work or that was reflected in the study? Its.
- [29:57]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
I mean, I think it's a little bit context dependent. Right. So during this period, which again was very high political salience period, I don't think it's likely that politics was not driving attention at a high level. Certainly that was quite pervasive. I think the concerns that most of the creators have is more the opposite. Right. The kinds of. For the apolitical sort of cultural ones. Right. Again, this was not partisan interventions, nothing like that. Right. The. It was a nonprofit intervention, but they were concerned about causing backlash with their audiences. They're concerned about getting things wrong or being attacked for being wrong, that kind of a thing. And so a lot of the intervention value is getting them comfortable, getting them having a capacity to do that well and accurately. So I think if you imagine recreating the study in a different context, where that wasn't the sort of heightened environment you could imagine, maybe it would look different. But yeah, it's hard to know.
- [30:53]
Bernard - Podcast Host / Interviewer
So, Nate, your paper also suggests that trust and parasocial connection are central to this. Can you talk a little bit about that? Maybe? What exactly is it? This comes up quite a lot in, even in academia, honestly, this not, not from the standpoint of how should we be studying it, but just what is that? What is it that thing that creators have that institutions or campaigns or legacy media and news increasingly do not?
- [31:17]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
So I think there's two dynamics that we're getting at in the paper that we're thinking about. One is to have the chance to have a persuasion effect of any kind. Condition for that is that you have to have seen the thing, right? And so that is threshold. That's obvious, but perhaps not always considered, which is why I think a lot of the most important work here for this more cultural dynamic happens because these people who are doing that kind of content creation have large followings and they're commanding attention. And so if they choose to use that capacity and that power to express something, then they have the ability to persuade. Right? And so if the audiences of those profiles are very different than the more political groups or other kinds of groups who are maybe to first, you know, first approximation, speaking to the choir, speaking to people who largely agree with them, their capacity for influence might be larger. The second dynamic I think you were speaking to in the question is particularly for repeated exposures where they sort of have, you know, for the channels we're following is more important or the legacy view history is more important. That repeated exposure might make make audiences more likely to believe them, more likely to take what they're saying on a kind of emotional level seriously. And depending on what they're saying, that might make what they're saying more credible and they make it more, more believable that what they're saying is not cherry picked or inaccurate. Again, in this research, it's a little bit hard to know exactly why things are happening. You just see the output effects. But again, the kind of indirect causal things seem to be consistent with what you would see if you were thinking about this as a parasocial connection theory for why the more apolitical creators are effective.
- [32:57]
Bernard - Podcast Host / Interviewer
There's an interesting set of thinking around why the news accounts didn't perform as quite so well on Twitter in general, which was that the news accounts didn't ever respond back. Right. And this is, I don't know if you've ever noticed, but they don't reply to you. And so you would have. I remember Elon actually gloating about this when he first bought the feed because I study influencers also. I would look at these differences and how they, how they communicate and you know, just what the, what the styles were and political influencers in particular. And one of the things that you would notice is that anyone would mock the New York Times about this. Oh, it's got no engagement, meaning it's got these massive follower accounts. But there's no back and forth when it tweets, it doesn't get the same amount of engagement as when, you know, cat turd tweets. And the reason that is in part I think is that when the algorithm is waiting responses, you know, when the algorithm is waiting engagement, prior engagement, when the algorithm is waiting, likelihood that there is going to be some back and forth that other people will be able to hop in on, right? To participate in, to make the platform feel like a participatory conversation. This is one of these areas where I feel like the news accounts in the feed, they're, they're not as well suited to the medium if the medium is trying to privilege conversation, if the medium is trying to privilege social engagement. And, and that becomes something that is a real challenge because it does mean then that you're going to be getting your news through the most, you know, charismatic, evocative person as opposed to an organization. I don't know if you have thoughts on that, but this was something that came up as I was reading.
- [34:36]
Filine - Researcher on Algorithmic Feed Effects
I think this is an excellent point. And I don't think, or at least I wouldn't read our study as like engagement maximize. Even if like certain kind of changes in political opinions were driven by like engagement, I don't think engagement is bad per se or a platform that drives, that drives engagement. And to a certain degree I think there's also like deeply funny, deeply informative content on social media. So there's clearly the potential of this being something fun and informative. I think the question is a bit like when it starts to replace other sources of information for certain types of information which might by nature be a bit less, less interesting or less, less engaging. I think it starts to have effects on, on the Democratic debate. And it's a bit the same discussion around fake news. Right. Sometimes reality just can't catch up with what you could potentially invent. So in that sense, right now I think we're a bit in this extreme where engagement is really like the bread and butter of these platforms. And there might be additional like specific kind of interest of like owners or platforms or other like big stakeholders, but the engagement maximization is clearly a big, big part of it. Maybe perhaps we're a bit in an equity room where there's just too much weight on that because the public information sphere, the public debate, engagement is not everything. Right. I think if we thought about it from a more like what would be the kind of wish list of how we would like a public debate going? I don't think that engagement would be the only thing that we would put on that list of the ideas that we would have about how a good Democratic debate would go on. And also this being said, a lot of it is not political. Right. And people share memes about cats, etc. So our concern is specifically about how are we organizing this huge marketplace for ideas that have an impact on politics.
- [36:36]
Bernard - Podcast Host / Interviewer
So we had these two kind of areas of focus. We have what did the machine rank? Then we have who do I let into my feed repeatedly, because I like them. Do you think about this is kind of a question for both of you. Do you think these papers point to two different forces or do you think one is upstream of the other? How should we be thinking about the intersection between these two things?
- [36:58]
Filine - Researcher on Algorithmic Feed Effects
Yeah, and so I think I've hinted at that at the very beginning and I think the two papers are very complementary because in some sense I think economists call would be measured sometimes sort of like a reduced form. So you measure the effect of the algorithm on these opinions. And then the next question is, but how? And so in our paper, we do go this extra step and we showed that one very plausible hypothesis that we can support by the data is it's the, it's really the, the accounts that you follow, it's these contents that you're exposed to. And I think that's exactly what Nate is showing in his paper. Right. If you, if you start following certain types of accounts, they, they will have an influence. There might be some limitations to external validity of this. Right. If the account is like super boring or something, and you might be asked to follow it, but it wouldn't impact you. But let's assume it's an account that is like reasonably engaging, if you're more likely to see it, there will be an effect. So I do think, taken together, they'll help us understand a bit better the mechanics. So these content creators matter. You can be, even if you're kind of pushed to follow new accounts through exogenous forces. In our case, it's the algorithm that you're assigned to by us through the experiment. In your case, if I understand it correctly, it's a researcher administered like induced. So the researcher induces this new following and we see that it really changes. And I think it does caution a bit against this like, you know, idea that recommender algorithms are just like helping you to find the content that you were interested all along. And in that sense they're politically neutral because they're just showing you what you wanted to see all along. I think both paper, they showed that what you consume online, even if changed like through random forces, it changes your political opinions. And I think this makes it just much harder to defend this political neutrality of algorithms. I think then when we take into account how big these platforms are, we know that they influence your opinions. It just raises some questions about whether we want to stay a bit in this kind of wait and see equilibrium. I mean, there's a lot of political action going on, but I think largely it's still relatively passive. Yeah, I think just these two things taken together, for me they highlight the need for a much bigger debate about how we want these platforms to function, and especially when it comes to politically relevant information.
- [39:34]
Bernard - Podcast Host / Interviewer
Did you see in the argument? The Substack newsletter, lakeisha. Lakeisha Jain wrote a very interesting essay called Twitter is not Real Life. Did you, did you happen to see this one go by? It's a for. Okay. For listeners who maybe didn't see this one. The Argument is the outlet. They did a series of graphs, series of charts from polling data that they pulled looking at x As a platform. Now this is a little bit after your study feline finding that X news consumers are notably more conservative than other platform audiences at this time. Right. So they note that for example ICE popularity on X is close to break even there even while they are wildly unpopular on other platforms. And overall that Donald Trump enjoys a high popularity rating still net popular among Twitter news followers. And the claim is not only that audiences self sort but also that X under musk. You know, it intersects with your. Your point that as X pushes users rightward, there is a. He kind of puts it as like a, a vibes problem for people over indexing on the platform. He's arguing that X at this point has become a conservative platform. This is reflected in work that we saw this at Stanford Internet Observatory back in the day. We saw like Gab's user base had gone back to there, right? Truth social user base had gone back to there because this was a platform where they could do things and get reach that they couldn't on those smaller niche platforms that they had kind of decamped to for a while. They now had a moderation environment favorable to them. And so it was a very interesting argument that he makes. But I think so they're sort of markedly different I would guess is the takeaway than the median voter. But one thing that that this raises is this question of when we look at platforms today, how should we interpret studies as? You know, I think there's still a bit of a legacy perception among many people that platforms are still where everyone is. Right? That Facebook is where everyone is, that Twitter is where everyone is. And increasingly when you actually look at the demographics on the platforms, that fracturing has occurred and that is not true. How do you both think about this?
- [41:43]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
I mean, I absolutely agree with that. I think there's also a massive distinction between the content creator profile or the active engagement profile of users on a platform like X versus the passive consumption group. Right. And so that is a recipe for real challenges representativeness when you do a survey context, if you can't balance for that or know the prior population amounts in some independent way. And obviously the platforms don't provide that kind of data at high fidelity by default really ever, if they ever did. And so it's very hard to do an assessment in that way with credibility. But I absolutely do think in much the same way the TV audience selecting which cable channel to watch, we're seeing a similar pattern in terms of what kind of channel and then what kind of universe within that channel you opt into. YouTube probably is the most pervasive from all, all groups. It's probably the most important channel, despite what people would say. But your experience as a recommendation experience can be wildly different depending on what part of the world you're. You're focusing on. And so I think, you know, the kind of power law dynamics of what experience is on the platform is the thing that I would point to there. Right. So there are, and whatever world of, of, of a network you look at, there are going to be a handful of very large channels or very large influence vectors of whatever the account type is or post for the. Based on the platform that at any given time are having lots of influence. And so that's sort of a dominant faction. But then within that any individual person is having these parallel feed experiences that are really important. And so both are true. Right. There are large effects that are having aggregate effects at max scale and then there's also, you know, individual effects. And so for any researcher or any person thinking about what's going on, you have to sort of account for, for both that, you know, each of those scales happening, which is hard because we don't have an independent reference frame to look at it with.
- [43:44]
Bernard - Podcast Host / Interviewer
I think the, there's also the argument that there's the elite effects phenomenon that happens on Twitter, right. Which is that there is still a disproportionate number even as the user base has shifted. Right. Even as the influencer base, as you note, has shifted. Right. The creators are a little bit more skewed that it does still shape political opinion by virtue of its role as a breaking news, a place that people still tend to gravitate to for breaking news. The fact that the left hasn't necessarily congealed in any one particular other alternative place. What do you think about the elite effects argument with Twitter as. As its continued, you know, salience is a place for, for political discourse not only in the US I know that this is true in Europe as well.
- [44:23]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
Yeah. I don't know if our study speaks to it. I could say as someone who spent a couple years of my life with weed deck open 100% of the time and felt the pain of that. I still think Twitter has an effect in much the same way for the left that CNN does. I turn CNN on when there's a election result or a war and pretty much never else. And I think that's kind of the way a lot of consumers now treat X when it's not their day to day. There's the daily users who are there. But then it's still a thing that you have on your phone that when there's a moment you want to see something you might go to. So I think, I think you're right that at high salience moments can have an effect. You know, there might be cases where that feed exposure can have an outsized effect in sort of framing a debate at a high salient moment. That said, I would be cautious at over extrapolating because I think we over index for those moments salience relative to all the rest of the time. I think a commonality of both these papers of papers is that there's lots of effects happening all the time everywhere. And your exposure to these platforms is sort of a layering effect of lots and lots of different nudges. And any individual nudge might be very small or might even be close to zero, but some of them are having effects and in aggregate they're having big effects. And so that point I think is the thing that is what's most important.
- [45:42]
Bernard - Podcast Host / Interviewer
What do you think about the implications from your respective findings on the what should be done front? Do you think that there are policy ramifications that come out of your takeaways here? Do you think that there are media literacy arguments or what do you want the either policymakers or regulators to understand about what you've found and where do you think platforms should be going?
- [46:04]
Filine - Researcher on Algorithmic Feed Effects
I'll jump in here with a very kind of somewhat self interested comment because doing a study like ours, it takes a lot of public research money, a lot of time and then we have one study to document the effect of a fleet algorithm that billions use every day in like one context with 5,000 people. And I don't think this is sustainable given that the impact that these platforms have. I think we need a much better infrastructure to create transparency. And some steps are being done in this direction with the DSA for example. But there's still this like general problem of like first of all who decides which which kind of questions are audited? And there should be some organized procedure of like understanding what are the questions that we want to audit as societies, how can we get the data to audit them? How can we potentially even like have some like an access to the code of these platforms and not just like selected questions of the code that the platforms themselves decide to share. But really like how can we get these insights? Because for us any study is just like one puzzle piece, right? I think there needs to be some much more systematic ongoing monitoring. There's the question about transparency for users, like what right do I have to Understand what will happen to me if I use the algorithm for a long time? Am I just like supposed to kind of figure out as I go or should I be able to choose much more? Should I be able to, I don't know, eventually bring along my own recommender algorithm that I can plug in in the platform? So I think there's many, many shades of this question about transparency and for me from a research perspective, so I'm not a policymaker, I'm a researcher. That's one of the things where I see a huge potential to create better knowledge, to create more knowledge that will help us and to, to create much more informed policies. And not just like the policies per se, but also like the enforcement instruments like in practice. How do we make it work? Not just what would we like to regulate, but really how can we also make this work in practice? How will it unfold specifically so that people, at the end of the day, they have better platforms?
- [48:15]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
I mean, I would absolutely agree with the transparency point. I mean in the American context, the Platform Accountability and Transparency act was sort of the closest that we got to a theory. Didn't get over the line, clearly for anyone who's followed that debate. I mean, I think more broadly we've been talking in this conversation about political persuasion. That's not the area that I would be focusing regulation on at all. I would be thinking about the health effects, given the attention and issues there where there's much more consensus and legacy from a product liability standpoint. So the frameworks that we looked at before, I think we should apply to that, which is to say, let's not worry about from a regulatory perspective, inputs that people are creating. Let's worry about the output effects of exposure. And there are ways to run randomized control trials and longitudinal assessments for population exposure, particularly for protected classes like kids, and have do no harm principles associated with that. And if a platform can figure out a way to not cause a problem, that dimension while optimizing for engagement without any other dimension, then great, good for them, they should do it. But if they can't, the reason why they should stop is not because engagement feeds are bad per se, it's because they would have an effect on a thing like kids health. And so we should be focusing on the actual, the actual thing.
- [49:36]
Bernard - Podcast Host / Interviewer
So holding policy and outside and you know, regulators and all that, holding all of that aside, if you had a magic wand, you know, how would you redesign the feed tomorrow? Or what would you change? Would it be an incentives for creators, the feed itself, the public's ability to choose and understand how both work. Like, what do you think is the one most impactful thing that we could be doing here?
- [49:56]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
If I had a magic wand, I would hope that the product leadership of all of these companies shift their emphasis on their optimization timelines from short term to long term and include metrics that are associated with societal health rather than individual time. And so societal health, to me, things like social trust or human connection metrics that are well grounded in the social science literature. So if they had a do no harm principle over the long term associated with those kinds of metrics, I think we would be in much better shape.
- [50:29]
Filine - Researcher on Algorithmic Feed Effects
I would like to add a small thing to this and I very much agree with this one. I would also add that I think many owners of the platforms now, their AI companies are the same ownership. They also have AI companies. So my magic wand would also focus the calculus of these companies when it comes to social media on the actual information sphere. Because one concern one might have now is that if you have a company that is also very, very active in other sectors of the economy in like artificial intelligence and more generally defense, etc. I think it's just much harder to understand this. There's some, like, can we think of them as some having like, some sense of like public service that you might have if you're just kind of active as some like editorial force, like a newspaper. So I would also like to see more. I mean, we're talking about the magic wand, right? So I don't have to be specific about how this exactly would go, but I would like to have these things a bit more separated because I feel like increasingly they're getting these kind of, of huge players that are active in many different sectors and they impact billions of people in like many sectors at the same time. And I think this makes it very, very hard to keep track of what's actually happening. What are, what are the real incentives, what is really happening. And so I would, I would find it helpful if this was it more clearly separated and in that sense, I think less concentration of power also.
- [52:03]
Bernard - Podcast Host / Interviewer
So then I guess to summarize our conversation, what we've gotten to is that what these papers suggest, kind of taken together, is that online political influence is not just about messages, right? It's about systems and relationships, what the platform decides to show, who people decide to trust, and how repeated exposure across both can shape what feels salient, what feels believable and what feels normal.
- [52:26]
Lawfare Founder / Gusto Advertiser
Right?
- [52:27]
Bernard - Podcast Host / Interviewer
It feels normal among a community. So then the question is not, you know, whether social media affects politics. It's where that power sits and what it means for a public sphere that is increasingly organized around feeds and creators and that interplay between them. So I want to thank both of you, Nate and Filine for joining us today. Thank you for a really thoughtful conversation. Great to have you both in the lot fair pod.
- [52:49]
Nate - Founder of Insight Studio and Researcher on Social Media Creators
Thanks for having us.
- [52:54]
Renee DiResta - Lawfare Podcast Host
The lafair Podcast is produced by the lafayer Institute. If you want to support the show and listen ad free, you can become a LawFair material supporter@lawfairmedia.org support supporters also get access to special events and other bonus content we don't share anywhere else. If you enjoy the podcast, please rate and review us wherever you listen. It really does help. And be sure to check out our other shows including Rational Security, Allies, the Aftermath and Escalation. Our latest Lockheart Presents podcast series is about the war in Ukraine. You can also find all of our written work@lawthermedia.org the podcast is edited by Jen Patyar with audio engineering by Kara Stone of gotwodio. Our theme song is from Alibi Music and as always, thanks for listening.
- [53:47]
Siemens Advertiser
Want to change the efficiency game? AI it automate tedious tasks to spend more time on the future Transform the everyday with Siemens.