← The Tech Policy Press Podcast
The Tech Policy Press Podcast
Promising Opportunities, Distinct Risks: AI and Digital Public Squares
00:50:37
Loading summary
Transcript49 lines
- [00:01]
A
Your film is now ready to be shown.
- [00:12]
B
Good morning. I'm Justin Hendricks, editor of Tech Policy Press, a nonprofit media venture intended to provoke new ideas, debate and discussion at the intersection of technology and democracy. Could AI help design better, more democratic platforms and online environments for public discourse? What are the opportunities, challenges and risks of deploying AI in contexts where people are engaged in political discussion? Today's guests are among the more than two dozen authors of a new paper on AI and the future of digital Public Squares.
- [00:41]
A
I'm Audrey Tang, Taiwan's Cyber Ambassador and previously Digital Minister.
- [00:47]
C
I'm Ravi Iyer, Managing Director of the University of Southern California Marshall Schools, Neely Center.
- [00:52]
D
I'm Beth Goldberg. I'm the head of the research and development team at Jigsaw, a semi autonomous unit at Google, and I'm also a lecturer at Yale's School of Public Policy.
- [01:02]
B
I'm excited to speak to all three of you tonight, and we're going to talk a little bit about a paper that you all are among the 27 authors of called AI and the Future of Digital Public Squares. It's got ideas in it, drawn from a discussion that involves at least 50 other thinkers and technologists last year, April 3, 2024, in New York City. I should disclose I was one of the individuals in the room for that discussion. Learned a lot from it and looking forward to hearing where you all ended up with the ideas that you developed there. But of course, this is really an effort to think about, can AI potentially be used in pro democratic ways? Can it help the health of the public square? So I want to find out first from each of you how you come at this work, what perspective you brought to that room, of course, in April of 2024, but ultimately then to the paper. And Audrey, perhaps I'll start with you.
- [02:04]
A
I'll start from 10 years ago, in 2014, when I began this work. At that year, only 9% of Taiwan citizens trusted our government, meaning whenever President Ma yingou spoke, over 20 million people disbelieved him in a country of 24 million. And so in March 2014, a trade pact with Beijing was rushed through, which would have invited them into our network and communication infrastructure. And so there was massive outrage, amplified by social media that just arrived on the scene. And so I helped young students peacefully occupy our parliament for three weeks. It's called the Sunflower Movement. So it's not just digital Public Square, it's also literally turning the parliament into a public square. And we call ourselves not protesters, but demonstrators to demonstrate new possibilities for democracy. To emerge in the digital age. And so, concretely speaking, we helped, for example, live streaming debates, building conversation networks, posting daily transcripts from open deliberations involving half a million people on the streets and many more online. And then it just worked. After three weeks, the speaker of Parliament agreed to people's policy demands. And so it was a very rare Occupy that converged instead of diverged. And so ever since, I've been working to make digital equivalents of that process so that people very polarized, very divided, can nevertheless agree on what's uncommon among them, a common ground, or what I call uncommon ground.
- [03:38]
B
And let me ask you before we come to Ravi, has your sort of faith in technology as a tool to create a healthy republic square or to do democratic engagement, has it been shaken to some extent over the last few years? Certainly we've seen major social media platforms present various challenges to democracy, various challenges to public discourse. How do you think about that versus the type of technologies that you've been working on?
- [04:07]
A
A key starting point from the Sunflower experience was that polarization is not an inevitable feature of social media. It is a consequence, a direct consequence of how platforms are designed. So our idea is to pilot new bridging tools such as the open source platform Polis start in 2015 to tackle the heated debate on Uber and so on and so forth. And already by 2020, we're seeing Taiwan's trust approval rate over 70%. So from 9% to over 70%, all it took is this consistent use of bridging systems to reflect people's will together through what I call pro social media, not anti social media. So to your question, No, I Talent's experience showed that it is possible to build social networks that are pro social.
- [04:59]
B
Ravi, that's a good place to bring you in. Sounds like Audrey's observations are somewhat similar to yours. After both spending time inside Meta and now more recently in the work you're doing, how do you come to the questions in this paper on AI and future of digital public squares?
- [05:17]
C
Yeah, so, I mean, I started my career as a programmer who didn't really know what he was doing with his life. And I eventually got a degree in social psychology and studied ended up studying moral psychology, which is the study of why do people do what they do? What are the values they have? Why do they come to have the values they have? I ended up working with John Haidt a bunch because we used to describe how people came to their values as opposed to judging them, which other groups were doing. And then we ended up doing A lot of work on polarization. My first work on polarization really was bringing liberals and conservatives together in the US and trying to study before and after what changed with them. And what we learned was that people don't really change their attitudes, but if you bring them together under the right circumstances, they do change their attitudes about the other side. Right. They still have the same political opinions, but they still. But they believe that the people on the other side are maybe not evil, not as bad as they think they could be, and they could have a beer with them. Eventually, I started working at Meta. I had this career in academia working on polarization. I also worked for this company called Rancor, and I had this, like, dual tech and sort of academic career. And somebody said, well, why don't you work at Meta? You can work on polarization. You can do a lot of good in the world. So I worked at Meta for about five years. I started off trying to define what is the kind of content that is dividing people and how do we, like, get rid of it. And I think the answer I came to was, that's actually the wrong question. There's always going to be conflict in the world. If you try to moderate your way out of it, you end up doing really awful things. You. You end up with the backlash that we see, and you end up over enforcing on people's authentic beliefs. The actual question that we should be asking is, are we incentivizing divisiveness on these systems? Are we almost paying people with attention to be more divisive? I was there when buzzfeed famously came to us and said, we're writing more divisive things because you make us right. We're doing it because it does well in your algorithm. Ben Sasse, conservative politician, he wrote a whole book about how politicians feel this push to be more divisive. And so I think we can design spaces that push people to be better, that people are pushing stuff that they're proud of, not things that they're doing just because the algorithm tells them to.
- [07:24]
B
Beth, I want to come to you and I think about Jigsaw and think about, so the legacy of Jigsaw where it started. Love to hear your perspective on how you've come to this particular moment, this particular work. But I just think about back to its founding, Jared Cohen, the ex US State Department policy wonk, the mission to use technology to tackle geopolitics. It seems like gone some twists and turns over the years in terms of what exactly that means, but this still seems to some extent right in the zone of that initial concern.
- [07:57]
D
Yeah, I couldn't agree more. Asking the question of how AI in a particularly large language models can help us be more democratic online feels part and parcel to why Jigsaw was created in the first place. I joined Jigsaw myself about seven years ago now, and I've been lucky to spend those years in a really interdisciplinary team. I think one of the things that's unique about us is we at Jigsaw, we've got engineers, product folks, we've got researchers and designers. We also work in really close partnership with a wide array of folks in academia and civil society. And so we've been coming at some of these questions, like using natural language processing for how to improve conversation spaces online for many years now. And we've been asking, like, how can we design tools that will help people want to engage in digital public squares and not just lurk or leave the public square when something becomes too toxic or too hostile? And we define digital public squares here really broadly.
- [08:54]
C
Right?
- [08:55]
D
This is not just traditional social media. We're also talking about comment sections, we're talking about private messaging apps, we're talking about anywhere that we're having large scale conversations today and doing the substance of democratic deliberation. And in those spaces, I think we came to this conclusion that we're really not fulfilling this opportunity to be having open democratic discourse, but we saw an emerging set of opportunities recently with the advent of large language models. We can really start to reshape and redesign these spaces, right? Not just to have more voices participate in this democratic discourse, but actually people that have more agency, more influence on those decisions, engage in more meaningful ways. So I. I don't need to tell folks listening to this particular podcast that our digital public squares today are not exactly these utopian ideals of pluralistic democratic discourse. Right? They're not exactly encouraging people like myself to. To stop lurking and start engaging. So I'm experiencing this too. But. But we wanted to really harness these opportunities with large language models that I mentioned. But we didn't want to do it alone. We wanted to develop the opportunities and mitigate the attendant risks that come with them, with a whole range of experts across sectors. So that brings me to the event you mentioned, Justin. Last April, we brought about 70 experts together to New York. And these were folks from academia, researchers, civil society, media, different tech company engineers and designers, and even some folks from governments, like Audrey Reid was there, as well as Ravi. And we asked really big questions about the different ways in which large language models could play constructive role in democratic discourse. We narrowed in on four different areas that we can talk a little bit more about today. But 27 of us then went on to write a position paper which, let me tell you, 27 co authors, what a choice. But I'm really glad we did it because we had this real diversity of opinions and perspectives and we were able to pull together not just a collaborative research agenda of where research fields should go, but also really prioritize opportunities for us as technologists, as civil society, as decision makers, to think about how we can be redesigning our digital public squares.
- [11:15]
B
So my listeners probably definitely agree that there are problems on the Internet at the moment, but there are probably multiple listeners thinking LLMs are the solution. Lots of problems with LLMs as well, around bias around original sin in their training data, all sorts of questions about the ways that they might, you know, harden some of the problems that we face simply by their deployment across the Internet. How do you think about that in going into a project like this, knowing that today's LLMs certainly aren't perfect and in fact in some cases are harmful? How do you think about that as part of the kind of condition for this set of questions?
- [11:59]
D
There's a whole range of mitigations that we're thinking about and that we thought about as a group for this paper and at that convening. Some of them are more technical, some of them are more social. When we think about some of the social mitigations, this starts with really deep and broad listening. Can we do more ethnographic style studies or deep interviews to understand people's relationships with AI, how they're understanding it, how the AI can be better interpreted, can be better explained to them. Now we also need to do broader listening to make sure that more people are actually able to use this stuff. It's not just getting concentrated in the hands of a few. I think some of the solutions are about the ways that we're building to be more transparent, more explainable in terms of why the AI is generating what it's generating. And then lastly, I think there's ways to build that are more interoperable and open source so that people can learn from each other, stress test what one another are building, contextualize these tools for different geographies and cultures more natively. Right. So we'll be able to actually build for different populations more effectively that way.
- [13:11]
A
I agree that not only the tools themselves need to be open source, the ability to work with open source language models goes a long way to address some of the fears around Essentially handing control over to proprietary language model providers. And I also would like to say that for example, in Polis, the tool that we use to uncover the uncommon ground, like 99% of this is good old fashioned AI. So it's K means clustering, it's principal component analysis. It really is not about language models. The only one page that uses language models as we speak on Polis GitHub is the narrative reporting and they use it in an extremely narrow way. That is to say, only uses a paraphraser so that multiple statements by multiple citizens can be summarized into a shorter statement, but all the output bits come come from the input. So you're not forcing the language model to hallucinate. And so I think this kind of very judicious use re narrowing of the scope of the originally maybe too broad capabilities of language model also shows a more reliable way to do evaluations and so on.
- [14:27]
C
I've done a lot of work on comment ranking and so I wish when I had done that work that I had access to the LLMs of today. I think the central observation is that people don't really like crappy comments. The way that people often will rank comments is like, what's the comment that got the most likes? Or what's the comments that gets the most replies? And oftentimes the comment that gets the most likes or replies actually ends up being a crappy comment, right? If I say f you politician, I get like a lot of likes. Maybe some people argue with me, you just don't have a good signal of what's a better comment. Whereas LLMs can actually give you a signal, it can give you like, okay, that's a comment that is, you know, expressing emotional support or curiosity or it's a very thoughtful comment, right? You can ask an LLM like these questions. It used to be really expensive to develop these. You could build a classifier for these kinds of constructs, but it was really expensive and you had to convince a ton of people to do it. Whereas now with LLMs it's a lot cheaper, it's open source. So I think you no longer have to live in a world where you're optimizing for applies and likes. You can now optimize for all these things that LLMs can give you.
- [15:30]
B
And so maybe we'll dig into some of those mitigations a little bit more as we talk about some of the application areas that you imagine in this paper. There are four. You look at collective dialogue systems which call bridging systems, which we've already mentioned today, community moderation, which is on the top of many people's minds at the moment. And then of course, proof of humanity systems. So we'll have to get into a little bit on each one of these. I think we'll spend most of our time on this idea of collective dialogue and bridging. But maybe Beth, collective dialogue systems. What is involved in collective dialogue systems?
- [16:09]
D
We're lucky to have Audrey Tang here, who's actually implemented a lot of these in practice. But I can give a bit of a high level overview. These are actually systems that take a lot of inspiration from peace builders and those who've done large scale negotiations in the past. But for example, this, these types of collective dialogue systems are taking technology platforms, some that you might have heard of, like Polis that Audrey mentioned, and they're taking that qualitative richness, a focus group or a large scale negotiation, and they're combining it with the quantitative scale pooling where you're able to get a large number of people's opinions all at once. What makes them deliberative is when you're able to then iterate and have people work on exchanging those ideas with each other, on finding these emerging areas of agreements and then negotiating them, refining them with, with one another, until you ultimately get to a place where you're really surfacing common ground and areas of disagreement between large groups of people. So there's lots of different types of technology platforms that can get there, some that are much more formal and look a little bit more like maybe an offline citizen assembly that just has some digital components. And then there are modes that look a little bit more just large scale surveys with sort of snapshots of people's opinions that, that get upvoted and downvoted, almost like a subreddit. I think these are super, super valuable. These collective dialogue systems are super valuable for helping us gather collective feedback, right? Like the collective intelligence of a whole group of people on a scale and with a level of nuance that was totally unimaginable before. Large language models, they're overcoming a lot of challenges of cost of information overload for decision makers and participants. And they're really making the most of large language models, ability to parse, analyze, summarize and find the common ground in huge quantities of data. I'll just flag, we've got a pilot that Jigsaw is running now, some partners running one of these collective dialogues in Kentucky and it's, I think it's a really cool example of how this technology can be applied in practice. It's a, it's a purple town. And they're having this conversation right now about what the future of their community should look like. And it's the type of thing that had a citizen assembly or a town hall get maybe a hundred people in the room. Right. But they're able to have many more times that engaging digitally by, by communicating with each other on polis, just like Audrey used in Taiwan. And then these large language models are going to help make sense and moderate this much larger conversation and then elevate those voices to decision makers in the community. So that's a bit of an example of how you could actually run these collective dialogues in practice.
- [18:56]
B
Audrey, maybe I'll just ask a little more about your experience. You mentioned polis. There's also other mentions of different systems in here. Remesh, make.org, all our ideas, Crowdsmart, Citizen Lab, other shots on the goal when it comes to collective dialogue systems, is there a kind of way in which you think about what LLMs open up for these systems and I don't know, potentially any bounds that you see for them, like how big can these dialogue systems get? Where do they start to break down and just become the Internet?
- [19:32]
A
Definitely. So I think a lot in terms of pro social spaces because this antisocial behavior is not a function of any particular person or even any particular community, but rather a function of how they're structured online. And the great thing about conversation networks is that it emphasizes the dialogue. So for example, the four of us, we're now getting real time feedback that as of which of our each statement resonates or not with the room and compare that with the more viral source of social media where people can talk alone to their screen and only the most viral, I don't know, 30 second clips gets amplified. And so you see a very narrow slice sample of what is the most outraged or more strange or things like that. But you don't see the resonance of it. And to your question, I think language models can help to provide this listening experience so that from surprising validators, from people who are unlike me. But actually we do have common ground. It has the way to paraphrase their output, their contents in a way that I can understand in my frame. And I think that is very like this transcultural capability is I think one of the most powerful. But constantly we need to push back against the intuition of this entire in silico deliberation in that we don't have to even listen to one another anymore, that we just talk to our avatars our LLMs and they do the deliberations for us. I think we need to push back that a little bit because that's going to the gym wanting to exercise our civic muscles, but instead we're just celebrating our robots doing the weightlifting for us. It's quite interesting. Pretty spectacular perhaps, but at the end of day, the civic muscle doesn't grow from it. So I think that is one of the bounds that we need to constantly weigh to see what kind of work is translational in nature that enhanced to human civic muscles and what kind of work could actually make it a trophy.
- [21:47]
B
Ravi, anything to jump in on here on collective dialogue systems?
- [21:51]
C
I think the simple analogy I sometimes use to the work I used to do when we brought liberals and conservatives together is you can design a space and start with what people agree on, or you can start with what people disagree on. And when we used to bring liberals and conservatives together, every group used to start, not by discussing healthcare or something that was contentious. They would start with talking about their families, talking about the local sports team, talk about the things that we all have in common. And then you could start get to the stuff that you disagree on. And in some ways the social media systems are designed to start with what we disagree on. That's what we, that's what you get first. Whereas a lot of these collective dialogue systems start with what you agree on. What is, what are the things that diverse groups of people both agree on? And for example, community notes, they surface notes where diverse groups of people agree on. And I think when you take that inspiration from the collective dialogue space and you import it into the social media space and you start thinking, what's a comment that diverse groups of people would agree upon? What if I put that first instead of one that everyone is like replying to? Right? What, how does that transform a space and how does that bring us closer together?
- [22:54]
B
Let's go into bridging, which is related, but this is a kind of a set of ideas that could potentially be used outside of collective dialogue systems and other systems for dialogue. But Ravi, I'll maybe stick with you. You've done a lot of work on bridging. How do you see LLMs being used for bridging?
- [23:16]
C
A lot of what I was just talking about as far as structuring a space so that it starts with what you agree upon is what the practice of bridging is online. So the simple version of bridging is to look for things that diverse groups of people both have some positive opinion about and try to surface Those things as opposed to the things that people are arguing about. And One thing that LLMs can do and AI can do is help figure out what are those things that diverse groups of people do agree upon. It can do it both by simulating the people. It can also simulate the constructs that those diverse groups of people agree upon. So the kinds of things that diverse groups of people find positive are tend to be things that are more thoughtful, that are more curious, that are more supportive. Whether it's trying to simulate the people and their reactions, or whether it's trying to actually simulate the constructs that those people would respond positively to. AI and LLMs can help create a space where we start with what we agree upon as opposed to what we disagree upon.
- [24:16]
B
You had a number of the folks that I think of as some of the experts on bridging in this conversation. There's folks who, you know among the authors, including Aviva Vadya and folks like Jonathan Stray, who's done a lot of work on depolarization. Beth, are you already using some of these bridging concepts in the work you're doing in Kentucky or elsewhere?
- [24:40]
D
Yeah, we are. So some of the systems that Ravi just described are part of these collective dialogue systems. Right. Some of the ways in which these dialogue platforms are identifying common ground is they are actually bridging. They're bridging these opinion groups or different groups who tend to disagree on those really polarizing topics. And they'll identify, oh, okay, maybe you guys all disagreed on the healthcare question, but there's actually a lot of overlap on this question about how do we invest in education. And so they're giving, in a way, decision makers. Maybe some of the lower hanging fruit of saying, hey, here's where there's already built in consensus in your community on where folks agree. The other space that we're already engaging on this outside of Kentucky, is something that we're calling content based bridging. So it's related to what Robbie just described of looking at what bridges groups and individuals. But that sometimes has the drawback of just identifying the stuff that everyone can agree on that's fluffy. So think like maybe groups are really polarized, but they all like cat memes. But we want to avoid a world where we're just a branking cat memes. So we spend a lot of time with researchers who look at the types of features of language that tend to connect people to one another. So the what are comments that build empathy or comments that build connection between groups. And that's stuff like curiosity, compassion, even reasoning and nuance. And so we were able to use large language models to actually build models to identify these nuanced attributes of speech. And then we can go ahead and uprank those, right? So if you've got comments that have a lot of nuance and reasoning, maybe a lot of curiosity and compassion, we can reward you for those types of comments and actually uprank those. It's not something that Jigsaw is deploying ourselves, but we've been working with a number of partners who run, say, common spaces or community spaces, and they're testing them out in their own communities now and saying, can we bridge divides in our community by changing the reward system? Right, Redesigning what our public square is rewarding and up ranking.
- [26:42]
B
Let's talk just a little bit about community moderation. Clearly, there's a lot in the news at the moment about community moderation after Meta's announcement on January 7 that it would try to move more towards a community moderation system looking a little like X. But I want to put that out of my mind a little bit right now just to hear what you think may be possible with community moderation in the future, because you're trying to think through what might be possible with new technology, but certainly also in new context. So not always situating the way we're thinking about community moderation necessarily in the examples that we have before us. But Ravi, I know you're somewhat a fan of the idea that community moderation could work better, could perhaps, maybe even work in. In in coordination with better fact checking. What do you think with when it comes to community moderation? What can do to help?
- [27:44]
C
I prefer instead of the word community moderation, I might just use the term feedback. And instead of having it centralized, where there's one group of people who decides what goes up or down or what people can and can't say. I think it's much better if there's a large group of people providing that. It's better for scale, it's better for legitimacy. And we all live in a world where we get feedback all the time, right? We all I see someone nodding or I see someone shaking their head, and I'm constantly adjusting my behavior to the feedback I get. And it's actually weird when we go into an online environment, we don't get that feedback. And that's where we get some of the problems we have in online discourse, where people are not getting signals that they're turning people off, they're dominating a conversation. And so we can create community feedback that look this is not something that people are appreciating. This is the voice that we want to elevate and I think community moderation is just the start of that. But I think the paradigm of content moderation is not going to solve the problems that we need it to solve. And hopefully if we start to move more towards this, this feedback system, we can get to where we want to go.
- [28:55]
B
Audrey, what about you? Have community moderation played a big role in the experiments that you've been working on?
- [29:02]
A
Yeah, definitely. I just published a paper with my co authors called Pro Social Media that summarizes what we have learned in Taiwan when it comes to open source communities that builds their own pro social media and what that may have a broader impact if adopted by especially decentralized but also now they increasingly, as you mentioned, centralized social media systems. And one of the key insight is to take this bridging idea but instead of applying it just after the fact that like community notes, which only takes place only after something divisive only goes viral and apply it to the main feed. And the great thing about applying to the main feed is that you can see for each post then it's popular with a community or with multiple communities that you're associated with and also to represent fairly the two sides or three sides of division that was your community is debating on this subject. And this basically means that instead of looking at one or two viral memes and mistakenly thought that was consensus when it probably was not, it's a caricature of a polarized camp. You can see a very clear representative of what is actually divisive and what actually is the underlying infrastructure of common knowledge that everybody now can know, that everybody else knows, that informs this debate in a way that's not polarizing but rather focusing. And so think about the bridging system we just mentioned, but applying it to the main social feed instead of just the community notes or the fact checking that was the main part of this paper. Pro Social Media and Beth, in the.
- [30:51]
B
Paper, when it comes to community moderation, the framing of the kind of question for inquiry is how can we empower community leaders to better guide the norms of discourse to support healthy, inclusive online communities? What does community leaders mean in your mind when it comes to community driven moderation?
- [31:13]
D
So these are often the folks who are already forming and shaping and then shepherding their online communities. Imagine a subreddit, discord server, even Wikipedia communities. They actually have a lot of these community leaders who are who are moderating community spaces online already where there is some Centralized moderation. But a lot of the work of shaping a healthy conversation space, shaping a healthy community is being done by these volunteer moderators. And so right now these folks are spending a lot of time and energy working with frankly, pretty clunky tools for the most part. There's often like pretty brittle collections of banned words or tools that are giving them binary. This crosses a line. This doesn't cross the line for each comment. And so it ends up being a lot of manual labor for these folks to be moderating and keeping their spaces safe. When my team did some deep ethnographic research with a couple dozen of these community moderators, they told us they really don't like this, this job of having to shield their communities from the trolls and the disinformation and the other sorts of hate speech. What they really enjoy is shepherding what they describe as more like the pro social elements of trying to foster healthy conversation, doing the stuff that, that's uplifting the good. And what they really wanted was more bespoke tooling. They wanted large language model type tools that allowed them to say, let's automatically filter out this type of content that isn't welcome in our space and then let's uprank this type of content that we want to reward and see more of in our space. And so we're just getting to the technology that's allowing them to have that level of nuance to really shape and steer the types of communities that they want to have online.
- [33:00]
B
I want to get to the last area of query here, which I think is maybe the most kind of forward looking one in a way, this idea of proof of humanity, where the question is, with advances in AI making it increasingly difficult to tell humans from machines online, how do we balance the competing interests of privacy, free speech and authenticity? So I guess the flip side of we can use LLMs perhaps to make better collective dialogue systems and platforms. How do we tell if it's humans that are engaged in that dialogue? This reminds me, I had a group of students a couple of years ago in my Tech, Media and democracy class. And we come up with various prototypes in that for ways to think about how to build a more democratic technology environment. And these students came up with a project which is basically the Internet, but only for people. And their whole idea was that eventually we're going to be in a place where it's going to be quite hard to tell sometimes who you're interacting with. But I don't know, how do we do that? How do we know that we're engaging with humans, but maybe also preserve some of the things which we know empower expression, particularly in difficult circumstances or in authoritarian circumstances. Things like anonymity or the ability to at least obscure your identity.
- [34:24]
A
I think there's a range between like pure anonymity, which makes it very difficult to tell a bot from a human, or pure, like real name systems where everybody have to digitally size everything, including video and live stream and things like that, which creates its own problems in terms of coercion and also power asymmetry makes whistleblowing much harder and so on. I think in the paper we develop a few ideas that is in the middle of this spectrum. So you can think of as meronimic systems. That is to say, it reveals a partial identity without revealing the full identity. A classic example is actually the age signals that many jurisdictions, including Australia, are now developing. The idea is that you need to show that you're over a certain age in order to, I don't know, drive or purchase addictive substances, including social media. But if you overly disclose the real name, then that creates a horde of problems, a lot of which will reside in state surveillance, basically. And so the Australians, like the Taiwanese, invested in the infrastructure structure of what's called selective disclosure, so that you can only, for example, prove that you're over a certain age, but not your birthday or where do you live, or you can prove that you have interacted previously as a human with this community without exactly specifying what was the piece of content that you have posted, and so on and so forth. So the idea is that everybody will then be able to establish some sort of proof of humanity credentials without overly disclosing really anything else besides what is required to show to the group of people that you are participating as a community member. So a lot of zero knowledge, cryptography and so on is going into this.
- [36:22]
B
One of the things that I definitely want to make sure we address and that I did find quite interesting in the conclusion of this paper. The paper seems to recognize that try as you might, as much effort as you might put into collective dialogue or content moderation or bridging. There are certain situations where the only answer is to send a pretty strong signal that someone is not engaged in meaningful or productive conversation. You have this paragraph where you say some discussions involve actors who attempt to inflame tensions, scapegoat others, engage in intimidation or amplify known misinformation. This is especially true when discussing issues germane to historically marginalized groups who may not wish to engage in communities where such tactics undermine their humanity. And dignity. Part of cultivating healthy public squares is de escalating such conflicts and removing bad faith actors. How do we draw the line there? Because you're talking earlier about the idea of sending signals, of trying to avoid, of course, that circumstance where you end up having to take such a drastic action as to essentially tell someone you're no longer a part of this particular collective dialogue. How do we think about designing systems that perhaps limit the number of times that we have to do that? And I don't know, Ravi, is this maybe a good question to put to you?
- [37:55]
C
I think the first thing we have to do is stop rewarding people for being bad faith actors. Right? So right now there is some amount of monetary reward in the form of advertising that you can get by being a bad faith actor and saying the most outrageous thing you want, regardless of political persuasion, it could be about health instead. Right. It doesn't have to be political. And so we just need to stop that incentive in order to expect that there will be less of that to have to clean up afterwards. The second thing is I think we need to have feedback or some way to hold people accountable. So it's, it's not. Most people, like most people in the world do not want. They're not trying to dominate a conversation. They are good faith actors. They, they care about other people, they're willing to listen if we just can do something. And we all know how to deal with that small group of people who are bad faith actors, right? We know how to deal with that in everyday life. We send them bad signals, we stop inviting them to our parties, right? How do we do that online? How do we stop inviting those people to our parties online so that the rest of us who are of good faith and who want to bridge divisions, who want to understand people on both sides can have a meaningful conversation. Right? And so if we can stop incentivizing the bad actors, stop like letting them dominate and then maybe stop inviting them to our parties, maybe we can start to have a better conversation.
- [39:15]
B
Changing those incentives might require a little more of a revolution than you're letting on. But Audrey, are you. How do you think about this and some of the systems you've built about removing bad faith actors?
- [39:28]
A
I was involved in the prototyping of VTAI1, so that was more than 10 years ago now. A way to build pro social systems in which the bad faith acts are de escalated but the good face parts are amplified, even though it comes from the same person. So you can see troth like gradually getting reformed and Actually, I practice this on a personal level too. My hobby is what's called troll hugging. So some people hug trees, but I hug trolls. And the way it goes is that during my cabinet tenure of seven and a half years as digital minister, sometimes I just see thousands of words from a person on social media, making ad hominem attacks, toxic attacks, and so on, on me and my identity and my policy, so on and so forth. But I can construe a constructive meaning out of maybe five words out of this 3,000 word rant. And so I would just focus on that and reply either through this group chat or through quotes, that is a quote that is only of the constructive parts and then engage very meaningfully with it. And so it shows the bystanders that the flaming words does not have any reach. Indeed, in the original design of VTAI 1, we simply limit the breach of those parts and then only the parts that can be constructive become the topic, become agenda setting. And so for many people, after a while they learned that only constructive acts of performance can get genuine attention that they crave. And that is what most trolls are coming up with because they really seek some sort of resource response. And the social media, the antisocial corner of it anyway, rewards the kind of behavior that are more fringe, more polarized and so on. So as soon as you can detect that, and I do use language models to help to find out those parts that are constructive and also construe a response. And so it reduces my emotional labor and then makes it easier to engage in a good faith way, the design.
- [41:48]
D
Of our spaces can set the norms for these actors. And so we can very intentionally redesign these spaces to either reward the bad actors or quote unquote, help incentivize them to become bad actors, or we can disincentivize that it's really a design choice that we make. For example, we know from a bunch of studies now that the comment that gets pinned to the top of a comment section often is the de facto norm setting for all the comments that are to follow. And so if that's a really trolly comment, you're going to see a whole bunch of trolley comments to follow because that's what people think is expected there and is welcome in that space. Right? That's the type of party that they're trying to have in that, I'm not space, to use Ravi's metaphor there. But if the party host of the facilitator of that, that comment space, maybe the creator who's moderating that, says, hey, that's not the type of party we're going to have. I'm going to pin this much more constructive comment that completely changes the conversation. Another good design example is removing the reply button. And I know for a lot of these collective dialogue systems they let you post comments and then uprank or downrank other folks comments, but they don't let you reply to them. That's a really intentional design choice. Macedon does this as well. And what it does is it encourages users to post their own individual ideas so you actually get a wider range of unique contributions. And it's discouraging people from just rebutting and trolling one another. And so I actually think that's that reply button is a much more powerful design choice than maybe we often think about.
- [43:26]
B
So my last question comes to this question around recommendations for future work. So you do make some recommendations at the end of this paper. Almost more kind of suggestions that this is somewhat uncharted space. We're talking about using artificial intelligence to shape human discourse. There's a lot of questions here about trust, about the extent to which people will trust these systems, will trust AI. I'm sure there's someone listening to this right now that says that's thinking. These people are talking about social engineering. They're talking about using artificial intelligence to change politics or change the way we engage in politics. You call for, of course, transparency. You call for diverse perspectives on this work. But I don't know, how do you think about the kind of real world circumstances where these types of ideas are being deployed? How do you avoid it simply being seen as another form of kind of technological meddling in human politics?
- [44:30]
A
Social engineering has a meaning in cybersecurity, meaning to use psychological manipulation to trick people into making security mistakes and giving away sensitive information. And that is indeed a failure mode for this kind of work, right? Because if you make a opaque language model and then you make opaque applications of it, and then you detect people's affiliations, the groups they associate with the most, and then you strip mine it only to sell individualized advertisement. That really is a kind of social engineering, which is by the way, also a business model. And what we're working on now is not just transparency in system design, but also making sure that people have the components of those tools so that they can deploy it themselves. What I like about, for example, Jigsaw's work on sense making is that it's not opinionated about which language models they use or which inputs modalities they prefer. It's just out there as open source, so that when people want to analyze a particular conversation, they get to determine exactly how it's done. And like I work with the collective intelligence project who basically look at the jigsaw sense making re implemented a large part of it. But because it's open source, we get to keep for example, the topic subtopic detection and things like that. And so basically the idea here is that it's like Lego bricks. And instead of designing one central monolithic system that all the community have to align to the logic of AI, this is very lowercase technology, civic tech that people can then take as individual Lego brocks to build systems to their particular community that hopefully federate together as we argue in the prosocial media paper. But even if they don't, that is still something that helps the community moderators, the people who chef the conversation, to reduce their cognitive labor.
- [46:37]
C
Any design choice you make is going to change a system. So there is some amount of engineering that you are doing that is going to change the system. And there is some amount that has already happened that has created the systems we have today. And there's some amount that you would be changing with some of the recommendations we're making. I think the big question is who are you engineering for? And so my hope is that a lot of systems are engineered for businesses to make money and they reward like tiny groups of people who enjoy these arguing online. And that's not most people, right? Most people stay out of it because it's not worth it to them. And can we engineer something that's for not for us, but for the user, right? Can we engineer something? And there's a lot of people who, you know, feel manipulated by these systems, they're turned off by these systems. And so regardless of whatever point of view ends up winning, just having something that people feel proud of, they're posting things they're proud of, they're learning new things, they're connecting with other people to some degree I'm agnostic about what they're actually connecting about or what they're actually learning. But just having helping them with their aspirations rather than engineering for a business or the tiny group of people that sort of wins the arguments online.
- [47:46]
D
A thorough line that I'm hearing in what both Audrey and Robbie are saying is getting people more agency. And if I want to think about some of the sub components of what does it look like to give people agency, especially if they're not developers, they're not doing what Audrey did and refactoring some of these open source Tools to their own liking. They're adopting the tech that's out there. What does it actually mean to give them agency? And I'll quote Zoe Weinberg here, who talks about three different types of agency that people can have. One is the choice over the tech that they use, so that they're not locked into one specific type of tech, but they can actually switch between them and understand their option sets next step in context. So being able to make more informed decisions, not just having to, to just take the tool as it's coming to you, you can actually go in and understand how to change the settings yourself. And lastly is control. We talk a lot about control over data, but actually control over like the whole experience that you're a part of, being able to opt in and opt out. So it's basically making sure that, yes, people are still excited by maybe the AI magic or the platform that they're a part of, the conversation they're part of, but it shouldn't feel like they're having a black box that's happening to them, that they're actually able to choose and have control over the tools that they're working with. One way that we can think about Getting to these three Cs is actually through co design. This is something that my team is thinking a lot about. How can we make sure that these large language models are being built with the affected communities that are ultimately going to use them? And I think that's where we need to start if we're ultimately going to be building tools that get to a place that feel transparent, explainable, accessible, et cetera.
- [49:34]
B
I look forward to seeing how this work plays out in the future, both from a research perspective, but also how it's put in practice. Both experiments that Jigsaw might run, things you might get up to, Audrey prototypes you might build. Ravi. I appreciate the 3 of you spending some time taking us through this. If folks want to find this paper, they can Google AI and the future of digital public squares. I'm sure they'll find it on an archive or certainly in the show notes for this episode. So I appreciate the three of you taking the time. Thank you so much. That's it for this episode. I hope you'll send your feedback. You can write to me at justinettechpolicy Press. Thanks to my guests, thanks to my co founder Brian Jones, and thank you.
- [50:25]
A
For listening.
- [50:34]
D
Tech Policy Press.