← The Tech Policy Press Podcast
The Tech Policy Press Podcast
Inocencia en Juego: An Investigation into Groups Targeting Children on Facebook
00:31:27
Loading summary
Transcript23 lines
- [00:01]
A
Your film is now ready to be shown.
- [00:12]
B
Good morning. I'm Justin Hintricks, editor of Tech Policy Press, a nonprofit media venture intended to provoke new ideas, debate and discussion at the intersection of technology and democracy. Last week on Tech Policy Press, we published the results of an inquiry into how adult Facebook users take advantage of Spanish language groups that form around celebrities and influencers to engage with users who indicate they are children, often below the age of 13. To investigate this phenomenon, Tech Policy Press joined the Latin American center for Investigative Journalism, or L clip, in publishing a report in a series of articles documenting how adult users using these public Facebook groups target children for sexual exploitation. The Innocence at Risk or Innocencia in Juego project, coordinated by El Clip, with participation from Czechiato, includes a report from Lara Putnam, a professor of Latin American history and director of the Civic Resilience Initiative at the Institute for Cyber Law, Policy and Security at the University of Pittsburgh, and a set of independent reports from journalists across Latin America investigating a pattern of behavior on the platform's public groups. This reporting collaboration stemmed from Putnam's research. In 2022, she published an analysis in Wired detailing public Spanish language Facebook groups where children were being targeted for sexual exploitation, although many of those groups she had identified were subsequently removed. In 2023, Putnam discovered a new trove of groups engaging in similar practices, which she documented in a report published by Tech policy press in January 2024. Later that year, Putnam approached Tech Policy Press with additional research documenting how popular Facebook groups centered around Latin American and K pop celebrity fandoms, YouTube stars and influencers became venues for child predation. Tech Policy Press and Putnam brought the report to Czechiado, which engaged with El Clip to coordinate reporting with journalists in Colombia, Venezuela and Argentina. This week, they published their reports in El Clip, Chechiado, Chronica Uno, El Espectador and Fact Checkiado. I'll note that in the course of our reporting, we presented a set of questions to META about these phenomena. A META spokesperson responded with a statement and a link to efforts the company takes to proactively address these and similar phenomena, which I've included in the show. Notes it reads, Child exploitation is a horrific crime. We work aggressively to fight it on and off our platforms and to support law enforcement in its efforts to arrest and prosecute the criminals behind it. Our policies prohibit child exploitation, inappropriate interactions with children, and the sexualization of minors. These rules apply globally in different languages, including English and Spanish, and across each of our platforms. Predators constantly change their tactics to evade detection. Our global teams and tools work to identify and quickly remove violating content. Unquote. In this episode, you'll hear my discussion with two of the principals on this project, the historian Lara Putnam and the investigative journalist Pablo Medina Uribe.
- [03:03]
A
I'm Laura Putnam. I'm a professor of history at the University of Pittsburgh where I'm also the director of the Global Studies center and I'm the faculty lead of the Civic Resilience Initiative of the University of Pittsburgh's Institute for Cyber Policy, Law and Security.
- [03:21]
C
I'm Pablo Medina Uribe. I'm a journalist and an editor at the Latin American center for Investigative Journalism, also known as Clip, and I specialize in investigating disinformation.
- [03:35]
B
Lara, I'm going to start with you. I want to ask you a little bit about how your now years long effort to look at this particular set of phenomena began. How did you first find these groups and why did you start to track them?
- [03:48]
A
So, just as you say, I'm a historian and a historian of Latin American gender migration and I've gotten involved in thinking about technological changes as well and looking at how social media is impacting broader cultural processes. But I'm definitely never intended to become some kind of specialist focused on either child harm or Facebook. I happen to be looking in, in Facebook groups. One of the things I was studying was grassroots organizing of different kinds. And so I'm in the 11th Ward of the city of Pittsburgh and I know that sometimes people form like political groups on Facebook with the name of the war that they're in. So I was like, oh, I should look and see are there political groups for the 11th, 12th, 13th ward of Pittsburgh? And I so I went into Facebook groups and I just looked for 11, 12, 13. And what came up was groups whose name was buscando novia de diez on sedos etrese Anne. So in Spanish I'm looking for a boyfriend or a girlfriend who's 10, 11, 12, 13. And it wasn't just one group, it was multiple groups appeared that had children's ages in the title and were explicitly looking for children of these ages. And then when I clicked through, there was not sort of childlike romantic content. There was a little of that, but it was super explicitly sexual content. There was like if you see your, the initials of your name, I'll send you a picture of my. And then, you know, an eggplant emoji or a pickle emoji. There were like anime style cartoons of naked people engaged in sexual acts. There were gamified things. Anyway, it was all content that struck Me as a mother of four as super appalling. And it was just shocking to me that this was available in a public Facebook group. But I thought, okay, well, now I'm going to do the thing that I should do and just report it online. And surely Facebook has, like, a very sensitive reporting system. So if someone says there's like, check sexual harm to children going on in a Facebook group, this will be taken down immediately. This somehow, this content must have somehow snuck in through what I was assuming were really strong, you know, filters. I mean, after all, Facebook knows everyone, you know, magically can suggest to you every person you went to high school with as a potential friend. So in my mind at that point, I was assuming that there were really sophisticated systems flagging the kind of thing that I was seeing. And then I found that I couldn't get it taken down. And I spent the next three months trying through different means to report items. And also I found that this was not just one group. There were multiple similar groups. So I kept reporting them. I started tweeting about this. I contacted people I thought might know people at Facebook, and I couldn't get anything taken down. And I ended up writing an article for Wired magazine, basically just because I thought, well, at least if I publish, Facebook will have to take action against these kind of groups. And in the short term, they did. But as the article that we've just published together shows, what I've just described for you took place in 2021, 2022. The Wired article came out in 2022. Here we are three years later, and close to identical content is still available.
- [07:01]
B
So for the listener that hasn't quite read your full report yet, and I hope they will click through to Tech Policy Press and look at it, as well as the links to the reports from Pablo and the team from El Clip and across various news organizations in Latin America. Can you just give them a brief sense of high level what we're talking about here? This kind of game mechanic, this mechanism of essentially luring children to talk about their age to engage on sexual matters on a false premise.
- [07:28]
A
Absolutely. And, you know, it's especially clear in the. The groups that I profiled in the article that has just come out with you. I profiled multiple dozens of groups encompassing over 2 million group members. And all of these are public groups that are focused around different, especially commonly focused around different kinds of celebrity fandom for, you know, child stars. And what you see in these groups is accounts, sometimes anonymous, very frequently sort of pretending to be the accounts impersonating These celebrities trying to establish relationships with children. And you can see exactly how they're trying to connect to children who are online and take advantage of the fact that in the current technological moment, there are lots of children with access to a phone or another device that has a camera and is connected to the Internet. And they're trying to convince those children to send them photos, to send photos of themselves. And you could already see how the, even just from the post, they're saying, oh, send a photo. Or I need girls age 6, 7, 8, 9, 10 to do dares who are daring or who are clever or who are hot or who are willing to do special things. And then one of the things that, that I found and is mentioned in the article is in some cases in replies to these posts, people who have interacted with these accounts reply posting, girls, don't trust him. It's not, it's not the star, it's your. It's an old guy who's going to send you pictures of himself naked, or it's someone who's going to ask you for pictures of yourself naked. So it's very clear that what's happening, and this is a pattern that I discovered has been well established by specialist researchers in this field in which bad actors, child predators internationally, meet children in the open online spaces and engage them through luring, through what sometimes is called grooming, to create a relationship. And then they move that relationship out of the public space and they move it into private encrypted channels, usually. And they increasingly pressure, manipulate, persuade children to send them photos of themselves, sometimes videos of themselves, sometimes by pretending to themselves be another child. There's, there's been some publicity in recent years about cases of this where young teenage or preteen boys in the United States are deceived by accounts that are presenting themselves as, as young as girls or girl teenagers that turn out to actually be, you know, men were part of a criminal ring based in, you know, different parts of the world who are then going to take those videos that, that a young man or young boy might have provided and then use that as the basis for extortion, threaten to expose them. There have been tragically, some suicides in the US Linked to this kind of international criminal activity. So again, it's taking advantage of the fact that you can present, you can create an accountant, present yourself, pretend to be another child, or pretend to be a student star and pull a child into a space, into private digital spaces where there aren't guardrails, and manipulate that child into performing acts that they obviously otherwise wouldn't be doing. And that is creating a, you know, permanent video or photographic record. And then you can, you know, sometimes blackmail them with that to continue the process of manipulation and pressure.
- [10:55]
B
Pablo, I want to bring you in now. You know, Laura just mentioned the scale of what she found, which was just by using Facebook's own search bar and being to just enter keywords and terms there. But you all have a little more investigative capability. At L Clip, at the time we started talking, you had access to CrowdTangle. Of course, now that's gone away. But I wanted to ask you a compound question, which is first, what is Clip's approach to investigative journalism and why did this particular set of phenomena appeal to you? To investigate?
- [11:26]
C
Yeah. So our philosophy at Clip is to do cross border collaborative investigations. So every investigation we do, we get partners, usually from more than one country and usually from Latin America, though we've partnered with people in the United States, like in this case, and also some people in Europe, and also we're working right now with some people in Australia and Indonesia, for example. But the main thing is to pool resources so we can all together do a better job than if we were all doing it by ourselves. And so the idea is to collaborate not only with the contents that we produce, but also with the resources that we have. So investigative resources. So maybe one outlet that's partnering has a very good data investigator, so he helps or she helps the other outlets in the investigation, or maybe one of them is really good at graphic design. And for this particular investigation, we were contacted by one of the partners, Chechya, which is a fact checking organization in Argentina and who I think had worked with Holicipress before. They were aware of Lara's previous research on this, and they were aware that you guys wanted to maybe try to expand this investigation a bit more. So they reached out to us because we have coordinated this sort of collaborative investigations. And I was particularly interested because the first thing that I saw is that there were a lot of Latin American users replying in the comments and those comments that Lara was describing a little while ago, because they self identified, because one of the questions they were supposed to answer is like say, your country and your age or something like that, and a lot of them mentioning they were from a country in Latin America and it was very spread out. So I thought it was very interesting that it was not a focalized phenomenon, but it was happening throughout the region and I'm sure it's happening in other countries. We only did this investigation in Spanish But I'm sure it happens probably in other languages. Actually, Lara mentioned in one of her pieces that it happened in Tagalog as well. So I'm sure that happens. But also I was interested in, because in many of the investigations we have done, we have encountered somewhat of a disinterest from the big tech social media platforms about Latin America or in general, about places that are not the United States or Europe. And the fact that Lara had reported this three years ago now and nothing had really happened, just made me think this is just another one of those instances. So maybe this is another thing we can try to investigate. So the users in Latin America are not on their own like that. So there's like someone who's trying to do something for the safety of users of social media in Latin America. So we had Chiquiado from Argentina, who I mentioned already, but we also had El Spectador in Colombia, Coronicado in Venezuela, and we had fact checkado, which is organization in the United States. But they work with Spanish content. They do fact checking in Spanish. What we found using Triangle, as you mentioned, was that the same patterns that Lara was just describing just kept repeating themselves. There's no stopping them. And it was a lot of new groups, a lot of new users, a lot of new content. It was being posted all the time. I think I did that research the week that Crowdtangle closed, which was in the middle of August last year. But when I did it, like the most recent post that hit the criteria was like one minute old or something like that. We found more fan groups of more bands, a lot of K pop bands and K dramas. We also found a lot more groups that were dealing with Pinninger romance or teenager relationships and stuff like that, which is some kind of way to disguise the real intent of having a low sexual content in there. But basically what we found is that there is no mechanism being activated inside the Facebook platform that was stopping this, because this was just continuously growing.
- [16:00]
B
Laura, in our diligence on this, we also talked to multiple experts who've been studying these issues across multiple other geographies. Well, as in the uk, Europe, United States, in addition to the experts that Pablo's partners talk to, I guess I wanted to ask you about what you feel we learned from that process. I think one of the things that really sticks out to me is just the degree of frustration people feel, the sense of constantly being told, you know, that meta is best in class, that it's done, all it can do to address these things while witnessing the scale of what Seemed to be mass vulnerabilities, real vectors for harm.
- [16:37]
A
Absolutely. I mean no one who we talked to was shocked, right? No one we talked to said, oh, I've never seen this material they looked through. They're like, yep, that's that the kind. So the kind of child sexual solicitation that we were finding. And that not only I should, I should add not only finding through super basic keyword searches, for instance for posts including children's ages or asking for children's ages, but also once I would find any one of these groups, if I simply would click on groups like this, recommended groups, the algorithm itself would suggest, oh, here are some other groups that are just like that and that would quite routinely themselves also be. Have been sort of flooded by predators. So it's very clear that the same structures that drive engagement on the platform are, you know, easily available, can be used for these predatorial ends by bad actors. Because it's not that I'm like more diligent than your average predator. And one of the things that was pointed out to us is that the bad actors internationally are very aware of where kids are. They're aware of where platform controls or oversight is weaker. In part this is sort of similar to what Pablo was saying about Latin America not getting attention from social media platforms as a space sort of meriting attention to protection for users. And at the same time, Spanish speaking users are almost certainly the largest number of any language group of users of for instance, Facebook internationally. And so you have people being targeted and you have a sort of an attractive for. For bad actors, you've got an attractive target and you've got the platforms that have created these engage. The tools for engagement are immediately and easily put to ill ends. And that was part, part of what came across through the. When we talked to folks, especially when we talked to folks who have had experience as working in trust and safety on the inside of platforms is that I began to understand the difference between things that look to the platform like tractable problems and things that were just off the table. So that design changes that would make an impact on putting in place sort of fundamental ways of, for instance, generating knowledge about what kinds of harm people are doing. One of the things we talked to one of the folks about was the reporting tools and the fact that the reporting tools are so, they're so clunky in addition to not giving results, they. It's like a seven stage click through process and then you have to wait and see whether you get the response or not. So as opposed to some Parts of the, you know, using a social media platform where you've got the smoothest possible user experience on reporting tools, you've got the clunkiest possible reporting experience. And you know, one of the questions that someone suggested to us that Facebook for instance never really answers is of the people who begin reporting violating content or disturbing content or content that they're, that they, that they feel should be removed. How many people actually finish that seven step cycle? So the kind of changes that would make it easier to report or the kind of changes that would make it that would put a breaker in for for instance accounts that are rapidly friending and by rapidly again super rapidly friending dozens, hundreds of underage users. Those the kinds of design changes that would be protective for children on the platforms would slow down engagement overall on platform. They would, they would hit the engagement numbers on the platform. And so rather than having a safety conversation focused on these design features that would kid, that would acknowledge that there are kids using Facebook and keep. And do more to keep kids safe or to provide real routes for response when kids aren't safe. Instead Facebook really has focused on and keeps the focus on individual items of content that end whether they fall on the line of being violating or not and taking them down. So Facebook, as you said, Facebook says, you know, insist that it's best in class with regard to for instance takedowns of explicit video and photographic child sexual abuse materials. And I, I recognize it's true that Facebook doesn't. The one kind of content you can get taken down if you see it on Facebook and report it is videos that explicitly have videos of naked people doing things if they seem like they could be children. If you report those, those are routinely taken down. And Facebook reports is a mandated reporter and reports to the national center for Missing Exploited Children millions of items of content. But that's the, the, the danger that we're the space of Facebook within the child endangerment and the child exploitative ecosystem is less as a place where individual items of quote unquote child pornography or child sexual abuse materials are exchanged. You don't need that you the. The dark web exists for people, for bad actors to exchange CSAM Facebook is a space where children are vulnerable to being reached and for these relationships to be created and taking steps against that would slow engagement on the platform.
- [21:48]
B
Pablo, would you be willing to characterize any of the observations from experts that the journalists spoke to in the Latin American outlets?
- [21:56]
C
So one of the things that most of the pieces reported was that there is little legal recourse in Latin American countries, when this sort of thing happens, and I don't mean about Facebook, but if a minor is sexually exploited or sexually abused online, most countries don't really have the laws in place that would have this grooming as a specific type of cybercrime. And so a lot of times the cases are really hard to investigate because there's no legal framework to do it. The only exception we found was in Argentina, where there is cyber crime law specifically about grooming. And it's still difficult there because it's somewhat new and like the cases that have. Have been investigated haven't been that much. So still a lot of things that happened. And also the experts that the, the journalists consulted said that people in Latin America are particularly vulnerable because of many things, but especially because the context, the socioeconomic context, there's a lot of people living in poverty here, unfortunately. So that leaves children even more vulnerable. And a lot of people don't really know that this can happen online, being both kids and their parents. So a lot of people don't take the necessary steps to prevent it. And that also leaves kids more vulnerable to this. And so we found out about a lot of cases that ended up with kids, unfortunately, being trafficked into international sex trafficking networks run from other countries. There's not from the cases we found on Facebook, but from the experts we consulted that said that it's something that is unfortunately not that uncommon for people to meet someone online and then being abused or trafficked. It's also really difficult, from what the experts told in this investigations, to create a framework to help this kid, like the victims of these crimes, sometimes because there's not enough government support, sometimes because it's hard to find these victims. And sometimes that's because people are scared of reporting that they happen because they. They don't really know where to go to.
- [24:26]
B
I want to ask you both, what's next? Lara, this has been a volunteer activity for you outside of the course of your normal work. Do you think you'll continue to work on this particular issue? And if so, do you have any thoughts about how?
- [24:40]
A
One thing I have started doing is working with a friend who's a law professor, and she's a specialist in business and human rights law and an amazing, brilliant person. Her name is Jenna Martin. If she's out there listening, and she and I were just talking through the, you know, stuff I was finding the articles I was writing, and we started realizing that there were points where her expertise as a business and human rights lawyer sort of helped me understand what was or wasn't possible with the current existing jurisprudence within the US and then beyond its borders. So she and I have written, we wrote an article together thinking through liability regimes in the Metaverse, where we, we tried to think through, put as one of the cases, the kinds of harm that exist online, the kind of sort of wholly online child sexual exploitation that we've been describing here. I mean, Pablo points to some cases in which people are met online by. By other children, are met online by adults who are sort of physically nearby enough that they can be then physically trafficked or in person meetings can happen. That's horrible. Obviously, that's horrible in its own unique way. But the fact that child children with, you know, cameras, with smartphones can be blackmailed by people halfway across the world, and that can happen at scale that, you know, humanity has never been in a place where that's the case before, where you've brought so many children into a space where they can be contacted by a nearly infinite number of potential predators. And that's just being scaled up with, for instance, AI that allows people to impersonate actors of different ages using translation, using video fakes, and so on. And so one thing that Jen and I developed as a idea or a thing to worry about is what we think of as the veil of scale. The way that bad actors can commit crimes or commit torts acting sort of behind the veil of scale, which is created by social media, where it's possible to create an account, commit great harm, and then disappear in a way that scales super rapidly because of the affordances of a platform. And law enforcement cannot scale at that speed. All the more so law enforcement and the capacity of reach of a developing country, a country like Venezuela or a country like Peru or Honduras is not going to be able to put, you know, 50 trained professionals on the, on the plane to fan out across the world to try to track down folks doing harm to kids from that country online. And so thinking through, there just aren't really vexed issues raised by the, the emergence of new kinds of crime and new kinds of harm that can be committed in these purely online settings. And so that's something that, you know, she and I are, as I said, we have one article coming out in the Yale Journal of Law and Technology, and we're writing another article that asks the question, well, you know, who could Mexican children who've been harmed in the kinds of groups that we're describing, what kinds of redress would they have if they can't, for instance, bring a lawsuit against Meta, because Meta is not based in their country, can they bring a lawsuit against their own government for failure to protect them? Given that, for instance, the government of Mexico's signature to the International Convention on the Rights of the Child has a whole series of laws that protect children's dignity and integrity, their right to live free from sexual harm. So could a child or a class of children in Mexico bring a lawsuit against the Mexican government before the Inter American Human Rights Court, hypothetically saying we had a right to be protected and our government didn't take those steps? I really want to get out of the business of looking at horrible things on Facebook because it's kind of soul killing. But on the other hand, when their kids being harmed, I feel like, and I know it's going on, I have an obligation to do something about it. So I'm trying to reach out through multiple different sources and do different kinds of things to try to move our collective dialogue forward on these issues.
- [28:52]
B
Pablo, what about you? I mean, L Clip is in the business of accountability journalism. Will you continue to pursue this story? I know you certainly plan to pursue tech accountability stories going forward.
- [29:03]
C
Yeah, definitely. We will keep pursuing those types of stories and also I think we're gonna keep an eye on what happens to these groups. After we contacted Meta, they deleted some of the groups, Some of the groups that we mentioned to them, actually, I don't know if they deleted them. They're unavailable now, so I don't know if they went private or they're suspended or what, but they're unavailable, so they're not public anymore. So we're going to keep an eye on that to see what happens to the other groups and the other kinds of posts that we're tracking. And we're going to definitely keep working on more stories about the possible harms to users from the, let's say, the big tech platforms not being very active in trying to moderate their content. This is something that really interests our investigations and we're not going to not just going to do it with Meta. There's a bunch of others that we're going to look into.
- [30:03]
B
I just want to thank you two for the opportunity to work with you on this. As ad hoc teams go, coming together without having worked together before, this is a great experience and I'm really grateful for all the hard work that both of you put into this, as well as to the journalists across the network that El Clip was able to pull together. Thank you so much.
- [30:23]
A
I mean, thank you, Justin. This. This dossier wouldn't have come together without you being there to support pushing this conversation forward, even when sometimes it feels a little bit like we're pushing a boulder up a hill. So can't thank you enough for the leadership you've shown on this.
- [30:38]
C
Yeah, Justin, likewise. Thank you very much for bringing us together, and it was really a pleasure to work with all of you and all of our colleagues that are not on this podcast right now.
- [30:49]
B
For more, see the show notes for links to these pieces on Tech Policy Press and L Clip. That's it for this episode. I hope you'll send your feedback. Write to me at JustInEckPolicy Press. Thanks to El Clip Check I Otto and fact check Iado. Thanks to my co founder Brian Jones and thank you for listening.
- [31:24]
C
Tech Policy Press.