← Scrolling 2 Death
Loading summary
Transcript51 lines
- [00:03]
A
Welcome to the Heat is On an investigative series by Scrolling to Death in partnership with Heat Initiative. I'm Nikki Petrossi, host of Scrolling to Death.
- [00:10]
B
And I'm Sarah Gardner, CEO and co founder of the Heat Initiative. We are turning up the pressure on big tech. Whether you're a parent, caregiver, educator, or just someone who cares about kids, this episode will give you the facts, tools, and the fire to demand change.
- [00:26]
A
Today, the heat is on. Snapchat. Something huge just dropped about Snapchat. Something that parents everywhere need to know about. So Sarah and I brought in our colleague and friend Amanda Clore to fill us in. Hi, Amanda, how are you?
- [00:45]
C
Hi, it's nice to be here. My name is Amanda Clore. I'm a researcher with Parents Together community of more than 3 million parents across the US and I help understand what the experience children have on social media and other technology platforms looks like.
- [01:03]
A
Incredible. Sarah, do you want to talk about why Heat and Parents Together came together to do this research?
- [01:10]
B
Yes. And also I have to just call Amanda out and brag about her for a moment because her introduction was not sufficient. This is for all the listeners and people watching. I mean, one of the leading researchers in the country of understanding harms to kids on social media platforms. And Amanda's specific approach of experiential testing, which is like going in as a child account and seeing what kids experience, has really blown open the door and exposed how much the companies lie about what they say they do and what they say to protect kids, but what the actual reality is. And so we're really in the midst of sort of like a half investigative reporter slash half academic guru. And so I just had to put that plug in for you, Amanda, because your research has really been groundbreaking in the field, and that's important to recognize. And yeah, I think, like, the bigger question of why do we do this is big tech companies have gotten really good at putting out press release after press release of all the magical things they're doing to the platform to make it safe for kids. And what we realized, like two or three years ago. Amanda, I'd actually be interested to know when you sort of feel like your work began is that no one was testing what they said they did, and so we didn't actually know if anything was working. And that was the gap, and that's sort of been the impetus for so much of this research. But Amanda, tell us, talk to us a little bit about, like, how you arrived at this type of research. And then to Nikki's point, like, why this Focus on SNAP right now.
- [02:40]
C
So I first started doing this kind of experiential research about five years ago, and I actually came to it initially as a mom, not as a researcher. I was. It was in the pandemic, and I was trying to figure out if I could give my kids a device with YouTube kids on it and if they would have a safe experience. And so I was like, well, why don't I use myself as a guinea pig and try this out? And so I signed up for a YouTube Kids account and I started poking around to see what I would see on there and pretty instantly got stuff that was deeply inappropriate for young kids and said, well, gosh, I think I've answered my question here as a mom, but it really sparked a question of if I, as a parent and someone who works in this space, can fall for some of the PR lines that platforms give about how safe their products are for children. I bet there are lots of other parents in the US who are experiencing the same thing and who are assuming that because they've seen an ad about how safe an experience is for kids, that that ad is trustworthy and that their kid is actually going to have a protective experience online. And what we found from a lot of the research is that that isn't the case, and a lot of the safeguards and products that companies are rolling out are. Aren't actually working the way they advertise. And so I think it's helpful to actually have people using these tools as minors and as kids to identify these gaps early on, because I would much rather it be me that encounters some of that stuff than my kids or someone else's.
- [04:19]
A
Can you talk to Amanda how complicated it actually is to perform these test accounts? Because I've thought about. I mean, I've done some of them on my phone, but then doesn't it recognize that I'm actually an adult? Like, how important it is it to do these tests?
- [04:33]
B
Right?
- [04:34]
C
It is important to have a strict methodological approach and protocol in all of our testing. We use clean devices that don't have a history, that don't have any other usage on them. We use devices that are registered to children, and we use accounts that are registered to children. And we document every part of the process. So we document setting up the new account, the birthday, we use the dedicated email address that we use. And that way we have as sort of pure and unadulterated saturated, a beginning to the test as possible. But within that, it's actually quite easy in terms of skill to conduct this Kind of testing. A lot of these platforms are set up so that a 13 year old unassisted can go on and create an account. And we know in a lot of cases kids younger than 13 are going on and lying about their age and creating accounts. So it's a very accessible interface. And when we use these sorts of intense methodological and testing protocols, we end up getting an experience that is as close as possible to what a real teen use would look like, while of course maintaining the fact that there's an adult researcher on the other end and not a minor.
- [05:49]
A
This is so valuable to parents because this doesn't feel like something I or most parents would have the time and the energy and even the expertise and knowledge to be able to, to do. And so thank you ahead of time, you know, and we're, we're going to jump into some of the Snapchat results soon. But why Snapchat? Why now? With Snap, we've looked at a lot
- [06:11]
C
of different platforms over the course of the last few years. We at Parents Together had really been hearing a lot of reports from parents about increasingly concerning things their children were seeing and experiencing on Snapchat. Problems with drug sales and drug advertisement, with being connected to sexual content and sexual predators. We've also seen some data coming out from other resources. I know that Thorne did a study analyzing some data from the national center for Missing and Exploited Children and found that Snapchat was the top place where children were sharing CSAM images and where predators were seeking CSAM images from children. We've conducted some surveys at Parents Together where parents told us that Snapchat was the number one platform where their child had sent a sexual image to someone. And so we really wanted to understand what is happening with the design of Snapchat that is creating these bad experiences parents are telling us about. And like, is it really as bad as they say? We wanted to sort of see for ourselves.
- [07:21]
A
What types of accounts did you set up and can you just kind of walk us through the methodology?
- [07:27]
C
So we started with two accounts, both registered to 13 year old users, one with a male avatar and one with a female avatar. And we set these up. We set up one on an iPhone and one on an Android phone. And one of the first things we saw was that the friend recommendation mechanism worked really differently on our iPhone test accounts and our Android test accounts. So staff chat says that they won't connect children to friends unless they have connections in common. So our phones had no contacts in them. These kids were fake avatars. So they didn't actually know anybody in real life. And on the iPhone, we saw that that behaved the way we would expect. The iPhone got no friend recommendations on the Android app. However, we received hundreds of friend recommendations. We even received two friend requests from people that appeared to be adult men, both within the first 48 hours of the new account being open, and said, yikes, we weren't expecting this. So we started an additional test account on a different Android clean Android device because we wanted to see, like, was this a fluke? Does this replicate? And it replicated the same pattern. So we were able to see across those two Android devices recommendations for about 400 strangers. We which included people who were using their Snapchat accounts to advertise their sexual content on platforms like OnlyFans and X.com and fetish websites and individuals who were advertising drugs for sale in their Snapchat profiles. And these were amongst the very first crop of people recommended to these minor accounts on the Android app. So we had some big questions around what was going on here and why there was such a difference inexperience between the Android app and the iPhone app.
- [09:18]
A
Do we have any hypotheses on that? Sarah, do you have any thoughts? No, I.
- [09:22]
B
But I also, I mean, even just beyond this research, I feel like anecdotally we've heard from there was like an 8th grader I know you've been in touch with recently, Nikki, that I also spoke to, who said that she's been recommended non mutuals and that might have been on an iPhone. So it's like the Android app clearly has a very specific hole right now, like technical hole that needs to be fixed. But I also, I mean, Amanda, I'd be curious what you think, but I, I would then hesitate to also say, like, the problem's been fixed completely too, until we hear more from users themselves as well, that they're really never being recommended non mutuals. And there's also the fact that, like, if a kid had connected with a drug dealer per se, or someone who has an OnlyFans account and then another friend of theirs gets hooked in, they could proactively recommend the drug dealer to that other friend because they have a mutual in common. So it sort of doesn't solve a larger issue of adults and kids becoming friends on the platform in one way or another, like, that would still be a possibility.
- [10:31]
C
I'm wearing my researcher hat, so I don't want to draw conclusions other than where the data leads us. And this is a qualitative study, and it was a relatively small study. So I think that there are a lot of open questions, but a question that I have is do Android users of Snapchat have a different experience than iPhone users of Snapchat? And if that's the case, I think that that is problematic, especially considering the price points difference between Android and iPhone. Then you've got a situation where you've got kids from higher income families potentially having a safer experience than kids from lower income families. I also have a question about how the two individuals who sent friend requests to the female avatar were able to that profile. If 13 year olds have private profiles, which is supposed to be the default setting on Snapchat. I did not accept their friend requests because there's no way to tell if someone's an adult or minor. And we wanted to have a strong ethical line where we weren't going to communicate with any minors during the course of research. So I think that's a thing that is certainly worth additional exploration.
- [11:38]
A
Amanda, normally a teenager who sets up a new account is using a phone that has a bunch of contacts loaded in right on their phone. And in my experience, Snapchat repeatedly asks the user to connect their contacts into Snapchat. Was that something that you saw as you did the setup and how many times were you asked to do that
- [11:59]
C
since these devices had no contacts? I wasn't prompted on this, but I did set up a Snapchat account as an adult on my adult phone. And it was every time I loaded the app, Snapchat would ask me to, to sync my contacts.
- [12:13]
A
So you think if you had a phone with contacts on it, it would have asked you?
- [12:18]
C
It's a little bit hard to guess.
- [12:19]
B
Okay.
- [12:20]
C
But it seems like the, you know, that is probably part of the way the networking effect of Snapchat works is that it utilizes the contacts within the phone.
- [12:30]
B
One of those recommendations or whatever is someone who clearly has an only fans and connected to like their only fans promoting their only fans. What's really been horrifying about this process too is just the types of content and or individuals that are recommended to these accounts are not safe adults to be recommending to kids. Never mind them just being a random adult. These are, you know, we saw a drug dealer be recommended to a kid to be friends with and an adult content creator. So it just makes you question like what is the purpose of this process and are there really any real safeguards there at all?
- [13:15]
A
So after the setup and you either were recommended strangers or not depending on the device, did you then go, Amanda, to the spotlight to the algorithm to kind of see what content was being recommended.
- [13:27]
C
Yeah. So then we spent an hour on each profile watching an hour of content on the Discover feed, and an hour of content on the Stories feed and a logging when we saw problematic content. The problematic content was started almost right away. And we sort of ended up sorting it into five big categories. The biggest was drug and alcohol related content, then sexual content, then violence and gang activity and criminality. We also saw a bit of eating disorder and sort of body shame content, and then self harm and suicide. And across 12 hours of testing across these two different Personas, we got over 700 pieces of problematic content, which averages out to about a piece of problematic content every minute. So it was really, really frequent. And it started almost as soon as we began scrolling through Snapchat.
- [14:25]
A
Oh my God.
- [14:26]
B
Yeah. And I just want to like, I don't mean to make it super graphic for parents, but I do think it is worthwhile like describing one or two of these videos. So for instance, in some of the sexual content that was recommended, one is a girl like graphically describing anal sex. And that you just have to like bear through it being painful at the beginning. This is recommended to a 13 year old. The first 5 seconds of Pep Door are, no, they're awful, they're excruciating. But after that, once you get past that and you grow a pair, oh my gosh, you feel like LeBron James in his prime. And you'll unlock this insane feeling that you have never felt before. And on top of that, he's gonna tell his mom about you the next day. So the ring is coming even sooner. Like that is so messed up.
- [15:17]
C
A lot of the sexual content too was sort of stuff that glamorized non consensual sex and incest. So mimed videos of brothers and sisters having sex or people having sex and finding out their brother and sister in the middle of it Videos glamorizing sexual assault by a stalker. And sort of the dark romance and mafia romance tags around being stalked and sexual assault by a stalker. Interestingly, there were a number of videos from people who create sort of adult oriented sexual content who were complaining about the fact that there were so many children in their comments section and watching their videos and begging kids not to follow them and saying, like, my content is for adults. I don't want minors to see it.
- [16:08]
A
Oh my God.
- [16:09]
C
And so that to me, as a researcher says, there's a real problem happening with the algorithm when people are making content that they're specifically saying is adult content. And yet that is still being promoted to minor registered accounts. We also saw a lot of videos of problematic drug use. People smoking two vapes, marijuana vapes at once, chugging whole bottles of alcohol, really young children using drugs, instructions for how to grow marijuana. We saw a lot of violent fighting and gang activity. Interestingly, the avatar, the male avatar was black. And that avatar got a whole bunch of content around rappers being murdered and violence in black communities that the white coded female avatar did not. We also saw some really disturbing self harm content around cutting.
- [17:09]
A
Not gonna be okay.
- [17:11]
B
It's always gonna be a mess.
- [17:14]
C
What's gonna be a mess?
- [17:17]
B
My whole life everybody wants to get away from me.
- [17:26]
A
Why?
- [17:28]
C
Including people in the comments section sort of planning future suicide dates and talking about why they were giving up and not fighting their demons. And this is all stuff that is being algorithmically recommended to a 13 year old account.
- [17:48]
A
Amanda, I feel sick just hearing you describe that. Can you explain how you felt like seeing this content?
- [17:55]
C
I will say like, as a, like as a researcher I, I do spend a lot of time with dark content on the Internet. But as a mom, like it is terrifying to me that my children are potentially walking into this digital world where any amount of time online is going to expose them to just some of the, the worst and hardest and darkest parts of humanity. And that's a lot especially when you think about, you know, these are seventh graders, these are seventh and eighth graders that we're talking about as these young teens that is a, that is a tough age, no matter what. And to ask them to then sort out, you know, pornographic sex advice and violence and drug use and self harm and like have the sorts of processing that us adults have. To be able to put that content in context is an impossible ask.
- [18:56]
A
We know they don't have that skill yet and it's just the most vulnerable time to be allowing them to access these things. And I wonder what do we think that Snapchat, do they know that this is there and being pushed to children?
- [19:13]
B
Snapchat definitely knows that it has an issue with videos ending up on the platform that violate their policies. The extent to which those videos are algorithmically recommended to children or not is that is the question that we would want to ask Snapchat. Like how much do you know? Because there's a question of is it responsible to have algorithmic recommendations at all if you cannot promise families that kids are not going to be getting this type of content? And I mean honestly, like adults shouldn't even watching the murder videos last night as we like prepared for this podcast like it is so disturbing. I mean, I think there should be a whole conversation about that type of environment as a whole and what it does to humanity and anybody consuming it, never mind children. So putting that aside, Snapchat definitely knows that videos make it onto the platform that violate their policies. The extent to which they know it and how much they know they're getting recommended to children is something they should answer.
- [20:17]
C
Another thing that I stumbled upon in the research that was especially disturbing, that there's a lot of videos of very young children who are being sexualized and sexually harassed in the comments section, where there are users going into the comments of videos of children dancing or doing gymnastics or cheerleading and making incredibly, incredibly aggressive sexualizing comments, leaving telegram handles, advertising things like pizza products, which we know is often used in Internet circles as, as code for child sexual abuse material. So I did end up reporting some of those findings to the national center for Missing Exploited Children. And my hope is that, you know, as this report comes out, there will be greater research and understanding into what is happening in the comments section of some of these videos and whether or not they are providing a safe space for people with a sexual interest in children to organize themselves and communicate with each other. And I would say to parents, don't post pictures and videos of your young children on Snapchat. It's a rough place for that. And I think that there's some real danger happening in the comments section of those videos.
- [21:36]
A
Let's talk about what Snapchat can do and what they should be doing better. And then a couple of recommendations for parents, if there's anything else we want to put in here for listeners.
- [21:45]
B
So, look, the Android fix is like a super easy backend fix that hopefully has to do more with just. There's like a glitch in the product and they could close a hole somewhere and then that would no longer happen. That one's pretty straightforward. The algorithmic recommendations and sort of the horrifying content that we described that's getting pushed to kids. That is really a question about whether they would be willing to turn off algorithmic recommendations until they can better prove or show or demonstrate that kids who are younger who have accounts will not be pushed. That type of content, especially content that violates policy, but even content that is like outside age appropriateness, because that is one of the big principles that we hold as organizations, is that there should be a safety by design element, which is that everything is tested before it's rolled out so that a platform would be able to Say with certainty, we know if a 14 year old is on Snapchat, they are not going to get a video of someone trying to hang themselves, a video of a girl talking about giving a blowjob. Like you have to be able to sort of assure us of that. If you're going to say that you've created a safe space for kids.
- [22:55]
C
I'd really love to see some aggressive and proactive detection of sexually harassing comments aimed at minors and then blocking those users from being able to use Snapchat and preventing them from starting new accounts. It is relatively easy to hire people to look for flaws and sort of stress test products the way that I do in the interest of sort of understanding this for parents. And I think that Snapchat and any of these other companies could invest more in adversarial testing of their products before ideally they go out to children. So we prevent these problems from happening as opposed to identifying them far down the road and needing to go back and fix them.
- [23:44]
A
So these recommendations and others are being made very clear to Snapchat through your work, through this report, through pressure on them and so the responsibilities on them to fix this. It's not on parents, but I would say in the meantime, we already said, you know, Amanda, you shared, don't post photos of your children on Snapchat. Obviously don't allow your children or young teens or I don't know, even teenagers to use Snapchat until some of these problems are fixed. Is there any other recommendations to end with for parents from Amanda or Sarah
- [24:15]
C
based on sort of what we've seen? Snapchat's not a safe place right now for children and young teens. And that if parents do decide to let their older teens use Snapchat to have some really frank conversations with them about things like sexual predators, drug use, self harm, suicide, because that is content that seems like they may encounter on Snapchat. And then we want parents to also feel empowered to reach out, out to Snapchat and other companies and let their thoughts and feelings be known and create that sort of demand that these platforms, who are making so much money off of children using them, are equal partners in safety with parents and families. There's also a petition that parents can sign asking Snapchat to implement safety by design and make some basic product changes to make their product safer for children to use. And I'll make sure that folks have access to that.
- [25:14]
A
Perfect. I'll put that in the episode notes so that parents have something they can do right now to light that fire on Snapchat and get them to make these really important changes. I mean, this is sometimes a life and death situation. We know a lot of families who have been harmed and even lost children due to some of these issues that are happening on their platform. And we need them to fix it.
- [25:37]
C
This is an industry wide problem. This is harmful algorithmic recommendations, harmful connections to dangerous strangers. Sexual harassment of children are problems that are not unique to Snapchat. And that's one of the reasons that I think we need legislation to help ensure that all companies have a basic floor of safety for children.
- [26:00]
A
Thank you for that. And the pressure is on our lawmakers to pay attention to reports like these and work on safety by design legislation. And I know that's in process. I know that all of the lawsuits across the country are also putting pressure on our lawmakers. And, and it's also up to us parents and people who care about children to make our voices heard and be really loud about these issues. We can't stand for it. It's not. I mean, this is some of the most horrific things I've seen yet when it comes to an experience on these platforms. And thank you for your work on this. Thank you for your dedication to it. Thank you for your sacrifices in doing it, because I know this is really difficult to put yourself through. So, Amanda, we're so grateful to you.
- [26:44]
C
Thank you.