Loading summary
Transcript48 lines
- [00:02]
A
Welcome to the Heat is on an investigative miniseries by scrolling to death in partnership with Heat Initiative. I'm Nikki Petrossi.
- [00:10]
B
And I'm Sarah Gardner. Today, the heat is on Apple.
- [00:22]
A
Apple revolutionized personal technology. But when it comes to child safety, Apple has fallen short.
- [00:28]
B
Its parental controls have long been criticized as confusing and inconsistent. Now Apple is rolling out new child safety features and promising better protections for families. But will they actually work?
- [00:41]
A
And while Apple promotes new safeguards, one big question. Why is the company still refusing to proactively detect and remove known child sexual abuse material that they store in their icloud? If one of the world's most powerful technology companies can't find a way to protect both private privacy and children, who will? So, Sarah, today we're going to narrow in on a specific problem that you and your team and colleagues have been trying to solve. Has it been a decade? I want you to take us back to when you first realized that Apple has a problem with child sexual abuse material.
- [01:20]
B
It was during my time when I worked at Thorne and we were auditing all the different ways technology companies were combating online child safety. And I honestly will never forget the day that I first learned that Apple wasn't doing any detection of csam. Because all the big companies at that time, and it's funny to hear the names, Facebook, Twitter, MySpace, Microsoft. Yeah, were actively engaged in a real conversation around how to set, find this kind of content and make sure it didn't exist on their platform. And I remember saying, oh well, what about Apple? Because they host a ton of images and videos. It was in the context of a conversation around companies that host images and videos. So another great example is like Dropbox. Dropbox also hosts videos and images and they do see SAM detection. And then I remember whoever I was talking to at the time being like, oh, well, Apple just doesn't engage on this at all. Like they just won't. That was such a gut punch because when we think about our lives and where images and videos are stored, we think about icloud. So for them to not be doing anything opens up a whole area where it's unprotected against this illegal content. And that really matters for a bunch of reasons, the most important of which is that there are real victims in these images and videos and they deserve removal and suppression and them taken down.
- [02:52]
A
And even if there was just one image or video on icloud, that's enough to need to do something about it. But we're talking about a larger scale problem that I don't Think people understand the scale of it. Can you give us an idea of how much we're talking about? Like, what is the scale of this type of material that lives in the cloud and on the Internet?
- [03:12]
B
Yeah, well, I'll just give you an example. There was a very famous. I think he was like an oncologist that was arrested in San Francisco a few years ago because an app made a report of him having CSAM somewhere, I can't remember, and they seized all his devices. And I think he had over like 2000 images of child sexual abuse in his icloud account. So that just goes to show you, like a lot of times the people who are sharing and trading this illegal, horrible content. It's not content. Evidence of a crime have a lot of it. And that matters because they're trading at a scale which just then means, like, it just proliferates around the open web. Every year, ncmec, the national center for Missing and Exploited Children, releases this report about how much of child sexual abuse has been reported to them. And it's been steadily above 15 million for like the last 10 years. A lot of those 15 million are duplicate images of the same series featuring the same child being shared over and over again. That should make people even more horrified that a kid who's experienced this has to live with the knowledge that their images and photos have been shared over like a hundred thousand times, five hundred thousand times. I know of a victim who knows that their abuse images have been found over 500,000 times. So the overwhelm of the scale of the harm, like, that's what we should really be focused on and being able to look those victims in the eye and say, we're doing everything possible to make sure that more people don't stop, see your images and videos.
- [04:49]
A
So I want to get to your conversations that you had with Apple during that time a decade plus ago. But before we get there, it's important to understand that Apple is not only storing the imagery, but making money off of that imagery being stored. Can you explain how that works?
- [05:08]
B
Yeah, I think this part's important because this is where Apple likes to sort of fudge it a little bit. So you, it's. You sort of like rent space in icloud. So, right. You pay to store your images and videos there, like the same way you would store something in a storage unit. The Apple is the one hosting the environment itself, right? They have the servers. They're in different server farms all across the US in the cloud means it's online, right? It's not only like on a physical device, it's, it's stored in the cloud. But that storage space needs technology underpinning in order to keep it there. And so the misconception that Apple wants you to feel is that, you know, your albums and all that, that that's like your private space that no one can get in and no one can see. And I think it's okay to say actually that space you're renting from us, so you can't store illegal content there. And maybe even more importantly, or just as important is you can't then share out an image from there. So that is where we could stem the tide of the flow of these images online is if they can't be in the cloud in the first place, then you also aren't as easily uploading them into different apps and stuff. So you'd have to have a hard drive and a desktop and be moving them around differently. And everything creates a barrier. Everything that creates a barrier of entry helps. The other thing I just want to add on finding known child sexual abuse material is that it can lead to finding a new kid who's being abused. So there have also been cases where someone has been found downloading and, and storing child sexual abuse material and they go to arrest that person. And as they arrest them, they look on their computer and they realize they're abusing their own child or a child in their life. So it leads to the actual recovery of new victims as well. That's why it's so critical. And it's so critical because victims who've already been found, whose images continue to be shared, they don't want to be part of the process of like normalizing and grooming other children. And they obviously just deserve justice as a whole.
- [07:20]
A
So other companies were proactively detecting, removing and reporting known CSAM back then, and Apple chose not to. And you, you went to Apple and tried to work with them on a solution. Can you tell us about those early conversations?
- [07:37]
B
Yeah, I'll tell you a little bit about the history. So they were just absent from all the child safety conversations, including detection of child sexual abuse material, which is why it was a huge surprise in August of 2021 when we got the call that they were shipping a solution kind of out of nowhere and they wanted support to do that, which, which as child safety advocates we wanted to give them. It was tough because we hadn't seen it, we didn't know anything about it, we didn't know how it would work. And so it's hard to just Jump into something blind and, like, defend it publicly when we're still trying to figure out what it was all about. The bigger issue is what Apple said about why they decided to do it, because then they changed their minds. So during that time period, they said, this is an abhorrent crime. These images and videos do not belong in our environments. They go against our terms of service. And Craig Federighi famously talked about in an interview with Joanna Stern that we feel very, very strongly that this is important to do, and this is. This solution is important, and we need to do this. 30 days later, they did a 180 and totally changed their minds. And we're like, never mind. So I'm just confused why there wasn't more questioning and, like, pushback on them of, like, well, why did you change your mind? And how does that mean you don't think it's an important crime? They've had various statements since then, and they all kind of circle around something like, this is an abhorrent crime, and we're so sorry about it, but, like, there's nothing we can do. I mean, it's just. It's a combination of negligence and lying and just hoping that no one's paying attention enough to call them out on it.
- [09:34]
A
And you guys have been calling them out on this for years, and we'll talk about what you've done in that capacity. At the time. What was their reason again, for not shipping that. That solution?
- [09:47]
B
So there's a lot of detail that gets really in the weeds. But unfortunately, in what they designed, part of their solution had an on device component, which meant that there was a part of the detection system that sat on the device and it didn't trigger until somebody saved it in icloud. Again, this is like very in the weeds. But the story that developed from that was that Apple was going to spy on you on your phone. That was like, the language that got developed was that Apple was now spying on you on your phone, which wildly misrepresents what was actually gonna happen. And unfortunately, knowing that there were some people at Apple that did not want this solution to go forward, I think they sort of added fuel to that fire. And like, once it sort of developed that narrative, they were like, oh, well, this could help kill it. So let's just let that happen instead of pushing back and defending it. I think that's one of the things that's, like, the most cowardice about this, is that they didn't defend their own solution. Like, I mean, and they're also Apple, so they Had a few weeks of sort of bad press around it, and then they were like, oh, my God, it's so bad. We have to stop. It's like, no, just keep going. I mean, why? It showed no backbone, no resolve around what they had decided. And I feel for the people at Apple who actually worked on the solution because I think they were really invested in it and they were doing really good work and they wanted it to happen, and they sort of saw all their hard work go out the window.
- [11:22]
A
And so you, as an expert on all of this, didn't see an issue. You weren't worried about user privacy related to this solution?
- [11:30]
B
No, not at all. Not with what they had designed, because it actually had a threshold of you had to have 30 known images of child sexual abuse material before it would even trigger a review of your account. So they had a very, very high, and many would argue too high of a, of a threshold for something to sort of go wrong.
- [11:55]
A
We're not talking about photos of, like, our kids in a bathtub. Not that type of photo.
- [11:59]
B
No. And let's talk about the baby in a bathtub thing, because this is a really important point that all parents need to understand. So. And I have a lot of feelings about babies and bathtubs. Overall. I would say having naked images of your kid and putting them online, even with, like, little stickers over them and stuff, isn't great because I know you think it's only your friends and blah, blah, blah, but you could be giving access to people who are not great, who are using those images in nefarious ways. And now with AI too, they can, like, remove the blockers and all of that. But that's about taking naked pictures and putting them on the Internet. Having photos of your kid in your, in the bathtub, on your phone is something that, like, every parent has. So I, I get that. So the reason that that would never be triggered in any of these solutions is because they're only looking for crimes that have already happened that have been documented by law enforcement, verified as child sexual abuse, and then fed back into the system. So the picture of you, your little Betsy in the bathtub is. That's not in the system. They're not looking for that. I will say the downside of that is that you don't find new kids because there are ways to have them look for very specific harm types that present and look a certain way. They could find new kids being abused and not to get too graphic, but, like, you can find a way to find images of, like, adult bodies with kids bodies interacting in certain ways, that would create a 99.999% chance that that's abuse. And yet many people, we don't use that technology because there is still much of too much of a fear of it being wrong. Others do, and I think they should because the kids who are being abused like that deserve it. But I really appreciate when parents ask me the baby in the bathtub thing because I love explaining it to put them at ease so that they're not worried about it and they can support a solution like this for other kids who have been abused.
- [14:00]
A
And we're still experiencing the privacy that we expect from holding photos of, you know, private photos of our children, safe photos of our children on our phones and in the icloud. And you mentioned AI and unfortunately that has advanced the problem with the ability to nudify children's photos. Can you tell us about how Apple is involved in the proliferation of AI generated deep fake imagery of children?
- [14:28]
B
Yeah. So Apple's involved because they are hosting nudify apps and deep fake AI apps in the App Store and making it really easy for people to download them on their phones, which then makes it easy for them to Dr. Photos that are in their photo library. And the thing that's really lame about this with Apple is that they're a trillion dollar company and somehow they're letting these apps get through that go against their policies. So that's the first big question is like why is that happening? And can't they manage their own app store efficiently? Feels like maybe it's because they want to accept more apps than reject because they make 30% of the download. What's happened since they've been hosting these apps is that it's taken nonprofits and researchers finding the bad apps and alerting Apple to them to make them take it down or like a really bad PR story. And then they don't put out any statement about it and they just kind of pretend it's not happening because they don't want people to realize that they were a part of the process. So that again to me is sort of the cowardice behind it is like own your thing. Like own the fact that you have the App Store, you make money off of it. Here are your rules. And then like live and die by them. Don't be wishy washy. And the fact that Grok wasn't removed from Apple's App Store earlier this year when Grok was known to be creating child sexual abuse images and videos and I've heard like rumors that there was a big fight behind the scenes between Apple and Grok and that. But, like, the fact that they didn't remove them is so atrocious.
- [16:06]
A
Well, it could be because they're making millions of dollars off of Grok and the new Defy apps being in the App Store, and you shared this the other day, that Apple has made an estimated $117 million from new DEFY apps and that's just the minimum number.
- [16:21]
B
Yeah, that could be the reason. And they also probably don't want to pick a fight with Elon Musk.
- [16:25]
A
I don't think anybody wants to fight with Elon. But sometimes when it comes to legal imagery and child safety, it's worth the fight. When a company knows that they're harming kids and victims, even adult victims, and they have refused to do the right thing. And you've. You did everything in your power to work with Apple to get them to do the right thing. You realized it was time to escalate, and so you took direct action against Apple multiple times. Can you kind of overview the direct actions you've taken against this company to try to get them to fix it?
- [17:03]
B
We've been to Apple park, their headquarters in Cupertino, over five times. This demand of removing known child sexual abuse material has always been front and center in all of those different actions. We've flown planes over them asking them why they don't detect child sexual abuse material. We've showed up with survivors and read testimonies about their experiences and why it's important to them that Apple take this seriously. That's another part of this is from that time period when they did announce a solution is they sort of talked a big game about, like, meeting with child sexual abuse material survivors, and we don't know who they are. We would like to know, like, if you're that interested, then meet with us again. Yeah, so that was another action we took. We've run mobile billboards and driven trucks around with messaging so that attention attendees at some of their conferences would see our demands and maybe engage with it or go inside and ask questions about it. We've made blimps of both Craig Federighi and Tim Cook and, you know, taking them around on little tours near Apple park trying to get people to engage with this message. But the last two have been the most kind of dramatic, which we deployed a big banner in September of 2025 that was 250 pounds and like 80ft by 50ft that had a picture of Tim Cook saying, new iPhone, same child sexual abuse and so that was a big statement that blocked the street. Again, trying to disrupt business as usual or really like, get the media's attention and get everybody's attention that this is happening so that Apple can't hide behind the walls and pretend like it's not happening. And then this last time, we escalated by showing up with a big sign that, you know, said, apple, this conference runs on child sexual abuse. John Ternus, who's the incoming CEO, what are you going to do about it? And then I was chained to a tree so that they couldn't remove me. And I read the testimonies of certain survivors and had the privilege of sharing their words with attendees as well. So we've taken a lot of steps to raise awareness around this problem. And I will say this last time, the amount of engagement that we got from people who were there and the media, it feels like we've earned their attention and also kind of their respect in a way that we know what we're talking about. We're not asking for something crazy. This is so basic. And the fact that Apple just continues to ignore it is shameful and wrong.
- [19:36]
A
In this last direct action, which was June 8, just a couple days ago, from when we're recording this, Apple was hosting a big worldwide developer conference within Apple Park. We were right outside. And so while we were outside and you were chained to a tree, reading victim statements and making sure everyone heard what the inaction of Apple has been, Apple was inside making some announcements about their new child safety efforts. And we actually. Did you hear that young man that walked across and yelled at us while we were doing it? He was like, they just spent 20 minutes of the keynote talking about child safety. Like, not in a positive way. He was telling us to, like, get out of there. But that was. We were encouraged to hear that because I think you said, like, that's something that they didn't used to do. So that was encouraging.
- [20:28]
B
Oh, yeah, I yelled. Well, first of all, you're not supposed to yell back, but I did say, great, that means it's working. Because, yeah, three years ago, child safety was like a tiny afterthought. On, like the third day, they would announce like, one update about screen time or something. So to put it in the opening keynote is a huge signal that Apple knows it can't ignore this genre anymore and that it has to, like, lean in a little bit. Otherwise it would get criticized too harshly for not addressing the fact that kids are a part of their ecosystem. So to me, that was a huge win. Even though the Changes they announced are not nearly enough. The fact that they dedicated that keynote time to these issues overall was a sign that it's working.
- [21:18]
A
And I don't know about you, but I've gotten a lot of questions from parents about what I think about these announcements of the child safety updates. And so I want to read them for parents and listeners really quick. Just briefly what Apple announced that they're going to roll out in the fall of this year, 2026. And then I would love your reaction as like somebody who's analyzed their actions and in actions over the years. And so what Apple announced at WWDC and in the media is that they're going to to offer redesigned screen time dashboard with easier parental controls. They're going to have ask to browse, which requires parental approval. Before visiting new websites, kids will need to get permission to add or contact new people in their messages, FaceTime and phone calls. Communication Safety will now detect and warn about violent and gory content. In addition to nudity, they have new app category time limits for social media, games and entertainment. Child accounts will come with more safety protections enabled by default. And new family safety resources and setup guides will be available for parents. And so what's your initial reaction to that announcement?
- [22:28]
B
My overall reaction is it's a little too little, a little too late in the sense that especially with the screen time piece was just ridiculous how long it's taken them to make the dashboard like more readable. I mean these were things that parents have been asking for for years. And so thank God they did it at this point. I mean, it's so wanting to just like jump for joy or pat them on the back is like not the energy that I have around it. But am I glad they did it? Yes. I think the Ask to browse is really interesting and much more impactful because that's about like kids visiting different websites. And the Communication Safety feature of the blurring of the nude images was already happening. So and is, I think now they
- [23:16]
A
added violent and gory content under that communication Safety feature.
- [23:20]
B
Right. So that's fine. I'd like to see the way it actually is rolled out because in the, in the older version it just basically asked you a question, are you sure you're comfortable sending this? And then it was like, yes, and then you just sent it. So it was a stop sign. But is it enough of a stop sign and did it stop sextortion from happening? I don't know. So I don't know if that's enough. What I really want in iMessage is a reporting button so that kids can report to Apple directly when bad things are happening to them or when people are sharing illegal images with them or when people are enticing them to take naked photos. So that's disappointing, but interesting. My high level is it's good to see movement. It's not enough. And on the very, very basic things like detection of known child sexual abuse and removing dangerous apps, it falls short.
- [24:12]
A
And celebrating a PR announcement is. We can't, we can't just celebrate them saying they're going to do something. And unfortunately, we've seen many different companies make an announcement about child safety features, most recently, I think, Meta and Instagram teens. And then once it rolls out, groups like ours will test it and often they fall short. Some of the safety features don't work or have been removed since the announcement was made. And so we can't, unfortunately, we just can't trust them at their word. Right.
- [24:45]
B
Let me tell you this example. I interact with some of the people at Apple about this and those individuals when they write me, they write that their titles are like privacy and child safety. And then when you look them up on LinkedIn, child safety is nowhere to be found. So are their investment in the child safety team? Who is it? How much investment is it? How are they prioritizing it? Are they really prioritizing it or are they just like doing the bare minimum and bandaging it and trying to tell the right people the right things to get out of it is an outstanding question that I have.
- [25:21]
A
So they're adding child safety to the job titles in the signatures of the emails sent to you, but they're really the privacy team?
- [25:29]
B
Yep. To be fair, I think they've posted for some new, like, child safety positions that actually include the words child safety. So that's exciting. But show us the list. Like, I look, if you have 35 people over there with child safety in their titles, I'd love to see it. Nothing would make me happier.
- [25:46]
A
What work are they actually doing? Like, are they actually doing child safety work? And are executives taking their recommendations or are they falling through, like, just not being heard?
- [25:56]
B
The signal that they did include it in the keynote is a pretty big signal. It could. It could be a shift. There could be a shift happening. I really hope there is. In the past, that was not what we observed, that there was like a strong investment in those teens and that it undercut a lot of the people there that were doing that work. I think those are good questions for Apple, but overall they're sort of wading in to the pool of like, acknowledging that they are part of the ecosystem of a child's online experience, which is positive.
- [26:30]
A
They're like the number one influencer of a child's experience on technology. They're the gateway.
- [26:34]
B
But they say they're only hard hardware. They're just the phone.
- [26:38]
A
Do you want to comment on the fact that Craig Federighi was one of the speakers at this event the other day and he was the one that had spoken publicly about the solution for detecting CSAM 10 years ago and then it fell through. Like, is. Is that interesting to you?
- [26:53]
B
Yeah, absolutely. I think he was also on a golf cart at one point when I was yelling about something he said. I wish that he would come out and just be honest about what his true feelings are around child sexual abuse detection and be transparent about their decision to not do it and why they can't come up with a different solution. Nothing would make me happier than him having to make more of a public statement about what happened before and not with words like, this will then unleash the dystopian dragnet of whatever the like. Give us specifics about why the most brilliant engineering teams at Apple can't figure this out.
- [27:37]
A
Yeah, because if you're going to spend time and resources rolling out improved parental controls and show us that you care about kids, what's more important than removing the sexual abuse of kids on your devices, on your platforms?
- [27:51]
B
It's like going uphill. We're climbing uphill. But I think it's because they generally see their responsibility to their users so that their mindset is like a child as the user of the phone versus them, having responsibility towards kids who are in images and videos that are horrific. And that is the mindset shift of that needs to happen. Because there's never been a company more primed to be able to come up with a solution of how not to host that illegal content. And there's a lot of discussion about how to do it. I mean, it would definitely take innovation, but, like, just literally don't let it be stored there, don't let it render there, have it disappear or, like, and report it. I mean, there's a lot of different ways that it could take shape. Sometimes I wish, and maybe we should do this at some point. I could read descriptions of what is in the images and videos that I'm talking about. Because when you actually, like, read the descriptions and everyone is horrified and then you say like, and that's okay with Apple, like, that really helps people understand what we're talking about because I worry that just talking about child sexual abuse material all the time takes away from what it really is and who's in it and what's happening in it. And just the shame that sometimes the survivors feel around it, which is so unfair because they're the ones who had to bear it and have gotten through it. And now it's. We're all afraid to talk about it. It's just not. That's not a world that I want to live in.
- [29:28]
A
It's uncomfortable to talk about. And you said something like most of the imagery is children under 10. And we're also talking about infants and babies in this imagery. And you know who it's more uncomfortable for? The victims. And if we continue to sweep it under the rug and keep it a secret that it's there and it's happening, then the predators win. And they continue to be able to share it.
- [29:54]
B
No. With impunity. Like they're not. Yeah, I appreciate you saying that. And it's true. And it's not comfortable, but that's why we do this work. And, you know, when I was standing there locked to the tree with like the 40 pound chain thing, it's like this is so small compared to, like, what they've had to go through. And so it's just like an honor and a privilege to be able to communicate their stories and their words to the people who need to hear them most in those moments. And that's why we do this work. It's really because of individual kids who we know who have gone through this, don't deserve it. And I think about, if it had happened to me, that I would want other people to care that this happened and not be okay with us with. With just like the injustice of being ignored.
- [30:48]
A
Is there anything that parents and listeners can do right now if they feel like they want to help out?
- [30:53]
B
Yes, there's a petition. We'll make sure to link it in all the right places so people can be a part of this fight. I think honestly, with Apple and child sexual abuse material, what I would ask. This is really an area where talking about it with one person would be so helpful because people do not know and raising it as a reason why the iPhone not the safest phone for kids. Helping parents understand that Apple has not prioritized child safety the way they think they have. So just making parents more curious about what's actually happening versus what Apple wants you to think is happening is what I would want your takeaway to be. Yes, Apple does make really great decisions around privacy of data for your kids. They're not selling their data, which is great, but the flip side of that is they're also not treating them like a child customer. They're sort of putting them in an adult category. And so that's why these changes they announced are important even though they're not enough. And then why they don't do the most basic things is, like, really concerning. So be curious, tell another person, and continue to really question whether an iPhone belongs in your kid's hand 100%.
- [32:08]
A
And please wait for groups like ours to test the new parental control system before just believing that it's going to keep your kids safe. Unfortunately, that's not where we're at and that's why we're here to help guide parents and give that assurance on what the best choices are for our kids and how you can get involved in improving child safety across the board. So thank you, Sarah, for all of that information. You're a wealth of info on this topic. I'm sorry that the fight has had to be so long and they didn't just do this from the start. Your. Your life would be much different. But here we are and you're still fighting. And I'm so grateful to you.
- [32:43]
B
Thank you, Nikki. Thank you for platforming this message and for everyone listening. I. I know it's not an easy topic and I always appreciate when people turn towards it, so thank. You.