Loading summary
Transcript51 lines
- [00:00]
Parent
It was so painful to get those emails. Calling the event rumors and speculation when a crime occurred was almost worse than the crime itself.
- [00:16]
Joseph
Hello and welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist founded company and needs your support to. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404 Media co. I'm your host Joseph and with me are the other 404 Media Co founders. The first being Sam Cole.
- [00:47]
Sam Cole
Hello.
- [00:48]
Joseph
Emmanuel Mayberg.
- [00:49]
Jason Kebler
Hello.
- [00:50]
Joseph
And Jason Kebler.
- [00:51]
Emmanuel Mayberg
Hello. Hello. Let's mention. Yeah. If you are in Los Angeles, we are doing an event at the LA Public Library like the central one on Sunday, May 31. I guess the best way to learn about it would be we'll like include a link in the show notes. But Sam and I as well as our friends Dexter Thomas and Geeta Jackson will be talking about AI and AI Slop and how it is changing the Internet and all that sort of thing. So it's Sunday afternoon. Please come out if you can. Link is in the show notes.
- [01:31]
Joseph
Yes, get the details there. All right. With that, we'll get to this week's stories. The first one is one that Sam wrote that she's been working on for a long time. It really paid off. It's a really, really deeply reported piece here. This, the headline is How Deepfakes Tore a High School Apart. I'm actually going to add a question to this, Sam, before we get into sort of the story itself and the specifics and what happened here. Just broadly, what do you think about deepfakes and schools at the moment and sort of the value of this sort of story? Cause I feel like we hear that deepfakes are all over schools. Boys are using it against girls. Non consensually, all of that. What was the idea here? To focus in on a specific example and case or what was the idea here?
- [02:27]
Sam Cole
Yeah, like you said, schools dealing with deepfakes is not a new topic. 44 has written about this quite a bit. We've been covering this as it comes up for the past several years. Emmanuel. Jason wrote a really good story based on police reports in 2024. So that's two years ago, how a school failed to report deepfake harassment among their students and how the police just kind of also bungled it. And everyone was kind of like who Dropped the ball and where. And since then, it's been something that it's just in the news so much lately, I think that people start to kind of stop paying attention to it. It's like, oh, another school is dealing with deepfakes. Another school is dealing with synthetic nudes and these nudify apps that we hear about all the time. So I think there's value in just taking a beat and breaking down. What is actually happening when this happens. It's not even that this particular school was an extraordinary case or that it was, like, really uniquely fumbling the situation, because these schools are failing students and the police are failing students in the same way over and over and over. It's a pattern. So I think there is value in just breaking down where exactly what points in this kind of system fail, where they actually are lacking in good response and, you know, breaking down actually, like, you know, at what point were the images talked about? At what point were they reaching the victims? What point were the. Was the school notified? Was the police notified? I think is really helpful, and I think that's the only way that you can actually prevent something like this happening again is learning from the ways that other schools have responded and help parents respond, help police respond. So that was the point of this story, is just kind of getting in basically from the moment where it started. So I think I reached out to one of the parents from a Facebook thread a couple months ago and was like, hey, I saw that this is something that you guys are dealing with in real time. It's not resolved. It's something that is actually still being litigated amongst you debated about what the response is. So I reached out directly to one of the parents, was like, hey, do you want to talk about what is actually happening? And then kind of follow through as this goes on. So that's where the story took us. And I would say it's still not totally resolved. It's still very much in progress, but it's the sort of thing that takes a long time to follow.
- [05:00]
Joseph
Yeah, exactly. And that is part of the story, the amount of time that is sort of elongated here. So let's start with the creation of the images themselves. What exactly happened? How many people impacted, like, right at the beginning? What is literally, like, the starting moment of this, you know, of these deepfakes taking over this school?
- [05:22]
Sam Cole
Yeah. So it was the first week of December, and among the freshmen at Radnor High School, which is in Pennsylvania, it's one of the top schools in the state. It's on The Main Line, which is a very affluent area, just to kind of paint a picture of the student body and what we're dealing with here. It's a well resourced school, I would say upper middle class, it's a good school. But in early December, these kids started talking amongst themselves on Snapchat, which has disappearing messages and elsewhere, talking about how one of the boys, one of the boys in the high school had made images of his classmates and they were discussing the fact that his parents caught him and he had to delete the app. And they were talking about how he had spent, you know, $250 on this app and, and on making these images and how he was caught. And his friends were like, you know, you're dead. Your parents are going to kill you if they find out what you were making with that thing. And so from there, the next morning, the girls who were in the images had to go to school and face other kids talking about them. Everyone had been talking all night about how these images were going around and like they were saying, you know, we've seen this, we've seen that. It was really unclear about what was actually being spread and what was actually just kids talking. But the girls were dealing with, you know, high school stuff. It's like the kids were, the boys were high fiving each other. They were like, you know, harassing them. The next morning after a night of just like being bullied over these images and they were really upset. They were texting their parents. I talked to one of the parents extensively about this and she was saying, you know, they, that her daughter was just like, this isn't okay. The kids are defending their friend. They're, you know, lying about what's happening. So that was kind of what they walked into the next day. And I don't know, I mean, I like, I remember high school. That's a nightmare.
- [07:30]
Joseph
Yeah, yeah, high school is a nightmare without AI enabled apps that can instantly nudify you or whatever. And now you have all of that and this. And just to clarify, what was the app capable of exactly? Is it one of these undressing apps? What is it?
- [07:49]
Sam Cole
Yeah, so I'm told. And again, I don't have definitive proof of this, but I'm told by the parents and by people who are familiar that the app was called Moovly, which we know what Movly can do because the Tech Transparency Project did a big report about nudify apps recently that I wrote about last month, where Movely was one of them. And Movely is one of those like those apps that on the face of it looks like, you know, put your face in Spider man or like in a cat picture. You know, it's like, it's. Or even like put yourself on the beach, stuff like that. But then once you pay up for the subscription, you can put someone's face into whatever you want. And it's, you know, it's like sexy undressing videos. And, you know, this is how they're like, marketing the. The app to paid users. So I, as I understand it, the images were, you know, like highly sexualized images of minors, of girls, young girls. And they were all between, I think, 14 and 15, so freshman girls. And it looked like nudes of them going around and being shown off by the boy who made them. So that's what we're dealing with. Moovly is still online. It still has the website, but I haven't been able to log in. I can't buy a subscription anymore. I think after the tech transparency project support, Apple took Movly off the App Store. But there are tons of other apps like this out there, tons of other apps specifically targeting young boys and men for this sort of content. It'll be like, make her boobs bigger, make her butt bigger, make her do this, make her take off her shirt, stuff like that. So that was Moovie's entire, entire gig?
- [09:51]
Joseph
Yeah. I mean, every day. I mean, essentially every day we learn about a new one. I think I got a text Yesterday from a 404 Media reader being like, hey, here's another app I got recommended on Reddit or something, and it's exactly the same thing.
- [10:03]
Sam Cole
And they're recommended on, like, social media. It's like on TikTok and on the other Instagram, Reddit.
- [10:10]
Emmanuel Mayberg
We've now done many stories about this. Like, Emmanuel's done a bunch of stories about the fact that these are being advertised often to kids, often on Instagram, often on TikTok. And the Apple App Store has removed a lot of these apps. There's been like, academic reports about this happening, like, over and over again, like, like systematic attempts to document this problem. And yet, like, every time I go on TikTok, I'm get still getting ads for similar apps. And there are often like, really blatant actually, like the advertising is very blatant, like what they're for, where it's like, make anyone nude or replace anyone's face with anyone else's. But then, yeah, again, when you look at the App Store description, it seems like innocuous or like an app that's just for fun. And they have not gotten a handle on this problem from what I can tell.
- [11:09]
Joseph
Yeah, it seems like it. So those images are made, they're spread around the school, as you said. The girls are obviously, I imagine, pretty traumatized by that, and it's really, really horrible. What happens next? Does the information sort of get to more parents? Like, sort of. What's the next stage of this? Because there's parents, there's the school, there's law enforcement. But which bit comes next? Exactly?
- [11:35]
Sam Cole
So the parents tried to get on top of this from the start. They emailed the school, they called the school, they were like, you know, please don't bring my kids into a one on one meeting with anybody until, you know, we have a chance to talk to you. You know, they're, they're having a hard time already. It's like, it's, you know, it's. They're an hour into the school day and they're already facing so much, just trying to get, get to their class. So the parents are just immediately in action. They're like, we have to get a handle on this. We have to figure out what's going on. What the school does is. Well, first they take too long, basically. They take several hours to get back to the parents. They say, we heard from a safe to say hotline, which is the. It's like a hotline where you can report harassment if you're a teenager or anybody for cyberbullying or a school shooting threat or something like that. But someone had reported to the hotline that this was happening. So the school was already kind of aware of it, but they were just kind of scrambling also behind the scenes. The school sent a statement which is in the story, but they didn't answer any specific questions about the response and things like that. But they sent an email. A lot of this is happening over email, honestly. They sent an email a couple days later where they addressed rumors. They called it rumors, which I think was upsetting and inaccurate because the images are real.
- [13:10]
Joseph
Well, they're not real, but they're real. Yes.
- [13:12]
Sam Cole
Right. Yeah, yeah, it's definitely happening. But the principal emailed and said there are rumors about AI generated videos that are going around. We understand how upsetting and serious the situation is. We're treating it with the highest level of urgency. You know, they're, they're trying to kind of get a handle on, on the response. They say that they contacted the police as soon as it was brought to their attention and that the police were actively involved. The Police also didn't reply to any of my or. They didn't, they didn't answer any questions specifically they didn't. They directed me to the DA's office which took over the case. But they said that there were supportive measures in place for the girls. So there's five girls in total dealing with this at the school. But those supportive measures were, you can leave class early, you can eat lunch separately, you know, you can isolate yourself basically from the, from your bullies if
- [14:15]
Joseph
you want, but not. But the school at that particular time, take the bully out or the boy who generated the images out.
- [14:25]
Sam Cole
As far as I can tell, he wasn't suspended or anything. He didn't show up to class the next day. I think his parents were like, you're not going to school the next day. But the response was definitely not great because it was isolating the girls. They were being punished for something that was happening to them. They were told that they could request accommodation. Basically they were like, you can come to the school counselor if you want and talk about this. Which is not. Again, it's part of the pattern that we see over and over at schools. They give the targets of this stuff the option to isolate and be away from their bully. But, but they don't actually respond to the actual trauma that's happening. So that's where that stood for a long time. And then the parents started going to school board meetings which is where a lot of the story was reported was just watching school board meetings and watching how the board is grappling with us. The parents are coming to school board meetings and saying, hey, we need to change the policies, we need to talk about the response, um, all this stuff. So.
- [15:37]
Joseph
And those are public, right?
- [15:39]
Sam Cole
Those, yeah, they're on YouTube. One of the biggest points in the school board meetings that parents kept coming back to was the way that the school and the police jointly were responding to, to these things.
- [15:50]
Parent
It was so painful to get those emails calling the event rumors and speculation when a crime occurred was almost worse than the crime itself. I mean if you're going to put out a statement like that, make sure the facts are right. I mean at a bare minimum, I mean the victim centered thing, great. But to say these images move and dance, I mean, for crying out loud, this is pornographic material. This is not Mickey Mouse, this is not Disneyland. We don't have Snuffleupagus moving around on a stage. This is pornography. And this email was so damaging and hurtful.
- [16:30]
Sam Cole
It was. The school administration and the police co signed this email and the email said that the police had concluded their investigation. So it had been a month, and said that there was an incident off school grounds outside of school hours, which again, kind of doing a little cover your ass. The student used an app that animates images, making them appear to move and dance, which downplays what actually is happening on this app pretty severely. You know, it's. It sounds like it's some kind of goofy, like, elf yourself app. It's not. It's realistic sexualized images. And then they say in this email that after the investigation, they found that there was no evidence shared with law enforcement depicting anything inappropriate or any other related crime, which was just galling to the parents and to the, the girls because they were saying there was no crime, they were saying there was an investigation, but they have all these screenshots. They have, you know, knowledge of these things happening over other apps, over discord, places like this. They know that the app was used, but the police just said say they didn't find anything, which is shocking.
- [17:45]
Joseph
So I guess just on that, before we get sort of the bigger takeaway question is like, what is the legality around this? Obviously, it will depend on the images, so it can be hard to say. Like, obviously possessing child abuse imagery is illegal in the United States, and that does extend to AI generated imagery. We've seen more and more cases of that recently. I guess there's a cyber bullying element here and a harassment element. Legally, where might this case fall into? And I appreciate stuff might be ongoing, but, like, where do you see it potentially?
- [18:25]
Sam Cole
Yeah, for sure. I mean, there is like a legal precedent for this is the, the frustrating thing. And that was what was really frustrating about this email, because they were like, there is like, something happened and we need to figure out what it is. And, you know, like, where are the, where are the images now? Like, did he delete them? He said he deleted them off his phone. Where are they? You know, you guys, like, the police relied on teenage boys to forfeit evidence is kind of the, the takeaway from this, which is crazy. You know, anyone should know that. That's, that's insane. And then just before we get into like the, the actual, the law here, just to close the loop on that email. A week later, after that initial email where they said nothing happened, the police released a statement and said that a juvenile offender was charged with the crime of harassment. So something did happen. And they kind of flip the story at that point. So, I mean, it's. I think it's hard because first of all, we're talking about minors, we're talking about images that the parents did not see and that apparently the police didn't try to actually find as is what's, what the parents believe at this point in Pennsylvania. Pennsylvania actually has pretty good laws against deep fakes. They, they criminalized deepfakes, malicious deepfakes. In 2024, they're, they're among the states in the U.S. they're a leader in this sort of thing. And then in 2025, a man was actually an adult man was charged with three 30 counts of felony possession of child sexual abuse material that were AI and AI generated. Child sexual abuse material has been a federal crime for years. So it's, there's a lot of confusion about like, you know, because it's fake, is it really a crime? It is, it's a felony. Child sexual abuse material is very serious. So I, the, I think a lot of the, the confusion and the hesitancy in these cases is because they're children, they're children making images of children. But you know, we have precedent of kids actually being charged with serious crimes. You know, 13 year olds being charged. I think it was 2023 where the first case of a child being charged with child sexual abuse material happened. They were arrested, they were charged with third degree felonies, which is, it's itself very wild. You know, it's like the, the kids are being failed in every direction here.
- [21:00]
Joseph
Right.
- [21:00]
Sam Cole
You know, a 13 year old being charged with a third degree felony is not, you know, that's not solving, obviously it was three years ago. That didn't solve the problem here.
- [21:10]
Joseph
Right. And they may not understand what they're doing is like. And they may, I don't think they do.
- [21:15]
Sam Cole
Yeah, right.
- [21:16]
Joseph
Or they may not understand that it's obviously a horrible thing to do to another human being because we're talking about literal children who don't. Yeah.
- [21:22]
Sam Cole
And the kid, the adults don't even seem to understand. You know, it's like no one is, no one is actually taking this as seriously as it is. And that's something that I talked to quite a few experts about this for this story. And you know, just bringing home the fact that this, this is a serious crime. It's being enabled by Apple and Google through the app stores. It's being pushed by social media just handed to them. And then when it, when they actually take the bait, they use these apps. There's just immediate handoff to police, which there should be, you know, police should get involved at that point. But there's no there's so little preventative education about this at this point. It's like when you get caught, you will get charged with a felony and your life is ruined. And these girls lives are ruined because they are so traumatized at this very tender age where they're now being, confronting sexual abuse material of themselves being spread in their school. They can't study, they're, they're in therapy now, you know, they're, they're leaving class early, stuff like this. So it's just in every direction. These kids are being let down by the adults in the room. And I don't think you can wait until there's a serious crime committed by a 15 year old or 14 year old or 13 year old to actually take some action on this. I think schools need to realize how serious this is and start talking to kids about it. And not just like, here's an AI class, here's an AI education class, which is what a lot of the schools end up doing. Actually talking about consent and sexuality and having some literacy around. What you're actually doing with these apps and what you're up against is a serious thing. So I don't know, it's just again, that's why this sort of story I think is important to really dig into and examine.
- [23:22]
Emmanuel Mayberg
Because
- [23:24]
Sam Cole
the failures start before it actually happens. It's long before that. It starts when you hand a kid a phone and say good luck.
- [23:35]
Jason Kebler
I think it's tricky because none of us would want to excuse the behavior. But I also agree that, I mean, imagine you're a kid. It is hard to understand that it's a serious felony when it's being pushed at you from the biggest social media platforms in the world. And then all you have to do in order to partake in it is download it from Apple from the Apple App Store, which is readily available on your phone. And it's just so frictionless. And it's like your way to arrive at a felony is so frictionless, which is not the same. It's not as if, I mean, we've reported on drugs being offered on social media and guns and things like this, but that is way more in the underbelly compared to something that's like a paid advertisement. And then I guess the other thing I wanted to say is that we report on all the many ways that you can produce non consensual content. And there's a spectrum, like there's people who roll their own models and make their own models and they're using their own GPUs and all that stuff is extremely hard and probably impossible to stop because you can just make your own DIY method of making non consensual content. And then there's people who are subverting existing image models and using all these hacks and that's more available. But whenever we write about one of these abuse cases in high schools or junior highs, it is never one of those solutions. It's always the lowest hanging fruit, which is there's an app advertised on TikTok. You download it from the App Store and you use it for money. And people often ask me, and I'm sure, Sam, you get the same question as well. It's like, what do we do about this? And it's like, that's a big question. And ultimately the answer is you can't stop it completely. You probably are never going to stop this completely. But what you can do is take care of the lowest hanging fruit, which is these apps on the App Store and the advertisements on social media. And we don't know what that world looks like because we haven't tried that really. These companies haven't really tried that. And maybe kids do get more sophisticated and start using their own GPUs to make this stuff, but the least we could do we haven't done, which is just taking care of these apps.
- [26:11]
Joseph
Yeah, that makes total sense. All right, let's leave that there. Everybody listening? Definitely. Go read the article for all the much deeper specifics as well. And sort of the play by play of what exactly happened. After the break, we're going to talk about a company called Bus Patrol. We'll be right back after this.
- [26:39]
Parent
Foreign.
- [26:45]
Emmanuel Mayberg
This episode is sponsored by Incogni. When people think about identity theft, they usually imagine some giant data breach making headlines. But the reality is a lot of the information that scammers use isn't stolen in a hack at all. It's already out there. Your phone number, address, email, age, family members, old accounts and things you would never even think about. Well, there's entire websites built around collecting and publishing that information. And the easier it is to find, the easier it is for scammers, impersonators and identity thieves to target you. That's why I've been using Incogni. I've been using Incogni for years because we're the target of a lot of these attempts to steal our data, to use it, to dox us, to harass us, et cetera. And Incogni has been amazing at helping us remove that personal information from the Internet by contacting data brokers. And people search sites on our behalf and demanding that they delete my data. And they don't do it just once. Incogni continuously monitors and sends removal requests because your information has a habit of reappearing online over time. One feature that I really like is custom removals. Let's say, like you find a specific website with your personal info on it. Instead of trying to figure out who to contact and how to get it taken down, you just give that to Incogni and then their privacy team handles the removal process for you. It's a huge differentiator because privacy isn't just about reducing spam or making sure that you don't get hacked. It's about shrinking the amount of personal information online that's available to people who shouldn't have it. The less information that's floating around about you, the better it is for you and the harder it is for bad actors to build a profile on you. Take your personal data back with Incogni. Go to incogni.com 404 Media and use code 404 Media to get 60% off an annual plan. Again, that's incogni.com 404 Media use code 404 Media for 60% off an annual plan today. As I've gotten older, one thing I've realized is that there's so many stories our parents never tell unless someone actually asks them. You know, they don't usually volunteer that stuff unless you bug them. Stuff like beyond the usual family stories. What was their first job? What were they afraid of when they were younger? What decision changed the course of their life? And the weird thing is you don't always think to ask those questions until years later. Or maybe you don't ever remember at all. That's why I love Storyworth as a Father's Day gift. Most Father's Day gifts end up in a drawer. Storyworth doesn't. It gives your dad a year long experience and gives your family a book full of stories he'd probably never think to tell on his own. Each week, Storyworth sends him a question about his life. He can then answer however he wants by email, through the web, with a voice recording, or even through a guided phone call. And you can help choose the questions too. Pick from their library, write your own, or let Storyworth create personalized questions based on his life. The coolest part is that you get each story as he tells it. Then, after a year, Storyworth compiles everything, his words, his photos, his memories into a beautiful hardcover book. Families have already created more than a million books with Storyworth, and they recently introduced an unlimited plan so you can give Storyworth Memoirs to friends and family throughout the year. I got a Storyworth last year for my family and it's been really amazing to learn all these things about, like their high school and college days that I never knew. So get something for the people that you love to hear in their own words about their memories before they fade. This year, give Dad a gift that captures who he really is before the stories get harder to remember. Father's day is Sunday, June 21st. Order right now and save up to $20@StoryWorth.com 404 Save up to $20@Storyworth.com 404 StoryWorth.com 400404 I recently had one of those moments where I opened Whatnot looking for one specific thing. I was actually looking for hard drives and 20 minutes later I was watching someone demo electronics. I was checking out vintage collectibles and finding brands I'd literally never heard of before. That's kind of the magic of Whatnot. It's the largest live shopping marketplace in the country, and it feels completely different from traditional online shopping because you're interacting with real sellers in real time. On Whatnot. You can ask questions, see products up close, and find deals you usually wouldn't find at retail prices. What I like most is that it feels more like a community or like an audience, kind of like an influencer type relationship than a storefront. Buyers connect with passionate sellers who actually know what they're talking about, and that's how you end up discovering products you weren't even looking for. And clearly people love it because Whatnot buyers spend more than an hour a day in the app. They're engaged, buying and coming back on the seller side. It's just as impressive whether you're running a business full time, selling as a side hustle, or just getting started. People selling on whatnot sell 10 times more than on other major marketplaces because they're building real relationships with buyers. I've genuinely found like a series of analog cameras, vintage cameras, and other photo gear on Whatnot that I wouldn't have discovered anywhere else. And I've just been amazed at sort of the variety of what I found and the great prices and deals I've been able to get. Download the Whatnot app today and get free shipping on your first order. Just search what not w h a t n o t whatnot in the App Store and start scoring amazing deals. Okay, and we're back we're going to talk about our favorite topic, which is automated license plate readers, but this time not about flock. We're going to talk about a company called Bus Patrol. This is a story Joseph wrote. It's called bus Patrol. Put AI cameras in tens of thousands of school buses. Now they want to give cops access. Joseph, first of all, what is bus patrol?
- [32:50]
Joseph
Yeah, so bus patrol is a company that is very focused on something called stop arm laws. So in the US when there's one of these quintessential yellow school buses, they will stop so kids can get out or get into the bus. And these stop signs will eject from the side of the bus. And that tells obviously cars nearby, hey, please don't overtake this bus. Because there's probably a bunch of kids coming in or out. And that would be really, really dangerous to the point where it's illegal to do that. Right. So what bus Patrol does is that it has these AI enabled cameras on the bus, I think to the side, maybe to the front as well. And what it will do is that when a car passes a school bus, when it has these stop arms extended, it will record that interaction or that potential violation. Right. Bus Patrol will then review that footage. It will then, if it believes there was a violation, send that to law enforcement. The police will double check it and say, yes, we think this was illegal. Bus Patrol will then send the ticket or the citation to that driver. You know, they're using license plate reader technology there to identify them and communicate with them. And then the person gets a ticket and they can go and dispute that in court or they can just pay it or whatever. But the attraction, at least the way it's pitched from bus patrol to counties and cities, is that you have all of these people maybe probably violating stop arm laws and you're not catching them. We can catch them. And we will also provide the mechanism of enforcement in a way to deliver the tickets to them. So it's kind of a win win for the counties and the cities and that sort of thing. That's what bus patrol does today.
- [34:54]
Emmanuel Mayberg
Yeah, I mean, I guess let's talk about that real quick. Just like as someone who has driven a lot, people do drive very recklessly around school buses and children can be quite unpredictable. And I see all the time the stop arms come out on a school bus and cars drive by it. And so the problem that they're nominally trying to solve is a good one. I think that's one of the worst traffic infractions that you can do is just speed Around a bus where the kids can't see around. Lots of kids die in this way. It's theoretically a limited problem, as in like it's a very specific thing that bus patrol is trying to solve, or at least saying that they're trying to solve. But now we're getting mission creep.
- [35:46]
Joseph
Yeah, I think mission creep is a good way to put it. So according to leaked bus patrol documents and a source familiar with the plans, Bus patrol intends to really rapidly and dramatically expand that data collection or surveillance. I mean, look, the stop arm stuff, you might not even call it surveillance because of the use case and because of the very limited application which is being used. But according to these documents, Bus patrol plans to turn those cameras into much more general purpose automatic license plate readers, which regular listeners will know. But I'll explain it. A automatic license plate reader just scans the license plate, the brand, the colour, the model of every vehicle that goes past a camera. Usually they're stationary, but in this case they're going to be on a bus, obviously. So what bus patrol is planning to do is turn those cameras into bus general ALPR cameras and collect much, much more data. It's not just about people violating stop arm laws anymore or speeding past school buses. It is general purpose and then giving that data, that license plate reader data to law enforcement or to law enforcement contractors. And according to the documents we've seen, Axon comes up a few times. Axon being the massive law enforcement contracting giant that does everything from cameras to body cams as well. Right. And tasers and all of this sort of stuff and all in one integration software and all of that sort of thing. So it really is expanding this pretty innocuous program to something much, much broader. And I think there's a quote right at the top of the story from an attorney involved in multiple license plate reader cases that just says, who would have thought that school buses would be turned into the mass surveillance state? And that kind of encapsulates what's going on here.
- [37:59]
Emmanuel Mayberg
Yeah, I mean, so what you are describing or what sort of like it seems they're proposing is basically like flock on wheels. You know, obviously school buses move. So I'm curious, sort of like, why is that potentially more powerful than what FLOC has?
- [38:19]
Joseph
Yeah, yeah. And the reason I kind of use the term flock on wheels, I don't think I did this in the copy, but I did it on Blue sky and in this Google Doc here is just like people know what the word flock means. And for better and for worse, I think people understand the word flock more than they do automatic license plate readers. I just want to stress flock is mentioned in this story because flock is mentioned in the documents. But Flock told me it does not work with bus patrol. So I don't want to conflate the two. It's just like a term so people understand it. But yes, normally a license plate reader camera is stuck on a pole somewhere. Right. It's like at a fixed point on a highway or an intersection or a cross section or something, and it's looking in one place and it captures all the vehicles that go past there. Obviously school buses drive around, they do not sit in a car park all day. Right. So as they're driving round, they're going to collect presumably much more information than an individual camera. Now, that's not to say all of the data that a school bus may collect may be worthwhile. I don't know, maybe they go down a particular road to drop off or pick up some kids and like there's barely any cars there and nothing happens there there or whatever. But the idea of having a roaming surveillance vehicle, I imagine could be quite attractive to law enforcement and it seems attractive to bus patrol to come up with the idea of sort of selling access to this data in the first place as well. Yeah, it just, it puts that capability on wheels and makes it into a portable sort of surveillance capability.
- [39:53]
Emmanuel Mayberg
Yeah, I think it's also, it's kind of interesting because it almost serves a duty different purpose than Flock, which are stationary cameras, where with Flock you kind of can catch a car in motion and track it around a city or from city to city. Whereas a school bus is likely going to scan parked cars, which, you know, is like something that Flock cannot really do unless you're parked directly in front of a Flock camera. And so it can kind of like find cars that are maybe like stationary, which could then lead to someone's house, theoretically, like depending. And also of course, like if the bus is driving next to someone that's moving, I guess it could track them as they move there. But it almost serves like a separate purpose to some degree. And yeah, it's like sort of part of this trend where we are seeing ALPR data being gathered in unusual ways, like new and interesting ways.
- [41:00]
Joseph
Yeah, so there's a couple. There's one that we reported I think last year at this point where there is a dashcam company called Nexar. And you know, these are just the dashcams that people put in the front of their, in their cars and they capture whatever there were at least exploratory plans around integrating NEXAR with floc. I don't know the current status of those plans, but that's what we reported at the time. And that would be sort of similar to this bus patrol thing, albeit not on a fixed route, like a bus. It would be, I don't know, people driving to work or people driving away for the weekend or whatever. It could be anywhere. The other one that comes to mind is sort of like the harbinger of this approach, which is a couple of companies called Vigilant and Digital Recognition Network. We covered drn, the latter one back at Motherboard, because the source had access to it and they performed a lookup with a target's consent. And we just saw how powerful this thing was. And the way that data was collected was that DRN partnered with repo men, people who are going around to tow vehicles and that sort of thing. They had a dash cam in their cars and that captured all of the vehicles that they drive by and all the license plates and exactly what you said, Jason. It was really good at finding parked cars. And we saw that because when this source of ours tested DRN and provided the results of the lookup, we found the target's car parked right outside their house. And then we found where else they drove. I think they drove somewhere on holiday or something like that, but it was exactly that. We could see where they lived and then where else they went. Which is something that a stationary camera may not get. Well, I guess it's obvious. Why are surveillance companies trying to get more license plate reader data from more unusual sources? I mean, I guess it's like, well, it's more data, so that's better, presumably. But I think it's kind of what you touch on in the. That these mobile cameras get something that the stationary ones can't. And it almost seems like a gamble or a hit and miss. It's like, dude, is the target going to appear in a school bus camera or something? Who knows? But we have the data, so we might as well look. And crucially, the cost of looking, beyond the financial cost, the cost of looking is basically nothing, because this stuff doesn't require a warrant. Because the argument is, is that this is publicly available data in a way, or rather, I'll rephrase, their cameras in public places, so law enforcement do not need a warrant to query them and that sort of thing. So, yeah, this is happening. The Nexon stuff was exploratory. This is like a plan with Nexon and the repo man. Stuff definitely happened as well, but yeah, it's coming from all over now.
- [44:04]
Emmanuel Mayberg
Yeah. And I think that this speaks a little bit to just like the expansion of AI for compliance. And, you know, this would be alprs, but we're seeing it with like, all sorts of cameras and sentiment analysis and all of that sort of thing. I think with ALPRS in general, I think cops are learning that they can be useful for a lot more things than originally thought. And with a company like Bus Patrol, again, it's like it was formed for this very one specific purpose and now they're finding like, well, we could actually expand it to do a lot of other things. Maybe we could, you know, make a lot more money doing this because instead of just catching cars driving past, you know, driving past a school bus that stopped, we can use it to find stolen vehicles, we can use it to find, like, fugitives. We can use it for all the other reasons that cops have been using ALPRs. We've also seen like, flock cameras used for, like, speeding enforcement, which is something they said that would not happen and has now happened in at least one case that we know of. And I know that there's also another company that put cameras on buses, like city buses, and did like, automated parking enforcement. So if cars were parked in bus lane, which again, is like another problem that needs to be solved, it. But it, but it's one of those things where it's like the way that it's being solved is via these, like, automated systems that, that are constantly collecting data, constantly surveilling. Like there's all sorts of, like, potential issues with it.
- [45:47]
Joseph
Yeah, yeah. That other company, Hayden AI, they do the bus lane compliance, as you say. I actually have some documents on them that I've been meaning to write up, but then this Bus Patrol one came along and it was like, oh, okay. They're actually doing the thing that I suspected that maybe another one of these companies would do. I'll also just say, before I take us to the subscriber section, is that there was a really good Bloomberg Businessweek investigation into bus patrol a few weeks ago at this point, and they did a bunch of public records requests, I think, to get a bunch of data. And they were finding that in at least one case. And I'll say that I think it was multiple, but I'll just say in at least one case, there was no evidence that the county that was using bus patrol actually had reduced number of violations or that sort of thing. And the county ended up paying like tens of millions of dollars to bus patrol. So it was a pretty good. That was pretty interesting as well. Even the original use case may not actually be effective. Which kind of relates to the thing you wrote about a while ago, Jason, which is like Flock says reduce crime by XYZ amount or whatever. And you look at the numbers, it doesn't really gel.
- [47:07]
Emmanuel Mayberg
I mean that's. That is the thing with like all of these surveillance companies is that they have usually a few high profile anecdot that prove their point. Like for Flock, Flock was used to find mass shooter in Boston, I believe it was. And like Flock loves to now talk about that. But if you look at sort of like crime statistics and all that sort of thing, it's like the studies that say that the surveillance technology is reducing crime are usually studies that are funded in some way by the surveillance companies. Like we've seen that with Flock and often those studies are like, you know, they're in very specific cities and they're looking at very specific types of crime. And then they kind of ignore a lot of the, a lot of the cases where it's like, well, it was used to violate this person's rights, it was used to stalk people. Like all of these things that we've seen over and over again with these systems. And so yeah, it's like there's a lot of kind of cherry picking going along and I think that the jury is still out as to whether these things actually reduce crime. I would think also these systems probably can be used to raise revenue. As in just like to issue tons and tons of tickets. But like we as a society kind of to decide like is that what we want to use surveillance for?
- [48:42]
Joseph
Yeah, in the Bloomberg one there was also they spoke to a couple of people who were like incorrectly ticketed by the system. But yeah, people can go read the details there. Let's leave that there. If you are listening to the free version of the podcast, I'll now play us out. But if you are a paying 404me, a subscriber going to talk about a really big crash out, big time crash out, you can subscribe and gain access to that content at 404 Media co. As a reminder, 404 Media is journalist founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our articles and an ad free version of this video podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. This podcast is produced by Alyssa Midcalf. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really does help us out. Here is one of those from Lanoush Incisive tours of media. Not talky. Even though it's talk, the intelligent questions turn up. Wary of banter, this was a breath of fresh air. Thank you so much. This has been 404 Media. We'll see you again next week.