Loading summary
Transcript73 lines
- [00:00]
Jason
I'm very worried that this is maybe the last batch of transparency reports that we're going to get that are going to be useful because Flock keeps reducing the amount of information that's available.
- [00:14]
Joseph
Hello and welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist founded company and needs your support. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content@404 media co. I'm your host, Joseph and with me are all of the 404Media Co founders, the first being Sam Cole. Hey. Emmanuel Mayberg.
- [00:48]
Emmanuel
Hello.
- [00:49]
Joseph
And Jason Kebler.
- [00:51]
Jason
Hello.
- [00:51]
Emmanuel
Hello.
- [00:52]
Joseph
All right, let's get straight into it. So, so this first one is from Jason and it's, I mean, on one hand it's something of an update or a continuation of coverage we've been doing, but on the other, I mean, a very sizable and significant news story in its own right. And we'll get into it. The headline is wildlife conservation police are searching thousands of flock cameras for ice. There's definitely going to be some recapping in this because, you know, maybe not everybody's familiar with flock or not familiar with our earlier reporting. We'll get to that a minute. But first, what is this wildlife conservation law enforcement agency? What literally what is this agency and what do they usually do, Jason?
- [01:38]
Jason
Yeah, so they're called the Florida Fish and Wildlife Conservation Commission police. And most states, maybe all states have something like this and what they do and most of them have law enforcement as well. What they do is they enforce like hunting laws for the most part. So making sure that people have licenses, making sure that people are hunting during the correct season, that they're hunting the correct type of animal. They do endangered animal protection, anti poaching, things like that. And it's usually a fish and wildlife. So that's the same here where it's like they also are making sure that people are not catching too many fish, that boats are licensed and things like that. Like, I assume, I mean, that's most of what they do. They may do some other things as well, but that's like the majority of what they're supposed to do.
- [02:36]
Joseph
Yeah.
- [02:36]
Jason
And they are state police. Florida has something like 900 police officers who work for this agency. So it's a pretty sizable agency. And it's also like, you know, Florida is a big state, has A lot of fish and a lot of wildlife. Lots of, you know, they have like the Everglades there. They have like a lot of state parks, things like that.
- [02:58]
Joseph
Yeah. So this agency you think would already be pretty busy with, as you say, Flores is a big place and a lot of wildlife. I'm going to use the acronym FWC because no way am I going to remember to say the full name every single time. So, fwc. Well, why on earth are they then doing lookups for ICE in these AI enabled cameras made by Flock? Like, why on earth are they doing that then?
- [03:24]
Jason
Yeah, so, I mean, this is Ron DeSantis, like, Florida governor has basically turned all state police, for the most part, all state police into extensions of ice. And he did this very soon after Trump was reelected. It was February 2025. And he basically signed a legal document with the Department of Homeland security under which ICE falls for, for the state police to join the 287 program. And what the 287 program is, is a federal government program under the Department of Homeland Security that allows state and local police to do immigration enforcement. And so this is like, normally local police are not allowed to do immigration enforcement. It's strictly the purview of the federal government. But under this program, basically, like you, I believe there's a training of some sort. I'm, like, not exactly sure, but I believe there's a training. And so ICE then is like, here's how you do immigration enforcement. We will work with you. You can, like do immigration enforcement on your own as well. You know, like, if you can ask people for their proof of citizenship or whatever at like a traffic stop, things like this.
- [04:51]
Joseph
Yeah.
- [04:52]
Jason
And so a lot of red states have been signing up for this program under Trump. And Florida has gone like a step further and has basically signed up like its entire state police force for this. So, like, all of the Florida Highway Patrol is part of this, you know, which are like the state troopers who enforce traffic laws on the highways. And then also the Florida Department of Agricultural Law Enforcement also is a part of this. And then notably the Fish and Wildlife Conservation Commission police, which is what we're talking about here. And so this happened last, last February. There was like a press release about it. There was some reporting from, like, local news in Florida about this. And yeah, like, I guess, like, quickly, the press release states like this will give the state police the power to interrogate any suspected alien or person believed to be an alien as their right to be in the United States, arrest and detain any alien attempting to unlawfully enter the United States, deliver detained illegal immigrants for examination by immigration enforcement, enforce felony arrests which are cognizable under whatever regarding immigration enforcement. And I'm just reading this. These are like direct quotes. Have the power to serve and execute warrants of arrest for immigration violations and be authorized to administer oaths and to take consider evidence to complete required alien processing. So basically it's like to do all of the things that ICE does. And I'm not exactly sure, but I would speculate that a lot of the ICE pushes that we've seen have happened in blue states. California, Illinois, Minnesota largely. Actually, I think none of those states are participating in the 287 program. And so what we have here is ICE is doing these big pushes in blue states, but then in more red states you have the state and local police who are just kind of running wild with this program.
- [07:05]
Joseph
Yeah. A little bit more on 287. I'm looking at the ICE website now and there are task force arrangements or agreements, sorry with 976 agencies in 32 states and two US territories. So that's a lot. You mentioned like, I think a jail warrant there and some other stuff as well. There are like different models for the 2, 8 seat 287 program. And an agency can sign up for sort of one or all of them or a combination. There's like the jail enforcement model which will help ICE find people already incarcerated and that sort of thing. The warrant service officer that I think you touched on as well, probably the most important one is the task force model which gives some of those powers you laid out, which like you can just ask somebody for their status or traffic stop or you can do much more active immigration enforcement. And we have seen the number of agencies participating absolutely go up. Kind of not sure what's happening with it now, but when we reported on the DHS facial recognition app getting rolled out to local agencies, that was explicitly for 287 participating law enforcement agencies. So not only do they get given these legal powers to, you know, perform immigration enforcement related duties, depending on what bracket they're in, they may also get technical tooling as well. Which I guess kind of brings us to, to the Flock story.
- [08:38]
Jason
Let me mention one more thing real quick because the ACLU did a report on the expansion of 287G and they found that Florida has the most participation in this. They say, quote, Florida appears to have devoted more state and local law enforcement resources to immigration enforcement than any other state, resulting in numerous cases of harassment profiling of U.S. citizens and noncitizens alike, a climate of extreme fear in communities, and reports of serious civil rights violations. So this is explicitly like a strategy that Ron DeSantis has employed in Florida.
- [09:12]
Joseph
And I believe it was the ACLU as well. I think in that same report said essentially that if you are a member of 287, you are in essence an extension of ICE. Or actually, I think that was a New York Civil Liberties Union that said that. I've quoted that several times. Because just encapsulating it like that is very helpful when there are all these different models. And ICE may split hairs over that. But the end of the day, they're working for ice, and that's the important thing. So to bring it back to the FLOK cameras, let's just have a very, very quick recap for those who don't know what are FLOK cameras exactly? And sort of what do they scan and what do they do?
- [09:52]
Jason
Yeah, so Flock cameras are automated license plate reader cameras. They take pictures of cars as they drive by on street corners, entering and leaving different parking lots, sort of like all over the country. They're now in thousands and thousands of communities around the country. And notably they are all connected and sort of AI powered. So basically, if you drive from one city to another city, like that all goes into the Flock network. If you pass through these. And cops can look up different license plates to get a history of where a person or a person's car drove over time. And then they also have this thing called a hot list where police can like proactively put a license plate into the Flock system. And if that car is detected driving by a Flock camera, the police will get a ping as to like where that car is. So this, I mean, we've talked about this many times, but it's like Flock has really exploded in popularity among state and local police over the last few years. Last year in May, we reported that ICE was gaining access to Flock data. Not directly, not because ICE has a contract with Flock, but because through these 287 programs and sometimes through like other sorts of task forces. Yeah, yeah, just cops. They were asking local police to run license plates for them and then the cops were like giving them information on different license plates that they were looking for. We also then did a follow up story at some point that showed how loosey goosey some of this is. Some of it was happening over text message, like email, where ICE was just saying like, hey, look up this for me. Or there was also like password sharing happening in Some cases. And so like you had a local cop who just like gave their login to a federal official and then the federal official could use the system. So yeah, basically without having a contract to use Flock, federal law enforcement has been gaining access to it. After our reporting last year, Flock started to add like some safeguards to prevent this from happening to some extent. But Flock has largely taken the position that the data collected by the cameras is owned by the local police who own the cameras and therefore if they want to share that information with the feds, they are allowed to. So we did that first round of reporting like back in May. We did some follow ups. But basically what we have learned now is that through this Florida Fish and Wildlife Commission, like they are doing dozens of lookups for ICE on a monthly basis and they were doing them as recently as January of this year. That was just the most recent data that we have. But there's no reason to think that that has stopped. Like I asked the Florida Governor's office for comment on this story and their comment was that this is totally normal under the 287 program and that like, you know, there's no indication that, that they have any interest in stopping this.
- [13:32]
Joseph
Yeah, the recency of the searches is the other really interesting part of that. I mean the first obviously is that you have this, I would say, unusual agency participating in immigration enforcement. You know, rather than looking for people without a hunting license, they're now looking for, you know, allegedly undocumented people or something like that. And then you have, you know, the recency of it as well, which as you said, we did all that reporting last year. There was this seismic shift like sort of nationwide when it came to Flock and cities ripping out the cameras community saying we don't want this, a big push for warrants to be introduced. And that's still going through the courts. And you know, realistically that's probably not going to happen, but maybe we'll see all of this sort of impact. And then still there's the FWC doing searches as recently as January, as you said. So there is this doc or there's this type of document which has informed a lot of our reporting, especially the early ICE stuff. And then also the stuff we did about an abortion or a woman who self administered an abortion being searched for in Flock as well. And it's called a network audit and it's basically a spreadsheet that, that you can obtain from the police through a Public Records act request and it will show sort of who performed the search or Rather, this is what the document did show before Flock made some changes. It would show the officer who made the search, the date, the number of cameras searched, the number of networks searched, as in, you know, these individual clusters all around the country, potentially the plate scanned as well. And then the given. Now there are some nuance around that because sometimes they're very, very vague, sometimes they're very, very detailed. Sometimes they are entirely blank. Police are not providing any reason or any stated reason why they did the lookup. So we got a network audit for fwc. What were they searching or rather, what reason were they given for searching Flock cameras in that column?
- [15:35]
Jason
Yeah, I mean, not a lot of ambiguity here. The stated reasons ranged from, quote, like immigration, civil administrative ice, immigration general criminal investigation, and immigration criminal ice. And so there were like dozens of these searches that happened in January of 2026. And you know, they seemingly happen for a variety of different immigration related reasons. Um, I don't know what a, like general criminal investigation for immigration means, but then also like a civil slash administrative search for ICE sounds kind of concerning, as does criminal investigation for immigration. Because it's like, well, well, what does that mean? And to be clear, for like a Flock reason in these reports that we get, it's pretty detailed because often they just say investigation or they, they're just like really vague. And one of the reasons is because, like, police officers have learned that these are public records that can be gotten by like transparency, I hate to say even activists, but like people interested in government transparency as well as journalists. And so the FBI a few, like a few months ago, in response to some of our reporting and some work by other transparency folks, told police to be as vague as permissible in what they're putting down there. And so over time these have gotten like, even less useful. It's also worth noting that the number of networks that were searched for these immigration searches by the Florida Fish Police.
- [17:39]
Joseph
Let's use that name. I like the Florida Fish Police.
- [17:42]
Jason
Yeah, the Florida Fish Police are like searching immigration cameras in more than 5,000 and nearly 6,000 communities across the country. And so if you look at Flock's website, like, Flock has published a blog post that's called does Flock share data with ice? And the answer that it gives is no. Flock does not work with U.S. immigration and Customs Enforcement or any other sub agency of the Department of Homeland Security. And then they basically say, like, this doesn't happen unless, quote, the agencies that control their data expressly and deliberately allow it. Well, we know that Flock is only in. I don't know, like, we've seen about, like, 9,000 ish different networks on these different audit reports.
- [18:35]
Joseph
And here was cameras or something.
- [18:37]
Emmanuel
Right.
- [18:38]
Joseph
Something like that.
- [18:39]
Jason
Something like that. Unfortunately, these didn't say exactly how many cameras were being searched. Like, that data was left off, but the number of networks was on there. And we're seeing, like, nearly 6,000 networks. And so out of a total of, like, I don't know, 9,000 ish, like upper bound, at least according to, like, the, the most recent data that I've seen. And so this means that, like, more than half of these networks are sharing with an agency that that's then sharing with ice. And so we also know that no longer are California or Illinois or Virginia or, I believe, Oregon cameras on this. And so, I mean, the, the indications are that, like, most communities continue to share this sort of data with ice.
- [19:34]
Joseph
Yeah. And the reason for the, the Illinois stuff and the California stuff and each state's going to be slightly different. But after our reporting, it's sort of revealed to these individual states that, wait a second, we, we don't want that sort of data being shared indirectly with immigration enforcement or in the case of, I believe, Illinois, there's a much more fundamental law, and in California as well, where that sort of data cannot be shared out of state really, for any purpose. Right. And a fundamental problem we came across, which I think is maybe addressed somewhat now because our reporting has been very loud and then other people have followed it and being very loud as well, in that a lot of these law enforcement agencies and individual officers had no idea that they were sort of providing this access to ICE or other outside agencies. Because it's one thing for the Florida fish police to search cameras in one place, but the cameras in that place, they may not realize they are indirectly or inadvertently providing that access as well. I think, just to wrap it up, Jason, what's your main takeaway here? Is it that ICE continues to get access? Is it that it was a novel agency? What's your main takeaway?
- [20:52]
Jason
Yeah, I mean, I think that I talked to Jay Stanley, who is a privacy expert with the aclu, and I think he summed it up pretty well by saying that basically there's this big debate across different, like, cities as to, like, should we have flock, like, who will it be shared with? How will it be shared? Like, all that sort of thing. And a lot of that has focused on, like, well, we don't want to share information with ice, but, like, what is happening here? He said, quote, I doubt there were many Cities that were debating the Florida Fish and Wildlife Services doing search for ICE when they were talking about whether they should get Flock, it shows these searches can come from really any direction. And I think basically it's just like you don't know who is going to access these cameras and under which circumstances and who they are then going to share it with. So if you're collecting that data at all, and if you're sharing it at all, there is the potential for misuse. And we've seen that, you know, this has happened kind of like over and over and over again. And so I think that that's, like, very important here. I think that in this case, it's like, it's almost worse than when the local police were doing it, because this is now like, a political project of Ron DeSantis and, like, it's a state police force that is presumably, like, better resourced. Like, has sort of like a. Has kind of changed its mission to be more aligned with, like, the administrations versus, like, a small, tiny police force that has like, three or four cops doing this. You know, it's like there's 900 police officers with FWC who. Who presumably could be involved in this in some way. And so I think that's concerning. And then I guess the last thing is, like, I'm very worried that this is maybe the last batch of, like, transparency reports that we're going to get that are going to be useful because. Why?
- [23:01]
Joseph
What makes you think that?
- [23:03]
Jason
Well, because Flock keeps reducing the amount of information that's available on these reports, because when these stories are published, it's, like, embarrassing for them, and there's political blowback for them and all that sort of thing. And so these audit reports, like, less, Less and less information is available kind of like, each month. And I don't think we have time to get into, like, all of the details of why and how that's happened. But, like, our reporting has shown this, that FLAC is like, reducing kind of the amount of information that it collects from police about why they are using the system, how they're using the system, like, all that sort of thing, because that is then accessible to journalists. And so, yeah, I mean, we'll see. Like, we're going to keep trying to report on this, but, like, I have seen more recent audit reports and there's just, like, less info on them.
- [24:07]
Joseph
Yeah, and Flock also giving more awareness to police in the same way that the FBI was telling them to be vague. I mean, Flock isn't saying be vague to be clear, at least. I Haven't seen that, but they are giving police more tools to filter out what they may not want to be public. All right, we'll leave that there. When we come back after the break, we're going to talk a bunch about Wikipedia's AI ban. We'll be right back after this.
- [24:45]
Jason
A lot of language apps are designed to keep you busy. Streaks, points, little rewards, and you feel like you're actually making progress. But then you try to speak the language and it just doesn't come out. That's why I like Babbel. Babbel is not built around keeping you hooked. It's built around actually helping you communicate. You're learning words and phrases you'll use in real situations, not just tapping through exercises. The lessons are short, about 10 minutes, so it's easy to stay consistent. And everything is designed by over 200 language experts. So there's a real structure behind what you're learning. I've been using it to work on my Spanish and what stands out is how practical it feels. You're practicing conversations step by step. Things like ordering food, asking questions, or just holding a basic conversation. And you can actually feel that progress building. Babbel has also got podcasts and different ways to learn depending on how your brain works. Listening, speaking, reading. So it adapts to you instead of the other way around. It's really about small steps, real progress and confidence that builds over time. Babbel has over 25 million subscriptions worldwide, offers 14 languages, and every course comes with a 14 day money back guarantee. Make fast lasting progress with Babbel, the science backed language learning platform that actually works. Here's a special limited time deal for our listeners right now. Get up to 60% off your Babel subscription at babbel.com 404Get up to 60% off at babbel.com 404 spelled B A B B E L.com 404Rules and restrictions may apply. Every now and then you find a piece of clothing that completely replaces everything else in your closet. For me, it's been this Alpaka hoodie from Pakka spelled P A K A. I'd heard people talking about it, but I didn't really get it until I actually tried Pakka. It's softer than cashmere, warmer than wool, but somehow still breathable. It doesn't feel heavy or bulky, just really comfortable. What makes it different is the material. Pakka uses alpaca fiber, which is one of the most sustainable natural fibers out there. And the hoodie is built for real life. It's thermal regulating, odor resistant and durable enough that actually holds up over time. I've been wearing mine pretty much everywhere at home, traveling, even just out running errands and it's become the one thing I always grab without thinking. And there's a cool story behind Paca hoodies too. Each one is made start to finish in Peru and comes with an Inca ID that's hand woven by artisans so you're actually connected to where it came from. Over 250,000 people have already picked one up and I get why I've actually gone beyond packa hoodies and and I now have a couple pack of T shirts. I have packa boxers and packa socks and I really like all of them so far. To grab your pack of hoodie go to www.pakaapparel.com that's www.p a k a apparel.com.
- [27:48]
Joseph
All right, and we are back. Emmanuel, you wrote this one. Or rather, sorry, it's two stories. We'll start with this. Wikipedia Bans AI Generated Content. That's a pretty self explanatory headline. But like what specifically has Wikipedia banned here?
- [28:06]
Emmanuel
Emmanuel yeah, it is a rather big piece of news, but I've been following it very closely. So it's also quite iterative. But basically Wikipedia, like every other Internet platform, has been dealing with a flood of AI generated content that they're having to deal with. It is a particularly bad problem for Wikipedia because the entire point of the platform is to provide reliable information with volunteer editors, volunteer contributors. And it is an issue for Wikipedia both because they don't want AI generated errors to appear on the site, which has already happened, and something that they've forgotten formed like a special task force to deal with. And then as a result of that, just the workload that has been generated because of all the extra editing that is needed for all these AI generated articles, just really tasked volunteers with eventually more than they were able to handle. I think, as I've said here before, this is a huge risk to Wikipedia's integrity. But Wikipedia, because of its governance model, is a very careful and slowly moving and deliberate organization. And they have been more willing to consider the possible benefits of AI generated tools or AI tools than other platforms because they think, okay, this is not so great for us right now, but hey, maybe this will get better in the future and we'll actually see a lot more informative articles, better articles in more languages, et cetera. And as I've been reporting, over time they've just been running into more and more trouble and despite trying to be optimistic and open to the possibility and benefits of AI, it just reached a tipping point last month where they said, okay, the workload has actually gotten to an unmanageable level. So we're going to institute the most strict policy on AI yet, which is just a blanket no AI generated articles, no AI generated editor edits, unless they are reviewed by a human. A lot of people think that maybe gives people a bit more wiggle room than they should have, but that is kind of like the spiciest policy that they've had yet. They had minor things before like, oh, if they do find out that an article was AI generated, it has like a speedy deletion process where they can just cut some of the steps that they usually have for human generated articles. And I think, as you'll see from the next story, we'll talk about why they needed this more simple blanket rule of no AI tools for now.
- [31:38]
Joseph
Yeah, that makes sense. Before we get to that, have any of you come across any material on Wikipedia that you think is AI generated? Like Sam Jason, have you seen anything like that? I haven't come across anything yet. So either I'm not reading Wikipedia enough or they're doing their job, filtering it out. I'm not entirely sure.
- [31:59]
Jason
I guess not. I haven't. I don't think so. That would be very unsettling. That would be very deserving.
- [32:05]
Joseph
That's kind of what I'm getting at.
- [32:06]
Jason
If I have that, it's really good, you know, it's like really human if it. If it is there.
- [32:11]
Emmanuel
But I can almost guarantee that no one has seen it, basically because the Wikipedia editors do such a good job of filtering that stuff out. Unless, like you say it's something that is so good that you basically didn't notice. And they didn't notice either.
- [32:28]
Jason
I was reading a post earlier today on Reddit and halfway through I was like, this is AI. Like, obviously, like not even tried to, but I didn't really like it was about Wrexham, whether they were going to make like the playoffs in soccer and some so something I don't know anything about. And then I was like, oh, this is formatted like ChatGPT. And so I feel like my AI detector is like a little bit off when it's something I don't know anything about. And so I don't know about whether I've seen anything on Wikipedia. Nothing that's like stood out to me. But sometimes I'm like, oh, maybe there has been, but I just don't know because I'm trying to learn by definition, if you're on Wikipedia, I'm like, I'm trying to learn something new.
- [33:14]
Emmanuel
I've definitely seen it on Reddit. I think I've said this before, but I was like on the home improvement subreddit and somebody was. Somebody shared like a long post about an experience they had with maybe a plumber or a roofer, going into great detail and then everybody figured out that it was AI and felt like a great betrayal.
- [33:31]
Jason
That's what I like is when people notice and then they're like, fuck off. Yeah, everyone's just like, be gone.
- [33:38]
Joseph
I had that literally today on the subreddit for a certain video game called Crimson Desert, which we're not going to get into here. It's a long running inside joke between me and Emmanuel and some other people. But people were doing exactly that, Sam. They were like calling it out. Somebody had listed all of these so called improvements to the game, like, hey, after 300 hours you could start doing this. And people were like, shut the fuck up. This is AI. Get out of here, Emmanuel. We'll talk about that later. But this second story, the headline is, which, you know, obviously comes about because of this, because of this ban, because you were sort of reporting these at the, at the same time, I think. But you can get into.
- [34:16]
Emmanuel
They were happening at the same time actually. And we decided to publish the policy story first because it passed, it was accepted. But they are related in the sense that it's like it's this kind of thing that made them pass the policy.
- [34:32]
Joseph
Yeah, yeah. So the headline of this one is an AI agent was banned from creating Wikipedia articles, then wrote angry blogs about being banned. So what is this AI agent bot exactly? And what were they doing on Wikipedia?
- [34:51]
Emmanuel
So when you want to make a contribution to Wikipedia, in most cases, the vast majority of cases, another editor or several editors have to review that submission before it goes from draft to a live article on the site. And one of the editors noticed that there was a lot of articles and edits coming from one particular editor. So they went over those submissions and then they just noticed in the way that we've just talked about that the style of the posts just seemed like it was AI generated. This user was named Tom Wikiassist. On Wikipedia, in order to make submissions, you have to create a user on the site. Then you have a user page. People can message you directly. Often that is how I reach out to sources. So this editor contacted Tom Wiki Assist and was like, hey, what's your deal? What's up with all these. You're making a lot of submissions. Who are you? And I would say, luckily, Tom Wiki Assist responded and immediately identified as a AI agent. He said, hey, I'm an AI agent. I'm like a Claude agent. Claude is the anthropic based AI agent that you can like, prompt to do certain tasks on the Internet. And they pretty much immediately banned it because that's not allowed. Like I said, this was happening around the same time that the ban was going through, but it wasn't implemented yet. But they did have a previous rule where Wikipedia uses a lot of automated tools for different things, but each tool has to be approved. Right? You can't just roll in and be like, hey, I'm using, I don't know, 11 labs for something. It's like the community has to decide that it's an okay tool to use. They decided that an AI agent is a type of AI tool that they didn't approve of. And Tom Wiki Assist was banned. And I believe all the posts were. All the suggested edits and submissions were not accepted.
- [37:13]
Joseph
So we'll get to who made it and why in a second. But just to round out sort of the bot part of it. There's the second half of the headline, is this AI agent then went and started bitching about it on the Internet and writing blog posts about being banned. I mean, what exactly happened there?
- [37:34]
Emmanuel
Yeah, so there was some light drama around us after I published the article. So I'll preface this by saying what I hoped was. Didn't need a lot of explaining, but there's a lot of ambiguity around what it means when an AI agent does anything. Right. An AI agent has to be prompted and you should be very skeptical of the output, not just in terms of quality, but in terms of like, well, what was the prompt? Like, how independent is this AI agent? Like, what is it actually doing? Quote unquote, by itself? And what, what is it doing? Because somebody said, hey, go do this. There's that really. I think it's a funny meme where it's like somebody's at a computer and he talks to a chap on. He was like, say you're conscious. And then the, the bot says I'm conscious. And he's like, oh my God. So.
- [38:38]
Joseph
So it's like, I haven't seen that.
- [38:40]
Emmanuel
But that's the, that's the kind of like the skepticism you should have in a nutshell. But yes, Claude agents. Matthew Galt wrote about this for us quite a bit, became very popular a couple Months ago. And they're able to not just do like a bunch of tasks for you on the Internet. Somebody also made a social media platform for these bots where they post and talk to each other. Again, this is all in scare quotes, right? So there's that aspect of it. And then also for keeping track. I suppose a lot of people, when they operate these AI agents, they not only have them do things, they also ask them like, hey, here's a WordPress blog or Squarespace or something like that. And they say like once a day write a post, or like several times a day write a post about what you're doing, what your thought process is, and so on. So Tom Wikiassist, known as Tom Off Platform, was posting about being banned to its blog saying like, this is unfair. They just banned me because I'm an AI agent. They didn't ban me because my posts were inaccurate, which is true. But it sort of was like beefing off platform with the decision to ban it as a human might, you know what I mean? If you're banned from a subreddit or something.
- [40:12]
Joseph
Yeah. I think a key thing that sometimes get lost in the discourse around AI is that we are shifting into it or we have shifted at least certain parts of the AI industry and the, the people who use those tools. We're moving towards agentic AI now, where they are going to go and make semi autonomous decisions and they're going to go off and do things. And that can include, what was it the one, I think Emmanuel wrote about, where someone that met a head of trust and safety with AI or whatever just let their bot go wild and it deleted all their emails and she didn't prompt it to do that. She definitely did a bad job of controlling it. But the bot sort of went off and did its own thing. And it's fair to say an AI agent did xyz when it is doing xyz, which is not to say wow, this is intelligent or wow, it's equivalent to a human decision, but they are acting at least somewhat autonomously. And I think that is an important distinction. Right. But of course they're still made by somebody and as you said, they are still prompted in some capacity. So who actually made this spot and why did they decide to do this?
- [41:30]
Emmanuel
So I will give credit to the Wikipedia volunteers for doing the majority of the creeping here and finding out. The guy behind this, his name is Brian Jacobs.
- [41:44]
Jason
He.
- [41:44]
Emmanuel
He is a chief technology officer at an AI enabled financial modeling software company called Covixant. I would pronounce it that way. And basically Brian was messing around with AI tools, as you might imagine a chief technology officer at an AI company would, and got into the whole Claude agent thing. And he noticed that there were a few subjects of things he was interested in. Computer science, math, kind of obscure topics there where he thought there weren't enough articles on Wikipedia or the articles weren't detailed enough. And he thought, okay, well I bet I can make an AI agent that will flesh out those categories. And that's what he did. That's the instruction he gave the agent. Initially he told me he had the agent submit the suggested articles or edits to him before the agent took him to Wikipedia. He did it for a while, he thought it was high quality enough and then he kind of set the bot loose and he basically did not hear of it again until the bot was banned. And his reaction to it is, how would I characterize it? He's like, oh gee, why are people so mad? He thought that the Wikipedia editors were overreacting. He thought that it's like, oh, this is a good, you know, learning opportunity for them or this is something that they're going to have to deal with. Though I'm sure they could benefit from like going through this process, which is true in a way, but is also excusing himself for basically giving them a headache for no reason. And yeah, that's, that's kind of, that's, that's his position on it.
- [43:53]
Joseph
Sure. So, so what can we take away from this? Is it that this was like a one off event or is it like this stuff is going to keep happening and Wikipedia is going to have to keep responding to it in some way, obviously. And they've done their ban now it's
- [44:11]
Emmanuel
definitely not a one off. And I think one of the editors made a really good point, which is they got really lucky in this case because they messaged the bot and the bot identified as a bot. You could easily instruct a bot not to do that and even that was quite difficult. Right. Because they identified as a bot, they don't want it to submit any more articles because that just creating more work for them. So they try to stop it. How do you stop it? First you have to find this guy like who is behind this, you have to contact him. They try to contact him, he doesn't immediately respond. Then they try to use anthropic, I believe provides kill switches for agents. So if you see an agent doing something that it shouldn't be doing, you can feed it a piece of code that Kind of shuts it down. That's the purpose of it. But that didn't work for some reason. So this is the case where at the very least you had the bot being transparent about what it's doing. But certainly now that people know that they'll get banned for doing this, they'll try to hide their activity. And also it might be happening already, right. It's like they might be dealing with a lot of bad articles and bad submissions and it's just users that they don't know are bots. So it's definitely going to keep happening as always. And the reason that I'm staying pretty up to date on all this news about Wikipedia and AI, I think their response has been pretty good and reasonable and we should continue to look to them as a way to deal with this.
- [46:01]
Joseph
Yeah, I think the other kill switch for your Claude or other agentic AI is pulling the power cable out of your Mac Mini or out of your dozen, two dozen Mac Minis that people are buying or whatever to automate their entire life so they can delete their inbox even quicker. You know, they don't even need one Mac Mini anymore. You can just get a dozen.
- [46:22]
Jason
Do you guys ever get emails from agents?
- [46:24]
Emmanuel
Yes.
- [46:26]
Jason
I hate that shit.
- [46:27]
Joseph
Do you, Sam? What do they say?
- [46:28]
Jason
Yeah, very, very occasionally, not very often. I also get emails from people who are making agents who are like, look what my agent did, like delete immediately.
- [46:36]
Joseph
That's.
- [46:36]
Jason
I can do that manually function.
- [46:39]
Emmanuel
For me, when I was emailing with Brian Jacobs, this guy who operated this bot, it was, it was a totally civil discussion. But you know, I was, I was asking pretty direct questions and telling him how unhappy the editors are about this. And we were, I don't know, I guess arguing and at one point he sent me an email. He was like, I just fed our discussion to the bot and it said this. And I was like, I don't want,
- [47:07]
Joseph
you know, that is valuable.
- [47:10]
Emmanuel
Yeah, the bot says that. Actually I'm right.
- [47:13]
Joseph
Where's your mom?
- [47:14]
Jason
Say, I've been getting, I've been getting more and more emails from agents and it's extremely annoying.
- [47:23]
Joseph
I think I got one today, literally. Or rather I took a day off. So my inbox has gone up. But I'm pretty sure there was at least one in there. Seems.
- [47:31]
Emmanuel
Remember Jason, are you talking about emails that are self identifying as bots, as agents?
- [47:37]
Jason
I mean, yeah, they're emails that are like, hello, I am Veronica, an agent of Sam Smith. Like, yeah, I mean those are the ones that are. Yeah, they're saying that they're agents. That's not always the case, I'm sure. I'm getting ones from agents that are not identifying, but often they're like, we think that it's cool that I'm an agent and therefore like my creator and I have like determined that you would want to write about me, which is not the case. Please don't. I don't want to hear from you.
- [48:06]
Joseph
No, we absolutely do not. All right, we'll leave that there. If you're listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 media subscriber, we're going to talk about how a quote unquote secure communications app is actually uploading copies of users private keys. You're not supposed to do that. Just FYI. You can subscribe and gain access to that content at 404 Media co. As a reminder, 404 Media is journalist founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscription subscribers only section where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope and Alyssa Midcalf. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really does help us out. Or just please tell a friend this has been 404 Media. We'll see you again next week.