Loading summary
Transcript70 lines
- [00:00]
A
Foreign.
- [00:04]
B
Hello, and welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist founding company and needs your support. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content@404 media. Cat CEO, I'm your host, Joseph, and with me are all of the 404 Media co founders, the first being Sam Cole.
- [00:36]
A
Hey.
- [00:37]
B
Emmanuel. Mabel.
- [00:38]
A
Hello.
- [00:39]
B
And Jason Kebler.
- [00:41]
C
Hello. Hello.
- [00:44]
B
Jason, do you briefly want to explain why your background looks.
- [00:48]
C
Why there's a mosquito net behind me? Yeah, I am in. I'm on Lamu island in Kenya, which is not where I normally record. I was invited to take part in this AI conference. AI and journalism conference, and specifically how to keep humans relevant in the age of AI. So I've taken a very long trip. Very lucky to be here, and I'm actually going to be doing a little bit of reporting while I'm here. Kenya is like a hotbed of data labeling, which is like content moderation for ChatGPT and AI tools, as well as training a lot of these tools. So I'm meeting with some folks here. We'll probably record some podcast material for the future. But this is. It's been a really very fun trip for me. Very lucky to do it. I've met a lot of really great Kenyan journalists, including a lot of people who focus on the same things that we do, like surveillance and privacy and things like that in a much different context. So hopefully we'll be able to have a little bit more info about that on future episodes and talk to some of them about some of the work that they're doing while I'm here.
- [02:06]
B
Yeah, that sounds really, really interesting. And do you want to take us through this first story?
- [02:12]
C
Yeah. So let's first talk about this story. It's called Elites and quotes the Palantir app ICE uses to find neighborhoods to raid. This is kind of the latest on our reporting on the surveillance tools that ICE uses, and specifically the latest in a series of stories that you've been doing about Palantir. Do you just want to give us some of the background about how you got started on the story before we jump into specifically what Elite is?
- [02:44]
B
Yeah, so. So, as you say, we've been covering ICE and its technological backbone and Palantir stuff for a while. You know, months ago, we first revealed that Palantir was getting much closer to ICE with a leak through Palantir and some public procurement records I looked at with this story I was hearing about something called Elite, and I didn't really know what it was. There were just rumblings of it. And then eventually I. I got this user guide which explained in much more detail what it was, what it was capable of. And of course, we'll get into that a little bit shortly as well. And then also this testimony from Oregon, which, again, we'll get into more specifics in a minute, but ICE and Customs and Border Protection officials talking really explicitly about not just this Elite tool, but also mobile phone, Fortify, the facial recognition app that we've covered, and some other stuff as well. And then, of course, I found the Palantir connection as well, which, frankly, when I figured that out, it was, oh, I have to write this quickly, because this is taking on a new sense of urgency. Not that it wasn't important before, of course, but as soon as you realize Palantira is involved, I mean, I think people are interested in. In that for better and for worse. Sometimes it seems sort of this omnipotent sort of bogeyman that can do anything. But I think it's really important to know what Palantir is doing during the second Trump administration. So this story sort of brought all of that together.
- [04:24]
C
Yeah, I mean, I think some of our first reporting on the Trump administration and on ice's activities were about some legacy contracts that Palantir had with the Department of Homeland Security to build these big databases for the government. Soon after things started, there was this reporting about something called Immigration os. And something that was never really clear to me at the time was like, what was. Or what is Immigration OS like? We had these contracts, we had this name, Immigration os. We knew that it probably had something to do with aggregating data and providing it to ice. But with elite, which, you know, there's all these names floating around. Like, I don't actually know if Elite is part of Immigration OS or if Immigration OS has become elite or whatever, but we now know about this program called Elite. So that is being used by ICE right now. What is it and what is its purpose?
- [05:25]
B
Yeah, it's a dedicated tool for ICE to basically acquire lists of targets or people, obviously. And I should say that maybe as a force of habit, I'll use the word target just because that's sort of the terminology that ICE uses in the user guide. But of course, I'm talking about people, individuals, you know, human beings. So I apologize If I lean on that term. But I remember when you were editing Jason, you were like, it's okay to use target a couple of times, but try not to use it during the entire copy, which I did, because I was just referencing the material, but just keep that in mind. So it allows EYES to draw up a list of people from a particular neighborhood, drill down on them, and then export these lists of people that they want to target or detain. It's meant to be basically the entire life cycle from, well, let's find a neighborhood. Okay, well, let's find who lives there. Well, now let's get a list of those people. Now let's have a supervisor approve this operation and then go out. And to make it a little bit more tangible, one part of it is a geospatial interface, or rather, it's a maps interface, Right? So ICE can kind of like the tool we spoke about last week, which is different, to be clear, they can draw a shape on a map, you know, a circle or a rectangle or whatever. The map will be populated with all of the people that ICE may wish to target there. You can then click on an individual person, bring up their dossier, and now we'll have their name, their date of birth, a photo of them, if that's available, and their current or known address. And this is the most important part, I think, because it also comes with a confidence score. Say it'll be 88.2 out of 100 or 79 out of 100. And that is based on where the data is coming from and its recency. And I've mentioned this before, but as we keep covering ICE and this technology and its data sharing agreements, one thing comes up over and over and over again. It's addresses. ICE wants them. It is like gold for them where they can go. We believe this person is at this location or this apartment block or this particular flat or apartment or whatever. That is what they're trying to get. We've seen that where they've hired bounty hunters and PIs to verify addresses. So this elite tool essentially brings all of that together. Now, I don't mean all of the data we'll get into where some of it comes from, but I don't know where it's all coming from necessarily. But it's to turn that data that ICE has going into this tool into something they can actually act on. It's not like an abstract technology product over here doing something that we can't really visualize or think about or sort of touch or understand. This is what it seems ICE is using to actually plan where to go and who to pick up.
- [08:38]
C
Yeah, I mean, that's part of the headline where it's like the Palantir app ICE uses to find neighborhoods to raid.
- [08:46]
D
Which I guess we'll talk a little.
- [08:47]
C
Bit more about this case in Oregon in a minute. But was there anything else from the user guide that you sort of wanted to highlight before we get into that?
- [08:57]
B
Yeah, I think it would just be some of the data sources. So again, with the addresses, there was a little bit in the user guide that said, hey, the Address Confidence Score is this new feature in Elite, and those addresses come from. It says the Department of Health, which is pretty interesting. You know, a lot of people aren't going to assume that data from the Health Department is going to be used for immigration enforcement, but obviously this is a different world. Now. There's mentions of uscis, which is part of DHS as well. Right. And then there's also a mention of Clear, which is hard to report when companies don't respond to requests for comment. We don't think it's Clear, as in the system lets you go through the airport quicker. Right. It seems to me that it's probably Thomson Reuters data product called Clear. Reuters has sold tons of data to the government in the past. They regularly contract with ice. We've covered that before, so it looks like they gained data from that as well. That being said, I don't know if that's all of the data sources, like, oh, remember all of that stuff about the IRS giving addresses to ice, could that be fed into here? I don't know, like, potentially. But those were the ones that were mentioned in the user guide, at least.
- [10:23]
C
Yeah. So, I mean, I know you had sort of like sourcing on this and that sort of thing, but the government actually has talked about the use of Elite in this case in Oregon. And you got testimony from that case from, I believe it was an ICE officer or maybe a DHS official. So what is the case in Oregon and what did you learn through that testimony?
- [10:48]
B
Yeah, so the case in Oregon is about a woman whose initials in the court records are mjma, if I'm recalling correctly. And she was detained along with around 30 other people during an immigration raid in Oregon in October. I think she was briefly detained, taken to a facility, then released with no conditions by ice. In the testimony, she said that she entered the US repeatedly on a tourist visa. She's now seeking asylum. So that's sort of the context of this case. And she's Suing the government to protect her constitutional rights. I believe that's all well and good, that's the context, but there's a ton of interesting stuff in the testimony of that case from ICE officials, from officials from Customs and Border Protection as well. And there are mentions of Elite in there where they say, this is the ICE official, I believe. Yes. You use Elite to find densely populated areas. Like, you're not going to go to a house that just has one pin on it. You're going to say, go to an apartment complex with a load of pins on it because there's probably going to be more people to arrest or detain there and in text messages, according to.
- [12:09]
C
Sorry. So when you say density, though, when you say density, it's not density of just like people living there. It's like density of suspected undocumented immigrants. It's like density of potential targets, right?
- [12:24]
B
Yes, density of potential targets. That's absolutely it. Now, you know, there are, of course, some questions it raises, such as, well, how reliable is the data that ICE would have on undocumented people? If they're getting data from, you know, DHS and uscis, isn't that going to be people legally in the country? And it's going to be very case by case and frankly, it's quite messy. But they have a lot of data here to go on and that's apparently what they're using it for. So that case, MJMA's case, it was actually reported briefly by local Oregon media in December because there was a hearing that month and there was a very brief mention of the mobile for the Mobile Fortify app being used on that woman and returning the wrong name twice. You know, we picked up on that and fleshed that out with some additional reporting as well. And of course, it tied into the Elite Palantir piece here as well. And I should just say that, you know, we left Vice well, because the company was falling apart, obviously, and it was going bankrupt. But one of the catalyzing events for me to, you know, leave and then make this with all of you was that one day I tried to get some court records for my job at Vice looking up some court case, which you do every single day, and it's supposed to cost 10 cents and couldn't do it because Vice wasn't paying its credit card bills. Apparently in this case, I email the court, I get a quote for $650 for these court transcripts, which is a lot of money. That's just how much transcripts cost. But we could just do it because we own the company now. If we're like, it's probably worth paying $650 to get this very, very important transcript, which has already result really important stories. It's just refreshing that we're able to do that. I just wanted to mention that.
- [14:28]
C
And we published the transcript too, so you don't have to pay $600, which is nice.
- [14:32]
E
Yes.
- [14:34]
C
So you mentioned Palantir a lot. Palantir is in the headline. We know Palantir made this, but the user guide you have actually doesn't say that. I don't think so. You had to sort of establish that link. Can you talk how you did that?
- [14:49]
B
Yes. So in the testimony and when I'd heard about Elite, it was just spelled either as a single word or an acronym or, you know, just by itself. But this user guide spelled it out and it's a really, really complicated one. I'm actually pulling up the Wikipedia. Somebody has added our article to the Palantir Wikipedia because I can't access the site right now on this screen. But it stands for Enhanced Leads Identification and Targeting for Enforcement. That is the spelled out acronym that was in the user guide. And I thought, well, what exactly does that relate to? So I googled it, literally, that phrase. The only result on the Internet at the time was an additional piece of information in a Palantir contract where it said Palantir was being paid 29.9 million for work on a few other things, but including configuration of enhanced leads, identification, targeting for enforcement. So it is clear that Palantir is working on this in some capacity now. Did Palantir originally design the system? Did Palantir build it from the ground up? Are they just setting it up and then somebody else is going to take over? That isn't clear. It just says they provided some sort of continuing configuration on this system. But for me, that's an incredibly strong link where I've never seen the acronym anywhere else except in this Palantir contract. I mean, I think the connection is pretty clear.
- [16:29]
C
Yeah, I guess. Just to round this out, what comes next?
- [16:34]
D
What's your takeaway from this?
- [16:35]
C
And then, uh, I mean, presumably we're going to continue to follow the use of this and other related tools, but. Yeah. What, what do you think this shows?
- [16:49]
B
Uh, I think for what comes next, presumably they'll put more data in there. Part of the user guide says it's integrating new data sources, so it sort of leaves open the possibility of adding more. But my main takeaway is that I kind of touched on it a little bit earlier, but we've been covering for months that there's all of this abstract technology being built and it's sort of happening over there in the background or in the shadows. And even when we've got leaks from inside Palantir describing the company's work with ice, it's always been quite abstract and obscure and you can't really put your finger on it because it's just dressed in all of these very confusing acronyms and technical jargon and all of that. This story, I think is the clearest example yet of what Palantir is building and what ICE is actually doing on the ground. It is clear that this tool is part of ICE's mass deportation effort because that is literally what it's designed for. And now we've seen that there is a through line between the tech the Palantir is making and what is happening on the ground. It's a combination of all those things. It's the user guide, it's the testimony from Oregon is the public procurement records. And yeah, I just think this is the clearest link between the company and what is actually happening.
- [18:10]
C
All right, let's leave that there.
- [18:12]
B
Yeah, yeah. And when we come back after the break, we're going to talk about one Emmanuel worked on about. Well, maybe just wait and see. We'll be right back after this.
- [18:31]
A
Foreign.
- [18:36]
E
It's funny what people will do all to hide the smell of the litter box in their house. I've seen some very strange purpose built contraptions in my time. Knew a guy who built a whole cabinet system out of wood just to contain his litter boxes and keep the smell from getting out of there. I don't have that problem. You know why? Because of Boxee. The pro in Boxee Pro stands for probiotics which stop the bacteria that cause odors so you never smell your litter box ever again. Boxy Pro keeps the box continuously odor free infinitely. Just gotta remember to scoop. All you do is top off the litter. You never have to dump out the whole box. Amazing clumping power that makes scooping easy. I've got three cats and I didn't have to build a special cabinet like my buddy because of Boxee Pro. If you're tired of switching litters looking for the one, get Boxee at box I E C A T dot com. It's the last litter you'll switch to. Enjoy 30% off with code 404Media@boxycat.com 404Media. That's B O X I E C A T dot com 404Media January is.
- [19:54]
D
Short and February is even shorter. If you're going to resolve to change your year, you better start now. That's why now is the time to start that business you've always wanted to with Shopify. Shopify has made it easy for us to set up and manage the 404 Media merch store. We're one of millions of entrepreneurs who have made the leap and with Shopify's easy to use tools, you can get a professional, well designed store up and out the door in just minutes. Shopify gives you everything you need to sell online and in person. You can build a beautiful store with customizable templates and even create email and social campaigns from the same dashboard. As you grow, Shopify grows with you. Handle more orders, expand to new markets, add new products, and handle the backend all from the same dashboard. In 2026, stop waiting and start selling with Shopify. Sign up for your $1 per month trial@shopify.com media. That's Shopify.com media hear your first this year with Shopify by your side at 404 Media. We know your personal information is everywhere. Online data brokers crawl the web to build profiles, selling your home address and phone number to anyone with a credit card. Trust us, we know this better than anyone else. This exposure leads to risks like doxxing and identity theft. That's why we recommend Deleteme, a hands free service that removes your personal info from hundreds of data broker sites. Their experts find and remove your information, providing a detailed Privacy report within 7 days. DeleteMe monitors these sites all year long to ensure your data stays removed. We feel like we're at increased risk because of the types of stories we cover and the public nature of our job. But the truth is, identity theft, threats and harassment can happen to anyone. I've been doxed. In fact, lots of us have been doxxed. And each time Deleteme has helped us pick up the pieces, get our data removed from shady websites, and otherwise clean up our online presence. I've been using Delete Me for years and every quarter I look forward to seeing what new, interesting and horrifying websites they've had my data taken down from. It's kind of a perverse little game. Take control of your data and keep your private life private by signing up for Deleteme now at a special discount for our listeners. Get 20% off your delete me plan when you go to JoinDeleteMe.com 404Media and use promo code 404Media at checkout. The only way to get 20% off is to go to JoinDeleteMe.com404Media and enter code 404Media at checkout. That's JoinDeleteMe.com 404Media code 404Media.
- [22:43]
B
All right, and we are back. Emmanuel wrote this one, as I said, and the headline is Instagram AI influencers are defaming celebrities with Sex Scandals. Honestly, I had to read the lead, like a couple of times just to get my head around it, because I'm sure it's very clear to you, Emmanuel Ray, Simple. But I've been out of the loop of sort of what these AI influencers are doing for a minute. And this kind of spun me out. To start, can you give us some examples of some of these images that are being posted? Then we'll get into the how and the why. But, like, what are we looking at when we're talking about this?
- [23:19]
A
Yeah, that's fair. It is confusing. I kind of had the same experience when I tried to write it down. We all look at Instagram and I think Jason and I especially get a lot of AI generated content and AI generated influencer content. And I look at it every day, so I'm steeped in it. And it all seems very obvious and regular to me until you try to describe what is actually happening. And then it's extremely weird and convoluted. But to your question, I think a good example for one of these images is, or the post, rather is it's an Instagram reel. You will see a very realistic looking image of a woman courtside at a basketball game, taking a selfie with LeBron James. And it will say how it started and then it will play a song and cut to an image of the Same Woman and LeBron James in bed, under the covers with their shirts off, all sweaty, hair tussled, obviously seeming like they've just had sex. And again, very, very convincing image. Sometimes it's an image, sometimes it's a slightly animated video. And that is just one example of many. Like any celebrity that you can imagine, there is a video of them with the exact same format and they are all over Instagram reels.
- [25:08]
B
Gotcha. But just to clarify, obviously the Mike Tyson or the LeBron James or whoever the celebrity is, they're not real. Obviously that is an AI generation of that celebrity. But also the other person who is allegedly taking a selfie with LeBron or whoever is also not real. I think that is, as you say, somewhat obvious to all three of you. Where you've looked at this AI stuff a lot, but maybe people who are just coming to this might be scratching their heads being like, wait, the influencer is not real either. Can you just briefly explain that?
- [25:46]
A
Yeah, in the vast majority of cases, I think there are out there are outliers and we can maybe talk about those. But in the vast majority of cases, the personality behind the account that is posting this content is itself an AI generated woman, female character, almost certainly created by a man. And Jason and I have done a ton of reporting about the people who create these accounts and looks very realistic, but yes, entirely AI generated.
- [26:26]
B
Gotcha well on that. Who is making these AI images then? Is it grifters? Is it people trying to make money? Is it people just looking for clout? Like, what's a profile of somebody who is making this?
- [26:43]
A
Yeah, so I think it was a year ago now. Jason and I wrote this article and have done a few follow ups since, but I think we're due for a refresher or a couple new stories about what's happening in this space. But it is a community of people who are kind of like in this hustle bro culture, get rich quick scheme culture, who are just into this latest fad in the same way that they are into dropshipping or cryptocurrency or what have you. You can plug in to this whole community that exists on Instagram and Discord and other places and learn how to AI generate a personality, gain traction with that personality on Instagram, and then funnel users via that attention to other places online where you can monetize that same AI generated personality by selling AI generated nude photos of them.
- [27:51]
B
So that's the play. It's to basically entice people with, oh, look, I have a sex tape with Mike Tyson or whatever. I know they don't say sex tape necessarily, but you see what I'm getting at? They say that and then drive them to like an OnlyFans or an OnlyFans equivalent. Is that the play?
- [28:13]
A
Yeah, that's always the. That's how you monetize the attention. Right. It's like you funnel people to specifically Fanview, which is an OnlyFans competitor. OnlyFans so far has pretty strict limitations on AI generated content. They do not want it on the platform, which I think is good and surprising, but we'll see how long that lasts. And Fanview basically offers the same service as OnlyFans, but has a more permissive policy, which essentially boils down to you can have a totally AI generated personality with AI generated Nudes, you just have to disclose that it's AI generated. So the flow is, you're on Instagram, you see an AI generated woman, you're interested, you go to the bio, you find a link to fan view, you click through, and then you can buy nudes or videos from this AI generated character, and it will stay on fanview that they are AI generated. And I think that's in response to our reporting because we flagged this happening to fanview a few times where those people weren't disclosed as AI. So now that tag is there, but it's at the bottom, it's in the fine print that it's an AI generated personality. And I think it would be very easy to miss. But, yeah, that's generally the flow. And the story is really what we call a story about the current quote, unquote meta in the same way that people discuss the Twitch meta. Right. It's like people who stream on Twitch or in the Twitch culture talk about how, like, what are streamers doing right now in order to get attention on Twitch? And it's the same thing on Instagram. And we've seen several metas, and this is just the latest one, and it is extremely common and extremely viral. I just had that classic thing where you see one of these posts and then because of the hashtags, because of the music that people use on the posts, and because of my engagement with it, I just got like an endless stream of these and seeing all these celebrities, I was just like, up one night looking at this on my phone, and we do this thing on Slack where we send ourselves messages just to like, remind ourselves stuff to blog in the morning. And I just, like, kept messaging myself, like 20 Instagrams in a row of. Of this exact thing.
- [30:52]
B
Yeah. That will bring me to my other question. But before that, I just want to ask Sam and Jason, have you, either of you come across this organically? Like, I feel like Jason's answer may be skewed because he's probably in the really bad bits of reels. But, Sam, have you come across. Is it like broken containment and come across your feed at all, Sam, or not yet.
- [31:13]
F
Not really. Which is surprising considering I assume the algorithm is basing my algorithm on my connections. Algorithms. Emmanuel sends me terrible things all the time, so it hasn't caught on to that yet. I don't know why I haven't seen it just in my normal personal scrolling. So, yeah.
- [31:33]
B
Jason, did you.
- [31:34]
D
Are you talking about the.
- [31:35]
C
The celebrity stuff in particular?
- [31:38]
B
Yes.
- [31:39]
C
So I. I hadn't seen this until Emmanuel sent it to me. But a huge amount of my reels in general are AI generated influencers. I feel like at this because I follow them and I buy their products. No, I mean, it's because I've written a lot about them, but I feel like there's all these niches within the AI influencer space. Like Emmanuel has written about the ones where they have been altered to look like they have down syndrome, for example. But I am now getting a lot of how to make your own vibes, like the hustle bro thing. And so it's like I'm getting a lot where it's like, starts as an AI generated woman and then the woman morphs into the man who made her.
- [32:36]
B
Oh.
- [32:37]
C
Which is like, I get a lot of jarring. Yeah.
- [32:41]
B
Very creepy.
- [32:44]
C
Creepy. And I think, I mean, Manuel mentioned it, but I think like, that will probably be our next follow up is sort of the like world that has, I don't know, of like men on yachts standing next to their AI influencer, who they are. Like, this woman doesn't exist, but she made me rich.
- [33:05]
B
Yeah. So the celebrity ones, they have gone viral. You know, they're pretty. Maybe even popular is the wrong word. They're just viral. I'll just stick with that term. Right. And a lot of people seemingly are seeing them. Do you think, Emmanuel, people actually believe it? Do they actually believe that, oh my God, LeBron James did XYZ? Or does it even matter whether people believe it or not?
- [33:34]
A
So they're definitely viral. I gave a few examples in the story and I just checked on it while we were talking. And yeah, many of these posts have millions and millions of views. I think the largest one I've seen is 20 million views. That was on Jon Jones video showing Jon Jones in one of these scenarios. Jon Jones being an MMA fighter. I don't know why that one took off. But very popular. Whether people believe it. Sam and I have discussed this a few times. It's something that we've attempted to report out. It's a more complicated question than you might imagine. If you go into the comments, you see broadly two types of posts. One of them is, I would say, thank God, people saying, clearly this is AI. And I think that shows that generally speaking, Internet users are becoming more educated about this type of content and they can just sort it out. They can see that it's not right real. But then the other bucket of posts is people just like posting fire emojis and saying, oh, my God, you're so beautiful. Please post More I love you stuff like that. And like, are those bots? Are those actual people? Is it like sock puppets that are just engaging with the account in order to boost its numbers? It's hard to say. I would, I would guess that there is a non negligible number of people who do believe this stuff, but we have not been able to quantify and scientifically prove that's the case.
- [35:19]
B
Sam, maybe just briefly on that, because as Emmanuel said, I remember this as well. You've tried to reach out to people, the people who look like old men who are replying to an AI influencer, like, wow, you're so beautiful, you're so gorgeous. Like, I think you tried that. Did. Did you ever hear back, like in general or.
- [35:41]
F
I sent so many DMs to so many people's dads that I gave up. It was crazy. And I think a lot of them probably were also bots. It's hard to tell, right? Because it's like, it's just like a picture of this guy who just looks like a normal kind of cringy guy and doesn't have a lot of like organic posts, but is like commenting on every AI models, you know, image on the platform that could be a real person. And probably a lot of them were real people. I didn't get anywhere with that. And I. It's something that I occasionally pick up is like when I see one of these floating to my feed and then I see a bunch of people commenting, I'm just like shooting off DMs to these guys. Maybe my DMs are going to like their requests or something and they don't know how to find them. Yeah, the most actual response that I got from anyone was a guy just started going through my profile and liking all of my posts and didn't answer me at all. And just liked every one of my posts from the past like three years. And then I made my profile private. So I don't know, it's like, I guess if you know someone who's really into replying to AI models posts in real life, ask them why and send them my way. If you're the child of one of these guys, that's your dad, please email me because I want to know what that's.
- [37:06]
A
Calling all uncles, please reach out.
- [37:09]
F
Calling all chopped dogs, please email Samu Co.
- [37:13]
A
The one thing I'll say that it's like good evidence that Sam, somebody is buying into this is that it is an ecosystem, right? It's like there are so many of these accounts, even if it's just like a handful of people who are making a ton of these accounts. And as we've discussed, they all funnel eyeballs to places where they can be monetized. And if you look at Fanview, those accounts do have subscribers. People do buy the images. I can't tell you for a fact that those people don't know or don't know that the images are AI generated.
- [37:51]
B
But they might not.
- [37:52]
A
Somebody is buying the images, you know what I mean? And this wouldn't exist as a phenomenon on Instagram if that wasn't the case.
- [38:00]
B
Yeah, maybe just the last thing I was going to ask is what do you think might happen now? And I bring that up because I think when you posted the article on social media, maybe you mentioned something about you reckon these celebrities might start finding someone to sue and that could be meta. Maybe I'm misremembering what you posted, but, like, do you think lawsuits might happen here because the celebrities didn't consent to this? Obviously?
- [38:29]
A
Yeah, that's a very good question. The thing that actually caught my eye and, and eventually made me want to write this because I've been looking at it for a while, is that I saw that post with LeBron. And as Jason broke the news a while ago, there was a very notable case where LeBron's lawyers reached out and sent a cease and desist to one of these groups that's creating AI generated videos of him. In this case, it was like, I apologize. It always gets like really stupid when we talk about this stuff. Talk about stuff. But it's like videos of pregnant LeBron in jail with P. Diddy and stuff like that. And I think that probably trickled up to him and he was like, okay, enough is enough. And they sent us cease and desist to that group. And then I believe Jason was in that discord and they actually stopped making those videos. So I was like, okay, let's. Let's highlight that this is happening still by. In another format. And yeah, I think, I think it's just a question of like how spicy and annoying it gets for these celebrities until they do something. But I definitely think they have standing to do something, whether they go after individual users or Instagram as a platform. I don't know, but it's like, it's pretty. Like it's not, it's not nice and it's unambiguous. You know what I mean? It's like the accounts are not flagged as like, this is parody or I'm joking. It's like literally they're lying about having affairs with celebrities in order to gain clout and sell, you know, like, content that could be defamation. It's. It's pretty bad.
- [40:13]
B
Jason, were you going to add to that?
- [40:16]
C
I mean, I was just going to say that, yeah, like, we have seen when there's, like, even the tiniest bit of pushback from some of these celebrities, like, in the form of a legal letter that they've stopped. And you kind of, like, hate to put the onus on the victims of this, but, like, John Jones and LeBron James and, like, I don't know, the people who are being put into this are not, like, what's happening is not, like, the most graphic type of NCII that we've seen. Like, it's not good. But at the same time, it's like, I do feel some of these celebrities could make quite a difference by, like, making a thing out of this, and I wish that they would, because we've seen, like, we've seen women who've been deep faked talk about this and talk about, like, how it affects them. You know, like, celebrities have. Have finally kind of gone on the record and talked about this and that. And we know, you know, sort of the really horrible things that some less famous women have gone through when they've discovered, you know, deep fakes of themselves on the Internet. Like, Sam's done a lot of great reporting on that, but, like, if LeBron starts threatening to sue these people, if Jon Jones starts threatening to sue these people, I think it would make a real difference, not just in terms of, like, they would stop making deepfakes of LeBron James, but, like, perhaps, maybe the platforms would start thinking about actually moderating this sort of thing. Maybe some of the communities would, like, shy away from stealing people's likenesses at the very least. Like, I don't think this industry is going to go away, unfortunately, but I do think just like, the Vegas of legal threats might actually do something. And it's weird that we've only really seen that one from LeBron James, like, a couple months ago, but not much else.
- [42:29]
B
Yeah. And as Emmanuel alluded to, it's bordering on defamation or, well, I guess under US Laws, it probably wouldn't be libel.
- [42:42]
A
Right.
- [42:42]
B
But I can definitely see some celebrities not responding to this very well. And, Jason, totally agree with you. Maybe it could be an avenue to some other change for, frankly, normal people as well. We'll leave that there. If you're listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 media subscriber, we're going to talk about a pretty big change around AI at Comic Con. You can subscribe and gain access to that content at 404 Media co. As a reminder, 404 Media is journalist founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the Subscribers Only section where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope and Alyssa Midcalf. Another way to support us is by leaving a five star rating and review for your podcast. That stuff really does help us out. Or please just tell a friend about us as well. This has been 404 Media. We'll see you again next week.