Loading summary
Sam Cole
Foreign.
Joseph
Welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist founded company and needs your support. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content@ 404 Media co. I'm your host, Joseph, and with me are the 404 Media co founders, Sam Cole.
Sam Cole
Hey.
Joseph
Emmanuel Mayberg.
Emmanuel Mayberg
Hello.
Joseph
And Jason Kebler.
Jason Kebler
Hey.
Joseph
So one small bit of housekeeping. We are having a party gathering event in Los Angeles, July 30, Wednesday at 6:00pm I'll put a link in the show notes, but Sam, anything else you want to shout out now while I'm just doing it? Or should people go to the event page if they want to see us hang out and talk about 404?
Sam Cole
Yeah, go to the event page and buy a ticket or email one of us for the code. If you're a supporter. If you're a subscriber in the supporter tier, which means you're paying supporter of 404 Media, you can get in free. And we have the password. So we'll have a post up on the site soon with the actual password for subscribers to get in. But until then you can just hit us up. But yeah, we're excited.
Joseph
Yeah, sounds good. Again, that's July 30th, Wednesday at 6:00pm Check the event page for all the latest information, but it'll be great to see some of our subscribers. Jason, where are you going?
Jason Kebler
Yeah, it's going to be at a place called RIP Space, which is like a hacker DIY space. We'll probably do some sort of like live podcast, live Q and A, something like that where we're sort of nailing down what it is we're going to do. Probably by the time you hear this, there'll be more information on both the event page and our website. But yeah, we'd love to see you. And there'll be a party after.
Joseph
Yeah, sounds good. All right, we have as ever, a bunch of stories to go through here. The first section is going to be a series of pieces written by Emmanuel, all about AI porn, Civiti Hugging Face. All of these different platforms are hosting it and there's been a ton of developments over the past few weeks and months in this world. So the first piece is. The headline of the first piece is a 16z backed AI site. Civitai is Mostly porn, despite claiming otherwise. So Emmanuel Civitai is this popular AI platform. We've spoken about it a bunch. You've done a bunch of investigations into it. We've especially covered how it's used to host models and generate pornography. What is Civitai's stance on that? Do they see themselves as this big provider of pornography or not?
Emmanuel Mayberg
Well, they definitely didn't used to. And we can kind of talk through where they landed these days. But historically, Civitai, which is a platform where people share AI image generation models, they've always allowed adult content, they've always allowed AI models that generate the likeness of real people. They just didn't allow people to post models or images that combine both of these things. Right. So no non consensual content. That policy was always there. A lot of my reporting has been about how they failed to properly enforce that initially on their own platform, but then especially off platform. Also, while I was talking to them for all of this reporting, whenever I managed to get their PR person on the phone or talk to anyone there, they said that my reporting is unfair because yes, there is adult content on the website and they support free speech and they think people should be allowed to have porn models and have models of real people. But that's really a minority of the content that is on the site. And I have said in my stories and I have told them directly that, that just like as a user experience, as someone who goes to their main page, if you look at the top models, the top images, the latest models, the latest images, it's just a never ending stream of adult content. But I didn't have the data to back that up. So I've always kind of presented what a user sees versus what they named especially. They made this case especially strongly after I published this piece that showed that Civitai's cloud computing provider called Octoml, had a lot of internal discussions about the kind of content they were generating for Civitai and they were worried that they were generating child pornography for Civitai. And that really freaked out the company. And they then went out a kind of like a mini PR tour with their CEO, Justin Mayer. And one of the things he did, I think the day of me publishing the story, right? So like in order to coordinate with the deadline that I gave them, he gave this exclusive interview to VentureBeat and he said, let me pull up the.
Joseph
While, while you're pulling that up. This is back in 2023, right? This is an investigation you wrote pretty on or 2024 maybe, but quite early on in our history. Right?
Emmanuel Mayberg
Yeah. So this is. End of. This is December 2023. And VentureBeat writes that contrary to those figures showing 60% of content on Civitai as not safe for work, a figure derived from 50,000 images, today, users on Civitai generate 3 million images daily. And the company says, quote, less than 20% of the posted content is what we would consider pg. 13 and above. So this is. Justin Mayer, the CEO of Civitai, explained to VentureBeat that the kind of audit that Octoml, this cloud computing provider, did on the content they were generating, they saw, they thought it was 60% of the content, was explicit. They say that it's less than 20%. Cut to. Recently, I wrote a story about Civitai changing a lot of its policies because of pressure from payment processors. So in May, they stopped, or they banned, rather, all models that generate the likeness of real people and also specific types of porn that the payment processors thought was too extreme. I wrote about this. I published a story. A researcher from the University of Zurich reached out and said that she has been scraping Civitai and she has, like, really comprehensive data on what exists on the platform. And we can dig into some of the numbers if you want, but basically it shows that their claims that it was Only less than 20% of the content are absurd. And the content is primarily used for adult content.
Joseph
Yeah, this is really, really interesting because you have your reporting over years at this point, where it's just like, there's a lot of porn on here. And I think you did a story a long time ago as well, which basically showed that one of the driving forces of capital and innovation in the AI industry is to develop porn, basically. And of course, we all know that from Sam being the original reporter, to cover deepfakes as well. It's always here all the time. So you have the reporting. You have. Not to undermine it or anything, but your experience of the website is more anecdotal, as you say, you log onto the website.
Emmanuel Mayberg
I don't have the data.
Joseph
Yeah, exactly. So I see a lot of. There's a lot of poor models on here. You finally have this data, and we'll talk about what those numbers are in a second. But you said this researcher was scraping Civitai. I don't know whether they got this granular review, but sort of. How did they do that? Did they look for all images that were marked as not safe for work, or did they grab videos and analyze them? Did they break down how they did the scraping? Exactly. Or do they not go the detail.
Emmanuel Mayberg
They didn't technically explain how they gathered all the data. I'm not sure what the specifics there are, but the data is composed of more than 40 million images that they took from the site. Each of those images has a bunch of metadata that Civitai itself generates. This isn't the researchers annotating the data or analyzing it or anything. It's data that Civitai itself tags as using its own rating system of like RPG 13r sorry xxx. Right. That's kind of its own method for tagging images. And then there's other tags that could include like names of celebrities or the type of sexual act that's in an image and things like this. They also did something similar to the models themselves. And they did that for 230,000 models I think are in the paper. I've been in touch with the researchers and I've looked at some data that is more comprehensive than that. I think they're up to like 400,000 as what they have. But it's not published yet in a journal or in a draft.
Joseph
Yeah. So it's interesting that it's based on the classifiers. Whereas Sifiti itself saying this is not safe for work, which. Well, that meme, that gif of they just admit it. It's almost like that tasteful. So the researcher scrapes that, has all these images, has all these classifiers very handily attached to them. Well, what's the research showing? What numbers are we talking about here when it comes to the prevalence of pornographic content on Civitai?
Emmanuel Mayberg
So in December of 2023 is when I published my story about Octoml. That's when Justin Mayer give this quote to VentureBeat. He says less than 20%. The researchers say that by October of 2023, 56% of all the images were tagged not safe for work or higher. Right. So that is already like more than double than what Mayor claimed. The actual number is likely higher because as both the researchers notice and as I have noticed and reported, not everything is properly tagged. Some people self identify their content as not safe for work, others do not and maybe try to get it under the radar. Right. If you have like some non consensual content or a non consensual model up on the site. Maybe the people who are doing that are not tagging it properly because they're trying to evade moderation. So the actual numbers are higher. Now I'm looking at a graph of kind of the distribution of adult content and safe for work content from 1Q23 to 4Q24. And Mayor was incorrect or misleading in his statement back when he made it and back when I was talking to him for my reporting. But from that time until now, the number of overall images and models on the site has absolutely exploded, as has the proportion of adult content. It's like, at this point, it looks like 80, 90% of everything that is on the site is adult content.
Joseph
I mean, that's staggering, right?
Emmanuel Mayberg
Yeah, I mean, it just, it shows the reason the hearing from this person and reading the paper was very validating is because it shows that my user experience of Civitai and what I thought the site is, despite what it was presenting itself as to the public and investors, it's just porn. It is just a porn site. I feel very comfortable saying that it is a porn platform. And actually, when I reached out to Justin Mayer for this story, he said he's standing by his VentureBeat comment, which I think are completely wrong. But he does now admit that somewhere around the beginning of 2024, the reason people use the site has changed. And I think that is probably because it's like you can't deny it anymore. Just like by looking at the numbers, by looking at the site, it's just undeniable that it is primarily used for adult content, or at least was until these policy changes.
Joseph
Yeah, I mean, that's really interesting. And then just the last question on this article, then we have a couple more to touch on related to this, but you brought up these policy changes and we'll go into more detail about the exact why of this with the payment processes in a bit. But basically, Civitai banned models related to real people in May, which is obviously a huge seismic change for any sort of AI platform. What happens when Civitai banned AI models based on real people? According to this researcher who's constantly scraping Civitai, did it fall off a cliff? What happened?
Emmanuel Mayberg
I don't think, first of all, I should say this is another very important number and another very interesting finding of the paper, because they just recently changed the policy. So the researchers have all these models in a spreadsheet with the metadata and also the links to where the models live. So once Civitai announced that it was removing real people content from the site, it was very easy for them to check how many models were removed overnight. And the actual number is more than 50,000, which is another kind of interesting. You kind of. When we, When I report about Civitai, it's like, you know, there's Taylor Swift and there's Natalie Portman and like, you know, all these big celebrities that have like multiple models. And then you see there's a bunch of YouTubers and lesser known Twitch streamers. But like, you're like, what is the actual number? And it's, it's much higher than I thought. It's like I can't even think. Like, obviously there are multiples, but, but like tens of thousands, tens of thousands of people have bespoke AI models to create their likeness. It's just like really kind of a shocking image, shocking figure. But to your question, I don't think they still, they have like post policy change usage number and post policy change, like content distribution. They can just see what was removed. That was posted before the end of 2024, I suppose.
Joseph
Yeah. So then this just leads on to two other stories and the title of this podcast is something like the A Exodus starts. And that's where this comes in, in that Hugging Face, another AI model hosting platform. You have a headline here. Hugging face is hosting 5000 non consensual AI models of real people. So are these some of the models that were on Civitai and then they got removed and now they've moved over here? What's the deal here?
Emmanuel Mayberg
Yeah, so basically, credit card companies come to Civitai. They say, hey, you have to remove all this content, all these real people models, or we're not going to work with you. They are not working with them anyway, still, even after they remove that content. Maybe Sam wants to talk about this, but we saw something similar with pornhub. It's kind of really hard to undo a payment processor cutting you off after they make that decision. And that's where Civitai is at the moment. But when they initially announced this, they were like, hey, we're making these models invisible. The people who made them can have access to them for an unspecified short time and then we're just nuking them off the site entirely. And the moment that they announced this, the Civitai community immediately mobilized and started this. Civitai, or actually the Civitai archiving project already existed in the form of a discord channel because there was other policy changes that kind of made people feel like something was coming. And they were starting to starting an effort to like archive anything because they were afraid that Civitai would remove it. Once this news came out, it really kicked into high gear and they started grabbing everything that they can in order to make it available elsewhere on the Internet. And because of this research that I talked about in the previous story. I had a spreadsheet with all the links of all these real people models where they used to live on Civitai. And the archiving project created a website where you take that link, you enter it into the website and it directs you to a mirror where that model was re uploaded to Hugging Face. So Hugging Face is like an AI tool and resource sharing platform. It's very popular. It's got like a multi billion dollar valuation, a partnership with Amazon. They've been very outspoken about wanting to be like an ethical AI company and what that means to them. I talk about that a little bit in the story. They don't have any specific policy against models that generate the likeness of real people, but they do have a lot of kind of vague language about AI being needing consent in order to be ethical. And I told Hugging Face that I have this data that I know of at least 5,000 models that are hosted on their site. And not just random models, models that I know were used to create non consensual porn models that were removed from a different, as we just discussed, porn site because they were used for this purpose. And I just haven't heard back from them after multiple attempts.
Joseph
You haven't heard back from the ethical AI platform company about the unethical stuff happening on their platform?
Emmanuel Mayberg
That's right, yeah. And I've talked to them before and they've replied to my emails and I don't know why they're stonewalling me about this one, but they are. I think I can say we're recording unusually we're recording this podcast on a Monday. By the time this podcast is out, my story will be out and we'll see if they have anything to say then. But I tried really hard to get any comment from them and I think if they, they asked and I would give them all the links and then they can decide what to do about them, but they were just not interested in doing that.
Joseph
And then we'll just touch on your last story very briefly. But yes, I just wanted to ask Sam for a moment. This last one is about payment processes and sort of the power that they have in the porn industry. Right. And for those who don't know, well, sorry, I'll ask this to you Sam. For those who don't know, what do we mean by a payment processor exactly? And why are they so important to the adult industry?
Sam Cole
So as Emmanuel just alluded to, pornhub has had this problem and a lot of porn sites have to abide by the rules of Payment processors, when you're talking about the different payment processors and gateways and banks, it's kind of it ladders up through the institutions that we're talking about. But so like, for example, Stripe has really strict rules against porn and sexual content. Visa and mastercard stopped processing payments for pornhub because there was allegations of abuse material on pornhub. You know, it's like Chase has rules about not safe for work content, adult content. It's because they're considered high risk, quote, unquote, high risk material. So in the same, in a similar vein as gambling or guns, they place sexual content in a similar kind of risk category as a bank. And then that trickles down to payment gateways and the processors and things like that. So, you know, it's like, it's. If it's working, ideally, what you get is like these banks are not having to be accountable for abusive content. Like what Emmanuel's talking about, what happens a lot of the time is the banks end up being like, acting as like moral arbiters for porn. So you have a lot of like consensual adult content creators caught up in the rules against abusive content. These payment processors don't want to take the time to sort out the difference in a lot of cases and just ban it all. And we've been talking about this for years happening to consensual and safe and legal adult content. And it's like these other platforms, like Sipitai and some of the other platforms that are now having trouble with payment processors, they're around and finding out at this point, it's like, this is. You thought you could kind of just like get away with doing whatever and you were above it because you were cool bros or whatever the logic is. I don't even know it was making a lot of money. So surely it was too big for them to give a. But I think now they're finding out that the same rules apply to them. And I don't know, we can't really put these things in the same bucket as legal, safe, adult content. But they do a lot of the time have to abide by similar rules.
Joseph
Yeah. And it's coming for them now. I guess. Just to wrap it up, Emmanuel, what is the impact from these payment processors deciding they don't want to work with Civitai or another AI platform or what happens after that sort of action has taken place and what's going to happen now?
Emmanuel Mayberg
Very quickly, I would just say I think last week I talked a lot about how the reason I was covering Civitai is that to me it seemed like a critical piece of Internet infrastructure in the entire practice of producing non consensual torn. And like what happens if you remove that? Where do people go? And something I saw people say after the Civitai announcement around real people is that they're just going to go to Tensor Art, which is a site I covered like I think the first week we launched. But it's a site that's identical to Civitai. Basically. It's the same services, same ui, you go there, you download models, you can generate images, yada yada. A lot of, a lot of people move their models there. For a while Tensor Art had like a tool that even helped people import their Civitai models automatically. And on Friday, Tensor Art announced that they are getting the same pressure from payment processors and they're no longer, they're saying temporarily. I don't know how they resolve this, but they're saying temporarily, no more real people content. They've disabled this Civitai importing tool and they're trying to come up with some sort of solution. I'm very skeptical that they can find a solution and still have the payment processors work with them because as I've said, sometimes even if you fix every problem, they don't want to come back. So I'm just going to continue to track, there's a huge vacuum now and how non consensual content is made online because of Civitai policy changes. And I'm just tracking where all this energy and all these people are going. And that's probably going to be some more reporting in the future.
Joseph
Yeah, totally makes sense. All right, we will leave that there. When we come back after the break, we're going to talk about one of Sam's stories about preserving history, very much in the vein of the Trump administration continuing to wipe things off the Internet. But this is now sort of IRL archiving. We'll talk about it in a minute. We'll be right back after this.
Jason Kebler
Time. Skills and money. When it comes to growing your business, we all know that engaging with our customers is crucial. But the question is always how? Most of us don't have the time, skills or money to put towards keeping up with marketing every day. We're too busy running our companies to begin with. Well, let me introduce you to Constant Contact. Your mini marketing team awaits. Constant Contact's award winning marketing platform is here to make marketing way easier and way more effective for small businesses like yours. The best part, you don't need to know anything about marketing. Your struggle with expensive, slow and unmeasurable marketing is over. With their all in one platform, you can create and manage attention grabbing campaigns in just a few clicks. Email, text, social media events, landing pages, you name it, it's all in one place. You also get automated sending, real time reporting and tools that actually help drive sales. So you're not just marketing your business, you're growing it. And if you ever get stuck, friendly phone support is included with every plan. That's why over half a million small businesses trust constant Contact to stay connected, top of mind and ahead of the competition. Constant Helping the Small stand Tall Every podcaster, content creator and business knows that in an increasingly fractured information ecosystem, it's getting harder to reach your audience. Constant contact can help. Get a free 30 day trial when you go to constantcontact.com try constant contact free for 30 days at constantcontact.com constant contact.com you know what doesn't belong in your epic summer plans? Getting burned by your old wireless bill. While you're planning beach trips, barbecues and three day weekends, your wireless bill should be the last thing holding you back. That's why I made the switch to Mint Mobile. With Mint Mobile, you can get the coverage and speed you're used to, but for way less money. And for a limited time, Mint mobile is offering three months of unlimited premium wireless service for 15 bucks a month. So while your friends are sweating over data, overages and surprise charges, you'll be chilling, literally and financially. Say buh bye to your overpriced wireless plan's jaw. Dropping monthly bills and unexpected overages, Mint Mobile is here to rescue you. All plans come with high speed data and unlimited talk and text delivered on the nation's largest 5G network. Use your own phone with any Mint Mobile plan and bring your phone number along with all your existing contacts. Ditch overpriced wireless and get three months of unlimited service from Mint Mobile for 15 bucks a month. I switched to Mint Mobile from one of the big incumbents and the only thing I noticed was a much smaller bill this year. Skip breaking a sweat and breaking the bank. Get this new customer offer and your three month unlimited wireless plan for just 15 bucks a month at mintmobile.com 404media that's mintmobile.com 404media upfront payment of $45 required, equivalent to $15 a month limited time new customer offer for the first three months only. Speeds may slow above 35gb on the unlimited plan. Taxes and fees extra. See Mint Mobile for details this limited edition inbound 9-3-5 brings attendees to San Francisco for a one time only west coast event with insights they won't find anywhere else. Just revealed, the inbound 2025 agenda is now live from the Agent AI workshop. From idea to agent and Dwarkesh on AI's future research backed bold predictions with Dwarkesh Patel. Explore 200 plus sessions all created for your growth. Get fresh perspectives on innovation from a dynamic lineup including Sean Evans, the host of Hot Ones, Creative Force Amy Poehler, tech reviewer Marcus Brownlee, and AI pioneer Dario Almadeh. They'll bring their unique approaches and expertise to inbound 2025. Cut through the noise with focused, actionable takeaways on the latest marketing, sales and AI trends that give businesses a competitive edge in today's rapidly changing landscape. Network with decision makers in San Francisco's AI powered ecosystem where innovative technologies are creating entirely new approaches to business experience. Firsthand how San Francisco's technology ecosystem is revolutionizing content creation, distribution and monetization through AI and innovative tech solutions. Secure your spot@inbound.com register and if you want the VIP experience at Inbound, don't wait. VIP tickets are almost sold out. Get exclusive perks that help you make the most of every moment, including a welcome party, exclusive networking opportunities and early registration. Access to limited capacity sessions. Registration for limited capacity sessions begins in August. Don't miss this opportunity to secure your ticket in advance. Start favoriting your must sees and be ready when reservations Open session reservations for VIP ticket holders begin August 5th and GA opens August 12th.
Joseph
All right, and we are back. Sam, you wrote this one. The headline is SaveOur Signs wants to save the Real history of National Parks before Trump Erases it. First off, can you just give us a little bit of context of what is the Trump administration doing to national parks and the history around them? What's the issue exactly?
Sam Cole
Yeah. So like many things these days that are just like bad news, dark news started with an executive order. In March, Trump issued this order called Restoring Truth and Sanity to American History. Already off to a really promising start with that title. The order mostly like targets the Smithsonian Museum Institution and that network. It makes some really bizarre claims about the museums being part of this like revisionist movement that they're trying to paint historical milestones in the negative light for acknowledging racism in American history, things like that. So the the order is mostly about that, but because it also mentions the Department of the Interior that catches national parks and monuments in that net also. So I can just read part of what the the order says because it's written like so many of these things in a very kind of slippery, vague way. Yeah, I'm sure it was lawyer to hell and back, but also very wordy. So I mean, who knows, you know, it's nothing was in all caps. So I'm sure it got at least one proofread. But yeah, it says that the Department of the Interior must take action, whatever that means, as appropriate and consistent with law, to ensure public monuments, memorials, statues, markers or similar properties within the Department of Interior jurisdiction do not contain descriptions, depictions or other content that inappropriately disparage Americans past or living, including persons living in colonial times. Interesting. Carve out. And instead focus on the greatness of the achievements and progress of the American people or with respect to natural features, the beauty, abundance and grandeur of the American landscape. So, I mean, if we can use our contextual thinking skills, take action, I think we can assume means remove or edit monuments and memorials and plaques basically at national parks or anywhere that's under the Department of Interior jurisdiction that say anything about anything other than how pretty the landscape and the monument is and not disparage people. Which would, you know, rules out quite a bit of American history if you can't really tell the truth about what was going on.
Joseph
Yeah. So purely hypothetical situation, you can have a sign that says, wow, trees look nice, mountains look good, but you can't say anything like, well, the indigenous people of this land are XYZ or something like that. Is that basically the thrust of it?
Sam Cole
Okay, yeah, I mean it's. And again, it's like this. The order doesn't really use any very specific examples for the national parks. But given the attack on like so called dei, the way this administration has been moving up until this point, we can assume that it means, you know, things like talking about indigenous people being slaughtered in this country, things like racism, civil rights and all of that, that actually built a lot of the. Helped build a lot of the park system. Black indigenous people, which played a huge part in building the parks. And you know, were many of them were there first things like that. It's just like there's a lot of history going on in the parks that we just are apparently no longer allowed to acknowledge. Environmental justice, women in leadership, things like that that the administration has targeted in the past already online, we can assume probably counts in this case too.
Joseph
Yeah, I mean, I haven't really been covered covering any of these sort of removal of history by the administration, but I don't know, that just drives me crazy. The idea that you were Going to delete history.
Jason Kebler
Jason, I'm curious if this stuff would be foiable. Theoretically, it would be. I think the crowdsourced aspect of it is very cool and makes it a group project. But theoretically, the National Park's offices should have copies of this somehow, like they had to go get them printed, et cetera. And theoretically, all of this could be obtained by foia, which is. Yeah, I don't know if they brought that up at all. I think that this is maybe easier than foia. Like, FOIA is a roll of the dice at this point, but, you know, a FOIA about this and maybe we'll file. Some would theoretically capture, like, attempts to take it down or like, alternate versions of the, like, what's being changed, et cetera. Like, if it were responded to appropriately.
Joseph
Yeah, totally. And I just looked it up and got the email address or the portal for the National Park Service. So we can do that or do it through Muckrock or whatever, but what these people are doing is sort of this other archiving project. So with all of those problems and that content text enter Save Our Signs. This campaign send launched. Well, what's the goal of this campaign, Sam, and who is involved, exactly? And how are they going about it? What are they doing?
Sam Cole
Yeah, so seeing the writing on the wall, so to speak, not to use a corny pun for this particular story.
Joseph
That was pretty good.
Sam Cole
The writing on the placards. Data preservationists from the Safeguarding and Research and Culture Project and the Data Rescue Project launched this web portal called Zero Signs. They're asking people, if you're at a park, if you're in a national park, to just take a picture of pictures of placards and signs and monuments and that they see as they come across them, to preserve them, to archive them, essentially, because they could be under threat. We don't, again, we don't really know what's going to go on. Like, the Take Action line is very vague, but, you know, it's. It's something where they're. The government is asking for visitors to fill out this QR code survey to report signs that they think are negative. So it's kind of like a counter protest to that saying, you know, if you're, if Trump administration is going to ask people to essentially snitch on signs that address, you know, anything other than beauty and grandeur of the American landscape and, you know, flag these placards that could fall under whatever the administration's definition of negative is, then they're going to turn around and say, you know, what, we're going to archive this and make sure it's preserved in case it goes anywhere, in case it is removed. And the negative, quote, unquote, negative content must be be taken again, using the words of the order, taken action against by September 17th is what the order says. So they're launching this to basically say we have to hurry up and preserve this in case something happens to it in the future. But, I mean, the Trump administration has totally gutted the Park Service at this point. It's. They barely have enough rangers to do basic administrative tasks in a lot of the parks. So this is just another task piled onto rangers and park workers that is silly and frivolous and totally wasteful. So we'll see what happens actually in September if anyone is around to take action against this negative content. But that's their goal. So it's worth taken seriously.
Joseph
Yeah. So I totally understand why the response to the executive order is crowdsourced. You need ordinary members of the public to go out and, hey, take a photo of this placard or signpost or whatever. Take a photo of that so we can archive it. It's, I don't know, unusual, weird, funny that the government has to crowdsource that bit as well, of the destroying bit because they can't even get the people to go destroy it in the first place. Or, I don't know. That's ridicul.
Sam Cole
So far, the responses to the QR code survey that the administration has put up on signs to ask people to snitch on negative content has been leaked to different outlets already. And the responses that people are putting into this survey are really funny. It's like there was one that was like, respectfully, go fuck yourselves. Someone put in there, the parks belong to the people, things like that. People are pissed. People do recognize a total frivolous waste of time when they see it, I think. Rarely do people show up to the parks and say, oh, this isn't, you know, perfectly optimistic about the beauty of nature. So I want something. I want my government to drop everything and do something about this. It's, you know, it's. You're there for a learning experience and an educational experience as well as the beauty. So, yeah, people are already. And that would be something funny to FOIA once. All the. Once it's had some more time to marinate, I guess, the QR codes and the responses to them, because I think people are trolling it at this point quite a bit. But that's not what the save our Science people are trying to do. They're trying to actually preserve what's on the signs.
Joseph
Yeah, I mean we might be able to still get some early on and FOIA them if the National Park Service still has a FOIA officer or public information officer at the end of the day. So the deadline to take action to remove all of these signs or whatever, potentially September. And then it looks like the Save Our Signs program is going to release the photos that they've crowdsourced in October, if I'm reading that correctly. Is there any indication of how many photos of signs they've got or how many people are participating yet? Or do we just have to wait and see whether people are trying to help help out here when October comes?
Sam Cole
I think we just have to wait and see. I mean I. They launched on July 4th. We were famously out, which is why we covered it a little late. But they were hoping to get people on holiday to take signs while they were out with their families and stuff. But I think it's a little early. It's only been about a week since they launched the project. But it would be cool to see, to see what they get back to see what they do with the data. They're going to make it public, like you said. They just are not totally sure how yet or they're still working on how that would work. But I think it's, I mean I think it's cool to get public participation in archiving like this. I think archiving feels kind of difficult or complex or unattainable for a lot of people. It's like, oh, these sites are going down. So there's this big archivist effort to preserve them or we have to kind of pull down the data and fill up a hard drive with it. And this kind of big effort that you have to be part of a community already to participate in a lot of the time is how it looks from the outside. But this is a nice like entry point into data preservation and archiving. You know, it's like you're outside already, you can just take a picture and put it on this website and you're helping preserve a piece of history when it's under threat. Which I think is. I think it's cool. I think it's a cool kind of muscle building exercise for people to participate in something like this.
Joseph
Yeah. And as you suggest, absolutely non technical as well. Usually you have to be pre technical to participate in some sort of archiving solution. You have to download a browser plugin or scrape something and this is just something you can do to archive IRL material. Well, I'm sure you'll revisit it in September and October and we'll file those FOIAs, but we'll leave that there for the moment. If you are listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 media subscriber, I said we're going to talk about something. Jason's going to rant for about 15 to 20 minutes so you can subscribe and gain access to that content. Hey. What?
Jason Kebler
That's not true. That's not true. We're gonna have a discussion.
Joseph
Yeah, of course we are. Okay. You can subscribe and gain access to that content at 404 Media co. As a reminder, 404 Media is journalist founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404 Media Code. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. This podcast was produced by Michael Hermes. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really helps us out. This has been 404 Media. We will see you again next week.
Release Date: July 16, 2025
Host: Joseph
Co-Hosts: Sam Cole, Emmanuel Mayberg, Jason Kebler
Podcast: The 404 Media Podcast
Episode Title: The AI Exodus Begins
The episode kicks off with Joseph welcoming listeners and briefly mentioning an upcoming event in Los Angeles scheduled for July 30th. Sam Cole elaborates on the event details, encouraging subscribers to attend by purchasing tickets or utilizing a special code for free entry reserved for supporters. Jason Kebler adds that the event will feature a live podcast and Q&A session at RIP Space, a hacker DIY space, followed by a party.
Notable Quote:
Joseph [00:38]: "Welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL."
Emmanuel Mayberg leads a deep dive into Civitai, an AI platform backed by 16z, scrutinizing its role in hosting and generating adult content. Historically, Civitai permitted adult content and AI models resembling real individuals, provided they avoided non-consensual depictions. Emmanuel's investigations reveal significant shortcomings in enforcing these policies, both on-platform and off-platform.
Notable Quotes:
Emmanuel Mayberg [03:07]: "When you look at the top models, the top images, the latest models, the latest images, it's just a never-ending stream of adult content."
Emmanuel Mayberg [05:43]: "Contrary to those figures showing 60% of content on Civitai as not safe for work... the company says, 'less than 20% of the posted content is what we would consider PG-13 and above.'"
Emmanuel references a study conducted by a University of Zurich researcher, which analyzed over 40 million images on Civitai. The findings starkly contrast Civitai's public statements, indicating that 56% of all images were tagged as not safe for work (NSFW), a figure that likely underrepresents the true prevalence due to improper tagging by users.
Emmanuel Mayberg [10:35]: "From that time until now, the number of overall images and models on the site has absolutely exploded, as has the proportion of adult content. It's like, at this point, it looks like 80, 90% of everything that is on the site is adult content."
In response to mounting pressure from payment processors, Civitai implemented significant policy changes in May 2024, banning models that generate the likeness of real people and certain types of extreme pornographic content. These changes led to the removal of over 50,000 models, revealing the extensive use of Civitai for creating bespoke AI-generated likenesses of celebrities and individuals.
Notable Quote:
Emmanuel Mayberg [14:22]: "A lot of people move their models there. For a while, Tensor Art had a tool that even helped people import their Civitai models automatically."
This policy shift didn't quell the demand but rather catalyzed an exodus of content creators to other platforms like Hugging Face. Emmanuel highlights that 5,000 non-consensual AI models of real people found a new home on Hugging Face, despite the platform's purported commitment to ethical AI practices.
Sam Cole provides an insightful analysis of how payment processors wield significant influence over platforms hosting adult content and AI models. Companies like Stripe, Visa, and Mastercard categorize adult content as high-risk, often leading to the severance of financial relationships with platforms hosting such material. This categorization forces platforms to adhere strictly to content guidelines or face financial repercussions.
Notable Quote:
Sam Cole [20:32]: "These payment processors don't want to take the time to sort out the difference in a lot of cases and just ban it all."
The impact of these decisions is profound, pushing platforms to either align with stringent content policies or risk losing essential financial services necessary for their operations.
Transitioning from the digital realm, Sam Cole introduces a pressing issue concerning the preservation of historical narratives within National Parks. In March, the Trump administration issued an executive order titled "Restoring Truth and Sanity to American History," aimed at reconfiguring how history is portrayed in public monuments and memorials managed by the Department of the Interior.
Notable Quotes:
Sam Cole [31:43]: "It says... 'to ensure public monuments, memorials, statues, markers or similar properties... do not contain descriptions, depictions or other content that inappropriately disparage Americans past or living...'"
Sam Cole [34:30]: "It's like... rules out quite a bit of American history if you can't really tell the truth about what was going on."
This order threatens to sanitize historical narratives by focusing solely on the grandeur of American landscapes and achievements, potentially erasing discussions around racism, indigenous histories, and other critical societal issues intertwined with these sites.
In response, Save Our Signs—a collaborative effort launched by data preservationists from the Safeguarding and Research and Culture Project and the Data Rescue Project—has initiated a crowdsourcing campaign. Through their web portal Zero Signs, they encourage visitors to National Parks to photograph and archive existing placards, signs, and monuments to safeguard them against potential removal or alteration.
Notable Quotes:
Sam Cole [37:46]: "If you're at a park... take a picture of placards and signs and monuments... to preserve them, to archive them, essentially..."
Sam Cole [44:03]: "It's a nice kind of entry point into data preservation and archiving... you're helping preserve a piece of history when it's under threat."
This grassroots effort not only empowers the public to participate in historical preservation but also serves as a countermeasure against potential governmental overreach in rewriting history.
The episode concludes with a brief mention of additional subscriber-only content, teasing a forthcoming discussion led by Jason Kebler. Listeners are encouraged to subscribe to 404 Media for exclusive access to bonus episodes and ad-free content, supporting the journalist-founded media company's mission to deliver investigative reporting and unique insights.
Notable Quote:
Joseph [44:51]: "As a reminder, 404 Media is journalist founded and supported by subscribers... you'll also get to listen to the subscribers only section where we talk about a bonus story each week."
Civitai's Shift Toward Adult Content: Despite public claims, data indicates that a significant majority of content on Civitai is adult-oriented, leading to policy changes under pressure from payment processors.
Migration of Content to Other Platforms: The removal of real-person models from Civitai has driven content creators to platforms like Hugging Face, raising concerns about the hosting of non-consensual AI-generated content.
Influence of Payment Processors: Financial institutions play a pivotal role in enforcing content policies, often categorizing adult content as high-risk and compelling platforms to alter their offerings.
Preservation of National Park Histories: In response to executive orders aimed at sanitizing historical narratives, initiatives like Save Our Signs are leveraging crowdsourcing to archive and preserve the true histories embedded within National Parks.
This episode underscores the intricate interplay between technology, content regulation, and historical preservation, highlighting the challenges and grassroots efforts to maintain integrity and diversity in both digital and physical public spaces.