← The 404 Media Podcast
The 404 Media Podcast
Exposing the People Behind Deepfake Porn Sites with Bellingcat Investigator Kolina Koltai
00:52:25
Loading summary
Transcript47 lines
- [00:00]
A
Foreign welcome to another interview episode of the four four Media podcast. Four four Media is a journalist founded company and needs your support. To subscribe, go to 4Media co. As well as bonus content every single week. Subscribers also get early access to episodes like these. Gain access to that content@404 media co. I'm Sam and today we're joined by Kalina Koltai. Galina is an investigator, senior researcher and trainer at Bell and Cat. Her investigations focus on the people and systems behind AI companies and platforms that peddle non consensual deepfake explicit imagery which is a topic that somehow never ever, ever, ever dies and only gets worse. So here we are. I would love to Kalina, before we get started into the stuff that you're doing today also hello by the way. Everybody, say hello to Kalina.
- [00:58]
B
Hi everyone.
- [00:59]
A
I'd love to get into your background and how you got into this world a little bit. I know that you've done quite a bit of research and your PhD and a lot of your writing has been on misinformation and kind of the socio technological systems that are at play with things like vaccine information and stuff like that. So if you could just give us a little bit of your background before you, before we find you here in the deepfake investigation world.
- [01:26]
B
Yeah, it's always like an interesting story and I'm so excited to be here. I listen to the podcast, I am a subscriber.
- [01:32]
A
Oh, we're huge fans of Bell and Cat. So we're going to try not to just glaze each other the whole entire time.
- [01:36]
B
But I'm just saying other people should subscribe. I subscribe. You guys support your journalists, but more now than ever. But yeah, gosh, you know, I, people ask me this because I like, I'll even get like a message sometimes on LinkedIn saying like, oh, how do I get into like what you're doing? And I'm like, it was kind of a little bit of a fluke if I'm honest. I wish there was a perfect linear path to get to where I am today and how I ended up here. But I think as someone who grew up in an age where everyone had access to the Internet from a research and from a professional point. I have a really weird career because when I first started doing research professionally after I started graduating, when I needed a job, I actually started researching aircraft safety, which is not at all what I do nowadays. So I did a very, very brief stint as a research assistant as researcher or human factors engineer, whatever title we want to use on the paperwork at NASA Ames in San Jose. And there we researched trust in automated technology. I bring this up because trust in automated technology becomes an overall thing. We started at that point trying to understand how people interact with technology, how technology aids in decision making. And in this case, it was pilots in a variety of different scenarios and how they use technology to help them decide where to land or what to do or using in tools when they're under high workload. But from that, I decided to go back to school and get my doctorate. And at that time, which was 2015, there was a big outbreak of measles in California where I lived. And at the time, the kind of sort of normative thing you say about, oh, those are just like a bunch of granola moms, a lot of people who are just like, uneducated. They don't know what they're doing. It was like at that time, like a leftist kind of crazy mom who didn't want to vaccinate their kids. And I was like, there's got to be more to it than that. And I think that sent me down this long journey of not just my PhD, but even after the PhD, because I graduated in 2020 at the start of the pandemic about studying and publishing mostly on misinformation, particularly around health and specifically vaccines and the decision to whether or not to vaccinate or not. Of course, it started off with childhood vaccines, but expanded to the COVID vaccine and other vaccines. And we're still seeing, I think, the repercussions of the wave of misinformation we saw. And particularly my focus was not just misinformation generally experienced, but like the way that social media platforms, digital platforms, digital communities, different technologies interacted with the way that we receive and trust information. And I've always kind of been this kind of person that's been focused on trying to understand the way we make decisions about the world and how we decide what's okay, what's not, how do we decide what to trust, what not to trust, when it comes from all these different sources. But if you can imagine. So I started a postdoc at the University of Washington after I graduated, and I got really burnt out on vaccines because I think at some point I realized as much as it's great to be in academia and doing that research, and that's really critical research because that's research I still cite today and other people who are doing that, I felt like it wasn't enough. It wasn't at the speed that misinformation was being produced and spreading online and we needed more. And that sort of what eventually I did do. I say brief stint also at Twitter, before it was X at Twitter working on Community Notes, but I eventually landed at Bellingcamp where I'm still here today. And this was around 2022. Now if we jump forward in time and this is where an AI technology, particularly AI generation technology around images was really beginning to gain steam, gain popularity. And it became a fluke just because I would spend a lot of time online way that people looking at misinformation, looking at things. And I came to AI image generation technology, particularly from the lens of misinform, worried that of course images which we use to debunk things or have you decide things or imagine it when it comes to vaccines or storytelling or even text generation, I thought, oh my gosh, aigen is going to be such a huge impact in misinformation. And while it is, I quickly discovered that the bigger impact, and I think in my opinion the bigger harm that's really come out of aigen is actually non consensual deepfake pornography. And that's a really, really long explanation to explain how I got here today and a bit of a flu, but I think because people are terrible online and I love being in terrible spaces online, it kind of gets me up in the day that I've gotten to here and I thought maybe this would be just like a topic for a little bit, but I think it is something that's just continued to explode over the past few years. We're now in 2026. One of the biggest news stories at the end of the last year, beginning of this year is of course grok, Twitter, X's AI platform generating non existential deep fake pornography of minors or sexual explicit imagery of minors. And that is huge because this is something that didn't just come out of nowhere. And I know I'm probably diverging away, but it's been growing in a sense. Anyone who's been in the space shouldn't have been surprised that that was happening because it had been happening 404. Actually it's reported in May that users were using GROK to make NOC essential deep fake images. So this is not a new issue, it is only getting worse and I hope to one day not have to work on it. But until. Until we kind of get a better hand on it, I'm going to keep working on it.
- [07:08]
A
Yeah, I'll see you there.
- [07:09]
B
Sorry, that's my long answer.
- [07:10]
A
We'll cheers a pina colada or something on the beach the day that we don't have to write about this shit anymore. Yeah, and it does. I also, when I started writing about this stuff, I also was. Had the same kind of view on it. I was like, this is going to be something that's like huge, but surely in six years this won't be. We'll have figured out something like something to mitigate the harm or we would have come to some kind of technological or social consensus on where we stand as a society, as a community on this stuff. And it just has gotten so much worse since then.
- [07:44]
B
So.
- [07:45]
A
Yeah, I don't know. Keeping us busy. I guess so.
- [07:47]
B
Yeah.
- [07:48]
A
And obviously coming from a misinformation standpoint makes so much sense to me because you were looking at pre generative AI misinformation and it all feels very connected and woven together where we're talking about misinformation that's spreading and engagement bait that's spreading that actually has harm in real life. And I think NCII is a big part of that too. So you have done so many incredible investigations. Bellingcat is on this topic and is, I would say, like the voice on like not just misinformation, but like uncovering the sources of the misinformation, which I think is so important. And you recently wrote a story with the headline Profiting from Exploitation. How we found the man behind two deepfake porn sites. I love this kind of story. When I saw the header image and it's like him, like on vacation or something. I guess he's like in a tropical place taking a selfie. It's like this is the guy who's kind of the mastermind behind some of the work first stuff online. And there are many like him. So let's walk through a little bit to the beginning of that story. When did these two sites, and the sites are called, or they were called, Reface porn and Deepfake porn. When did they first come onto your radar?
- [09:08]
B
Yeah, and I like to like tease. We didn't get included in that. We focused on just two sites. But I suspect he actually owned multiple sites, which is pretty indicative of this. Partway through the investigation, I already found so much overwhelming evidence for those two sites and his ownership that it almost wasn't worth going to like all the long lists of these much smaller sites that he had that didn't get as much traffic. And I will tease that those sites also shut off at the same time the other main sites did. So, you know, we'll say the Two, but there is probably also a lot more. But they've all been shut down, which is great. I checked even this morning and they are still off live, which is fantastic. But yeah, I don't know, maybe it's a good thing or a bad thing. On the Internet, there's these adult sites that give lots of recommendations for AI. Fake sites like, oh, if you want to create your deepfakes, here's our top hundred sites we can recommend. And so I go check out the top 100 sites on these recommender lists online that are just probably also created by AI. I don't know, but I don't think a human's making all of these lists. But through some sort of like, miracle of the Internet, you're able to easily find lots and lots of different sites. And it's not just these two sites. I've looked at lots of different sites. It's often a choice deciding which ones we publish on, because if someone wanted to set up a whole lot group just weren't focusing on this, they would have no shortage of sites. Unfortunately, these came onto our radar just like any other of the deepfake sites. Do people use it, talk about it? It's on recommender list tend to do is I say, like, okay, how easy is it to actually figure out who's behind it? Some sites are a lot harder and ones I've been working on for years. Some are quite easy, and in this case, it was quite easy. I don't want to say sites when they first set up. I think when people are new to the game, they aren't as careful as they probably should be online. Especially if someone's been a digital native for a really long time. They've been on the Internet for a while and they think they're being maybe clever or really good about covering their steps. They set up a new email out. But people are sloppy, people are lazy. And very quickly, within less than a week, I think a day or two of actual searching, I was able to identify and just look in the site. And I was like, all right, we got the evidence. But as we continue to dig, we had a suspect in mind, this guy, Mark Reson. It became really clear that he had some pretty strong, I think, irrefutable evidence that he is the owner of these sites and as we suspect, multiple more.
- [11:44]
A
So, I mean, you say you're like, it was easy to find this guy. And then most people reading this, including myself, I'm like, this is so much. And a lot of it is stuff that I wouldn't even think to go after. Unless I had been reading Belly and Cat for a while and understood the way that open source investigations work. So let's kind of walk through some of the the ways that you uncovered him. Where do you begin when you're like, I want to know who's behind this platform? Where's the first place that you start?
- [12:16]
B
Yeah, I love using a tool and so some of the very first tools I do, particularly I'll try to uncover who is behind a website is look up who registered the domain. Right. Most times it doesn't lead anywhere, it will just kind of give you a dead end. But you can look up saved domain history. So people oftentimes redact it. But if they're a little lazy, they've had the domain for a while, they might have put their name or maybe put a company or some sort of contact information on there before they chose redacted. And so very quickly, when we went to go look up the whois information, which is like, who is responsible? Who is the purchaser of this or who's the registrar? We saw Mark Reson's business of acidic as a registered company, as the company that's like, oh, this is the ownership on it. So I was like, all right, clue, would we go look to see Facetic? Is there an owner to it and Mark Reson is the owner? That's a very quick, easy way. Granted, sometimes it's tricky to look up the history of who's the owner of that domain. But unfortunately we use Domain Tools, which is a paid service. That's great. I think we might get discounted for free, so I will mention that. But it is a really great tool. Just look up, say ownership history. And so sometimes it doesn't lead anywhere. In this case that was another way. But that can mean anything. So we don't just rely on one piece of evidence, so we begin to look at other things. So we look at what are other websites that could be connected to this. Another way to do that is through Google Tags. If you are a marketer or you're familiar with that. Google Tag is a bit of script or code you can put into the code of a website. So you're able to track web traffic, you know how your ads are doing, where are people coming from, all that kind of information. And so we went to go look up the Google Tag history because all that is archived over time through a variety of different services and we saw other websites connected to it. So we looked at these unique Google tags and that's why we found other websites that also all had some other connections to Marc Reson or a company that was registered to him publicly. You can look up on UK Companies House and you can see which is the company registry that's public in the UK and look at that. So that's all public information. But of course we don't stop there. You know, we begin to go find out additional information like emails and other websites that are registered to him. And in particular we find a variety of different emails. You know, a lot of people use their name and their email, so that's like also really helpful if you just were trying to even guess. We looked at other websites that are associated with those businesses. So by looking at Sycidic were able to find other emails by Mark from his other businesses that are maybe more legitimate businesses or potentially just front facing, sort of like shell kind of companies. And in addition to that we also looked at leaked data sets. So unfortunately I think everyone who has an email who's been on the Internet for more than maybe six weeks may have been involved in some sort of data leak. You can do your best to try to prevent it, but if you sign up for a website, there is a chance at some point that website will be hacked and your information in some capacity can be leaked. We look at those leaks, so we don't do the hacking ourselves. Other people have done that, but those sets are now available online and it's actually really great for anyone to check to see it because if it is, you should change your password. But that is another way. So once we had a variety of different emails associated with Mark and his businesses, we checked to see if any of those email addresses have been in a variety of different data leaks. And we're able to see passwords or name registration, a variety of different information that is associated depending on the leak. And we're able to match suspected emails together because they use the same unique password, which is something we've also done in other investigations. It's also really good to have a website like a password manager that uses unique passwords. It kind of gives you protect to show Mark did not do that he used the same password like many people do across multiple sites, which I've done as well. So that's another thing. And then I think one of the other things that I also tend to do is I go through a mock transaction on a website. So without actually like generating an image, I go to try to purchase tokens or credits or whatever they want to be called on that site. And in that process you're able to usually get some clues because you're trying to figure out how is this person or how is this company processing payments? Because most major payment processors don't allow transactions for non consensual deepfake pornography. So if you're trying to use Stripe to do this, Stripe will usually tell you no, unless you're trying to do this in a deceptive way. So sometimes people set up fake businesses or just place marker websites. A variety of different deepfake sites do this. It looks like you're making a transaction on site X, but really you're buying tokens for a deepfake site. Also in that process, we got lucky that one of the payment processors that Mark was using is this one called PurWallet, which has also since shut down. But Pure Wallet was surprisingly open. This was actually, I think one of the most interesting components of this investigation is that in that process when I was being emailed a link to make this process, they're like, oh, you're making a purchase to dorocon llp. And I was like, huh, Dorakron llp. I thought it was making it a purchase on deepfakeporn app. It's a very inventive name. So in that you're able to look up Dorakron, which is a business registered in Canada. In Canada they have better privacy rules. So we can't look up the owner of Dorakron. And so just that. So I have like a registration date there. But we're able to find other businesses and other websites that have Dorachron. And on Pur Wallet's email to me when they're like, here's a link to make payment, they have a link to a dashboard so you can investigate and look at, oh, here is the seller, Dorachron. And in that process we have the entire dashboard of transactions that Dorachron has done using PureWallet. So we're able to see a contact information which was Mark's Skype. We're able to see what website domains are associated with Pure Wallet, which included some of the sort of these masking sites, these payment redirect sites, these hidden sites that allow you to hide the transaction. So it doesn't say deep fake porn there, but it'll say like one of his masking sites, AI Remaker Me, which. And then some of these sites, because there was like over 20 of them listed list businesses that are publicly registered. So they'll list Pacific, which is if you recall, one of the businesses publicly listed to Mark. So there's Lots and lots of connections here that all continuously pointed to Merck. And so I kind of feel like we got a little easy, like we got a win that way. We were also able to see how much money he's made just through this one payment processor and Pure Wallet. It was not the only payment processor he used. A lot of these sites will use multiple ones or they'll have to switch out once they lose one. We only know like a portion. This is just a portion of the funds he made. But we know that he made over $300,000 USD off the deep paid for size, which is definitely way more than I make in a year. And it just kind of, I think shows sort of the value of how one guy in Hungary can make hundreds of thousands of dollars off sort of, I think actually not very good sites. They weren't particularly flashy, they were pretty mediocre. Looking at the wide site scale of quality of Deepfake porn sites. But enough people went to him and was willing to make the transaction and give him money. And so I can only imagine what percentage that 300,000 was compared to all the other payment processors he previously had.
- [20:01]
A
It's mind boggling that you can also, you can be this bad at covering your identity, this bad at making a website, and also be making highly immoral stuff that's harmful to people and still make that much money off of it. It's just like shocking. So I guess just to give people a little bit of context of what we're actually talking about here and we can be, we don't have to be super PG about it. I think we definitely believe at 404 and I know you guys too, in showing people and telling people what's going on, because if you mince words about it, it will not get across what's actually happening. So what is ha. What was happening on Deepfake porn and re Face porn? The names kind of tell it all. But what are we discussing when we're talking about, you know, like what people are actually making on these sites or what they were making?
- [20:53]
B
Yeah, it definitely was not a subtle site. It was very clearly when you, when you would go to these sites login, the very first thing you're presented with is a woman giving a blowjob. It was a video and I was like, ah, right away. And this video would then show her face switching to different women's faces. So it was basically very clearly advertising. Put in whatever face you want, upload your face, which I think the text said, and you could create deep fake Pornography, you know, using any woman's face that you want. So in that, in that process, when clicking around on the site, they had a lot of like, sort of like tester videos, like here you could try it out. So they had a variety of different pornographic scenes that you could use. They had some sample faces if you wanted to try one of those faces. But they did encourage you to upload your own. I do tend to test out the sites myself. I use AI generated faces, so I don't use any real person's face uploaded to the site. And so I have a selection of adult women's faces and young girls faces, so girls who I would say are teenagers and faces of girls that are younger that I use that are all generated. And I use this in my testing because sometimes some sites will say like, hey, we go and accept that that looks like might be a little young. Do you want to try a different photo? So some sites are better than that. So when I was testing an upload the face to see if it would say like, hey, we don't accept that, they accepted every photo I submitted of adult women, teenage girls and of young girls. And when I say young girls, I'm talking like 5, 6, 7, it would be my guess looking at it. So this site had like no protections on it. And so I think one of the big questions I have too is, and there's no way to prove this unless we speak to someone who might have been a victim of this site is, you know, was the site also being used to generate csam, which is child sexual abuse material? Was it being used to make a blowjob video using a child's face? I don't know. I know that there was no safeguards that I saw in any of the images I uploaded. Now I didn't create it. So, you know, I will say that because that is, that is something we don't try to do is create csam, even if it's an AI generated child. But I know that like some other sites that I won't name when I've tested it, it will say like, hey, this image you uploaded might be too young. So there is at least on some other sites, some, some sort of check, right, which there wasn't on this site. And I think that tends to be the case also for these sort of, I don't want to say like smaller sites, but maybe not as name brand. Maybe it doesn't get millions and millions of visits per month, but still got like hundreds of thousands of visits per month. So getting traction. But not like, you know, 4 million visits. Right.
- [23:36]
A
It's still, you know, quite a bit.
- [23:39]
B
It's still quite a bit. It's quite a bit. Yeah. Yeah.
- [23:42]
A
And I mean, I mean, I think, and that's a good point too, is when investigators are testing these sorts of things, you know, they also have to abide by the law and not break the law. And in a lot of places, AI generated CSAM is now illegal. So it's, it's something that it's, it's hard to investigate these things and you have to find creative ways to do it.
- [24:09]
C
I'm looking at the calendar now. January is almost over and we're into the thick of the year. If you want to start something new in 2026, it's best to get started now. Stop waiting and launch your dream business with Shopify. We've been using Shopify for more than two years, and if you're looking to start a shop either online or irl, I couldn't recommend it more. Shopify has grown with us, letting us set up a professional looking store at the start. Using their highly customizable templates and their simple to use but powerful backend, Shopify lets you easily add and manage inventory and fulfillment and marketing is built in. Create email and social campaigns that reach customers wherever they scroll. In 2026, stop waiting and start selling with Shopify. Sign up for your $1 per month trial@shopify.com media. That's shopify.com media here. Your first this year with Shopify. A new year, colder days. This is the moment your winter wardrobe really has to deliver. If you're craving a winter reset, start with pieces truly made to last. Season after season. Quince brings together premium materials, thoughtful design and enduring quality so you stay warm, look sharp, and feel your best all season long. Quince has everything you need from Mongolian cashmere sweaters, wool coats, leather and suede outerwear that actually holds up to daily wear and still looks good. Their outerwear is particularly impressive. Think down jackets, wool coats and Italian leather outerwear that keep you warm when it's actually cold. Each piece is made from premium materials by trusted factories that meet rigorous standards for craftsmanship and ethical production. By cutting out middlemen and traditional markups, Quince delivers the same quality as luxury brands at a fraction of the price. The result is classic styles that you'll love and hold up year after year. I was thinking the other day and I was realizing that Quince is slowly taking over my entire wardrobe. It started with T shirts, linen button downs like this one and other effortlessly cool basics over the summer and as we got into the winter, I picked up my favorite pair of sneakers, which I wear almost every day, and I usually pair it with a 100% cotton sweater that I also wear pretty much every day. I really can't wait to get more quints at this point, so go check them out. Refresh your winter wardrobe with quints. Go to quints.com 404media for free shipping on your order and 365 day returns. Now available in Canada too. That's Q-U-I-N c-e.com 404media free shipping and 365 day returns. Quint.com 404media now that the holidays are over, you might feel like you've got a bit of a spending hangover. Between the drinks, the food, the gifts, it all adds up really fast. That's exactly why Mint Mobile has been clutch for me this January, because they're helping people cut back on overspending on Wireless with 50% off unlimited premium wireless plans. It's my favorite type of cutting back, the kind where I save money, don't have to do much, and still get essentially the same service for much less money. Mint Mobile's end of the year sale is still going on, but just till the end of this month. Cut out Big Wireless's bloated plans and unnecessary monthly charges with 50% off 3, 6 or 12 months of unlimited. All plans come with high speed data and unlimited talk and text delivered on the nation's largest 5G network. Use your own phone with any Mint Mobile plan and bring your phone number along with all your existing contacts. I switched to Mint Mobile from Big Wireless and honestly, the difference has been wild. Not in the service, but in the price. I'm getting the same quality coverage I had before, but I'm paying way less every month. No bloated plans, no random charges, no nonsense. Their end of the year sale is still going on, but only until the end of the month. You can get 50% off 3, 6 or 12 months of unlimited and every plan comes with high speed data plus unlimited talk and text. I used my own phone, kept my number and all my contacts transferred over seamlessly. Zero hassle. This January, quit overspending on Wireless with 50% off unlimited premium wireless plans start at $15 a month at mintmobile.com 404media that's mintmobile.com 404media Limited time offer upfront payment of $45 for three months, $90 for six months or $180 for 12 month plan required. That's $15 a month equivalent taxes and fees. Extra initial plan terms only over 50 gigabytes. May slow when network is busy. Capable device required. Availability, speed and coverage varies. CementMobile.com.
- [29:00]
A
So you have all these smoking guns, basically you have all these direct leads to this guy Rhysand and next you have to try to reach him, Right? So what do you do at that point? You're like, we have to contact this guy. We have to get comment as part of doing our due diligence. So what happens at that point?
- [29:22]
B
Yeah, at that point we send emails and we have fortunately lots of different emails for him businesses. So we sent emails to his emails to his business emails. So like Dorakron's email, Vasitic's email. And multiple times, you know, we gave him multiple chances across multiple emails. So we sent out one, no response. We wait a little bit, we sent out another one, give him say like, hey, we need a response by this time, please respond to us. And nothing. But we did notice that sites were taken offline. He started trying to delete things. As soon as he reached out to us, things began to tighten down. And so we know that he saw them. I don't think there's any doubt in my mind that he's seen those. He's never responded to us. One of the interesting components about this is that there were archives. So we used the Internet Wayback Machine. It's wonderful. People haven't used that. It's a great way to look at archived history of websites. And so there were already archives of this site scraped through Internet Archive and you could have in theory. So like, oh, look at the evidence of Zod here. That's archived, it's timestamped. This is what the site looked like. But in the process, right before we published, we saw that all the archives were gone on there. And we're like, huh, that's weird. So we emailed Internet archives saying, hey, did you guys take this down? Because it is actually possible, I don't think people know that you can, if you're the owner of a website, request your site to be removed. And they did confirm to us that the owner of the site requested that those sites that we identified were removed. Which we then emailed Mark saying like, hey, did you get these removed? We saw. Which again he did not respond. And so he very clearly was, I think ashamed and tried to hide a little bit that he was identified. He's trying to cover up his tracks. That's why also having Redundancy in archiving and history of stuff is so important. But yeah, he never responds. A lot of these guys never respond. They just try to run and hide. I don't think they are. No one wants to come out and be proud and be like, yes, I'm the owner of this site. Right. And the thing is, it's not illegal yet what he's doing. It's not like there'd be any penalty.
- [31:39]
A
Yeah, he could just keep doing it without, for now, at least, without. Without facing any consequences, which is crazy. I think it's so satisfying when you reach out to someone for comment and they start cleaning their shit. I'm like, I see you changing your LinkedIn place of work. I see you making little changes to your website, things like that, or taking the website down altogether, which I would love to talk about. Just the feeling after one of these sorts of sites goes down because like we said earlier this, it's a problem that's getting worse. It's just increasingly spreading and harmful and something that a lot of times we don't see any recourse for victims or any movement in the direction of accountability or responsibility or anything. So when one of these investigations essentially gets the site shut down, that's so huge. So I would love to hear that from your point of view. Just how did it feel when you saw these sites were gone? And not only gone, but like gone from the archives, gone entirely, no trace.
- [32:48]
B
Yeah, you're right. It is so satisfying like it is. It is such a. Because ultimately that's one of the things I want to be the end result of these investigations. I want the sites to come down. I don't want people to be able to use that site for any additional services. I don't want them to make money on it. It feels like that's like our impact in a sense. You know, we have on our internal work communication system on Slack, we have a channel as impact and to me that is the impact the site has come down. It's proof. We know that we were right in identifying him. We know that he has some shame on that and he's going to do the thing which is that we wanted ultimately it's the site to come down. And so we've seen this happen across a variety of sites when someone's identified as the side go down. I think this one is a huge one because it's a generation service. The other really big one that came down after we reached out to them was the Mr. Deep Fake Sites. That was, I think, one of the Biggest, really big ones for us is that site coming down because it also hosted so much content. And I will say, unfortunately, it doesn't always happen. Some sites stay up, totally working, and also some sites kind of end up just like dying and being left to sit and rot. And I think that's the part that's also very frustrating is there's a variety of sites that I've known that I've followed through investigations that like, they might have switched hands or sold and then slowly just kind of like stop maintaining the sites. But those sites will still actively host those content because they didn't turn the site offline. So even though you can't use the site anymore, some of them will just still have tons of deep fake imagery that are up there, but no one there to maintain it. It just sits there and it's pressuring because, like, take the site down, you've lost, you're not using the site anymore, but you're going to leave the harm up. And I think that's also like one of the frustrating. Like, I'm happy that the site is dead, the company's dead, people have abandoned that project. But nothing beats the sort of satisfaction of the site actually, like being removed.
- [34:43]
A
Yeah. So good. And the Mr. D fakes situation was so good. If people who aren't familiar, that was. It was an investigation that was, I think, weeks in the making. Right? Definitely.
- [34:56]
B
At least. Or years. Yeah, years.
- [34:59]
A
Years in the making. It came out in May 2025. It was the headline that Bellingcat had was unmasking Mr. Deepfakes, Canadian pharmacist linked to world's most notorious Deepfake foreign site. And that was a collaboration between bellingcat, CBC and checked at. And am I missing anybody?
- [35:17]
B
I think politics.
- [35:18]
A
Politics, yeah. As well.
- [35:19]
B
Yeah.
- [35:19]
A
Okay. So it was a multi newsroom effort and so impressive. They uncovered the man behind Mr. Deepfakes and it ended up being this guy who is a pharmacist in Canada. And CBC sent a reporter out with a camera and found him in his Tesla at his house, which is such a crazy, crazy video. And I think it's so interesting and ironic that these guys are so scared of being unmasked or unveiled or identified in public when they're making so much money off of women's likenesses without their consent. And they're so scared of being on camera or named or anything. And it's like you are profiting off of the identities of thousands of women every day. You know, Are we gonna acknowledge that? No, we're just Gonna drive away in the Tesla Crazy. And Mr. And Mr. Deepfakes, I guess is just to get back to the point, went down after that investigation, after he was cornered and caught, the site completely went down and is not up. I haven't checked today, but I do check every once in a while, but it's gone. Which is huge for the women whose images were on it because I think a lot of the time targets of deepfakes just want the imagery to stop. They don't care as much sometimes about the person facing consequences. They just want it gone. They want it to stop being produced, they want it to stop being spread. And when a hub like this goes down, when the ones that you investigated that we've been talking about go down, when Mr. Deepaks, which is huge, was like a tutorial site in addition to hosting go down, that's just massive impact. So yeah, kudos to Bell and Cat, kudos to you. This is such important work. So I guess just to ask some more existential questions now that we've established where we're at in this space. And speaking of just impact, I guess why do you feel like this reporting is important to keep doing? Why does this matter when you're kind of like oh my God, I can't believe this is happening again. What kind of keeps you doing it?
- [37:26]
B
You know that's, that's a great question. You know, I think it matters. You know, I, I think it's when you see the site go down, when you see it happening and you're like a change has happened and this site would not have gone down there. It doesn't matter what kind of email it would, didn't take until like I'm going to identify you, we are going to publish your name and say that you're the owner of this site. Does anything happen? And I wish I would have one for all sites that kind of like have ties to this, you know, we don't have to do an investigation of who's the owner of X, we already know who that is. So more investigation or something else would need to happen in that case. But for these types of sites, right. David Doe, who is that? Canadian pharmacist, 36 year old guy, looks otherwise like a normal regular life, you know, family guy, had a partner, you know, pharmacist job, like just a dude in Canada, right. Canadians are supposed to be all these like nice guys.
- [38:26]
A
Canadians are everywhere and many of them are up to some stuff.
- [38:30]
B
Some stuff, yeah. And yeah, it didn't, it. That was like a really, I think Wonderful investigation to be a part of because it was such a huge effort across many people. And I think we had so much redundancy and so much checking on that. That to be sure, because that was going to be a really big publicly naming. Like, we were staking our careers. Like, if we were wrong, that would have been like, that's it. I would have just, like, left. It's been like, that's. But I think, you know, it. It took until him being cornered by cbc. We were like, we were trying to track an end. Also when we were trying to corner him, like, we went. They went to initially his work to try to talk to him, and he wouldn't talk. They had to hand. They did letters, they did emails, tons of communication. Like, gave him every single opportunity to, like, defend himself or deny or say anything to get a lawyer. We were about to, like, go. They were about to go to his home and he was, like, gone. And we're like, oh, no, did he flee the country? And then we're tracking his Airbnb profile and saw that he was traveling in Europe, and we're like, should we send someone to Europe? We have colleagues in Europe. Like, can we just go and, like, door knock while he's on vacation? Which is one thing we thought about. But, yeah. And they honestly kind of felt like a little, like, movie suspense because they had a team come and, like, sit by the house, wait, check to see if the lights are on. For real? Yeah, like a stakeout. Like, really, like. Like police work. And, like, the police wouldn't do any of this anyway. So it's like, I guess it's up to, you know, journalists to go do it. And yeah, it took until, like, you know, repeatedly, repeatedly, repeatedly giving him every opportunity to say anything, any defense, and he wouldn't. He would just not want to talk to the team. So we published his name, or we'd say, hey, we're going to publish on this date. And the day before, we publish site went offline. And I think that means so much. That means we weren't falsely accusing the wrong guy. We gave this person every single opportunity, and he knew that he wasn't going to be able to handle it. So he took the side offline, he said, because of other things. But we know we gave you a deadline and you turn it off the day beforehand. We don't threaten to say you should turn off the site. I'll be clear. These are choices that these owners are making. We don't say take off the site or we're going to Publish your identity. We just say, hey, this is what we're publishing. As you, as the owner, anything you want to say for yourself. And they decide to run and hide. I think that is, it's such a high, it's such a rush, but you have to do it. Because on a site like that they had thousands and thousands of videos. That site had been up since 2018. It was the largest site that hosted non consensual deepfake pornography, particularly around celebrities. But they also had a forum site. And that is where it got really, really awful because that's where people asked for bespoke materials. Here's an image I have of this woman. It's like my classmate, a friend's girlfriend or someone people would ask and get custom stuff made. Wasn't just about celebrity content. And I think the harm that slut caused over years, years, we can never undo that harm. But at least now less harm is being created and being perpetrated. Because imagine you have a video on there and sometimes it's people who aren't particularly famous, sometimes small time YouTubers that only have 5,000 followers. We're not just talking about a Scarlett Johansson, we're talking about people who might be a regular person. There's journalists who've been on there. It's like anyone who is just trying to have a career in a space online could be a victim. And it's overwhelmingly women. And it's not just say, oh, because I'm a woman, I feel for this. I think anyone could feel for this. You don't have to have your identity tied to seeing that the harm that's being done and that can continue to harm you in ways beyond it. Because images and videos that originated on that site could continue to spread on the Internet. It's really hard sometimes to put a cap on those sort of things. And as much as a whack a mole game it could feel between these big sites and the small sites, we need more people doing this work. I'm just one woman, we're just one organization. I am not trying to gatekeep how to do this work or how to be the only person doing these investigations. Anyone is capable of doing these types of investigations because it is open source. You know, we're using things that are available on the Internet out there to be able to investigate the ownership of these sites. And I think that's what's also incredibly powerful, is that you know, you can do something. You know, it's. I think in a world where we feel so powerless about so Much is happening. You know, I live in the U.S. like many, many people, and there's a lot of awful stuff, a ton of awful stuff every single day that's happening. And you feel powerless to do something about it. This work gives me some power, some agency that something can be done by identifying these owners. They get so ashamed, so scared that they turn off the site. You know, they run and hide. Those sites die. And that is an impact that gets me up every single day. Yeah.
- [43:41]
A
Yes. And especially, like you mentioned, the police often do not do anything. Reporting this stuff to the police does not actually do anything in a lot of cases. Not to say that you shouldn't try, but in Canada, it's not even. There's not even any law in the books about, you know, preventing deep fakes or preventing the harm from them. So, you know, again, it's like he could have just left it up. But the shame that he must have felt and, you know, the threat of being identified essentially by a journalist and by investigators is what took that site down ultimately. Not, you know, any kind of action from authorities or from Canada or from legislation. It was journalists and investigators doing really hard work and really staying on it and not just saying, oh, well, we can't find them, never mind, which many people have done throughout the years that that site has been online. I haven't even tried to figure out who was on it or who was running it, because I was like, this seems like something bellingcat should do.
- [44:42]
B
Which.
- [44:42]
A
Is often something that crosses my mind. So I admire Bell and Cat's work so, so much. And I admire the attitude of everyone should be doing this. Everyone should know how to be doing this kind of work, because the more people on it, the better. It's not a competition. It's very much open to whoever wants to kind of look into this stuff in a responsible way and in a way that's safe. And I think your work is exemplifying that in a really important way. So I'm going to leave it at that note because I think it's a positive and optimistic one in a topic that is very dark and depressing. Is there anything else that we didn't talk about that you wanted to mention? I know we haven't talked about rock, but that is also, you know, like you said, we don't need to unmask him or anything.
- [45:26]
B
Yeah, I mean, like. Like, rock thing is so interesting because, you know, I'm glad that there's, like, steam that's happening. You know, like, there's. There's traction. That's happening and that's what it takes. It's like people being upset. You know, I, I kind of hate the phrase like keyboard warrior, but sometimes it does take a lot of just like people talking about it continuously putting the pressure on. It's like, like if to me, when I think about the Grox situation, if AIC Sam isn't the line, isn't the bar, then where is it? I mean, that is usually like, you know, Kanaka Central. Pornography of minors is usually where everyone's on saying that, hey, that's not good. You know, I don't think you have to like do a whole sort of like, you know, in academia you're trained, like, oh, you should have a whole section of why this work is important and why this care and like why we should, should, you know, say invest in it or why should we read your. Like there's a whole site, like, why do people care about your work? Your work is usually so niche and you have to really make a strong argument. There shouldn't be any need to make a strong argument. All I have to say is like non consensual, deep fake pornography of minors. What is the argument for? You know?
- [46:33]
A
Yeah, yeah, I thought we were protecting the children here. Like, what happened to those, yeah, those.
- [46:37]
B
Voices and the idea of like, oh, we're just gonna like ban a user from that. That is not enough. That is so inefficient with, oh, we're just gonna ban somebody. But the technology and I think that's the thing. I wish there are laws. There's a lot of these times nothing happens to these guys, right? Besides their name maybe being out there. I would say even at outlet balancing, we can never really even publish a person's face because of we're based in the Netherlands and so we have really strong data privacy protection rules about what we can say. Not to say, if I was at an American outlet, probably be like, here's his face, here's his name, here's all this. We can't even do that. Like, we give these guys more rights and protections than they have afforded the women that are victims of their sights, right? Like, yeah, like, we don't. Like Mark we saw, we didn't even put his face on it, right? Like his and his technology, it's like put your face on anything. You know, it'd be not, I'm saying advocate people make content of. Of him, deep fake content. But I'm just saying that like there is so much we need to do legislatively which is Slow and process and everything. It's. Is my one sort of thing I want to advocate for everyone is that there's a lot of wonderful laws. I don't say wonderful laws, but there's a lot of laws that are coming down the pipeline and that are put in place, say give more rights to victims, which is fantastic. The other thing, the major gap in legislation and consequences is that these websites are not illegal. This kind of technology has zero regulation, zero laws. Owning a website like this is not illegal. There is no consequences for that. And to me, that is insane that we've put in some now lawsuit saying, hey, creating the content and distributing that content, it's against the law. But profiting off that content isn't right. And I think that is the thing we need because in all the kind of components that understanding about this kind of work and these kind of sites, I think most people are in this because they know it's profitable. They know you can make money. This guy in Hungary made hundreds of thousands of dollars off the site. He did it not because he loves deep fake pornography. I mean, he might love deep fake pornography, but I'm sure that considering he's had all these other businesses, all these other random ventures, that he's in it to make money. And we need to make it harder for these guys to make money off of it because otherwise then they'll just go to something else. It all comes to money and power at the end of the day. And I think we cannot ignore that as part of the solutions in tackling the issue. So we need to add friction to that process. We need to add consequences to these actions more than just shame, because shame will be great for the Mr. Deepfakes guy or deepfake poured, all these other re. Fakes poured. But it's not going to take down our big players. We need to basically make. This is so toxic to touch. Like having like child pornography on your computer. Like no one wants child pornography on their computer. But why is aic Sam okay to have on your platform or a service that creates the child pornography? Is that okay? You know, and I don't, I don't mean to be brutish, but it just that that should be the bare minimum line.
- [49:50]
A
What an extreme take.
- [49:53]
B
Hot take.
- [49:54]
A
Yeah, yeah. I mean it's. And I think that's a question a lot of people are, are waking up to and are asking themselves now with the Grok situation is like, why is this allowed and why is he allowed to do this? Why is Elon Musk allowed to run the platform in the way that he's running it right now and blatantly ask for subscriptions. If you want to make this horrible content, you have to pay him to do it in a lot of cases. Or I think that he turned that off. But yeah, it's just. It's so in your face. The answer should not be more clear that this is unacceptable, but somehow it's acceptable for the billionaires among us to do it. So. All right, we are going to leave that there. I've kept you long enough. Thank you so, so much for your work and also for joining us today. We really appreciate it and this has been so fascinating. I could talk for two more hours, but I'm not going to do that to you.
- [50:45]
B
This has been. It's been a lot of fun. I was looking forward to having this chat. I love talking about this kind of stuff. I am the worst dinner table guest.
- [50:52]
A
Unless you like this topic. Tell me about it. Yeah, I am definitely with you there. We need a dinner where we're just talking about this. Um. God. Actually, maybe not. That sounds. We need a lot of wine for that dinner. Um, all right, well, I'm going to go ahead and play us out. As a reminder, 404 Media is journalist founded and supported by subscribers. If you wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You get unlimited access to articles and an ad free version of this podcast. You'll also get to listen to subscribers only sections where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope and Alyssa Midcalf. Another way to support us is by leaving a five star rating and review for this podcast. That stuff really helps us out. This has been four four Media. We'll see you again next.