Front Burner Presents: Deepfake Porn Empire

Summary

---

Front Burner Presents: Deepfake Porn Empire

Episode 1: The Dawn of Fake Porn
Date: April 6, 2026
Host: Sam Cole (Reporter, 404 Media, Guest Host of Understood)
Featured Voices: Cutie Cinderella (Streamer), Walter Schreier (Engineering Professor, Notre Dame), Ian Goodfellow (AI Researcher), TMFU (Anonymous Photoshop Porn Creator), Lux Luker / Carrie Pearson (Early Fake Porn Community Founder)

---

Main Theme

The episode traces the origins and explosive growth of non-consensual deepfake porn—a billion-click industry built on AI-generated images and stolen identities, chiefly of women. Host Sam Cole chronicles the technical, cultural, and social history of "fake porn," from 19th-century photographic trickery to today's AI-powered deepfakes, focusing on the story of streamer Cutie Cinderella as emblematic of the human toll—and the systemic indifference—surrounding this evolving online abuse.

---

Key Discussion Points & Insights

1. Personal Fallout: The Cutie Cinderella Incident

[01:26–05:06]

• Cutie Cinderella wakes up to find herself trending on Twitter after a fellow streamer, Atriok, accidentally exposes tabs of deepfake porn—including fakes of women he knows—on his live stream.
• The images go viral across Discord, Reddit, and Twitter, flooding Cutie's inbox before she even grasps what’s happening.
• Cutie describes the shock:
  • "It is so convincingly my body, but not my body. And holy shit, it hits you like a truck. You feel so violated." – Cutie [04:54]
• The episode frames Cutie’s story as a turning point in public awareness of deepfake porn’s victim impact.

---

2. A Brief History of Fake Photos and Digital Manipulation

[06:55–10:33]

• Walter Schreier contextualizes fakery, dating back to the earliest days of photography:
  • "Basically, as soon as the camera is Invented in the 19th century, people are faking their photos." – Schreier [06:55]
• Examples:
  • Hippolyte Bayard fakes his own death for publicity in 1840.
  • The Cottingley Fairies (1917) deceive even Sherlock Holmes’ creator, Arthur Conan Doyle.
• The digital era begins with Photoshop (1990), democratizing powerful image manipulation—immediately co-opted for pornography.

---

3. Predecessors to AI: The Photoshop Porn Subculture

[10:33–14:41]

• The 1990s-2000s: Usenet groups and early websites cultivate communities sharing celebrity nude “fakes,” led by figures like Lux Luker (Carrie Pearson).
  • "We all love seeing the heads of some of our favorite stars and other prominent people placed on top of a nude model. The sheer godlike power of exposing them to the world for our own fantasies..." – Lux Luker [12:35]
• These forums established rules against minors—but also chilling lists of girls’ birthdays to know when they become “eligible.”

---

4. Hollywood, CGI, and the Blurring of Reality

[15:04–16:39]

• Public trust in images further erodes with advances in CGI: films like Jurassic Park (1993) and Furious 7 (2015, Paul Walker facial replacement) unsettle ideas about what’s real vs. digital.

---

5. The Technical Breakthrough: Generative Adversarial Networks (GANs)

[18:31–22:03]

• In 2014, Montreal grad student Ian Goodfellow invents GANs, allowing AI to generate photorealistic images:
  • "The generator makes images that aren't real at all, and the discriminator doesn't know what's real or fake... Eventually, the generator learns to make very realistic images.” – Goodfellow [20:47]
• Goodfellow hopes GANs will solve scientific problems—not realizing their transformative (and sinister) potential for porn.

---

6. Deepfakes Emerge: The Pornification of GANs

[22:16–25:38]

• Open-sourcing GANs code enables rapid, uncontrolled diffusion—Reddit user “deepfakes” posts the first AI-generated celebrity porn videos (Gal Gadot, Scarlett Johansson, etc.) in late 2017.
• Sam Cole recalls first breaking this story. The tech world shrugs:
  • "Within the computer vision community, Deepfakes comes out. No one really wants to comment on that ... it's not our problem." – Schreier [25:29]

---

7. Public Panic: The Wrong Deepfake Problem

[25:52–28:25]

• Authorities and tech media focus on political deepfakes (“fake Obama”/Jordan Peele, Zelenskyy surrendering, Trump nuclear threat)—while deepfake porn is ignored, despite being the overwhelming majority of usage.
  • "People were paying attention to deep fakes, just not porn." – Sam Cole [25:52]
  • "Some people are genuinely fooled. Right. Which is. Which is a problem. Right. But for the most part... they're almost always released on, like, an anonymous account..." – Schreier [28:04]

---

8. Deepfake Porn: The Human Cost

[28:25–34:51]

• For victims, what counts is the violation, not the technical unreality.
  • "This is what pain looks like. This is what it looks like. The Internet, the constant exploitation and objectification of women. It's exhausting. It's exhausting." – Cutie Cinderella [30:12]
• Cutie faces relentless harassment, resurfaced trauma, family shame, and persistent online abuse. She explains the long-term impact and the futility of control:
  • "Something that's really important to me, is it to be known that like I am not opposed to sex work. I just don't want to be a sex worker." – Cutie [32:24]
  • On explaining to her family and emotional fallout: [33:02–34:51]

---

9. Aftermath and Moral Reckoning

[34:51–36:57]

• The Atriok scandal marks a turn: someone caught paying to view these images is forced to apologize, yet the deeper chain of creation remains hidden.
  • "The fallout came quickly. Atriox streamed a tearful apology, his wife in the background, also tearful. Later, he donated $60,000 to a law firm to help with takedowns..." – Sam Cole [35:53]
• Despite apologies, most content remains; Cutie and Atriok continue to work together, underlining the necessity of professional survival amid unresolved harm.

---

10. The Scope Today: Anyone Can Be Targeted

[36:57–38:24]

• The episode notes the shifting targets of deepfake porn—from celebrities to ordinary people. The makers remain anonymous; the reviewed storylines hint at a global kingpin, “Mr. Deepfakes.”
  • "Anyone can make a deepfake of anyone, which means the person uploading it could be your neighbor, your coworker, your best friend. And the person in the video could be you." – Sam Cole [36:57]

---

Notable Quotes and Moments

• On personal violation:
  • "It is so convincingly my body, but not my body. And holy shit, it hits you like a truck."
    — Cutie Cinderella [04:54]
• On early technology’s moral sidestepping:
  • "Within the computer vision community, Deepfakes comes out. No one really wants to comment on that ... it's not our problem."
    — Walter Schreier [25:29]
• On emotional trauma:
  • "It was the same feeling, like where you feel guilty, you feel dirty, you feel what just happened... it makes that resurface."
    — Cutie Cinderella [33:02]
• On ubiquity of threat:
  • "There is no woman in the world who is safe from this technology."
    — Jamie (Podcast Host) [37:37]

---

Important Timestamps

| Timestamp | Segment/Topic | |:--------------:|:----------------------------------------------------| | 01:26–05:06 | Cutie Cinderella’s discovery and initial reaction | | 06:55–10:33 | History of fake photography and Photoshop era | | 12:35 | Lux Luker on the early fake porn community | | 18:31–22:03 | Ian Goodfellow invents GANs | | 22:16–25:38 | Reddit’s “deepfakes” and the first AI porn videos | | 25:52–28:25 | Political deepfakes and public panic | | 28:25–34:51 | Victim experience, family impact, and online abuse | | 34:51–36:57 | Atriok’s apology and the limits of accountability | | 36:57–38:24 | Escalating anonymity, ubiquity, and coming investigation |

---

Tone and Approach

The episode is investigative yet personal, blending deep reporting with the raw, unfiltered voices of both victims and creators. The narrative refuses to sensationalize, instead foregrounding the lived pain, professional complicity, and systemic failures that allow this abuse to proliferate. Technical history is woven with culture and emotion, pushing listeners to engage with the uncomfortable reality: anyone can be a victim, and the chasm between technological optimism and harm remains vast.

---

Looking Ahead

The closing moments lay out the season’s quest: tracing the global investigation to the Canadian kingpin behind the world’s biggest deepfake porn hub, “Mr. Deepfakes,” and the efforts to hold such actors accountable.

---

For listeners seeking to understand deepfakes—not only as a technological marvel, but as a tool of violation and exploitation—this episode offers a clear, urgent, and empathetic primer on how we got here, and what it means for all of us.

---

Transcript

[Paylocity Advertiser]

When everything is moving all at once, your workforce, your tech stack, your business, you don't need more tools. You need one solution. That's why Paylocity built a single platform to connect hr, finance and IT with AI driven insights and automated workflows that simplify the complex and power what's next. Because when everything comes together in one place, growth comes easy experience. One place for all your HCM needs. Start [e-mail] 1

[CBC Announcer]

this is a CBC podcast.

[Jamie (Podcast Host)]

Hey everybody, Jamie here. We have a special bonus episode for you all today from the brand new season of Understood, a Deep Dive mini series feed covering everything from crypto chaos to the rise of tech oligarchs and the broken promises of the Internet. This new season is called Deep Fake Porn Empire. Deepfake porn is a billion click industry built on stolen faces. While the people making it hide theirs behind. SCRE host Sam Cole traces the decades long rise of synthetic porn, the targets who are fighting back, and the global investigation that led to its Canadian kingpin. Sam's been on the show a bunch of times, a fantastic reporter and this season is really excellent. Now here's the first episode, the dawn of Fake Porn. If you like it, all four episodes of the season are available to follow right now. Have a listen.

[Cutie Cinderella (Streamer)]

Foreign.

[Sam Cole (Host and Reporter)]

It's January 31, 2023 when a streamer who goes by Cutie Cinderella wakes up and something strange is happening. She's trending on Twitter, which even when you're Internet famous like Cutie is, is really never a good sign.

[Cutie Cinderella (Streamer)]

And I'm like just waking up, trying to figure out what is going on. And then I get a call from one of the women involved that she's like, yo, have you seen everything? I'm like no. And then she really broke it down for me and we went through and then that's the first time I got on Twitter and I was like, oh my God. Like.

[Sam Cole (Host and Reporter)]

Cutie Cinderella is a twitch streamer.

[Cutie Cinderella (Streamer)]

We're doing something fun today. I think it's genuinely been a few years since we have done an Ask me anything about me Cutie Cinderella. Hi, that's me. Hi guys. So we're going to do that today.

[Sam Cole (Host and Reporter)]

Streaming is pretty simple. You broadcast yourself live, usually playing video games to thousands of people watching in real time. Oh boy.

[Cutie Cinderella (Streamer)]

Oh sorry.

[Sam Cole (Host and Reporter)]

Cutie got her start playing League of Legends. It's one of the biggest video games on the planet. A kind of online gladiator arena where teams battle it out.

[Cutie Cinderella (Streamer)]

Yeah, you're dead.

[Sam Cole (Host and Reporter)]

Oh my lord, he be camping from there. She Became known for bringing people together, co hosting podcasts, arranging meetups in real life.

[Cutie Cinderella (Streamer)]

Say hi to chat guys, Happy girls chat.

[Sam Cole (Host and Reporter)]

Cutie Cinderella is used to being online, and she's used to people talking about her online. But she wasn't prepared for what happened that January morning in 2023. It all started the day before with a guy who goes by Atriok. Atriok is in Cutie's circle. They've streamed together. So he's a colleague, a friend really. And he was live on Twitch when he wanted to check a different window.

[Walter Schreier (Engineering Professor)]

What time is it? Where I am?

[Lux Luker (Carrie Pearson, Fake Porn Community Founder)]

What time is it? Pst.

[Sam Cole (Host and Reporter)]

That's the moment that upended Cutie's life. Atriok hits Alt tab and for barely a second, every window he has open flashes on screen to the thousands of people watching. Among them a YouTube video of lovely Day by Bill Withers. A Google search for Jennifer Garner in Catch Me if youf can, his recent Uber Eats orders, and a website selling porn. Not just any porn, but porn of women he knows, women he works with, including Cutie Cinderella. Someone takes a screenshot and posts it online.

[Cutie Cinderella (Streamer)]

And then it just created a wildfire.

[Sam Cole (Host and Reporter)]

The pictures ripple across. Discord, Reddit, Twitter. And by the next morning, Cutie's phone is blowing up.

[Cutie Cinderella (Streamer)]

I was already getting DM'd, the photos or reply to my tweets and stuff like that, before I even had a full grasp of what the hell is going on.

[Sam Cole (Host and Reporter)]

A bunch of the messages have links, and when she clicks through, she finds a porn site she's never seen before, never even heard of. And she's on it, her face, doing things she never did. Because here's the thing, it wasn't Cutie, not really the porn. It was a deep fake.

[Cutie Cinderella (Streamer)]

It is so convincingly my body, but not my body. And holy shit, it hits you like a truck. You feel so violated.

[Sam Cole (Host and Reporter)]

Deepfake videos, which are manipulated using AI, can make someone appear as though they're saying or doing something that they're not. Tech experts are sounding alarm bells over the rapid spread of AI generated explicit images of women online.

[Ian Goodfellow (AI Researcher)]

We're moving into a future where you

[Walter Schreier (Engineering Professor)]

really won't know what's real.

[Sam Cole (Host and Reporter)]

Online, we've seen a significant increase in deep fakes.

[TMFU (Anonymous Photoshop Porn Creator)]

And as our research has shown, 96%

[Sam Cole (Host and Reporter)]

of these are pornographic. I'm Sam Cole. I'm a tech journalist at 404 Media, where I write about sex and the Internet. And I've been reporting on deepfake porn since the very beginning. And here's what I've Learned deepfakes didn't just come out of nowhere. They were built by people on platforms, inside subcultures. They were allowed to spread while governments dragged their feet and tech companies shrugged. And at every step, someone profits, while the targets, almost always women, pay the price. So how did we get here? And if you follow the trail of deepfake porn all the way to the source, who does it lead to? This is understood. Deepfake porn empire Episode 1 the dawn of Fake Porn. Deepfakes, you've seen them everywhere, AI videos of politicians, celebrities, random people on TikTok. Sometimes so seamless that you might not even realize you're looking at a fake. And we have this idea that it's a fast moving new threat. But fake images, yeah, they're not exactly new.

[Walter Schreier (Engineering Professor)]

Basically, as soon as the camera is Invented in the 19th century, people are faking their photos.

[Sam Cole (Host and Reporter)]

This is Walter Schreier. He's a professor of engineering at Notre Dame, where he researches AI the tech, but also the cultural history. And when it comes to fake things online, he wrote the book.

[Walter Schreier (Engineering Professor)]

I am the author of A History of Fake Things on the Internet. I think a lot of folks today believe film photography was rather cut and dry, right? It's like you took the photo and it captured an objective picture of reality. But that was never really the case.

[Sam Cole (Host and Reporter)]

In fact, the very first fake photo showed up in 1840, made by an early camera inventor.

[Walter Schreier (Engineering Professor)]

This inventor, Hippolyte Bayard, is in France and he's really, really annoyed that his camera process is not receiving the attention he thinks it should.

[Sam Cole (Host and Reporter)]

So he comes up with a publicity stunt.

[Walter Schreier (Engineering Professor)]

He fakes his own death in a photograph, writes a little story on the back of this photo, right, Protesting all this, explaining why he died. But again, it was all this sort of hysterical exaggeration, the kind of thing we associate with the Internet today.

[Sam Cole (Host and Reporter)]

From there, fakes only multiplied. One of the most famous came decades later in 1917.

[Walter Schreier (Engineering Professor)]

So you have two cousins in the English countryside that have access to a camera, and they produced a remarkable series of photographs where they appeared to be, you know, in different natural settings. The woods with a bunch of dancing fairies.

[Sam Cole (Host and Reporter)]

Yeah, like little winged Tinkerbells.

[Walter Schreier (Engineering Professor)]

Now, the fairies generated this enormous debate. Notable figures get involved in this debate, most prominently Sir Arthur Conan Doyle, the author of the Sherlock Holmes series.

[Sam Cole (Host and Reporter)]

And when the creator of the most famous detective in literature saw the photos, he was fully convinced they were real.

[Walter Schreier (Engineering Professor)]

He absolutely defended these photos till he died. Now, flash forward. Several decades later, one of the cousins steps forward and says, you know what? Those were just cardboard cutouts. They were fake.

[Sam Cole (Host and Reporter)]

The incident became known as the Cottingly fairies. And a century later, we're still at it, still faking images. Only our tools got a serious Upgrade. In the 1970s, digital cameras entered the scene.

[Walter Schreier (Engineering Professor)]

Flash forward, then just a couple more years, you get the appearance of Photoshop, a piece of software that has stood the test of time.

[Sam Cole (Host and Reporter)]

In 1990, Adobe was best known for making printer software and tools for font nerds. Photoshop put them on the map and got them on the news.

[Unidentified Commentator on Photoshop Usage]

Technology makes it difficult, maybe even impossible, to tell what's real and what's not. Recently the issue of altering photographs came up when Playboy added its logo to a picture of Roseanne Arquette without her knowledge or her approval. And then there was a huge publicity outcry when TV Guide tacked Oprah Winfrey's head onto Ann Margaret's body.

[Sam Cole (Host and Reporter)]

Suddenly, the kind of image editing power once reserved for million dollar studios was available to every graphic designer and everybody else.

[Walter Schreier (Engineering Professor)]

If you look at the actual examples of how people were using the earlier tools, right, like Photoshop, in very bad contexts, it was almost always pornography.

[Sam Cole (Host and Reporter)]

And we were able to track down a guy who was making Photoshop porn in this era.

[TMFU (Anonymous Photoshop Porn Creator)]

I actually started playing around with this stuff at home, strictly privately. And yes, that obviously already involved nudity. It was very clumsy and even limited to grayscale images. But at the time it was cutting edge and easier to suspend disbelief than taping a cut out physical picture of a head over a Playboy centerfold.

[Sam Cole (Host and Reporter)]

I guess this is tmfu, that's his username and he has a digital footprint in the fake porn scene going back decades. TMFU didn't want to be interviewed and he wouldn't share his real name. But when we emailed him, he responded with a multi thousand word manifesto. Those emails are what my colleague is reading from and the story goes right back.

[TMFU (Anonymous Photoshop Porn Creator)]

It got all started when Lux Luker, a Canadian guy, libertarian type, created a group dedicated to celebrity nude faking in 1996.

[Sam Cole (Host and Reporter)]

Lex Luker's real name was Carrie Pearson and he ran two Usenet groups dedicated to fake nudes of celebrities. Quick refresher. Usenet was basically the early Internet's version of Reddit. Think of it like a message board, but run through email.

[TMFU (Anonymous Photoshop Porn Creator)]

I'd say there were probably a good hundred regular people from all over the world sharing their Photoshops. This was at the time of Britney Spears, Baywatch Girls and every American sitcom girl you can imagine.

[Sam Cole (Host and Reporter)]

Lux Luker also ran a website which is still archived all these decades later. Www.layerofluxluker.com Lux's website is peak 90s Internet. A black background decorated with neon naked lady silhouettes, half a dozen different shades of neon text and comic Sans everywhere. All of it dedicated to fake celebrity porn and the community churning it out. My sound designer is going to read from it.

[Lux Luker (Carrie Pearson, Fake Porn Community Founder)]

We all love seeing the heads of some of our favorite stars and other prominent people placed on top of a nude model. The sheer godlike power of exposing them to the world for our own fantasies is a healthy, and thanks to the easy availability of advanced image manipulation programs, relatively easy task to undertake.

[Sam Cole (Host and Reporter)]

When TMFU found Lux Luker's website, he'd found his people.

[TMFU (Anonymous Photoshop Porn Creator)]

When you can share a particular interest like that, especially a naughty one, with other people who are also fascinated by it, it gets an entirely different dimension. I probably spent every night participating in the newsgroup, working on fakes and posting them, learning, commenting on fakes by others, exchanging tips. I probably finished and posted 150 fakes in the first year. It completely consumed me.

[Sam Cole (Host and Reporter)]

The celebrity fakes groups did have rules about who they could fake and how. You can still read these on Luxx's website.

[Lux Luker (Carrie Pearson, Fake Porn Community Founder)]

No real nudes of celebrities. These have plenty of their own groups. No advertisements for commercial porn photo sites, also known as spam. No fakes of bestiality, and no fakes of minors. Artists will not post or take requests for celebs under 18.

[Sam Cole (Host and Reporter)]

If that feels somewhat reassuring, it doesn't last. Right below that warning, there's a chart listing underage celebrities, all teenage girls with the years they would turn 18. Lux died in 2004 from complications related to diabetes. This means that we can't ask him how he'd look back on the ethics of all of this now, but he had a statement about it on his website.

[Lux Luker (Carrie Pearson, Fake Porn Community Founder)]

Are these pictures ethical? My personal opinion is that it would be unethical to present them as being a real representation of the person or the activities depicted. A projection of personal fantasy, rightfully identified as such, should not cause us to lose any sleep. Except, of course, to masturbation.

[Sam Cole (Host and Reporter)]

In 2003, the year before he died, Lux Luker told Wired that he believed there were about 300,000 celebrity porn fakes out there. And even once he was gone, the community kept growing. Meanwhile, everyone else was discovering digital fakery with a little help from Hollywood Boy

[Cutie Cinderella (Streamer, personal statements)]

in my head being right all the time.

[Walter Schreier (Engineering Professor)]

Yeah, I remember seeing Jurassic park when it first came out in the theater and there was this game to play. You know what Scene was shot with a puppet versus what scene was computer graphics? Some of the T. Rex shots, right when the kids are in the jeep, keep absolutely still.

[Sam Cole (Host and Reporter)]

This vision's based on movement.

[Walter Schreier (Engineering Professor)]

You know, it's like the head, the puppet, right? Was it the body? That's cg. You know, there were a lot of questions. And when you have that kind of ambiguity, you're doing a really good job with the computer graphics.

[Sam Cole (Host and Reporter)]

Jurassic park wasn't the first film to use CGI. But in 1993, the film proved computer generated creatures could look real or real enough. And that opened the door to another holy grail. Digital actors.

[Walter Schreier (Engineering Professor)]

Hey, thought you could leave without saying goodbye.

[Sam Cole (Host and Reporter)]

The most famous early example happened in 2013 when Paul Walker died midway through filming Fast and Furious 7. The studio needed a way to finish the movie, so they got Paul's brothers and CGI'd his face over theirs.

[Walter Schreier (Engineering Professor)]

What do you think? Parking brake slide right up to the school?

[Sam Cole (Host and Reporter)]

Philosophically, it kinda really creeped people out, but visually it worked.

[Walter Schreier (Engineering Professor)]

This is where you start to get these debates around, like, what? What is real? Like what do you trust? Right? And then you really do start to wonder, you know, like, where could this veer off course?

[Sam Cole (Host and Reporter)]

But there were limitations. The dinosaurs, Paul Walker's face. This was labor intensive. It was all hand animated by teams of people, frame by frame.

[Walter Schreier (Engineering Professor)]

So that's when you start to see researchers in this space discussing the possibility of. Of automating this process. Right. Again, turning to computer vision and moving away from the sort of, you know, very manual process of computer graphics.

[Sam Cole (Host and Reporter)]

And that's where this story tips into something new. You've got Hollywood's painstaking process on one side, testing the public's tolerance for fakes, fake celebrity porn, photoshoppers on the other. And soon a brand new technology about to smell smash through both.

[CBC Announcer]

This episode is brought to you by Progressive Insurance. Do you ever think about switching insurance companies to see if you could save some cash? Progressive makes it easy to see if you could save when you bundle your home and auto policies. Try [e-mail] Progressive Casualty Insurance Company and affiliates. Potential savings will vary. Not available in all states.

[Helena Merriman (BBC Journalist)]

If journalism is the first draft of history, what happens if that draft is flawed? In 1999, four Russian apartment buildings were bombed, hundreds killed. But even now, we still don't know for sure who did it. It's a mystery that sparked chilling theories.

[Walter Schreier (Engineering Professor)]

Helena.

[Helena Merriman (BBC Journalist)]

I'm Helena Merriman, and in a new BBC series, I'm talking to the reporters who first covered this story. What did they Miss the first time the History Bureau, Putin and the apartment bombs. Listen on BBC.com or wherever you get your podcasts.

[Sam Cole (Host and Reporter)]

The Future arrived in 2014 at a Montreal microbrewery where a grad student named Ian Goodfellow was getting drunk.

[Ian Goodfellow (AI Researcher)]

I don't want to be someone who goes around promoting alcohol for the purposes of science, but in this case, I do actually think that drinking helped a little bit.

[Sam Cole (Host and Reporter)]

That's him talking on the Lex Friedman podcast today. Ian Goodfellow is an AI researcher at Google DeepMind. But at the time, he was a PhD student at the University of Montreal. He was basically done. He just handed in his thesis. And that night, out drinking with friends, they kept circling the same question about something they were all studying generative models. Here's Walter Schreier again, author of a history of Fake things on the Internet.

[Walter Schreier (Engineering Professor)]

All a generative model is, is an algorithm that generates data that is new.

[Sam Cole (Host and Reporter)]

At the time, there were a few generative models that produced images, but the results looked wrong. Blurry, distorted half apples, misshapen faces. What researchers wanted was a system where you could dump in data and it would invent totally new images, ones that looked real. At the time, Walter Schreier was also a PhD. He was at Stanford working in a similar field. And he remembers this being the big goal.

[Walter Schreier (Engineering Professor)]

Yeah, I think all the researchers wanted was useful output, like photorealistic output. That's all they were looking for.

[Sam Cole (Host and Reporter)]

So this was the problem. Ian and his friends were debating that night how to build a program that could do this.

[Ian Goodfellow (AI Researcher)]

And then finally, when I was arguing about chocolate models with my friends in a bar, something clicked into place. And I started telling them, you need to do this, this, and this, and I swear it'll work. And my friends didn't believe me that it would work, but I believed strongly enough that it would work that I went home and coded it up the same night. And it worked.

[Sam Cole (Host and Reporter)]

It worked, worked. Ian invented a system that could produce new photorealistic images. He called his invention Generative adversarial networks, or GANs for short. And the breakthrough was two systems playing a game. One makes fakes, the other tries to spot them.

[Ian Goodfellow (AI Researcher)]

You can think of it as kind of like an art critic at the start. The generator makes images that aren't real at all, and the discriminator doesn't know what's pretty real or fake. But when they play the game against each other, where they have to try to fool each other all the time, eventually the generator learns to make very realistic images.

[Sam Cole (Host and Reporter)]

Within months, Ian Goodfellow published a paper.

[Walter Schreier (Engineering Professor)]

I remember specifically my lab mate showing me this paper. He's like, look what Goodfellow just released. It was like what? It was like. This is incredible. I mean, it was basically a turning point for the entire field.

[Sam Cole (Host and Reporter)]

When Ian cracked this technology, he had big dreams for what his invention might unlock.

[Ian Goodfellow (AI Researcher)]

I thought about a lot of important problems in science that I could solve that would help people.

[Sam Cole (Host and Reporter)]

He was excited about how Gans could be used to help design medicine, how they could fix bias in data. Like if there was an underrepresented category, a Gan could create realistic examples to help fill in the gaps. And then there was what they could do for teeth.

[Ian Goodfellow (AI Researcher)]

Specially trained technicians spend about two weeks to make each patient's dental crown. Now, with generative adversarial networks, it's possible to design them basically instantly and then 3D print them.

[Sam Cole (Host and Reporter)]

But no one was really thinking about porn.

[Walter Schreier (Engineering Professor)]

No, nobody was thinking about por.

[Sam Cole (Host and Reporter)]

Here's the thing about science, researchers love to share, make their findings reproducible. That's how the field moves forward. So after his paper was published, Ian Goodfellow quickly made the source code for gans available on GitHub, the public code sharing platform.

[Walter Schreier (Engineering Professor)]

So as the technology just got more accessible, you see amateurs on the Internet taking this code and doing things researchers never imagined would happen because they expected only other researchers to use the open source code. Surprise, surprise, that did not happen. And so you see the appearance of like the original deepfake algorithm. You know, it's like a pseudonym on Reddit saying, I have this new porn generating AI system, check it out. And you know, it spirals from there.

[Sam Cole (Host and Reporter)]

I remember that spiral very clearly because I was the journalist who broke that story. It was December of 2017 and I was a reporter at Motherboard, that's Vice's tech outlet. And my editor found a post in a subreddit called Celeb Fakes where people were doing the old school Photoshop thing. The post he sent me was by a user named deepfakes. This was a play on deep as in deep learning, like Gans, and fake as in celebrity fakes. This one guy would go on to inadvertently name the whole phenomenon, but at the time, he was just some rando. Deepfakes had posted a video. It was of actor Gal Gadot, you know, Wonder Woman, lying on a bed, waving a sex toy around. I clicked and the page opened to what looked like a porn video of Godot, only it wasn't quite her. It was her face stitched onto a porn performer's body. I Researched porn and the Internet for a living, and I'd never seen anything like it. At that point, deepfakes had posted porn videos featuring the faces of Scarlett Johansson, Maisie Williams, Taylor Swift, Aubrey Plaza, and now Gal Gadot on Reddit. The tech wasn't seamless. Gal's face glitched, and every now and then you could see the real actor's face under hers. But I remember thinking, this is going to get so much worse. So I sent him this message, do you think this technology could be used with bad motivations in the future? And he got back to me, every

[CBC Announcer]

technology can be used with bad motivation. I don't think this is any different

[Sam Cole (Host and Reporter)]

than recreating Paul Walker in Fast and Furious. I wrote it up for Motherboard and in 2017, became the first journalist to publish a story on AI porn. My headline was AI assisted fake porn is Here and We're All Fucked. When my story came out, I'm not sure what I expected to happen. Outrage, maybe. And there was some of that, but mostly what I got were shrugs.

[Walter Schreier (Engineering Professor)]

Within the computer vision community, Deepfakes comes out. No one really wants to comment on that. It makes the field look bad, obviously, to sort of just like, you know, it's not our problem.

[Cutie Cinderella (Streamer)]

Right.

[Walter Schreier (Engineering Professor)]

We didn't. We didn't create it. We're not. We're not the deepfakes pseudonym on Reddit, you know, and that. That's just not. That was not a good response. And again, I think also the hype about the political stuff, right? Like the. The field was happy to continue to talk about that.

[Sam Cole (Host and Reporter)]

People were paying attention to deep fakes, just not porn. These manipulated images can pose a very real national security threat.

[Ian Goodfellow (AI Researcher)]

Fake videos could become a real and present danger to our democracy.

[Sam Cole (Host and Reporter)]

The presidential election season is ramping up, and so are the warnings about deepfake technology being used to disrupt the campaign. In 2018, BuzzFeed and Jordan Peele made a fake video of Barack Obama. Obama looked straight into the camera and called Donald Trump the total and complete dipshit. Of course, none of it was real. Jordan Peele provided the voice. The AI filled in the lips. It was a warning. This is what's coming. The warning landed. Newsrooms ran headlines about the end of trust. Congressional hearings predicted fake candidates and fake wars. Darpa, the Pentagon's research arm, launched a massive media forensics program. Tens of millions of dollars were poured into building deepfake detectors. Algorithms that could spot tiny lighting inconsistencies, missing frames, spliced pixels. During Trump's first term, the panic went Nuclear analysts went on cable news to freak out about a hypothetical apocalypse triggered by a hypothetical deepfake of Trump. Trump saying he'd launch nukes at North

[Walter Schreier (Engineering Professor)]

Korea, which is like the most implausible scenario in foreign politics, if you ask me. The hardest thing to do is, like, launch a nuclear missile, right? Like an anonymous account online. Right. Like saying things is not gonna, like, trigger World War III.

[Sam Cole (Host and Reporter)]

Of course, political deepfakes do happen. There's a notorious YouTube ad featuring a deepfake of Canadian Prime Minister Mark Carney trying to lure people into a shady investment scheme.

[Ian Goodfellow (AI Researcher)]

It's a special investment system backed by the government and designed for the average citizen to earn Money.

[Sam Cole (Host and Reporter)]

And in 2022, a deepfake of Ukrainian President Volodymyr Zelenskyy surrendering to Putin went viral. But nobody in the trenches threw down their weapons and ran off the battlefield. Because political fakes are very quickly delegitimized.

[Walter Schreier (Engineering Professor)]

Some people are genuinely fooled. Right. Which is. Which is a problem. Right. But for the most part, again, I don't think that's quite the case with a lot of this material. They're almost always released on, like, an anonymous account or through a pseudonym. Right. It's not a major media outlet reporting it as fact. You know, if it were real, we would have heard about it through a legitimate source.

[Sam Cole (Host and Reporter)]

But for the people targeted by deepfake porn, it doesn't matter if it's delegitimized or not. It doesn't matter that it's not technically real, whatever that means. The violation feels the same. Which brings me to something else calling non consensual deepfakes porn, even when they're technically pornographic, is not quite right. It's like how there's no such thing as abusive porn or non consensual porn that's sexual assault or rape. But the phrase non consensual AI generated sexually explicit images is a mouthful. So deepfake porn, for better or worse, is the name that stuck. And the scope is massive. Back in 2018, researchers at the cybersecurity firm DeepTrace found that 96% of all deepfake videos online were pornographic and non consensual. Five years later, a 2023 study by Sensiti, an identity verification company, had the same findings.

[Walter Schreier (Engineering Professor)]

And yet it takes, like, several years after the appearance of deepfakes till we're really talking about the problem of porn. Right?

[Sam Cole (Host and Reporter)]

And the moment that really happened. 2023 with Cutie Cinderella.

[Cutie Cinderella (Streamer, personal statements)]

This is probably the stupidest thing I've ever done.

[Sam Cole (Host and Reporter)]

That morning after finding out that she'd been deep baked and that the videos were all over the Internet, Cutie Cinderella had to decide what to do. For her, the answer was almost inevitable. She's a live streamer, so she live streams.

[Cutie Cinderella (Streamer, personal statements)]

And I'm sure everyone in the world would tell me not to go live right now, but I want to go live. Because this is what pain looks like. This is what it looks like. The Internet, the constant exploitation and objectification of women. It's exhausting. It's exhausting. Fuck Atriok for showing it to thousands of people. Fuck the people dming me pictures of myself from that website. Fuck you all. And I think you guys need to know what pain looks like, because this is it. This is what it looks like to feel violated. This is what it looks like to feel taken advantage of. This is what it looks like to see yourself naked against your well being, spread all over the Internet.

[Sam Cole (Host and Reporter)]

And Cutie's pain, her anger, it goes viral. The streaming world was shaken to its core recently. Her name is Cutie Cinderella.

[TMFU (Anonymous Photoshop Porn Creator)]

With a heartbreaking and infuriating video that she posted.

[Walter Schreier (Engineering Professor)]

AI Deep faked porn of his streaming co workers.

[Sam Cole (Host and Reporter)]

Cutie's stream and the story about what happened, it's written about in Wired, the Washington Post, Business Insider, Entertainment Tonight. The list goes on.

[Walter Schreier (Engineering Professor)]

But the ramifications from this are going to last much longer.

[Sam Cole (Host and Reporter)]

Hello? Oh hi. Can you hear me? And shortly after that, she talked to me.

[Cutie Cinderella (Streamer)]

I know a lot of the women a part of this are refusing to do interviews and it's because they don't want this a part of their narrative. I don't want this part of my narrative either, but plus, it's the only option I have to hopefully do something about in the future. I hope that in 10 years when my niece is more on the Internet, she doesn't have to deal with something like this just for assisting as a woman on the Internet.

[Sam Cole (Host and Reporter)]

For Cutie, the violation of being deep faked wasn't the end of the invasion.

[Cutie Cinderella (Streamer)]

The harassment has been relentless. You go to my YouTube cards, you search my name, you do anything. It's. It's just there and it's like it sucks. Something that's really important to me, is it to be known that like I am not opposed to sex work. I just don't want to be a sex worker. The problems could send and it's just fucking miserable that people could take whatever they want from you and turn it into whatever they want.

[Sam Cole (Host and Reporter)]

Beyond the harassment, the videos and the comments everyone could see. Cutie told me about all the things that happened behind the scenes, the, the dominoes, the deep fakes knocked over, the things they dredged up.

[Cutie Cinderella (Streamer)]

I said this in one interview before, but I was, I was sexually assaulted as a child. And it was the same feeling, like where you feel guilty, you feel dirty, you feel what just happened. And it's bizarre that it, it makes that resurface. I genuinely didn't realize it would. Yeah. When I first heard about it, I think I was born a whirlwind. I didn't get hit with it psychologically until I saw the photos. I, I think I potentially could have been someone on the Internet that was lacking empathy before it did happen to me.

[Sam Cole (Host and Reporter)]

Yeah.

[Cutie Cinderella (Streamer)]

Yeah.

[Sam Cole (Host and Reporter)]

It's so scary how like, you know, even if like years and years later and you, you're like, I'm, I'm healing and then something like that can just be like right back there.

[Cutie Cinderella (Streamer)]

It did another layer to it my family seen went so viral that my family saw it and some people went out of their way to send it, potentially some of the girls, the photos were used as blackmail to try to get stuff out of them. It's just like, it's been so gross. And none of this is stuff that we've created. These aren't even photos of ourselves. But I'll tell you what, my 65 year old dad, it'd be a hard time explaining to him that that's not real.

[Sam Cole (Host and Reporter)]

Have you had to like explain it to your parents?

[Cutie Cinderella (Streamer)]

I've literally avoided calling my dad. I've had conversations with a few of my cousins, one of my aunts, as well as my sister. And you know, they all get it, but they also think that the Internet's evil. And you know, they, it's so sad because of all the good things I've done in this career. I've done so much. I've raised tons of money for charity. I've made community events that tried to highlight people that maybe haven't had the opportunity to be highlighted. I've done so much, but this is what my family now knows as my job.

[Sam Cole (Host and Reporter)]

For many people, news coverage of Cutie Cinderella's story was their first time hearing about Deepfake porn. Or if they had heard of it before, this was their first time coming face to face with the real emotional impact it had on the people who were used in. Was also one of the first times someone, in this case Atriok, the streamer who exposed the porn site was caught and called out for watching.

[Cutie Cinderella (Streamer)]

Was a shock, genuinely a shock to see a face behind someone that paid for this and someone that saw this. And so I think the taboo of that is one reason that it got so large. I think also there's even more taboo because he is my boyfriend's best friend. My boyfriend was in his wedding party and so the fact that he was able to scroll past naked photos of me and have no emotional or visceral reaction to exit out of the page. Right. Bizarre.

[Sam Cole (Host and Reporter)]

The fallout came quickly. Atriox streamed a tearful apology, his wife in the background, also tearful. Later, he donated $60,000 to a law firm to help with takedowns and legal support for the streamers who'd been deep faked. Cutie filed takedown notices. Some of the content came down, not all of it. The site Atriok had been looking at pulled the videos offline and posted an apology. Today, Cutie and Atriok still stream together. By all appearances, they've moved past it. There's a cynical way to look at this. He got wrapped on the knuckles and she had to get over it to keep her career in a space they shared. But there's also a hopeful one that people can apologize and come back from this. But here's the thing. Yes, Cutie saw who was watching deepfakes of her, but she never found out who made them. For years, fake porn has been thought of as a celebrity curiosity, a weird fringe corner of the Internet. Pop stars, movie stars, streamers, as if that was just the price of fame. But anyone can make a deepfake of anyone, which means the person uploading it could be your neighbor, your co worker, your best friend. And the person in the video could be you.

[Cutie Cinderella (Streamer)]

We were still living together, so we

[Sam Cole (Host and Reporter)]

still had to share content. We still had to share a bathroom. I was terrified.

[Walter Schreier (Engineering Professor)]

The reality is it's very easy to anonymize yourself online.

[Jamie (Podcast Host)]

There is no woman in the world who is safe from this technology.

[Sam Cole (Host and Reporter)]

I'm not just gonna like sit here and take it. This season of understood, we are burrowing into the digital world of non consensual deepfake porn. The people impacted, the political battles and legal loopholes. And we'll follow the trail because behind the people watching and making deepfake porn, there's someone else. A spider at the center of a web. A kingpin. An investigation that begins in Denmark will ignite a chase across three countries and four newsrooms as a CBC journalist closes in on the very real person behind the world's largest fake porn website, Mr. Deep bakes himself. Mr. Deepfakes.

[Walter Schreier (Engineering Professor)]

Mr. Deepfakes Mr. Deepfakes all the man behind Mr. Deepfakes Mr. Deepfakes is the most notorious non consensual deepfake porn site in the world.

[Sam Cole (Host and Reporter)]

He has been using women's faces for years, exploiting them, earning money on them.

[Helena Merriman (BBC Journalist)]

Now it's his turn to be on the front page.

[Sam Cole (Host and Reporter)]

In this episode you heard archival tape from CBC and Cutie Cinderella Ludwig React Max Mintberry Crunch the Kim Commando Show Leduc Leadership, the Today show from NBCUniversal, Jurassic park from Universal Pictures and Amblin Entertainment Furious 7 from Universal Pictures Lex Friedman Podcast Preserve Knowledge the AI Podcast Rework association for Computing Machinery, Fox News, Buzzfeed, NPR, Dextero, Penguins and Another Body My AI Nightmare Deepfake Porn Empire is written and produced by our showrunner, A.C. rowe and me. Your host, Sam Cole. Arman Agbali is the producer. Sound design by Julian Uzieli, who also voiced Carrie Pearson, AKA Lux Luker. TMFU was voiced by Greg Kelly. Our story editor is Veronica Simmons and our Executive producer is Nick McKay. Blokos. If you enjoyed this episode, make sure you check out previous seasons of Understood, like season two the pornhub Empire, which I also hosted. I'll take you inside the rise and reckoning of one of the biggest porn platforms in the world. You'll find that wherever you found this podcast, so go check it out and make sure you hit follow. Then meet me back here for the next episode of Deepfake Porn Empire.

[Jamie (Podcast Host)]

That was the first episode of Deepfake Porn Empire. If you like what you heard, all episodes are available right now. Just search for the Understood feed wherever you get your podcasts. There's a brand new season coming later this spring, so be sure to follow the feed so you don't miss an episode.

[CBC Announcer]

For more CBC Podcasts, go to CBC CA Podcasts.