Deepfake Porn Empire | E1: The Dawn of Fake Porn

Summary

---

Podcast Summary: Understood (CBC)

Episode 1: The Dawn of Fake Porn
Published: February 17, 2026
Host: Sam Cole

---

Episode Overview

The first episode of "Understood: Deepfake Porn Empire" traces the origin and explosive rise of deepfake porn—a billion-click, billion-dollar industry built on AI-generated, non-consensual images, with women overwhelmingly targeted. Host Sam Cole explores the multi-decade history of image fakery leading to today's deepfakes, devastating personal stories (most notably streamer Cutie Cinderella), and the technology’s shadowy creators and profiteers. This episode sets the stage for an investigative journey—culminating in a hunt for the Canadian kingpin at the center of the world’s largest deepfake porn site.

---

Key Discussion Points and Insights

1. Cutie Cinderella's Viral Deepfake Ordeal (00:49–05:00, 29:23–35:30)

• Incident: In January 2023, Twitch streamer Cutie Cinderella discovers she’s trending—only to realize a friend and colleague, Atriok, accidentally exposed on stream that he visited a site selling deepfake porn of women he knows, including Cutie herself. This triggers a viral social media storm.
• Emotional Impact:
  • Cutie shares the horror of seeing AI-generated pornographic images with her face, saying:

    "It is so convincingly my body, but not my body. And holy shit, it hits you like a truck. You feel so violated." — Cutie Cinderella (04:11)

  • Cutie’s live emotional response:

    “Because this is what pain looks like. … Fuck the fucking Internet. Fuck the constant exploitation and objectification of women. It's exhausting.” — Cutie Cinderella (live stream, 29:47)

• Ramifications:
  • The event draws mainstream media attention: Wired, Washington Post, Entertainment Tonight, among others.
  • Cutie recounts ongoing harassment, family fallout, and the overwhelming feeling of being violated and helpless to erase the content.
  • On legacy:

    “I've done so much … But this is what my family now knows as my job.” — Cutie Cinderella (34:14)

Timestamps

• 00:49–05:00: Intro and breakdown of the accidental on-stream reveal
• 29:23–35:30: Cutie discusses the aftermath, harassment, family impact, and forced public reckoning

---

2. The Long History of Image Fakery (06:12–17:00)

• Historical Precedent: Professor Walter Schreier unpacks how image fakery dates back to the dawn of photography:
  • 1840: Hippolyte Bayard fakes his own death in a publicity stunt.
  • 1917: The Cottingley Fairy photos fool even Arthur Conan Doyle.
• Digital Revolution:
  • 1970s–90s: Portable digital cameras and Photoshop make image manipulation ubiquitous.
  • Early internet communities form around fake celebrity nudes, largely via Usenet and niche websites. "Layer of Lux Luker" community cited as foundational.

Notable Quotes

"Basically, as soon as the camera is invented in the 19th century, people are faking their photos." — Walter Schreier (06:12)

---

3. The Rise of Photoshop Porn, Online Subcultures, and Questionable Ethics (09:36–13:58)

• Community Building: Early internet forums like Usenet host “celebrity fake” groups governed by loose ethical codes—prohibiting minors and commercial content, but with glaring contradictions.
• Artist Perspectives:

  "I probably spent every night participating in the newsgroup, working on fakes and posting them ... It completely consumed me." — TMFU (Greg Kelly, 12:15)

“Are these pictures ethical? ... A projection of personal fantasy, rightfully identified as such, should not cause us to lose any sleep. Except, of course, to masturbation.” — Lux Luker/Carrie Pearson (13:39)

---

4. Hollywood’s Influence and the “Uncanny Valley” (14:20–16:21)

• Film and CGI: Movies like "Jurassic Park" and “Fast & Furious 7” showcase cinematic fakery, making the public comfortable with digitally generated, nearly-real images.
• Painstaking Processes: Hollywood's digital wizardry is contrasted with the brute-force, community-driven efforts online.

---

5. GANs and the Scientific Breakthrough (18:06–21:35)

• Birth of GANs: In 2014, Ian Goodfellow invents Generative Adversarial Networks (GANs), a leap in AI image generation, in a drunken “Eureka” moment:

  "I went home and coded it up the same night. And it worked." — Ian Goodfellow (19:42)

• Intended Uses: Helping medical, scientific, and dental fields. Porn was not considered.

  “No, nobody was thinking about porn.” — Walter Schreier (21:38)

---

6. Open Source Gone Awry and the First Deepfakes Scandal (21:51–25:27)

• Open Source Release: Goodfellow releases GANs code on GitHub for researchers.
• Unintended Consequences: Hobbyists quickly adapt GANs for pornographic uses; "deepfakes" as a term and technology emerges in December 2017.
  • Sam Cole breaks the story for Motherboard/Vice:

    "AI-assisted Fake Porn is here and we're all fucked." — Sam Cole paraphrasing her 2017 headline (24:37)

• Community Reaction: Initial apathy or denial from the tech community.

---

7. The Focus on Political Deepfakes, and Neglect of Porn’s Harm (25:27–28:00)

• Media Panic: National security and political ramifications dominate the deepfake conversation—fake political ads and apocalyptic scenarios.
• Non-consensual Deepfake Porn:
  • Despite media focus on politics, 96% of deepfake videos online are pornographic and non-consensual (study cited at 28:00).
  • The term “deepfake porn” becomes the label for these violations—though Cole points out its limitations.

Notable Quotes

“For the people targeted by deepfake porn, it doesn’t matter if it’s delegitimized or not. The violation feels the same.” — Sam Cole (28:00)

---

8. Deepfakes’ Real-World Cost: Pain, Trauma, Powerlessness (31:34–35:30)

• Personal Toll: Cutie draws parallels between her experience and past sexual assault, highlighting re-traumatization and helplessness:

  “It was the same feel—it … makes that resurface. I genuinely didn’t realize it would.” — Cutie Cinderella (32:37)

• Ripple Effects: Content often resurfaces, is used as blackmail, and affects careers, reputations, and family relationships. For many, it overshadows accomplishments.

---

9. Accountability, Taboo, and (Partial) Aftermath (34:50–36:13)

• First Accountability Moment: For the first time, someone (Atriok) is not only caught consuming deepfake porn of acquaintances but held accountable. He issues public apologies, donates to legal takedowns, and continues streaming.
  • Cutie’s reaction:

    “Genuinely a shock. … He is my boyfriend’s best friend. … The fact that he was able to scroll past naked photos of me and have no … reaction … Bizarre.” — Cutie Cinderella (34:50)

• Persistent Issue: Most creators remain faceless; victims rarely know who generated or distributed the content.

---

10. Setting Up the Investigation: Finding Mr. Deepfakes (36:13–End)

• Anyone Can Be a Target: Sam Cole highlights how this technology endangers all women, not just celebrities:

  “There is no woman in the world who is safe from this technology.” — Sam Cole (37:12)

• The Hunt Begins: The rest of the season promises an international investigation to unmask "Mr. Deepfakes," the shadowy operator of the world’s largest deepfake porn site.

---

Notable Quotes & Memorable Moments with Timestamps

• “It is so convincingly my body, but not my body. And holy shit, it hits you like a truck. You feel so violated.”
  — Cutie Cinderella (04:11)

• “Because this is what pain looks like ... Fuck the constant exploitation and objectification of women. It's exhausting.”
  — Cutie Cinderella, livestream (29:47)

• "Basically, as soon as the camera is invented ... people are faking their photos."
  — Walter Schreier (06:12)

• "When I was arguing about generative models with my friends in a bar, something clicked ... and I went home and coded it up ... and it worked."
  — Ian Goodfellow (19:42)

• "For the people targeted by deepfake porn ... The violation feels the same."
  — Sam Cole (28:00)

• “I hope that in 10 years … my niece … doesn’t have to deal with something like this just for existing as a woman on the Internet.”
  — Cutie Cinderella (31:55)

• “There is no woman in the world who is safe from this technology.”
  — Sam Cole (37:12)

---

Important Segment Timestamps

• 00:49–05:00 — The incident and Cutie’s shock/trauma
• 06:12–13:58 — Early history of photo-fake culture, Photoshop, and online porn subcultures
• 18:06–21:35 — Ian Goodfellow invents GANs
• 21:51–25:27 — GANs’ open-source proliferation and the first AI-powered deepfakes
• 29:23–35:30 — Cutie's account of aftermath, harassment, and public anger
• 36:13–End — Forward-looking: promise of a global investigation into “Mr. Deepfakes”

---

Structure & Tone

The tone is empathetic but direct—combining personal testimonies, journalistic investigation, technical explanations, and cultural history. Cutie Cinderella’s emotional story is not sensationalist but instead underscores the human stakes of deepfake technology. Sam Cole’s authoritative, compassionate narration weaves together firsthand trauma, technical evolution, and the unresolved question of accountability.

---

Summary Takeaways

• Deepfake porn is the culmination of decades of technological progress and cultural permissiveness around image manipulation and online exploitation, with non-consensual images overwhelmingly targeting women.
• The technology driving deepfakes was not intended for exploitation, but rapid innovation and open-source ethos made its misuse inevitable.
• The personal impact on targets is devastating, with reputational, psychological, and social consequences that are difficult—if not impossible—to erase.
• Cutie Cinderella’s viral reckoning forced a broader conversation about deepfake porn’s real-world cost and briefly held a consumer accountable, but the real creators remain largely anonymous, with industry-scale profit and little oversight.
• The season promises a deep dive, not just into victims’ stories but also the power structure, economic engine, and legal holes that make the deepfake porn empire possible.

---

Transcript

[BetterHelp Ad Narrator]

BetterHelp online therapy bought this 30 second ad to remind you right now, wherever you are, to unclench your jaw, relax your shoulders, take a deep breath in and out. Feels better, right? That's 15 seconds of self care. Imagine what you could do with more. Visit betterhelp.com randompodcast for 10% off your first month of therapy. No pressure, just help. But for now, just relax.

[Brijesh Dave]

This is a CBC podcast.

[Sam Cole]

It's January 31, 2023, when a streamer who goes by Cutie Cinderella wakes up and something strange is happening. She's trending on Twitter, which even when you're Internet famous like Cutie is, is really never a good sign.

[Cutie Cinderella]

And I'm like, just waking up, trying to figure out what is going on. And then I get a call from one of the women involved that she's like, yo, have you seen everything? I'm like, no. And then she really broke it down for me and we went through. And then that's the first time I got on Twitter and I was like, oh, my God. Like.

[Sam Cole]

Cutie Cinderella is a Twitch streamer.

[Cutie Cinderella]

We're doing something fun today. I think it's genuinely been a few years since we have done an Ask me anything about me. Cutie Cinderella. Hi, that's me. Hi, guys. So we're gonna do that today.

[Sam Cole]

Streaming is pretty simple. You broadcast yourself live, usually playing video games to thousands of people watching in real time.

[Walter Schreier]

Oh, boy.

[Cutie Cinderella]

Oh, sorry.

[Sam Cole]

Cutie got her start playing League of Legends. It's one of the biggest video games on the planet. A kind of online gladiator arena where teams battle it out.

[Cutie Cinderella]

Yeah, you're dead.

[Sam Cole]

Oh, my Lord.

[Walter Schreier]

He be camping.

[Sam Cole]

From there, she became known for bringing people together, co hosting podcasts, arranging meetups in real life.

[Walter Schreier]

Say hi to chat, guys.

[Cutie Cinderella]

Happy girls, trip.

[Sam Cole]

Cutie Cinderella is used to being online and she's used to people talking about her online. But she wasn't prepared for what happened that January morning in 2023. It all started the day before with a guy who goes by Atriok. Atrioc is in QT's circle. They've streamed together. So he's a colleague, a friend really. And he was live on Twitch when he wanted to check a different window.

[Lux Luker (Carrie Pearson)]

What time is it?

[BetterHelp Ad Narrator]

Where I am?

[Lux Luker (Carrie Pearson)]

What time is it? Pst.

[Sam Cole]

That's the moment that upended Cutie's life. Atriok hits alt tab and for barely a second, every window he has open flashes on screen to the thousands of people watching. Among them, a YouTube video of lovely Day by Bill Withers. A Google search for Jennifer Garner in Catch Me if youf can, his recent Uber Eats orders, and a website selling porn. Not just any porn, but porn of women he knows, women he works with, including Cutie Cinderella. Someone takes a screenshot and posts it online.

[Cutie Cinderella]

And then it just created a wildfire.

[Sam Cole]

The pictures ripple across Discord, Reddit, Twitter, and by the next morning, Cutie's phone is blowing up.

[Cutie Cinderella]

I was already getting DMed to the photos or reply to my tweets and stuff like that before I even had a full grasp of what the hell is going on.

[Sam Cole]

A bunch of the messages have links, and when she clicks through, she finds a porn site she's never seen before, never even heard of. And she's on it, her face, doing things she never did. Because here's the thing, it wasn't Cutie, not really the porn. It was a deep fake.

[Cutie Cinderella]

It is so convincingly my body, but not my body. And holy shit, it hits you like a truck. You feel so violated.

[Sam Cole]

Deepfake videos, which are manipulated using AI, can make someone appear as though they're saying or doing something that they're not. Tech experts are sounding alarm bells over the rapid spread of AI generated explicit images of women online.

[Ian Goodfellow]

We're moving into a future where you.

[Walter Schreier]

Really won't know what's real.

[TMFU (Greg Kelly)]

Online, we've seen a significant increase in deep fakes, and as our research has shown, 96% of these are pornographic.

[Sam Cole]

I'm Sam Cole. I'm a tech journalist at 404 Media where I write about sex and the Internet. And I've been reporting on deepfake porn since the very beginning. And here's what I've learned. Deepfakes didn't just come out of nowhere. They were built by people on platforms inside subcultures. They were allowed to spread while governments dragged their feet and tech companies shrugged. And at every step, someone profits, while the targets, almost always women, pay the price. So how did we get here? And if you follow the trail of deepfake porn all the way to the source, who does it lead to? This is understood. Deepfake Porn Empire Episode 1 the dawn of Fake Porn. Deepfakes, you've seen them everywhere. AI videos of politicians, celebrities, random people on TikTok. Sometimes so seamless that you might not even realize you're looking at a fake. And we have this idea that it's a fast moving new threat. But fake images, yeah, they're not exactly new.

[Walter Schreier]

Basically, as soon as the camera is Invented in the 19th century, people are faking their photos.

[Sam Cole]

This is Walter Schreier. He's a professor of engineering at Notre Dame, where he researches AI the tech, but also the cultural history. And when it comes to fake things online, he wrote the book I am.

[Walter Schreier]

The author of a History of fake things on the Internet. I think a lot of folks today believe film photography was rather cut and dry, right? It's like you took the photo and it captured an objective picture of reality, but that was never really the case.

[Sam Cole]

In fact, the very first fake photo showed up in 1840, made by an early camera inventor.

[Walter Schreier]

This inventor, Hippolyte Bayard, is in France, and he's really, really annoyed that his camera process is not receiving the attention he thinks it should.

[Sam Cole]

So he comes up with a publicity stunt.

[Walter Schreier]

He fakes his own death in a photograph, writes a little story on the back of this photo, right, Protesting all this, explaining why he died. But again, it was all this sort of hysterical exaggeration, the kind of thing we associate with the Internet today.

[Sam Cole]

From there, fakes only multiplied. One of the most famous came decades later in 1917.

[Walter Schreier]

So you have two cousins in the English countryside that have access to a camera, and they produced a remarkable series of photographs where they appeared to be, you know, in different natural settings. The woods with a bunch of dancing fairies.

[Sam Cole]

Yeah, like little winged tinkerbells.

[Walter Schreier]

Now, the fairies generated this enormous debate. Notable figures get involved in this debate, most prominently Sir Arthur Conan Doyle, the author of the Sherlock Holmes series.

[Sam Cole]

And when the creator of the most famous detective in literature saw the photos, he was fully convinced they were real.

[Walter Schreier]

He absolutely defended these photos till he died. Now, flash forward. Several decades later, one of the cousins steps forward and says, you know what? Those were just cardboard cutouts. They were fake.

[Sam Cole]

The incident became known as the cottingley fairies. And a century later, we're still at it, still faking images. Only our tools got a serious upgrade in the 1970s. Digital cameras entered the scene.

[Walter Schreier]

Flash forward. Then just a couple more years, you get the appearance of Photoshop, a piece of software that has stood the test of time.

[Sam Cole]

In 1990, Adobe was best known for making printer software and tools for font nerds. Photoshop put them on the map and got them on the news.

[Unknown Commentator]

Technology makes it difficult, maybe even impossible, to tell what's real and what's not. Recently, the issue of altering photographs came up when Playboy added its logo to a picture of Roseanne Arquette without her knowledge or her approval. And then there was a huge publicity outcry when TV Guide tacked Oprah Winfrey's head onto Ann Margret's body.

[Sam Cole]

Suddenly the kind of image editing power once reserved for million dollar studios was available to every graphic designer and everybody else.

[Walter Schreier]

If you look at the actual examples of how people were using the earlier tools, right, like Photoshop in very bad contexts, it was almost always pornography and.

[Sam Cole]

We were able to track down a guy who was making Photoshop porn in this era.

[TMFU (Greg Kelly)]

I actually started playing around with this stuff at home, strictly privately. And yes, that obviously already involved nudity. It was very clumsy and even limited to grayscale images. But at the time it was cutting edge and easier to suspend disbelief than taping a cut out physical picture of a head over a Playboy centerfold.

[Sam Cole]

I guess this is tmfu, that's his username and he has a digital footprint in the fake porn scene going back decades. TMFU didn't want to be interviewed and he wouldn't share his real name. But when we emailed him, he responded with a multi thousand word manifesto. Those emails are what my colleague is reading from and the story goes right back.

[TMFU (Greg Kelly)]

It got all started when Lux Luker, a Canadian guy, libertarian type, created a group dedicated to celebrity nude faking in 1996.

[Sam Cole]

Lux Luker's real name was Carrie Pearson and he ran two Usenet groups dedicated to fake nudes of celebrities. Quick refresher. Usenet was basically the early Internet's version of Reddit. Think of it like a message board, but run through email.

[TMFU (Greg Kelly)]

I'd say there were probably a good hundred regular people from all over the world sharing their Photoshops. This was at the time of Britney Spears, Baywatch Girls and every American sitcom girl you can imagine.

[Sam Cole]

Lux Luker also ran a website which is still archived all these decades later. Www.layerofluxluker.com Lux's website is peak 90s Internet. A black background decorated with neon naked lady silhouettes, half a dozen different shades of neon text and Comic Sans everywhere. All of it dedicated to fake celebrity porn and the community churning it out. My sound designer is going to read from it.

[Lux Luker (Carrie Pearson)]

We all love seeing the heads of some of our favorite stars and other prominent people placed on top of a nude model. The sheer godlike power of exposing them to the world for our own fantasies is a healthy and thanks to the easy availability of advanced image manipulation programs, relatively easy task to undertake.

[Sam Cole]

When TMFU found Lux Luker's website, he'd found his people.

[TMFU (Greg Kelly)]

When you can share a particular interest like that, especially a naughty one, with other people who are also fascinated by it, it gets an entirely different dimension. I probably Spent every night participating in the newsgroup, working on fakes and posting them, learning, commenting on fakes by others, exchanging tips. I probably finished and posted 150 fakes in the first year. It completely consumed me.

[Sam Cole]

The celebrity fakes groups did have rules about who they could fake and how. You can still read these on Lux's website.

[Lux Luker (Carrie Pearson)]

No real nudes of celebrities. These have plenty of their own groups. No advertisements for commercial porn, photo sites, also known as spam. No fakes of bestiality, and no fakes of minors. Artists will not post or take requests for celebs under 18.

[Sam Cole]

If that feels somewhat reassuring, it doesn't last. Right below that warning, there's a chart listing underage celebrities, all teenage girls with the years they would turn 18. Lux died in 2004 from complications related to diabetes. This means that we can't ask him how he'd look back on the ethics of all of this now. But he had a statement about it on his website.

[Lux Luker (Carrie Pearson)]

Are these pictures ethical? My personal opinion is that it would be unethical to present them as being a real representation of the person or the activities depicted. A projection of personal fantasy, rightfully identified as such, should not cause us to lose any sleep. Except, of course, to masturbation.

[Sam Cole]

In 2003, the year before he died, Lux Luker told Wired that he believed there were about 300,000 celebrity porn fakes out there. And even once he was gone, the community kept growing. Meanwhile, everyone else was discovering digital fakery with a little help from Hollywood.

[TMFU (Greg Kelly)]

Boy, do I hate being right all the time.

[Walter Schreier]

Yeah, I remember seeing Jurassic park when it first came out in the theater and there was this game to play. You know, what scene was shot with a puppet versus what scene was computer graphics. Some of the T. Rex shots, right, when the kids are in the jeep.

[TMFU (Greg Kelly)]

Keep absolutely still suspicions based on movement.

[Walter Schreier]

You know, is like the head, the puppet, right? Was it the body? That's cg. You know, there were a lot of questions, and when you have that kind of ambiguity, you're doing a really good job with the computer graphics.

[Sam Cole]

Jurassic park wasn't the first film to use CGI. But in 1993, the film proved computer generated creatures could look real or real enough. And that opened the door to another holy grail. Digital actors.

[Walter Schreier]

Hey, thought you could leave without saying goodbye.

[Sam Cole]

The most famous early example happened in 2013, when Paul Walker died midway through filming Fast and Furious 7. The studio needed a way to finish the movie, so they got Paul's brothers and CGI'd his face over theirs what do you think?

[Walter Schreier]

Parking brake slide right up to the school.

[Sam Cole]

Philosophically, it kinda really creeped people out, but visually, it worked.

[Walter Schreier]

This is where you start to get these debates around, like, what? What is real? Like, what do you trust?

[Ian Goodfellow]

Right?

[Walter Schreier]

And then you really do start to wonder, you know, like, where could this veer off course?

[Sam Cole]

But there were limitations. The dinosaurs, Paul Walker's face. This was labor intensive. It was all hand animated by teams of people, frame by frame.

[Walter Schreier]

So that's when you start to see researchers in this space discussing the possibility of automating this process right again, turning to computer vision and moving away from the sort of very manual process of computer graphics.

[Sam Cole]

And that's where this story tips into something new. You've got Hollywood's painstaking process on one side, testing the public's tolerance for fakes, fake celebrity porn, photoshoppers on the other. And soon, a brand new technology about to smash through both.

[BetterHelp Ad Narrator]

BetterHelp online therapy bought this 30 second ad to remind you right now, wherever you are, to unclench your jaw, relax your shoulders, take a deep breath in and out. Feels better, right? That's 15 seconds of self care. Imagine what you could do with more. Visit betterhelp.com randompodcast for 10% off your first month of therapy. No pressure, just help. But for now, just relax.

[Brijesh Dave]

The news doesn't wait, so why should you? I'm Brijesh Dave, host of Big Headlines, your new daily news headline podcast built for busy Canadians from the team behind the Big Story. Every weekday at 11am Eastern, we'll break down the biggest stories in Canada and around the world quickly. Clearly just your snapshot into the headlines that matter to Canadians in Eastern. Easy to digest bites, perfect for your commute, your lunch break, or that five minute reset between meetings. Listen to Big Headlines now on your favorite podcast player.

[Sam Cole]

The Future arrived in 2014 at a Montreal microbrewery where a grad student named Ian Goodfellow was getting drunk.

[Ian Goodfellow]

I don't want to be someone who goes around promoting alcohol for the purposes of science, but in this case, I do actually think that drinking helped a little bit.

[Sam Cole]

That's him talking on the Lex Friedman podcast today. Ian Goodfellow is an AI researcher at Google DeepMind. But at the time, he was a PhD student at the University of Montreal. He was basically done. He'd just handed in his thesis. And that night out drinking with friends, they kept circling the same question about something they were all studying. Generative models. Here's Walter Schreier again, author of A History of Fake Things on The Internet.

[Walter Schreier]

All a generative model is, is an algorithm that generates data that is new.

[Sam Cole]

At the time, there were a few generative models that produced images, but the results looked wrong. Blurry, distorted half apples, misshapen faces. What researchers wanted was a system where you could dump in data and it would invent totally new images, ones that looked real. At the time, Walter Schreier was also a PhD. He was at Stanford working in a similar field. And he remembers this being the big goal.

[Walter Schreier]

Yeah, I think all the researchers wanted was useful output, like photorealistic output. That's all they were looking for.

[Sam Cole]

So this was the problem. Ian and his friends were debating that night how to build a program that could do this.

[Ian Goodfellow]

And then finally, when I was arguing about generative models with my friends in a bar, something clicked into place. And I started telling them, you need to do this, this and this, and I swear it'll work. And my friends didn't believe me that it would work, but I believed strongly enough that it would work that I went home and coded it up the same night. And it worked.

[Sam Cole]

It worked. Ian invented a system that could produce new photorealistic images. He called his invention Generative adversarial networks, or GANs for short. And the breakthrough was two systems playing a game. One makes fakes, the other tries to spot them.

[Ian Goodfellow]

You can think of it as kind of like an art critic at the start. The generator makes images that aren't real at all, and the discriminator doesn't know what's real or fake. But when they play the game against each other, where they have to try to fool each other all the time, eventually the generator learns to make very realistic images.

[Sam Cole]

Within months, Ian Goodfellow published a paper.

[Walter Schreier]

I remember specifically my lab mate showing me this paper. He's like, look what Goodfellow just released. It was like, what? It was like, this is incredible. I mean, it was basically a turning point for the entire field.

[Sam Cole]

When Ian cracked this technology, he had big dreams for what his invention might unlock.

[Ian Goodfellow]

I thought about a lot of important problems in science that I could solve that would help people.

[Sam Cole]

He was excited about how GANs could be used to help design medicine, how they could fix bias in data. Like, if there was an underrepresented category, a GAN could create realistic examples to help fill in the gaps. And then there was what they could do for teeth.

[Ian Goodfellow]

Specially trained technicians spend about two weeks to make each patient's dental crown. Now, with generative adversarial networks, it's possible to design them basically instantly, and then 3D print them.

[Sam Cole]

But no one was really thinking about porn.

[Walter Schreier]

No, nobody was thinking about porn.

[Sam Cole]

Here's the thing about science. Researchers love to share, make their findings reproducible. That's how the field moves forward. So after his paper was published, Ian Goodfellow quickly made the source code for gans available on GitHub, the public code sharing platform.

[Walter Schreier]

So as the technology just got more accessible, you see amateurs on the Internet taking this code and doing things researchers never imagined would happen because they expected only other researchers to use the open source code. Surprise, surprise, that did not happen. And so you see the appearance of like, the original deepfake algorithm. You know, it's like a pseudonym on Reddit saying, I have this new porn generating, you know, AI system. Check it out. And, you know, it spirals from there.

[Sam Cole]

I remember that spiral very clearly because I was the journalist who broke that story. It was December of 2017, and I was a reporter at Motherboard, that's Vice's tech outlet. And my editor found a post in a subreddit called CelebFakes where people were doing the old school Photoshop thing. The post he sent me was by a user named deepfakes. This was a play on deep as in deep learning, like Gans, and fake as in celebrity fakes. This one guy would go on to inadvertently name the whole phenomenon, but at the time, he was just some rando. Deepfakes had posted a video. It was of actor Gal Gadot, you know, Wonder Woman, lying on a bed, waving a sex toy around. I clicked, and the page opened to what looked like a porn video of Godot, only it wasn't quite her. It was her face stitched onto a porn performer's body. I research porn and the Internet for a living, and I'd never seen anything like it. At that point, deepfakes had posted porn videos featuring the faces of Scarlett Johansson, Maisie Williams, Taylor Swift, Aubrey Plaza, and now Gal Gadot. On Reddit. The tech wasn't seamless. Gal's face glitched, and every now and then you could see the real actor's face under hers. But I remember thinking, this is going to get so much worse. So I sent him this. Do you think this technology could be used with bad motivations in the future? And he got back to me.

[TMFU (Greg Kelly)]

Every technology can be used with bad motivation. I don't think this is any different than recreating Paul Walker in Fast and Furious.

[Sam Cole]

I wrote it up for motherboard and in 2017 became the first journalist to publish a story on AI porn. My headline was AI assisted. Fake porn is here and we're all fucked. When my story came out, I'm not sure what I expected to happen. Outrage, maybe, and there was some of that, but mostly what I got were shrugs.

[Walter Schreier]

Within the computer vision community, Deepfakes comes out. No one really wants to comment on that. It makes the field look bad, obviously, to sort of just like, you know, it's not our problem.

[Sam Cole]

Right.

[Walter Schreier]

We didn't. We didn't create it. We're not. We're not the deepfakes pseudonym on Reddit, you know, and that. That's just not. That was not a good response. And again, I think also the hype about the political stuff, right? Like the. The field was happy to continue to talk about that.

[Sam Cole]

People were paying attention to deepfakes, just not porn. These manipulated images can pose a very real national security threat.

[Walter Schreier]

Fake videos could. Could become a real and present danger to our democracy. The presidential election season is ramping up.

[Sam Cole]

And so are the warnings about deepfake.

[TMFU (Greg Kelly)]

Technology being used to disrupt the campaign.

[Sam Cole]

In 2018, BuzzFeed and Jordan Peele made a fake video of Barack Obama. Obama looked straight into the camera and.

[TMFU (Greg Kelly)]

Called Donald Trump the total and complete dipshit.

[Sam Cole]

Of course, none of it was real. Jordan Peele provided the voice. The AI filled in the lips. It was a warning. This is what's coming. The warning landed. Newsrooms ran headlines about the end of trust. Congressional hearings predicted fake candidates and fake wars. Darpa, the Pentagon's research arm, launched a massive media forensics program. Tens of millions of dollars were poured into building deepfake detectors. Algorithms that could spot tiny lighting inconsistencies, missing frames, spliced pixels. During Trump's first term, the panic went nuclear. Analysts went on cable news to freak out about a hypothetical apocalypse triggered by a hypothetical deepfake of Trump saying he'd launched nukes at North Korea, which is.

[Walter Schreier]

Like the most implausible scenario in foreign politics, if you ask me. The hardest thing to do is like, launch a nuclear missile, right? Like an anonymous account online, right? Like saying things is not going to, like, trigger World War III.

[Sam Cole]

Of course, political deepfakes do happen. There's a notorious YouTube ad featuring a deepfake of Canadian Prime Minister Mark Carney trying to lure people into a shady investment scheme.

[Ian Goodfellow]

It's a special investment system backed by the government and designed for the average.

[Sam Cole]

Citizen to earn Money. And in 2022, a deepfake of Ukrainian President Vladimir Volodymyr Zelensky surrendering to Putin went viral. But nobody in the trenches threw down their weapons and ran off the battlefield. Because political fakes are very quickly delegitimized.

[Walter Schreier]

Some people are genuinely fooled. Right. Which is. Which is a problem. Right. But for the most part, again, I don't think that's quite the case with a lot of this material. They're almost always released on like an anonymous account or through a pseudonymous.

[Brijesh Dave]

Right.

[Walter Schreier]

It's not a major media outlet reporting it as fact. You know, if it were real, we would have heard about it through a legitimate source.

[Sam Cole]

But for the people targeted by deepfake porn, it doesn't matter if it's delegitimized or not. It doesn't matter that it's not technically real, whatever that means. The violation feels the same. Which brings me to something else. Calling non consensual deepfakes porn, even when they're technically pornographic, is not quite right. It's like how there's no such thing as abusive porn or non consensual porn. That's sexual assault or rape. But the phrase non consensual AI generated sexually explicit images is a mouthful. So deepfake porn, for better or worse, is the name that stuck. And the scope is massive. Back in 2018, researchers at the cybersecurity firm DeepTrace found that 96% of all deepfake videos online were pornographic and non consensual. Five years later, a 2023 study by Sensiti, an identity verification company, had the same findings.

[Walter Schreier]

And yet it takes like several years after the appearance of deepfakes till we're really talking about the problem of porn, right?

[Sam Cole]

And the moment that really happened. 2023 with Cutie Cinderella.

[Cutie Cinderella (Live Stream Excerpt)]

This is probably the stupidest thing I've ever done.

[Sam Cole]

That morning, after finding out that she'd been deep faked and that the videos were all over the Internet, Cutie Cinderella had to decide what to do. For her, the answer was almost inevitable. She's a live streamer, so she livestreams.

[Cutie Cinderella (Live Stream Excerpt)]

And I'm sure everyone in the world would tell me not to go live right now, but I want to go live. Because this is what pain looks like. This is what it looks like. Fuck the fucking Internet. Fuck the constant exploitation and objectification of women. It's exhausting. It's exhausting. Fuck Atriok for showing it to thousands of people. Fuck the people dming me pictures of myself from that website. Fuck you all. And I think you guys need to know what pain looks like, because this is it. This is what it looks like to feel violated. This is what it looks like to Feel taken advantage of. This is what it looks like to see yourself naked against your well being. Spread all over the Internet.

[Sam Cole]

And Cutie's pain, her anger, it goes viral. The streaming world was shaken to its core recently.

[Brijesh Dave]

Her name is Cutie Cinderella.

[TMFU (Greg Kelly)]

With a heartbreaking and infuriating video that she posted.

[Walter Schreier]

AI Deep faked porn of his streaming co workers.

[Sam Cole]

Cutie's stream and the story about what happened, it's written about in Wired, the Washington Post, Business Insider, Entertainment Tonight. The list goes on.

[Walter Schreier]

But the ramifications from this are going to last months much longer.

[Sam Cole]

Hello? Oh, hi. Can you hear me? And shortly after that, she talked to me.

[Cutie Cinderella]

I know a lot of the women a part of this are refusing to do interviews and it's because they don't want this a part of their narrative. I don't want this part of my narrative either. But it's the only option I have to hopefully do something about in the future. I hope that in 10 years when my nieces more on the Internet, she doesn't have to deal with something like this just for existing as a woman on the Internet.

[Sam Cole]

For Cutie, the violation of being deepfaked wasn't the end of the invasion.

[Cutie Cinderella]

The harassment has been relentless. You go to my YouTube cards, you search my name, you do anything. It's. It's just there and it's like, it sucks. Something that's really important to me, is it to be known that like, I am not opposed to sex work. I just don't want to be a sex worker. The problems could send. And it's just miserable that people could take whatever they want from you and turn it into whatever they want.

[Sam Cole]

Beyond the harassment, the videos and the comments everyone could see, Cutie told me about all the things that happened behind the scenes. The dominoes, the deep fakes knocked over, the things they dredged up.

[Cutie Cinderella]

I said this in one interview before, but I was, I was sexually assaulted as a child. And it was the same feel it like where you feel guilty, you feel dirty, you feel what just happened. And it's bizarre that it, it makes that resurface. I genuinely didn't realize it would. Yeah. When I first heard about it, I think I was born a whirlwind. I didn't get hit with it psychologically until I saw the photos. I. I think I potentially could have been someone on the Internet that was lacking empathy before it did happen to me. Yeah. Yeah.

[Sam Cole]

It's so scary how like, you know, even if, like years and years later and you, you're like, I'm healing. And then something like that can just be like right back there.

[Cutie Cinderella]

It did another layer to it. My family's seen went so viral that my family saw it. So people went out of their way to send it, potentially. Some of the girls, the photos were used as blackmail to try to get stuff out of that. It's just like it's been sent. So gross. And none of this is stuff that we've created. These aren't even photos of ourselves. But I'll tell you what, my 65 year old dad, it'd be a hard time explaining to him that that's not real.

[Sam Cole]

Have you had to like explain it to your parents?

[Cutie Cinderella]

I've literally avoided calling my dad my. I've had conversations with a few of my cousins, one of my aunts as well as my sister and you know, they all get it but they also think that the Internet's evil and you know, they. It's so sad because of all the good things I've done in this career. I've done so much. I've raised tons of money for charity. I've made community events that tried to highlight people that maybe haven't had the opportunity to be highlighted. I've done so much. But this is what my family now knows as my job.

[Sam Cole]

For many people, news coverage of Cutie Cinderella's story was their first time hearing about Deepfake porn, or if they had heard of it before, this was their first time coming face to face with the real emotional impact it had on the people who were used in was also one of the first times someone, in this case Atriok, the streamer who exposed the porn site was caught and.

[Cutie Cinderella]

Called out for watching was a shock, genuinely a shock. To see a face behind someone that paid for this is someone that saw this. And so I think the taboo of that is one reason that it got so large. I think also there's even more taboo because he is my boyfriend's best friend. My boyfriend was in his wedding party and so the fact that he was able to scroll past naked photos of me and have no emotional or visceral reaction to exit out of the page. Right. Bizarre.

[Sam Cole]

The fallout came quickly. Atriox streamed a tearful apology, his wife in the background, also tearful. Later, he donated $60,000 to a law firm to help with takedowns and legal support for the streamers who'd been deep faked. Cutie filed takedown notices. Some of the content came down, not all of it. The site Atriok had been looking at pulled the videos offline and posted an apology today. Cutie and Atriok still stream together. By all appearances, they've moved past it. There's a cynical way to look at this. He got wrapped on the knuckles and she had to get over it to keep her career in a space they shared. But there's also a hopeful one that people can apologize and come back from this. But here's the thing. Yes, Cutie saw who was watching deepfakes of her, but she never found out who made them. For years, fake porn has been thought of as a celebrity curiosity, a weird fringe corner of the Internet. Pop stars, movie stars, streamers. As if that was just the price of fame. But anyone can make a deepfake of anyone. Which means the person uploading it could be your neighbor, your co worker, your best friend. And the person in the video could be you.

[Cutie Cinderella]

We were still living together, so we.

[Sam Cole]

Still had to share content. We still had to share a bathroom. I was terrified.

[Brijesh Dave]

The reality is, it's very easy to anonymize yourself online.

[Sam Cole]

There is no woman in the world who is safe from this technology. I'm not just gonna, like, sit here and take it. This season of Understood, we are burrowing into the digital world of non consensual deepfake porn. The people impacted, the political battles and legal loopholes. And we'll follow the trail, because behind the people watching and making deepfake porn, there's someone else. A spider at the center of a web. A kingpin. An investigation that begins in Denmark will ignite a chase across three countries and four newsrooms as a CBC journalist closes in on the very real person behind the world's largest fake porn website, Mr. Deepfakes himself. Mr. Deepfakes. Mr. Deepfakes.

[Walter Schreier]

Mister Deepfakes. All the man behind Mr. Deepfakes. Mr.— Deepfakes is the most notorious non consensual deepfake porn site in the world.

[Sam Cole]

He has been using women's faces for.

[Cutie Cinderella]

Years, exploiting them, earning money on them. Now it's his turn to be on the front page.

[Sam Cole]

In this episode, you heard archival tape from CBC and Cutie, Cinderella, Ludwig React, Mintberry Crunch, the Kim Commando Show, Leduc Leadership, the Today show from NBCUniversal, Jurassic park from Universal Pictures and Amblin Entertainment Furious 7 from Universal Pictures. Lex Friedman, podcast Preserve Knowledge, the AI Podcast, Rework association for Computing Machinery, Fox News, Buzzfeed, NPR, Dextero, penguins and another body. My AI nightmare. Deepfake porn empire is written and produced by our showrunner, Acey Rowe, and me, your host, Sam Cole. Arman Agbali is the producer. Sound design by Julian Uzieli, who also voiced Carrie Pearson, AKA Lux Luker. TMFU was voiced by Greg Kelly. Our story editor is Veronica Simmons and our Executive producer is Nick McKay. Blokos if you enjoyed this episode, make sure you check out previous seasons of Understood, like season two, the pornhub Empire, which I also hosted. I'll take you inside the rise and reckoning of one of the biggest porn platforms in the world. You'll find that wherever you found this podcast, so go check it out and make sure you hit follow. Then meet me back here for the next episode of Deepfake Porn Empire.

[Brijesh Dave]

For more cbc podcasts, go to cbc ca podcasts.