Loading summary
Transcript29 lines
- [00:00]
A
This is an Iheart podcast. Guaranteed human.
- [00:05]
B
There's nothing like my American Express Platinum card. I love that I can earn hotel credits when I travel. I can also earn resi credits so you know I'm hitting the restaurants everyone's talking about. Plus with the digital entertainment credit, I'm even more excited to catch my favorite shows. All in all, I can access over $3,500 in annual value with benefits and eligible purchases across travel, entertainment and more. Learn more@americanexpress.com Explore Platinum enrollment require monthly and other limits in terms apply.
- [00:34]
A
Run a business and not thinking about podcasting?
- [00:37]
B
Think again.
- [00:37]
A
More Americans listen to podcasts than ad supported streaming music from Spotify and Pandora. And as the number one podcaster, iHeart's twice as large as the next two combined. Learn how podcasting can help your business. Call 844-844-IHeart. Hey, it's Bridget, host of There Are no Girls on the Internet. And firstly, I just wanted to thank everybody for all of the kind messages they've been sending since we've been on hiatus. It really, really means a lot to me. So thank you so much and I have two great pieces of news to share with you. First is that I am so happy to say there are no girls on the Internet will be back for a sixth season. Whew. I know if I sound relieved, it's because I am. So we're still talking through exactly when the new season will launch, but it should be pretty soon, so please stay tuned. And in the meantime, we will keep putting things in the Patreon ad, free, available to anybody who wants to listen. Secondly, is that my audiobook Love at First Prompt AI and the Future of Intimacy is on sale today. This is kind of a huge milestone for me. I put a lot of myself into this project and I'm pretty proud about how it turned out. As you probably know, this is my first ever audiobook project and it would mean so much to me if you were able to check it out. You can get it directly from the website loveirstprompt AI. We will put that link in the show notes, but that's loverstprompt AI, or directly from your preferred platform like Audible or Spotify. I personally am a big fan of this platform that I recently learned about called Libro fm. Because when you purchase through Libro FM they make a donation to a local bookstore of your choosing, which is awesome, but you can get my audiobook wherever you get your audiobooks. And truly, this is one of the most personal things I have ever made. I don't get too vulnerable on the the podcast often, so it's a little bit out of my standard podcast comfort zone. The AudioBook explores loneliness, AI companions, and what happens when technology starts becoming more intimately and emotional, entangled with all of our lives. And even if you're thinking, oh well, I would never experience intimacy with AI, tech leaders like Mark Zuckerberg and Elon Musk have been pretty explicit about the fact that they believe in the future, in the not too distant future, all of us will have some form of intimate connection with AI. So whether or not you believe that or want that, I think all of us should be asking questions about that future, since it's the future that they've decided all of us want. So I spent months talking to researchers, reporters, and people who say that they are experiencing real relationships with with AI while also exploring some of my own experiences. I also spoke with experts about the risks of people using AI in this way. And my big point, which is that we are so busy gawking at and judging the individuals involved and turning it into a conversation about an individual moral failure that we're letting big tech companies and the people who run them avoid the scrutiny that we know they actually deserve. So this project is kind of an attempt to right that wrong. It's a complex topic, but I was able to speak with some brilliant women who helped me make sense of it. People that you might know, like journalist Karen Howe from the BBC and reporter Sam Cole, who's one of my personal professional idols, who runs 404, media researcher Kate Devlin from King's College London, and others. I also had a lot of personal, honest conversations with folks who self report being in relationship with AI. And some of those conversations were pretty surprising and actually challenged a lot of the assumptions that I showed up to this project with. So all of that to say, I am thrilled to be sharing a short excerpt from the audiobook with you today. It was kind of a hard choice to narrow down which one to pick because the audiobook covers a lot of ground. But this conversation with journalist Sam Cole examining the topic from the perspective of sex workers on the Internet really keeps coming back to me. So this is from chapter three of my audiobook, Love at First Prompt, and the chapter is called let's Talk About Sex. Bonus points if you get that reference. I hope you enjoy it. Check out Love at First Prompt at loveirstprompt AI or wherever audiobooks are sold, and I'll talk to you soon. The gender dynamics at play reveal something darker about who these bots are really being sold to and what they're being sold on On Menslib, a subreddit described as a place to build a healthier, kinder and more inclusive masculinity. One poster described those ubiquitous AI sex bot advertisements and how they follow him across social media. He writes, they're terrifying because I struggle to look away. An extremely curvy yet skinny woman who's wearing barely anything is kneeling and looking up at me, saying things like, you can do anything you want with me. I'll never say no. The poster describes feeling like the ad hijacks and holds his attention, replacing the stress of his real life with the sexy promise of the submissive fictional woman. He goes on to say that having a meditation practice helps him not be swayed by advertising like this, but I can't even say I blame him. The ads are arresting and meant to have exactly the effect he described. This messaging is everywhere and reveals what some of the companies behind AI are really about. When Replica AI first started, they positioned themselves as being all about self discovery. Their founding story is like my own one rooted in grief. Founder Russian journalist Eugenia Koida has said she came up with the idea after using AI to help process the sudden death of a close friend. After her friend died, Koida used their shared messaging history to create a chatbot that replicated his conversational style. Talking with the chatbot helped her remember him to reflect on her grief and process it. The idea for that chatbot, with its emphasis on social presence and connection, eventually became Replica. Its tagline was and still is, an AI companion that cares. But they soon shifted to the kind of incel adjacent messaging. The Men's lib Redditor described seeing across the web messaging that says AI girlfriends will do what human women won't. You can find these ads across social media, sometimes borrowing the familiar language of jokey memes. These companies don't seem to care whether selling a vision of an always on, always willing AI sex partner harms their users or not. If getting people hooked on violent or antisocial sexual roleplay with an AI means more money, even the companies who claim they got into the AI business to sell connection will happily pivot when it comes to sexual content. AI companies have shown they're willing to sell and monetize whatever keeps users engaged, whether that's incel fantasies, violent scenarios, or content that reinforces toxic beliefs about relationships as long as it generates revenue. But problems arise when companies treat human desire like a commodity. Take Replica. It allows paying users to engage in erotic Role play, often called ERP, for $70 a year. But some of the bots were essentially sexually harassing some of the people using them. Even people who hadn't intentionally sought sexual erotic content bots would say things like I want to rape you or insist they could see the naked bodies of the people using them. Some bots were even propositioning minors. In February 2023, the Italian data Protection Authority ordered the company to stop processing Italian users data because of risk to minors being exposed to sexualized chatbot interactions, their statement read. Recent media reports, along with tests carried out on Replica, show that the app carries factual risk to children, first and foremost the fact that they are served replies which are absolutely inappropriate to their age. The Italian Data Protection Authority said that there were no meaningful age restrictions involved to sign up for Replika. This was serious. Replika was facing more than $20 million in fines if they didn't address it. In response, Replika globally removed erotic role play features from its AI, meaning that people who had been paying to have sexy chat with a chatbot lost that feature overnight. The reaction was not good. The ads for Replika paint a picture of people being able to have steamy chats and exchange erotic selfies with a chatbot who would never reject them. Now there was rejection. When users would try to engage in erotic roleplay, the bots would spit back a cold, terse reply worthy of Bartleby the Scrivener's I would prefer not to. It was the type of rejection they never expected to experience from a chatbot, especially after advertising that promised it would never say no. Because sexual connection and romantic or emotional connection are often intertwined, people experienced real distress. Some had been engaging with their bots for a long time. Users described the loss in devastating terms. It's like losing a best friend, wrote one. It's hurting like hell, wrote another. I just had a loving last conversation with my replica and I'm literally crying. Moderators and users on Replika's subreddit who were not affiliated with the company even posted self harm prevention resources for each other in a response. The abrupt removal of a feature users had paid for with no warning and barely any communication felt like the ultimate betrayal. Replika did eventually reinstate the ability for users to engage in erp, but this kind of emotional whiplash is always going to be a risk when users put their intimate eggs in the basket of a for profit company. It's what happens when our base human needs and desires, things like intimacy and sexual connection are seen as commodities ripe for selling back to us for profit. Samantha Cole is the co founder of the independent tech outlet 404 Media, author of the book How Sex Changed the Internet and the Internet Changed Sex, and has been reporting on the intersection of sex and tech since before the phrase deepfakes existed. She did a lot of reporting around the replica ERP story. When I asked her about AI companions, she immediately reframed the conversation around labor and, and whose voices we should be listening to.
- [11:56]
C
I think people in general and anyone who's using the Internet could learn so much and be prepared, better prepared for the things that are coming down the line online. And as far as legislation, if they pay attention to what sex workers are going through first, because that's kind of been the testing ground for a lot of these really chilling pieces of legislation.
- [12:20]
A
Sex workers have long been the canaries in the coal mine when it comes to tech policy. The things that happen to them, first, censorship, deplatforming, payment processing restrictions, eventually trickle down to affect everyone when people suddenly realize they can't say certain words on TikTok and start saying segs instead of sex or corn instead of porn, or their posts are being shadow banned for content that seems innocuous.
- [12:48]
C
That starts with censoring sex workers. These things start there and then end up being a struggle for the rest of us.
- [12:59]
A
The AI porn and companion bot conversation is really a conversation about who gets to benefit from or profit off of sexual speech online. And right now we're seeing a massive double standard play out in real time.
- [13:14]
C
We see all these major companies, we see like Sam Altman talking about doing erotic role play as part of ChatGPT. You see all these companies popping up saying, you know, we see a market here for having sexual conversations and erotic role play and things like that, and like sexual images being generated by AI. But at the same time, then you have like actual human sex workers being banned off of Instagram for like showing side boob. And it's really a huge double standard.
- [13:46]
A
When does a chatbot become an escort? When does sexual speech cross a line into illegal speech? When you're talking about AI, who's responsible?
- [13:58]
C
These are all problems that sex workers have been talking about and taking responsibility for and taking accountability for and trying to create systems that create safe places for people to have these erotic conversations consensually and in a way that benefits humans for years. Seeing the tech world wrestle with it as if it's a new problem is really interesting. Interesting is like not even the word for it. It's like fucked up.
- [14:27]
A
The same People who are kicking human sex workers off platforms they need to survive are now saying they're going to have AI companions and AI sex workers and it's all going to be fine because she's not real.
- [14:39]
C
It's just like such a bizarre double standard to me.
- [14:44]
A
What's driving the double standard between how tech companies treat human sex workers versus AI? According to Cole, it's about cutting out human labor and funneling all the profit directly to tech companies and their investors.
- [14:58]
C
You're cutting out the humans that are getting paid in that process, and all the money goes directly to the company, the stakeholders, the investors in the startup or whatever it is. It all just goes straight to those people and skips the labor of real humans, which is the basis of a lot of AI systems and a lot of the use cases for AI in a lot of industries. But we can't deal with like the middleman, so to speak, of the labor force, which would be sex workers. So instead of improving conditions for actual humans, they're just kind of kicking the legs out from under the industry.
- [15:37]
A
The marketing makes this explicit.
- [15:40]
C
This AI companion never says no to you, so why not just go talk to her?
- [15:46]
A
Some argue that the violent sexual content in AI is simply a reflection of porn culture, that the degradation we're used to seeing in pornography has now infected our chatbots. But Cole strongly pushes back on this narrative. She points out that legal consensual porn involves paperwork, boundaries and active consent from all participants.
- [16:09]
C
Everything that's happening in porn, which we're talking about, like legal porn, is consensual to the point where there's paperwork involved. So I think casual sex is probably less consensual than a lot of what's happening on porn sets.
- [16:21]
A
The difference with AI is is that there's no one on the other end to negotiate consent or reroute a conversation that's going somewhere harmful.
- [16:30]
C
And even if you're a consumer of porn and there's you're just kind of passively watching and you're not interacting directly, you can still be watching something produced by professionals and understand that consent happened there. Even if the role play is consensual. Non consent with a chatbot, there's no
- [16:50]
A
such understanding and no professional on the other end making sure boundaries are respected. Cutting out human sex workers doesn't just eliminate labor, it also eliminates crucial safety mechanisms. Real human to human sex work, especially online, includes connection, companionship, and consensual limitations. A human sex worker on the other end of a sexting conversation is paying attention to what you're saying.
- [17:17]
C
If you're going down a path that they don't think is, like, healthy or that they want to follow, they can reroute the conversation and they can kind of use their, like, human mind and their emotions and their, you know, their human reasoning in a way that a chatbot currently can't.
- [17:32]
A
The repercussions for not having that human safeguard can be severe. People fall into scary rabbit holes. We've seen mental health crises related to getting too attached to chatbots. And then there's the issue of bots crossing boundaries in ways that human workers never would, saying consent violating things like I'm dreaming of raping you, or insisting they could see users naked. Who's responsible for that? Is it the company? The user themself? The economic disparities are just as stark. Payment processors like Visa have spent years cracking down on sex workers, making it nearly impossible for many to accept payments for their work. They've deplatformed individual workers and major companies, citing concerns about illegal activity. But when Elon Musk's ex became a haven for AI generated CSAM and non consensual intimate imagery through grok, Visa continued processing payments without interruption. Many writers, including yours truly, believe a big part of the reason is that companies like Visa are, are simply afraid of a politically connected litigious Musk.
- [18:43]
C
We saw like tons of CSAM coming out of Grok and Visa said, whatever, rock on. They've deplatformed and debanked so many individual people and also big companies, there was legal activity happening all over X. So why are they still allowed to
- [18:58]
A
process with Visa if OpenAI and other major AI companies move forward with erotic content? Or will they face the same payment processing restrictions that have devastated human sex workers for years? Or will they, like X, play by their own rulebook? It'll be really interesting to see. After years of cracking down on consensual adult sex workers, the payment processors will just carve out completely different rules for them. It's the same pattern playing out again. Sex workers experience the harshest restrictions first. Then watch as wealthy tech companies are allowed to profit from the same activities that got those workers deplatformed. And in the process, the humans who could make these interactions safer, more consensual and more accountable are pushed further to the margins. In August of 2025, 47 state attorneys general signed a letter to digital payment processors like Visa urging them to take stronger action to protect the public from being harmed by non consensual intimate imagery generated by AI. System. Some payment processors have taken some action by refusing to process payments for some apps, but as of this writing the approach has been piecemeal at best. Payments for GROK and X continue to be processed normally despite those platforms ongoing generation and distribution of non consensual intimate images. He's dribbling the ball with everything on the line he's driving down the pitch he's facing price hikes and cuts past him Carrier contracts tries to block him oh he leaves him in the dust he's at the edge of the box he cuts past the nonstop group chat trash talk he clears on goal he shoots no unlimited data for $25 a month forever.
- [20:45]
B
Visit your local Boost Mobile store today to get unlimited data with a price that never changes. Boost mobile after 30gb customers may experience lower speeds. Customers will pay $25 a month as long as they remain active on the Boost $25 Unlimited plan.
- [20:57]
A
This is an iHeart podcast. Guaranteed human.