A (3:58)

Support for this show comes from SC Johnson. We've all been there Choosing not to wear your new white shoes because there's a 10% chance of rain. Bending awkwardly over the tiny coffee table to enjoy a sip of your latte, not ordering the red sauce. Those feelings of dread are what we call stainxiety. But now you can break free from your stainxiety with Shout's Triple Acting Spray that has stain fighting ingredients to remove a huge variety of stains so you can live in the moment and clean up later. Just breathe and Shout with Shout Triple Acting Spray. Learn more@shoutitout.com this is a Monday.com ad the same Monday.com designed for every team. The same Monday.com with built in AI scanning. Scaling your work from day one the same Monday.com with an easy and intuitive setup.

B (4:51)

Go to Monday.com and try it for free. It's true that some things change as we get older.

A (4:57)

But if you're a woman over 40 and you're dealing with insomnia, brain fog, moodiness and weight gain, you don't have.

B (5:03)

To accept it as just another part of aging. And with MIDI health, you can get help and stop pushing through it alone. The experts at MITI understand that all.

A (5:13)

These symptoms can be connected to the.

B (5:14)

Hormonal changes that happen around menopause, and.

A (5:17)

MIDI can help you feel more like yourself again. Many healthcare providers aren't trained to treat.

B (5:23)

Or even recognize menopause symptoms.

A (5:25)

MIDI clinicians are menopause experts. They're dedicated to providing safe, effective FDA Approved solutions for dozens of hormonal symptoms.

B (5:35)

Not just hot flashes. Most importantly, they're covered by insurance.

A (5:40)

91% of midi patients get relief from.

B (5:42)

Symptoms within just two months. You deserve to feel great. Book your virtual visit today@joinmidi.com that's join M I D I.com it is on Renee, Hani and Casey. Thanks for coming on. On.

C (6:01)

Hey, Kara.

A (6:01)

Hey, Kara.

B (6:02)

Hey. This is a topic I really wanted to do a show about. I'm sort of incandescent as a parent and as a person who's watched this go on and so I have feels about it. So just get ready. But thanks for being here.

C (6:14)

It's good to be with you.

B (6:15)

Grok has been in the news for weeks because it's facilitating the creation and distribution of deeply disturbing sexualized images. Some of them are non consensual deep fakes and potentially even see Sam. But see Sam, deep fakes and revenge porn are serious problems that they've plagued the Internet for years. This is not a new thing. So before we dive into the issues with Grok, let's talk about the scope and scale of the problem. Let's hear first from Renee, then Casey, then Hani.

A (6:39)

Okay, so as you note, this has been an issue for a very, very long time. Grok is not the first model to be capable of doing this. One of the things that I think we should talk about first, it began to happen on Grok late December, so maybe we're almost a month into this issue. It began producing what you might call nudified images. So nudification, the ability to do what's called inpainting, to change an image that already exists, is a technology that again has been around for a while. It's got a lot of other legitimate use. It became something that on Grok, you could see it done very visually in the public eye. So this was something that had previously been done much more in platforms and communities that were dedicated to this kind of thing. Right. So there were always, you know, communities that you could go and find discord servers, places where users would do this. Then there was a spate of apps that began to pop up that the app stores were playing whack a mole with, where kids started doing this to harass their classmates. So there was this, these sort of small scale little apps that were where this was happening. But Grok really made it something that was very front and center. People were being harassed in their own replies. And it began to be something that took that harassment out of the dark corners of the web, which was the threat model that a lot of people who have studied this for a very long time, that was, that was how, how we had previously seen it manifest. And it took it and became something where it was happening at around almost 6,700 posts per hour was the stat that some of the research who tracked it took it to.

B (8:16)

So they essentially just amplified it.

A (8:19)

Yeah, it became the place where you could create, distribute and honestly normalize. It became something that all of a sudden was normalized. You could just go and you could do it. And that was what was different about it.

B (8:28)

Okay, Casey?

D (8:31)

Yeah. I mean, as Renee notes, nudification has been a problem going back many years. In the recent past, though, you couldn't get one of the frontier models to generate a photorealistic image of a naked person. You kind of have to hack your way around it and deal with a subpar image. But nonetheless, there were tons of unscrupulous app makers who were buying ads on Meta and who were buying ads on Apple and Google's Play stores, getting these apps into the hands of thousands of people who were using them to terrorize, especially young women. And it was a bad problem. And it did lead to things like the Take It down act, which did pass here in the United States. But as Rene notes, what has happened on Grok has just taken it to an insane new degree where you, you don't have to go scrounge around on the dark web to find one of these apps or use a subpar model. You can now use one of the most powerful models that we have and harass people directly in their own replies in public.

B (9:30)

Right in public. And so this is this spicy mode, honey, is that what it. Explain that.

C (9:35)

So first, I think Renee got it exactly right. What GROK did is they centralized the creation, the distribution and the normalization of this content, which now dates 10 years back. In fact, the very term deepfake comes from the moniker of a Reddit user who used the then nascent technology to create the first non consensual intimate imagery. So this is neither unintended or unexpected and it was also preventable. So your point, Kara, about the Spicy mode is OpenAI's ChatGPT, Google's Gemini have pretty good guardrails and they have guardrails on the semantic inputs, what you type in. They won't allow certain types of prompts. They have semantic guardrails on the output. They will analyze the output image to determine if it shows a child, if it's sexually explicit. And Grok decided we don't want to do that. Hence the spicy mode. So this is neither. You know, sometimes we have technology where people misuse it in a way that is unintended or unexpected. This is neither. This was. We knew exactly what was going to happen. Elon Musk knew what he was doing and he allowed it to happen. And it's been awful. And Rene also made a really good point here, is that this stuff has been around for a long time, but it was sort of.

B (10:42)

Yeah, talk about that. The larger, broader.

C (10:44)

Yeah, let's talk about that. So we're seeing this up and down the stack. I mean, there's hardly a day goes by where I don't see an article about some school kids creating this type of images of their fellow classmates and then of course, sharing it and distributing it. People are using it to weaponize against reporters, against academics, against politicians, against people they just don't like. And it is, you know, there's multiple fact parts to this. So there's obviously the people who are creating the content, there's the tech they're using, there's the platform that empowers them. But then we shouldn't also forget there's an entire ecosystem that is enabling this. Advertisers still advertising on X, Apple and Google are still hosting these apps. Ads are being delivered by Google and Facebook. There's an entire ecosystem that is monetizing and profiting from this, not just the technology companies that create the content. And we shouldn't forget that entire Silicon Valley ecosystem is profiting quite handsomely from the creation and distribution of this content.

B (11:43)

Including of csam.

C (11:44)

Including csam, which is, you know, look, maybe reasonable people want to have a conversation about non consensual imagery. I don't think there's much of a conversation, but. But there's no conversation around csam, right? It's illegal and it's awful and it's dangerous. So this is not, you know, we're not talking about constitutional rights, we're talking about illegal constitution.

B (12:03)

Right. We've agreed on a long time ago and Casey, a lot has happened in just a few weeks. Foreign governments have blocked grok, other ones are investigating. So is California's Attorney General. The UK is working on passing a new law to make illegal to create non consensual intimate images. At first, Elon mocked the controversy. Then Xai made GroX, added a premium feature, meaning you had to pay for it, I guess. Now Xai says they've instituted guardrails to stop it from nudifying images with limited success. Give. Give us an Update, there is an.

D (12:31)

Article in Bloomberg by Kurt Wagner saying that there is still a trickle of images that the public Grok account is still producing. So I think whenever Grok says that guardrails are in place, that is a statement we should view with deep suspicion. It continues to be the case that the standalone Grok app, so this is just something that you can download onto your phone and is rated for children 13 in the Apple App Store, you can use that to make nude images of women. And if you go onto Reddit, there are now thriving subreddits where people are just competing to see who can make the most explicit porn using Grok. So no, there really are not meaningful safeguards in the way that I think most regulators would want to see here.

B (13:15)

And where are regulators right now?

D (13:18)

Well, you know, as you know, we have started to see regulators in a number of countries step up. The countries with the strictest anti porn laws have been the fastest to act. So Grok is now just banned wholesale in Indonesia and Malaysia. But if you look in the European Union, they're planning to take action. The UK is. Even the United States Senate passed a law designed to prevent some of the harms that we're seeing from Grok, although we'll see if it passes in the House. So this has managed to get the attention of regulators around the world. But you know, an ongoing question with Elon is what sort of regulation, what kind of fine, what kind of threat actually results in the harm being addressed? Cuz I think it's just been the case over and over again that that's really hard to.

B (14:01)

He just ignores it now. Renee, you've studied CSAM online for years. In 2023 you contributed to a paper that predicted that AI generated CSAM would be a problem. Looking back, what did you get right and what else didn't you see coming besides Elon's heinous personality?

A (14:18)

Well, I think the willingness to fight about it actually was what has surprised me most about this recent debacle. The, you know, I posted about this on Substack and I got some comment and notes like well, this is all just a plot to censor X. And I thought, really, is that where we're going here? We're using the censorship like the mental stop word? Censorship is where we're going with this. But yes, I mean that is in fact where we're going with this. So I think that it is that term that he has chosen to use for Any regulation is censorship. Any barrier of any kind is censorship. Any restriction is censorship. And now we are using that term to apply to the generation of images of underage girls, right? Is censorship. Anything that throttles that. And it works very well, unfortunately, because I think that term, that word, you know, it has a couple different meanings, right? It refers to legally, it refers to, like, kind of three different things, right? There's factual claims about speech suppression saying, like, this content was taken down by a private platform. That's one meaning of the word. It's a legal claim about US Law, right? A violation of the First Amendment, which doesn't apply to private platforms at all. But nonetheless, they use it in that way. And then it's a normative claim about free speech values. And that is the expansive way that Elon is trying to use it at this point to normalize what is happening on his platform. And that is how it is being used. It is the idea that it is an affront to your free speech rights to not be able to generate nude images of other people. And that's what surprised me.

B (15:56)

Including children.

A (15:57)

Including children.

B (15:58)

And so that's kind of a NAMBLA kind of argument.

A (16:01)

Correct. So seeing that expansion is where I was like, man, I really, you know, we used to have these fights about content moderation, right? The people who would scream at you that content moderation is censorship, but then they would say, but not csam, okay, that's not censorship. That's fine. And there would be this hard line where they would say, like, okay, but we'll concede on this point. And now that's off the table now, right? Now that's censorship too. We just took that off the table. We've expanded it to include that now also. And that's because they have to keep expanding that term to encompass any possible conceivable restriction. And that's what you're seeing here as Ofcom has gone after Ofcom being the British communication safety regulator, as they have gone after Elon, naturally, he is reframing this as, oh, they just want to censor free speech on X and they're.

B (16:46)

Mad at me for that and the other thing.

A (16:48)

So that has been what has surprised me. Nothing related to the actual content. I mean, you know, when Stanford Internet Observatory existed, we used to go and we would present at the Crimes Against Children conference and we would see examples of inpainting, examples of all of the different ways in which, you know, the actual technology has manifested and changed over the last, you know, even just five years. Right? The advances in which this kind of content creation has shifted. So nothing about the actual technology is Surprising here, nothing about the ways people try to get around guardrails, but the mainstreaming of it and then the application of censorship as a normative thing and the aggression in this way. And the aggression has been very surprising to see.

B (17:25)

So Hani, you developed technology that can flag previously identified images of csam. You've developed tools to identify AI created images. But if Grok and X help users create and distribute sexualized images, minors, there's obviously no technical solution for that. Even if tech companies make good faith effort prevent their chatbots from creating sexualized images in the first place, AI will inevitably exacerbate what was already a very serious problem. Talk about how these AI tools complicate the efforts to fight it. For example, deepfakes can be used to extort minors. Someone could attempt to make an authentic piece of CSAM look like it's generated from AI in order to avoid scrutiny from authorities. There's all manner of problems here.

C (18:04)

Yeah. Okay, so there's many sort of threads to pull on here. So let's see a few things. So first of all, when the national center for Missing and Exploited Children NCMEC has previously identified and categorized csam, we extract what's called the perceptual hash and then we can prevent the re upload of that content on things like Facebook and Instagram and TikTok and YouTube. That's called PhotoDNA. That's the work that I worked on with Microsoft back in the day.

B (18:29)

Right, and all the companies have been cooperating on that.

C (18:31)

Yeah, many of the companies, I mean, I mean I would say it was kicking and screaming, they were cooperating. Now most of the big tech companies are on board, including X by the way. But it was not an easy ask because as Renee was just saying, nobody wanted to open the doors of any content moderation because the thought was, oh, if we open it for csam, well what else are you going to come for? So it was a fight back in the day to do this back in the early knots. But okay, here we are. The problem of course with AI generated content is that it's not been previously seen. And in fact you can produce, I can produce hundreds of images on my laptop every minute. And so there's this flood of content that is coming on the Internet that we simply don't have the volume or the capacity to look at every single piece of content and flag it. That's number one. Number two, is that okay, maybe if the tech was very, very good, we could deal with the public facing version of this. Things that are posted on action Facebook. But what you are seeing, as you pointed correctly, is that that bad guys are taking innocent photos of children, nudifying them, sending them in, things like snap and discord and private messaging that we don't have, we're not privy to and then extorting them. And those kids are taking their own lives, you know, destroying their lives. And it is horrific. What we are seeing and this is happening from organized cyber criminals from around the world. And we have seen story after story after story. And here's the thing is when you look at these technologies, everything we always see, and this has been true of Silicon Valley, as you know, for over 25 years, is safety is always an afterthought. We always try to backfill safety in.

B (20:08)

It's not even a thought.

C (20:09)

It's not even a thought. I'm being generous here, right? You can't backfill safety. You can't do what X is trying to do now. Being like, okay, we made a mess. We've normalized this, We've given everybody a taste of it. Now we're gonna try to sort of backtrack it. It's not going to work. I'll add one more person, by the way, if I may, to the list of folks that are going after X which Australia's esafety commissioner Julie Inman Grant has opened an investigation and they doing really, has consistently doing really good work to try to hold these tech companies accountable for this. But this is a mess that was both preventable and not unexpected.

B (20:43)

We'll be back in a minute. Support for this show comes from Acorns. A lot of people focus on where their money is today, but Acorns is the financial financial wellness app that cares about where your money is going tomorrow. And with Acorns potential screen, you can find out what your money is capable of. Acorns is a smart way to give your money a chance to grow. You can sign up in minutes and start automatically investing your spare money, even if all you've got is spare change. I haven't tried Acorns yet, but Scott has. Scott, tell me about it. It's a wonderful way to start investing. It rounds up to the nearest dollar and then automatically invests in a low cost index fund. It's a great way to start investing and it's easy to use if you're not super se us. It's fantastic and it's, it's all about establishing good habits and financial security at a young age pretty seamlessly. Thank you for that, Scott. That's a great thing about Acorns. It grows with you. Sign up now with Acorns and boost your new account with a five dollar bonus. Investment join the over 14 million all time customers who have already saved invested over 27 billion dollars with Acorns. Head to acorns.com cara or download the Acorns app to get started. Paid non client compensation provides incentive to positively promote Acorns tier 2 compensation provided potential subject to various factors such as customers accounts, age and investment settings. Does not include Acorns fees. Results do not predict or represent the performance of any Acorn portfolio. Investment results will vary. Investing involves risk. Acorn Advisors LLC and SEC Registered Investment Advisors. View Important disclosures@acorns.com Cara. Support for on with Kara Swisher comes from Groons. If you're looking for a health goal that you can actually stick to, you might want to check out Groons. Groons is a simple daily habit that deliver real benefits with minimal effort. They're convenient, comprehensive formula packed into a snack pack of gummies a day. This isn't a multivitamin, a greens gummy or a prebiotic. It's all of those things and then some at a fraction of the price. And bonus, it tastes great. Grunes ingredients are backed by over 35,000 research publications, while generic multivitamins contain only seven to nine vitamins. Groons have more than 20 vitamins and minerals and 60 ingredients which include nutrient dense and whole foods. That includes 6 grams of prebiotic fiber which is three times the amount of dietary fiber compared to the lean greens powders and more than two cups cups of broccoli. It's a daily snack pack because you can't fit the amount of nutrients Gruns does into just one gummy plus. That makes it a fun treat to look forward to every day. Kick off the new year right and save up to 52% off with the code Cara@groons.co that's code Kara K A r a at Grunsgruns Co. Let 2026.

A (23:50)

Be the year you stop fantasizing about.

B (23:53)

Starting a business and actually do it with Shopify. Setup is fast with Shopify's built in.

A (23:59)

AI tools, they can write product descriptions and headlines and help you edit product photos in 2026.

B (24:06)

Stop waiting and start selling with Shopify. Sign up for your $1 per month trial and start start selling today at shopify.com Vox go to shopify.com Vox that's shopify.com Vox hear your first this new year with Shopify by your side. So, Casey, when it comes to livestream, we're actually in a legal gray area in the US at least. The Take It down act makes it illegal to publish non consensual intimate visual depictions, both authentic and deep fake. But it doesn't take into effect until later this year. More importantly, what qualifies as an intimate visual depiction isn't clear cut. The Senate just passed, as you said, the Defiance act for the second time, which allows victims of non consensual explicit AI images to sue. But you run into the same message you do with the Take It Down Act. It's not clear if the AI platforms themselves are liable realistically. Talk about the legal recourse of victims of Grok's deep fake binge have right now.

D (25:12)

Well, I mean, it's terrible because it places the onus on the victim to go out and get justice for themself, right? Like they have already been attacked in public and, you know, suffered in who knows what way. And now starting in May, there will at least be some avenue through the Take It down act through which they can go and try to get some sort of, you know, justice done. But, you know, my, my fear here is that this is going to be yet another horrible customer service experience from a tech platform where you go onto a website and there's a form that you fill out and you send it in and you just pray that someone answers it. Right? So I, I actually right now, you know, do not have a lot of confidence that this is going to feel like it is getting justice for any of the victims. And if you think about, you know, the sorts of people that are often targeted in these attacks, it is often women. It is often women who are, you know, politicians or leaders of some other type. And these sorts of attacks are used in an effort to silence them and to sort of drive them out of public life.

B (26:14)

Speaking of which, Ashley St. Clair, the former conservative influencer and mother of a child with Elon, has sued X over these deep fakes of her and it's countersued, saying she violated the terms of use by actually suing for her abuse.

D (26:28)

Yeah, well, in particular cause she didn't file in the, in the Northern district of Texas, you know, which is just a frivolous, you know, legal action designed to make this, you know, more expensive and been horrible for her. But my point is here, by the time that it comes to go fill out the form on the website that says hi, like I've been a victim of a horrible attack, the damage has really already been done here. Right. And there's. And my fear is that this is just going to drive a lot of women in particular off of social platforms, which of course serves the interests of the fascists who run x dot com.

C (26:54)

Okay, can I add one more thing too? Is that the Take it down.

B (26:57)

Go ahead.

C (26:57)

The take it down has a 48 hour window. That's an attorney on the Internet. The half life of a social media post is about 90 seconds. The 48 hour window is completely absurd. The content has been online. It's been online for two days and it's spread wildly. There's no taking it down at this point. So the whole thing, I think, is a fool's errand.

B (27:17)

So now Google, as you mentioned, Renee, Apple and Google could have kicked X off their app stores for violating their guidelines. So far, Tim Cook and Satya Nadella have done nothing. They've said nothing. Renee, you wrote on threads that the app stores wouldn't do anything. Let's have you read what you posted.

A (27:31)

Okay, here we go. It's too big to regulate. It's not a two bit neutify app. And Elon will howl censorship about second order consequen. He uses the word even when Europeans fine him over blue check fraud. Similarly, congressmen that talk a big game about protecting kids with Zuckerberg are afraid of Elon. Josh Hawley and Marsha Blackburn should be speaking up.

B (27:48)

So talk about why they aren't and why they're upset about Elon in particular.

A (27:52)

So the. So as we were talking about a little bit earlier with the small two bit neutify apps, there is one use for those apps. We all know what they are for. They are for harassment, that is, and they are often made by kind of indie developers. And so when they are, you know, when the mole is whacked, nobody complains. Or if they complain, you're not gonna hear about it. Nobody really cares when Elon complains. The world hears about it. Right? There is an entire army of people who will come out and will howl censorship right along with him. As I mentioned again, we have now seen the expansion of that term to cover, you know, your perceived right to generate news of other people. As we've seen, you know, Ashley St. Clair had the audacity to complain. She had her blue check revoked. She was demonetized. Right. It was immediately made clear that that is the complaining. When you do things to Elon's companies, all of a sudden you get tariffed. Right? There are people who get their visas revoked because they have the audacity to you know, to, to implement, to, to implement fines on X for dishonest business practices. Right. When X doesn't adhere to the terms of the Digital Services act, that becomes an international issue. So the dynamics around regulating X are fraught to begin with because Josh Hawley.

B (29:08)

And Marsha Blackburn have been very vocal on this.

A (29:11)

Right? So, so to them in just one second. But the other thing I'll say about this though is that the other, the realistic point though is that the integration of GROK and X is a challenge, right? So Malaysia and Indonesia block grok. The challenge though is that people use X to communicate. It is a speech platform. And when you block X, you are blocking a lot of other types of speech that have nothing to do with notification. And that is realistically one of the reasons why when I say it's sort of too big to block, it has a lot of other issues that go along with it. And so when you block the notification piece, you're, you have a lot of other kind of second order effects. But you also have Marsha Blackburn, who, if you've heard her speak in tech hearings, she gets very, very upset about two things. One, AI doing bad things, right. You've heard her complain about, you know, where they say things like they make up accusations about conservatives and things like this. You'd think she'd be very upset about them nudifying women, right. She's a strong proponent of defending women. Two, defending children. These are people who support cosa. Josh Hawley has a very strong Christian populist sentiment in his kind of ethics underlying why he wants to see tech regulated. Constantly writing letters to Zuckerberg asking him to account for himself. And the two of them were entirely silent about this. The person who spoke up was, was Ted Cruz, who spoke up, but then also promptly posted a photo of himself with Elon. Right. So you have this, this tension here, right, which is he knows this is wrong, but also is very, very afraid of being seen as challenging Elon, who is a major Republican Party donor. Right.

B (30:41)

He's just given $10 million to a candidate in Kentucky. For example, Kroc isn AI chatbot with vulnerabilities around DPIX, obviously, or CSAM obviously. Xai seems to care less, but creating robust guardrails for AI chatbot image generators is legally perilous. In order to fix any of the vulnerabilities, you have to find them. That means red teaming the chat bots, in other words, trying to get them to actually generate CSAM images in order to identify loopholes and close them. But because of the ways the laws are written, anyone red teaming a chatbot for CSAM might actually be breaking law. Hani, you've testified about this issue in csam. What would you like to see to help create a more robust infrastructure to fight it? Because that's one of the things that they, that's the problem here. Just like Ashley Sinclair suing and then being sued for suing because she's not supposed to sue under the terms of service.

C (31:30)

So I do think, you know, we should add another component to this that as somebody who has worked in this space and has viewed this content as part of my work, it is horrific. You don't want people looking at this. Even the fake stuff is gruesome and is the thing that nightmares are made of. But I do think it is necessary and we do need robust red teaming in. We need mechanisms to make sure, and you're 100% right, that you know, we continue to follow the path of move fast and break things and you know, look, some of these things are really hard problems, you know, and nobody's expecting perfection here. Nobody is saying you can never ever, ever do this. But when you make it so easy, easy, it is really inexcusable. And I think there are many, many things we can do. From the guardrails on the training data, this is work that Renee has done on semantic guardrails on the input, semantic guardrails on the output, really vigorous red teaming. But the thing of course that you know, is that none of this is good for profits. Right. And so what we need here is really good either liability or regulatory overhead to create an incentive for the companies to do better. Because what we know is left to their own devices, they won't do better. And so it's on us as the public and as regulators and the courts to force that upon them.

B (32:41)

Yeah. Now they've also created a media narrative. So let's take a step back and analyze the larger implications for the debacle. Elon explicitly created GROK to be, quote, anti woke. And the issue with the GROK image edit is besides making it exceptionally easy to create these sexualized deepfakes, it's made it equally easy to use those images to humiliate and harass victims. As you've all noted, this is part of a much larger problem of bullying and denigrating women online. Casey, you've mentioned this is part parts of the larger trend of using harassment. As you said, silence women, push them out of the public discourse. I'd love to hear each of you talk about how this grafsko fits in this larger systemic issue of harassment and silencing. Casey, you start and then Renee and then Hani.

D (33:21)

Well, I mean, look, women have never had an easy time online, right? Like going back to the earliest days of Twitter, you know, women were also having a terrible time. And you know, under Jack Dorsey and Dick Costello, the company also massively underinvested in tools that would just, you know, keep women not just safe, but also just feel comfortable participating. I think the fresh shock here is just that the company itself makes a tool that can strip a woman down to her underwear and shove that into her mentions and the entire executive leadership team is just laughing it off. Right? I mean it's like, it's like a level of moral depravity I just didn't think we would get to in my lifetime. But now here we are.

A (34:02)

Renee I think I would just co sign that. I think again, it's the, it's where leadership has gone. It's the moral degradation on that front. I think we had seen a trend from 2015 with Gamergate and attitudes around how should we think about harassment and the fact that if you think about free expression, there was a balancing act, a recognition that certain types of speech push other types of users out of the conversation. And so you wanted to try to maximize that equity, to create that environment that allowed the greatest possible number of users to continue to contribute. So saying we're going to allow these people to express themselves by creating nude images of those other people. Right. Means that those other people are going to exit and go somewhere else. They're going to cease contributing, their speech is going to be stifled, they are going to be chilled. But it's almost like we've sort of reverted back 10 years into that, into that view of equities and conversation as we've just decided that there's a, you know, kind of new sheriff in town and this one just doesn't care.

B (35:09)

Yeah, doesn't care more than doesn't care. Thinks it's hysterical, honey.

D (35:12)

But, but remember, Kara, remember you're not allowed to create an account on X that posts the recent whereabouts of Elon's plane. Cuz that would make him unsafe and it would make him feel like he can't speak. But if you wanted to strip down any women on, on X, that's fine.

B (35:26)

Fair point. Casey Hani.

C (35:28)

Yeah, I'll add two things here. One is while women are the brunt of a lot of this, there's. It's also all underrepresented groups have been targeted for years and continue be targeted, including children. And so that umbrella is actually quite large of people who are being harassed online. The second thing, and I want to sort of reemphasize something Renee said. I always like it when people like Elon Musk talk about free speech, because what he really means is free speech for me and people who agree with me. If you are a true believer in free speech, you want a platform that brings all voices, not just the voices of people who agree with you. And by allowing this type of content, you are, in fact attacking free speech for lots and lots of people. And so if you want to be a free speech person, fine, be. But let's at least be consistent with that worldview.

B (36:11)

Oh, you've noticed the hypocrisy.

C (36:14)

Shocking, isn't it?

B (36:16)

Shocking. It's so. It's been that way forever, actually.

C (36:18)

I know, and I know I'm stating the obvious, but it's. It's. You know, I. I sort of really. That hypocrisy in particular really frustrates me because first of all, I think there's lots of restrictions on free speech. That's first of all. But this is not a free speech issue. This is absolutely a red herring that is used to silence.

B (36:36)

That is correct. So the right obviously has also been obsessed with pedophilia. I was walking across Union Square park this weekend, and the QAnon people are still there, still there, still protesting something. Pizzagate and QAnon were about supposed pedophile rings, obviously. So is the Epstein story, though. That one is grounded in reality. And then there's the fixation with the drag queen story hour and calling LGBTQ IA people groomers. And yet we see this casual attitude towards Elon Cross and Grux exploitation, harassment of minors online.

C (37:05)

It's never been about pedophilia. It's never been about this. That's the point, right? We don't like some people. We're gonna tag them with something that seems like it's impossible to ignore. And then when your own guy gets tagged with it, we are silent. This has never been about protecting Twitter, right?

B (37:20)

This is what Elon did to Yoel Roth. And as you all know, Yoel was the head of safety at Twitter before he just quit. And then the minute he quit, Elon attacked him for being a pedophile, which that's his go to pedophile. Pedophilia is Elon's go to trans or pedophilia is his go to Renee, you and many other researchers who study online Hate disinformation and misinformation. And CSAM have essentially been harassed out of a job by MAGA lawsuits, congressional letters, funding threats, and now deportation threats. In the process, they've gutted organizations that research ways to fight CSAM and deepfakes like your former employer, as you noted just earlier, the Stanford Internet Observatory. What's the current state of play when it comes to research on csam? Sam?

A (38:02)

Well, I mean, losing SIO is a really big deal. When I got this, I got a text from a reporter. Gosh, might have been like New Year's Day, I think, talking about some of what was happening and saying, hey, I've got this. I have this tip, this person saying they think they have AI generated csam. I don't know what to do with it. And I said, well, you know, don't look at it.

B (38:27)

Right, right, right.

A (38:28)

But then there's this question of like, okay, where do we. We send it? And I was kind of dming with my former colleagues trying to figure out, all right, well, who can we send this to? We used to have special mechanisms for dealing with it, and obviously there's NCMEC and a bunch of other places. But as far as researchers who are still in the space, this question of who are the research groups that are still equipped to do certain types of work on this, I think there are different groups that had different specialties that are no longer operational in part because of the funding cuts. Other things where, you know, saying the, the rather gleeful dismantling of research centers because maybe they studied misinformation, or maybe they had the audacity to say the 2020 election wasn't rigged and stolen, you know, which it wasn't. Right? I think some of my. Right, exactly. Well, I mean, SIO shut down as a result of that. I think my. I think I'm heading into year three of the litigation. Two court cases again against us. And Jim Jordan sent yet another letter to Stanford complaining about researcher speech. You know, they, they had a dinner apparently with funny of Julian Mongrant and a few people, which Michael Shallenberger and Jim Jordan reframed as some sort of censorship cabal. So they sent another letter demanding emails to try to get to the bottom of that. And so the harassment continues, the stupid conspiracy theories continue. And meanwhile, the question of who does important work work to actually solve real problems. You know, the centers that used to really specialize in that get shut down do not exist.

B (40:08)

And they continue to ask someone like you, especially that ever idiotic Michael Shellenberger, honey, Your work has been somewhat controversial though in the industry. People that are not Michael Shellenberger, like Mike Masnik, the founder of techdeck, feel you're too willing to compromise the privacy rights of everyone who's in order to try to catch people who create or distribute. See Sam, give us your best face version of your critics argument and then explain the trade offs between privacy and child safety online and why you think they're worth it.

C (40:36)

Well, first let's acknowledge that there is tension here between privacy and safety. And that's not a strictly online thing, it's an offline thing too. When I go to the airport, you know, they get to look through my bags and my person. So there's a trade off we have to make. And so one position to say is, is absolute privacy for everybody and safety is a second third thought. The other is privacy for nobody. And it's, you know, that's a problem also. So I, I think reasonable people can disagree here. My position has been this, when we talk about privacy, we should talk about privacy for everybody, not just you and me. Because when I talk about safety measures for children, that is a privacy issue for the children. And so I reject this notion that I'm throwing privacy under the, the bus right now. The argument that I've heard that is, goes something like this, that if you do this, go searching for child sexual abuse material, well then you will enable people to do xyz, the slippery slope argument. And the reality is that when we built photo DNA back in the day, we heard those same arguments and it just was not true. People said to me in my face, you are going to destroy the Internet, you will destroy speech. And it was not true. I reject the notion that we can't have reasonable safety measures and reasonable privacy. This notion that everything absolutely has to be end to end encrypted and nobody gets to do anything. I think sort of defies the sniff test when it comes to the physical world. Nobody would say that when you go to the airport, nobody's allowed to look in your bag because it's a privacy issue. So what I would like is for this to stop being a conversation about privacy and safety because this is an issue of privacy for everybody and a safety for everybody. And I, I think as we have done in the physical world, we have to find a compromise. And I think that that is not a conversation that has been particularly healthy in the last few years.

B (42:28)

Yeah, I don't know why it shifts so drastically when it comes to online child pornography. I find it strange, I find it strange too.

C (42:35)

And here's the thing. I mean, maybe 20 years ago we could talk about an online world and an offline world. There's no distinction anymore. There's no online and offline world. There's the world, this one world. And we have to start coming to grips with that.

B (42:48)

We'll be back in a minute. Support for on with Kara Swisher comes from Indeed. Right now there's a talented person out in the world who can help take your business to the next level. But finding that person doesn't need to be a grind. Just use Indeed Sponsored Jobs. It boosts your job posting to reach quality candidates so you can connect with the exact people you want factors faster and it makes a big difference. According to Indeed data, Sponsored jobs posted directly on indeed are 90% more likely to record a hire than non sponsored jobs because you reach a bigger pool of quality candidates. Join the 1.6 million companies that sponsor their jobs with Indeed so you can spend more time interviewing candidates who check all your boxes. Less stress, less time and more results with Indeed Sponsored Jobs and listeners to the show will get a $75 sponsored job credit to help get your job the premium status it deserves@ Indeed.com on go to Indeed.com on right now and support our show by saying you heard about Indeed on this podcast. Indeed.com on terms and conditions apply. Hiring do it the Right Way with Indeed. Support for the show comes from Cervel AI. If you ever wonder why it takes long for your IT department to to get the big picture things done, just look at the sheer number of tasks that they have to do day in and day out. Your IT team wastes half their day on password resets, access requests and onboarding and the more your business grows, the more requests pile up. But with Servl, you can cut about 80% of your help desk tickets in order to free up your team for more meaningful work. Unlike other legacy players, Servl was built for AI agents from the ground up. Servil AI writes automation in seconds. Your IT team just describes what they need in plain English and Serval generates production ready automations instantly. Plus, Serval guarantees 50% help desk automation by week four of your free pilot. But try it now because pilots are limited. SERVL powers the fastest growing companies in the world like Perplexity, Merkor, Verkada and Klay. Get your team out of the help desk and back to the work they enjoy. Book your free pilot@servol.com Cara that's that's S-E-R-V-A-L.com Kara RFKJR HHS Secretary recently announced.

A (45:19)

New federal dietary Guidelines Declaring war Today.

D (45:24)

Our government declares war on added sugar.

A (45:27)

Also ending war.

C (45:29)

We are ending the war on saturated fats.

B (45:32)

In his tenure as head of hhs, RFK has ended the war on protein and declared war on ultra processed foods. And on the old food pyram.

D (45:41)

You can see the food pyramid here. It's upside down, a lot of you will say, but it was actually upside down before and we just righted it.

A (45:51)

The Trump administration has pushed companies to eliminate food dyes and in the first bill signing of the new year says.

B (45:58)

Schools can now serve whole milk.

C (46:00)

Whole milk and it's whole with a.

A (46:02)

W. For those of you that have.

B (46:04)

A problem on Today Explained the push from the top to revamp the American diet. Today Explained drops every every weekday. Let's wrap up talking about accountability or lack thereof. Really. There may not be any accountability around this whole ordeal. It boosted Grok's downloads and Secretary of Defense Pete Hegseth announced that Pentagon will be integrating Grok into its workflows. And nothing happened. With Apple and Google around not doing anything they didn't get shame they were not. They never responded. So what's the takeaway from this whole episode? It'll be just as another grimy chapter of the story of X and Grok. Like the Mecca Hitler tweets or explicit Taylor Swift deep fakes going viral on X. Or is there any possibility of real change?

D (46:51)

I think that to the extent there's going to be real change that's just going to come from outside the United States, I do think that X's existence is just going to become more complicated in more countries around the world. It's going to become more of a headache for Elon. So maybe we'll see see some measure of accountability there. If you're looking for any sort of silver lining here, though, I would just say that the fact that Grok has to stoop to these levels just show you how unsuccessful it is compared to its rivals. Like the introduction of its porn chatbot last year was just explicitly a way to try to boost engagement for what is essentially the fourth or fifth place AI company. So to me this is just a sign of increasing desperation over there to get something, something working. And yeah, so that's what I would say about that.

B (47:42)

And then dress it up like free speech essentially.

D (47:45)

Yeah.

B (47:45)

So honey, a review by Wired magazine shows Grok is creating violent sexual images and videos with minors in them. That is much worse than what's being posted on X. We assume the malicious actors are going to find ways to use AI to create illegal deepfakes in csam. What sort of AI tools can you imagine would be useful in combating the distribution of these images while respecting, protecting these privacy rights you talked about and minimizing false positives that can tar an innocent person's reputation, for example?

C (48:11)

Yeah, yeah, yeah. Well, first, like everything in cyberspace, this is an adversarial system, right? Like spam, malware and ransomware. It's a give and take and it's going to be a back and forth. There are, I mean, if you go to OpenAI's ChatGPT or Google's Gemini and you try to put in the same prompts that you put into Grok, it won't work. So we have an existence proof that we can have reasonable measures in place. And this is the way it works. You solve 80% of the problem relatively quickly and then you start chipping it away with a combination of red teaming and responsible reporting mechanisms. And we, we get to a steady state the way we have with spam. It's a mostly manageable problem these days. But that takes effort and it takes years and years and years to do that. And it takes will and it takes leadership. And my thing, if I can come back to the question that you asked Casey, is I think there's, there's one positive thing that has come out of this, which is the real, really loud outrage from the media about what is happening. And I think article after article after article just really hammering. And I think that's good to see that there is some consequence to this, number one. Number two is I don't think regulation is going to get us out of this just because it takes so long and it's so imperfect in the way it's crafted. And of course the lobbyists are going to beat the hell out of the regulation before it sees the light of day. I think where this starts to get traction is with lobbyists. So putting aside how Elon Musk is threatening people, but if we start suing these companies back to the dark ages, they will internalize that cost and they will make safer products. And so I'm looking to the courts for relief here because I think that's the fastest way to get relief.

B (49:46)

You're absolutely right. You have sued the bastards. That's what I said. But Renee, we're talking about very dark, disturbing trends. And the underlying assumption behind your work is that we can have a better Internet. So paint a Positive picture. Let's imagine a world ten years from now where there's a momentum towards creating, creating a safer online world. Something I've talked about a decade ago. Speaking of which, one of less hate and harassment. What does it look like? What would you need to do? What are the pieces that you need to put in place?

A (50:12)

I mean, some of it is, as Hani mentions. There is the courts, right. There's the FTC also, which I haven't heard anybody mention, but I know.

B (50:20)

Oh, please, feel free. Yeah.

A (50:21)

Well, baronsoka and some other folks are writing about consumer protection powers. I don't know if this FTC will do it again because of the unfortunate integration of Elon into the Republican Party and the political tensions there. But that's another potential option.

B (50:34)

Brandon has an enormous crush on him. It's a little bit untoward, frankly, but.

A (50:39)

One of the things that we saw.

B (50:40)

I may create an image or something.

A (50:46)

There's the exit versus Voice phenomenon, right? Users have voice. They can say, we don't like this, we don't want it. You should, I would hope, see more conservatives on the platform saying, this isn't appropriate, this isn't right, this isn't normal. That's what I would have wanted to see instead of people falling, falling in line behind the censorship, the censorship bullshit. But the other thing is exit. You can leave. There are other platforms, you can go there. Right? Threads is a little weird, but it exists. You know, Blue sky is a little weird in some ways, but it exists. There are, There are alternatives. And users can vote with their feet and at some point they're going to have to. And just to say, this isn't normal. And that's the other option. And that's where you just say, there are other places to go, there are other places to be. There's Instagram, there's TikTok, I mean, there. Which didn't ever actually get banned, right? There's. There's so many different, so many different opt. It's not that they're necessarily good, but they. There are alternatives. And that user. User outcry in the United States in particular has done more to move platforms in various directions with their policies than any regulation, because we haven't had any regulation here. And that is, I think, the last lever really available to say that this isn't okay. We don't want it, it's not moral, it's not ethical, and we should be doing something different.

B (52:02)

Yeah, yeah. I do my best by calling it a Nazi porn bar, as you know, all the time, but it Seems to have caught on a little bit. So if we take a step back, this episode with Grok is just the tip of the iceberg when it comes to issues created by unregulated AI. I'd like each of you to finish up by talking about what else is coming our way. What worries you most? What is the thing you're thinking about next if you had to anticipate? I mean, I think we have anticipated that Elon will do the very worst thing given the choice. But what worries you the most? And what do you think is coming our way? Kasey, then Hani, and then Renee.

D (52:37)

To me, the worst aspect of Elon's underinvestment in content moderation and basically just like, rejection of it as something that anyone should do, is that it has just lowered the bar for every other player in the industry. Like, Meta is so much worse now than it was before Elon bought Twitter. And it's just because they feel no pressure to do even half the stuff they used to, because they look at what Elon gets away with and they think, well, you know, what does it matter if we're gonna get 10% of our revenue from literal scams this year? Right? Or you look at Roblox and all the issues that they've had with child predators, and they just look at it as, ah, that's just a compliance function. Let's go ahead and do the bare minimum. So to me, that's the real issue here, is that when there is someone who is so publicly able to get away with worst things imaginable on their platform, everybody else follows suit because it's cheaper.

B (53:31)

That's a very good point. Is there something coming that worries you?

D (53:34)

I mean, the thing that worries me is just that we're already in a world where people have trouble distinguishing fact from fiction. The underlying models continue to get more powerful. There's a huge contingent, particularly on the left, that thinks that AI is fake and a bubble and is all going to collapse tomorrow when OpenAI can't pay a bill. And so I think a lot of us are just sort of sleepwalking into a future where this, the media environment becomes ever more challenging and dangerous. And it doesn't seem like many people in power want to do much about that.

B (54:04)

Good point, Hany, then Renee.

C (54:06)

Sure. I mean, a lot of things keep me up at night these days, but here's Bill Gates had this great line many years ago. He said, we tend to overestimate what tech will do in one year and underestimate what it will do. In 10 years. And that's been true for quite a while. I think in the age of AI that 10 years is probably closer to 5 years and maybe 3 years. I think the world is going to look very different in three years. And here's the thing I'm looking at, particularly for 2026, which is the, the age of agentic AI is upon us. So we, everything we have talked about for the last 45 minutes or so has been humans in the loop, right? Humans are powering this in terms of creating the content and then sharing it and liking it. But soon what we are going to see is fully deployed agentic AI, state sponsored cyber criminals, individuals. We'll have agents doing the work for them, fully automatic, 24 hours a day, no sleep, no moral, no fear of arrest. And that's going to be a very weird world that we are entering in. And I think cyber security in general is going to explode with attacks from agents that are working 247 and adapting very, very quickly. And we've already started to see attack from AI, written malware and ransomware. And I think this, the world of agentic AI, more likely than not in the next 12 to 18 months is upon us. And I think that is going to change an awful lot.

B (55:27)

So supersize. This supersize is a good and huge problem. Okay, Renee?

A (55:32)

Yeah, I would concur on that. I think it's the prevalence of fraud. I think the financial industry is beginning to realize that systems that had built using voice as an authenticator, insurers have been using voice as an authenticator. A lot of systems that were a lot of money invested thinking that this was going to be secure, the realization it is not secure, secure and efforts to try to find things that are the privacy trade offs with things like biometrics that people are very uncomfortable with, particularly in the hands of governments. And yet at the same time the recognition that there have to be mechanisms for credentialing in some way in part to mitigate fraud. Right. So what are those trade offs going to look like? What does a privacy protecting identity layer look like? Again, recognizing that people don't want the government to control it and yet the private sector options are not great either. So that's an area that I've been looking at a lot lately and wondering what on earth happens on that front.

B (56:25)

And what worries you the most in this area?

A (56:29)

You know, we've got another election cycle coming up right now. I think those questions around trust and legitimacy are going to play out very heavily in the race. You know, I've never been in an AI election, Doomer. I think there's always been that it's going to matter in some way. We should prepare, we should recognize it as a threat and yet not overreact. And so again, yet another cycle coming up, gearing up for that because we have an incredible fractured trust environment here and we've seen the technology advance a lot over the last two years and we're going to see what that looks like going into this next cycle.

B (57:08)

All right. But we can be sure Lilong will do something repulsive anyway. Never underestimate, underestimate his repulsiveness. Anyway, goodbye and thank you so much. Renee, Hani and Casey. It's a really important topic. I'm gonna keep coming back to it all year, around chatbots, around kids, around suicide, around everything, because they're not gonna do anything about it. As you said, the media certainly can. I really appreciate it.

C (57:33)

Thanks, Karen.

A (57:34)

Thanks, Karen.

B (57:34)

Thanks, Kara. Thanks. Today's show was produced by Christian Castor Rousselle, Michelle Eloi, Megan Burney and Kalyn Lynch. Nishat Kurwa is Vox Media's executive producer of podcast. Special thanks to Katherine Barner. Our engineers are Fernando Arruda and Rick Kwan and our theme music is by Trackademics. Go. Wherever you listen to podcasts, search for on with Kara Swisher and hit follow. Thanks for listening to on with Kara Swisher from Podium Media Media, New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.