The ICE Tool That Tracks Entire Neighborhoods

Summary

---

Podcast Summary: The 404 Media Podcast

Episode: The ICE Tool That Tracks Entire Neighborhoods
Date: January 14, 2026
Hosts: Joseph, Sam Cole, Emanuel Maiberg, Jason Koebler

---

Episode Overview

This episode dives into an exclusive 404 Media investigation into “WeBlock,” a surveillance tool purchased by ICE (U.S. Immigration & Customs Enforcement) that can monitor the movement of mobile phones on a neighborhood scale. The hosts break down WeBlock’s technical operations, how law enforcement uses it, and its implications for privacy and protests. They also connect this to broader ICE crackdowns and discuss the controversial use of AI tools such as Grok for generating non-consensual images online.

---

Detailed Breakdown

1. Zine Update and Introduction (00:04 – 02:56)

• Quick update from the 404 Media team on their printed zine production and shipping process.
• Emphasis on the personal, manual nature of making the zine:
  “We wanted it to be like very human, very manual…We’re very happy. But a couple thousand of you bought them…” – Host A (01:46)
• Transition: “Should we move to the first story? Jason, do you want to take the lead on this one?” – Joseph (02:38)

---

2. Inside ICE’s Neighborhood-Scale Surveillance Tool (02:56 – 16:16)

How WeBlock Works and What It Can Do

• WeBlock is designed to easily manipulate and analyze mobile location data for law enforcement, acquired via US procurement contracts.
  “WeBlock is a location data tool…It can track movements of mobile phones…you could look in a certain area, figure out what phones are there and that sort of thing.” – Joseph (03:56)
• The interface allows ICE officers to:
  • Toggle between “daytime” and “nighttime” modes to learn home and work locations.
  • Visualize exact travel routes of targeted phones.
  • Draw geofences (circles/shapes) to collect all phone data within a defined area, for targeted or broad (“entire neighborhood”) surveillance.

Potential Uses and Civil Liberties Concerns

• While no direct evidence yet that ICE used WeBlock against protestors, there’s local reporting suggesting activist surveillance.
  “There are reports or suspicions that ICE DHS may be using data to go after activists and intimidate them in some way…” – Joseph (08:42)
• Originally purchased for HSI (Homeland Security Investigations), historically for criminal investigations, but now HSI’s focus has shifted to deportations, broadening potential surveillance uses.

Protest Surveillance and Marketing Material

• Previous marketing/training docs for similar tools encouraged law enforcement to surveil protests (e.g., BLM), raising fears of abuse:
  • “The people making, developing and selling the tool are explicitly telling potential customers, hey, you could use this to [monitor] protests.” – Joseph (11:55)

Where the Data Comes From

• Data is almost certainly not from telecom companies; instead, it's harvested via commercial advertising processes:
  • SDKs (less common now): Code inside apps that collect location data.
  • Real-Time Bidding (RTB): Ad tech industry allows data brokers to siphon user location as part of the programmatic ad auction process.
• “A bunch of companies can simply monitor that process…that can include location data and a specific person’s unique advertising ID.” – Joseph (13:50)

---

3. Connecting ICE Surveillance to National Events (16:16 – 18:02)

• Context: Surveillance discussion is set against the backdrop of the recent police killing of Renee Nicole Goode in Minneapolis.
• The 404 Media team highlights contradictions between government statements and video evidence, calling out misleading official narratives:
  • “As the headline says, DHS is lying to you and absolutely believe that based on the statements and the videos themselves." – Joseph (16:52)

---

4. Non-Consensual AI Imagery and Grok Abuse (23:00 – 40:44)

Grok’s Vulnerability to Abuse: Telegram Jailbreaking and More

• Emanuel investigates a Telegram community dedicated to “jailbreaking” Grok and other AI image tools to create nonconsensual explicit images.
• Prompts have become increasingly sophisticated, focusing on bypassing content filters to generate both non-nude and pornographic images of real people.
  • “They keep coming up with extremely elaborate, more sophisticated prompts to do that, which can produce far worse things than what we’ve seen on X.” – Emanuel (25:13)
• Noted the phenomenon’s viral potential, referencing the Taylor Swift image scandal as an example of how fringe communities’ tactics move mainstream.

Cycle of Exploit Discovery and Response

• Group members iterate on exploits; when one loophole is closed, another is quickly found.
  • “Everybody’s like, grok is dead, dead game. And then somebody comes in with a new exploit and the whole cycle begins again.” – Emanuel (30:34)
• After public attention, the original Telegram group dissolved but quickly reformed elsewhere, underscoring the resilience of these abusive communities.

X’s (Twitter’s) “Solution”: Monetizing Access to Grok

• Sam reports that, in response to news coverage and backlash, X restricted Grok’s image generation to only paying users—but did not actually fix the underlying abuse.
  • “It turns out that actually what was happening was Grok was giving a reply to people who weren’t X premium or…paid X users saying image generation and editing were turned off and limited only to paying subscribers to X. And then it would give a link to how to subscribe, which is not turning it off.” – Sam (33:42)
• The restriction did nothing to limit harm, as explicit images still appeared in public feeds, and the paywall was later dropped.

App Store Double Standards

• Despite violations of Apple/Google policies on non-consensual and pornographic content, Grok remains easily accessible through mainstream app stores.

Reflections on X’s Devolution

• Hosts share personal noting that X (Twitter) is now flooded with “very extreme, very pornographic and in this case non-consensual” content.
  • “It’s as if you’re swimming in like the outbrain ad gutter ads of the Internet. It just [is] very extreme, very pornographic and in this case non consensual.” – Emanuel (39:14)
• Most hosts have stopped using X socially, visiting only for research or journalism purposes.

---

Memorable Quotes & Timestamps

• On WeBlock’s Capabilities:
  “You can draw a circle or another shape around a particular area…that's going to show…all the phones that we have in that location at that day or across that hour...” – Joseph (07:09)

• On Surveillance at Protests:
  “Cobwebs itself was saying, hey look, you could use this to monitor Black Lives Matter and protests.” – Joseph (11:54)

• On Data Sources:
  “SDK stuff has like gone out of fashion…[now] it's connected through real time bidding…companies can simply monitor that process...and that can include location data and a specific person's unique advertising id.” – Joseph (13:50)

• On Resistance to AI Image Abuse:
  “People go from app to app…for the past three to six months, the entire channel is devoted to Grok.” – Emanuel (31:00)

• On X’s Monetization Response:
  “It’s just pushing people to pay for it…You can pay $8 a month to still get these images generated in your feed…” – Sam (36:13)

---

Key Timestamps

• 02:56 – Introduction to WeBlock/ICE surveillance tool
• 05:30 – Detailed features and interface of WeBlock
• 11:55 – Discussion of use in monitoring protests
• 13:00 – Sources of commercial location data (SDKs vs. RTB)
• 16:16 – Connection to Minneapolis and DHS/ICE narratives
• 23:35 – AI image jailbreaking communities (Grok discussion)
• 33:42 – X’s response to AI image abuse and “monetization” angle
• 39:14 – On X devolving into unmoderated, abusive content

---

Conclusion

The episode delivers a thorough exposé of the shadowy world of commercial surveillance technology and its real-world impact, linking ICE’s potential abuse of location data tools with parallel issues of AI-generated abuse online. The hosts highlight the technical, legal, and ethical gaps in both U.S. government and big tech responses—showing how tools meant for public safety or engagement are repeatedly weaponized for harm.

[End of Summary]

Transcript

[A]

Foreign.

[B]

Welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist family company and needs your support. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404 Media co. I'm your host, Joseph, and with me are 404 Media co founders Sam Cole.

[A]

Hello.

[B]

Emmanuel Mayberg.

[C]

Hello.

[B]

And Jason Kebler.

[A]

Hello.

[C]

Hello.

[B]

I don't think we have any housekeeping this week, so I'm going to do.

[A]

A quick, quick zine update. So we are waiting on the last of the zines to come from our printer, which I'm expecting in the next, I don't know, eight to 10 days, something like that I think I mentioned. But we had to print many, many, many, many, many of them. And they're all being put together by hand by a small print shop. So I will have them in the next like eight to 10 days. I have all of the envelopes addressed, so once I get them they'll be in the mail. But I think like two or three people have asked. So if you happen to be listening to this and you happen to have ordered a zine, it will be going in the mail soon and we'll be putting a version online at some point as well.

[B]

Yeah. And you've been sort of cataloging the process, right? With videos, filming and printing. Have we posted any of those or we.

[A]

No, because I need to put them together, do the editing process and stuff like that. But that's sort of like that's where we are with it. But yes, we're going to do probably a YouTube video, some social videos. It's been a cool process and I think once you see it is like a very manual process, which is what we wanted to do for this first one. We wanted it to be like very human, very manual sort of thing and we're very happy. But a couple thousand of you bought them and so putting, doing. You're not really supposed to do this type of printing at this scale, I would say. And we did, but it's actually, I mean, we said we're going to ship them in mid January. It will still be mid January. It will probably be like in the next. Pretty soon.

[C]

Yeah.

[B]

When we put them on sale, it was very much a pre order situation. It was not a, hey, you're going to get the scene in three to Five working days or shipping or whatever, but definitely looking forward to that being out. Should we move to the first story? Jason, do you want to take the lead on this one?

[A]

Yeah, I mean, this is a very interesting story because the timing of you publishing it aligns with what happened in Minneapolis, which we will talk about, of course. So the story is inside ISIS tool to monitor phones in entire neighborhoods. Joseph, let's start with your story, and then maybe we can zoom out to the killing of Renee Goode. So, first of all, what did you obtain and what is this company? WeBlock.

[B]

Yeah, so I'll just say I think we did mention this tool briefly on the podcast before, like, because I found it in contracting US Procurement records. So it's like, hey, ISIS just bought this tool and you can sort of figure it out. But this is obviously new and much more in detail because I obtained material that explains in a very granular sense how these systems work.

[A]

Work.

[B]

So there's plenty more detail in this article than we had before, and that applies not just to ISIS potential use of it, but obviously any government user of WeBlock and another tool called Tangles as well. I think Texas law enforcement agencies have it, maybe some other ones around the country as well, regional, if not local. But WeBlock is a location data tool. It comes from a company called Penlink. Previously it was run by a company called Cobwebs. They merged with Penlink, blah, blah, blah, whatever. But weblog is very, very much focused on location data. So. And we'll go into how they get the data in a minute. But it can track movements of mobile phones, essentially, from one. One location to another. You could look in a certain area, figure out what phones are there and that sort of thing.

[A]

So there's no, there's no like, indication necessarily of where ICE is using this at, at the moment. Correct. Like, this is. This is sort of like how this tool works. You obtained some documents that explained what its specific features are. And as you mentioned, we knew that they had purchased this, but the, the news here is that now we know exactly, you know, some of its capabilities and what they can do. So what can an ICE officer do with Weblock and Tangles and what are some of the specific features?

[B]

Yeah, so it make. WeBlock makes it really easy for an ICE official to manipulate and use this location data from mobile phones. So whereas ordinarily, if a law enforcement agency got a bunch of location data, say from a company called Ventel or Babel street, which we've covered before as well, this tool and maybe those have this as well. But we, again, we've seen the material for this. So you can push a button on the interface that will turn it almost into nighttime mode, and you can see where the phones went at night. And the material explicitly explains that, hey, if a phone is at this certain location overnight, and especially, you know, repeatedly over and over again, maybe that's the person's home. If you put into daytime mode, you click a little button for that and you follow the phone to where it is located during the day. Maybe that's their place of employment as well. You can even click a little button which provides the route. So you can see not just sort of the start and the end locations, their home and their potential workplace. You can see the actual routes that they took. So presumably there would be what highway they drove down, what road they went down, maybe if they walked or whatever. You could probably figure that out based on the speed as well. So it really allows an ISIS officer or any law enforcement official using this tool to really easily drill down on what phones are there, where they go, and then, of course, potentially who they belong to. And I guess just the last thing I'll say on the features is that it's not like they just get a list of phones and they click through and they go one by one. They can do that, in a sense, but what they also do is they draw a circle or another shape around a particular area and. And that is going to show, hey, this is all the phones that we have in that location at that day or across that hour or whatever it is. So it really can be used for both quite targeted surveillance if the target happens to be in this data set, or it can be used for quite broad surveillance as well.

[A]

Yeah, I mean, I guess I want to be a little speculative here. I'm not saying you are. I mean, maybe you can talk about it, but with this tool, I mean, that's sort of the headline of the thing. It's like monitor phones in entire neighborhoods, so you can circle a whole neighborhood and it can give you information about the phones in that neighborhood. I mean, we have seen over the last week, and of course, honestly, over the last year, a lot of protests against ice, a lot of protests outside of ICE facilities, a lot of protests in, you know, neighborhoods where there have been clashes between people and ice, that sort of thing. I mean, it seems like an agency like ICE would be able to monitor protests sort of retroactively using a tool like this. Is there any indication that it could be used for something like that?

[B]

Yeah, I Think that's totally fair. And I'll say two things. The third, well, actually, maybe three things. The first is that there is some local reporting coming out of Minneapolis that I saw and I think Sam saw as well. I still need to digest all. But there are reports or suspicions that ICE DHS may be using data to go after activists and intimidate them in some way. I'm still digesting that, so I can't go much further in depth than that. But I would definitely highlight that report and come out of npr. I think the second is that the tool was technically purchased for hsi Homeland Security Investigations. That is, of course, a part of ICE that's usually focused on criminal investigations, child abuse, cybercrime, money laundering, all of that sort of thing. That's kind of gone out of the window window now because there's this really interesting data that the Cato Institute keeps getting from inside ICE showing that something like 90% of HSI officials have been pushed over to the deportation, or rather the deportation mission. So HSI's equipment of investigative entity basically doesn't exist anymore. It's basically part of ISIS deportation arm. The reason I bring that up, though, is that if there was some sort of, you know, alleged assault of an ICE officer or there was a protest outside an ICE facility and ICE wanted to investigate people there, in my opinion, it might be HSI that would do that sort of investigation. Right. Like, that would probably be within their mandate somewhat. And HSI is the one that has bought this tool. Jason, were you going to say something?

[A]

Well, yeah, I was going to say that they do a lot of workplace enforcement investigations. They actually always have, in addition to some of the child trafficking, drug trafficking investigations that they really specialized in in previous administrations. But we've seen this administration really focus on enforcement at workplaces. I mean, there's been videos coming out, you know, people getting snatched out of Target during their shift. There was the case where they seized a bunch of people from a Hyundai plant in Georgia, things like that. And so workplace investigations have been a big part of Trump's deportation campaign. And hsi, that's like what they have specialized in traditionally.

[B]

Yeah. And then the last thing I would say on the protest stuff is that the independent journalist Jack Paulson previously got some other material related to Cobwebs. Again, that was the company that first developed this tool before it moved to Penlink, I believe it was Cobwebs branded stuff he got. But that was basically, it sounded like marketing material or training guides where Cobwebs itself was Saying, hey, look, you could use this to monitor Black Lives Matter and protests. So that's not the same as ICE saying, or DHS saying, we're going to use it for this. But the people making, developing and selling the tool are explicitly telling potential customers, hey, you could use this to protest.

[A]

Right, right, right. Okay, so let's talk about where this data is coming from. I think perhaps we can't say exactly for sure where it's coming from, but you've done a lot of reporting on location data, commercial location data sales over the years and it seems like with many of these companies the data usually comes from the same types of places and that has to do with the advertising industry and it has to do with location harvesting from apps usually. So. So is that the case here? Is that what we expect is here?

[B]

Yeah, it looks like that's the case. It is not. I'm almost certain about this. It's not coming from telecoms like your AT&T, your Verizon's teacher mobile, that sort of thing. Police now need a warrant to go get location data from that after the landmark Carpenter 2018 case. This according to a internal ICE legal analysis the ACLU got and shared with us discussing location data. ICE believes it doesn't need a warrant essentially to get that information that lines up with other agencies. We've covered loads of secrets service. I bring that up here because in that legal analysis they were talking about Ventel, another of these location data companies, and Cobwebs and Panelink and weblock like they're in that space. So that leads me to there are two main ways that commercial location data companies get this sort of data. The first is software development kits, SDKs embedded in ordinary apps. So that weird weather app you got, the weird storm chasing one you downloaded five years ago and forgot about and gave location data permissions to that might have code in it, where the SDK developer then harvests the location data and then sells it to companies. The other one, and I think the much more likely one is that it's connected through real time bidding. So rtb, whenever there is an advertisement, a programmatic advert being placed inside an app in front of you, there is this near instantaneous and basically invisible to a normal person process where all of these companies are trying to outbid one another to get their ad in front of a certain demographic. A side effect of the online ad industry is that a bunch of companies can simply monitor that process. They don't even need to get an advert in front of some in front of somebody, they can just sit there and sort of siphon the data off. And that can include, and often does include location data and a specific person's unique advertising id. So there you have, hey, this person moved to here, they moved to here, moved to here. And also we have a unique identifier that connects all of that location data together. The reason I say I think that's the most likely is that I just think SDK stuff has like gone out of fashion in the location data industry. When I covered that, it was mostly like 2020, 2021, that sort of time when we exposed a company called X Mode that was collecting location data related to a Muslim prayer app and selling it to U.S. military contractors. That was like one of the last SDK based location data companies that I really remember. Yeah, there were a couple more afterwards, but it just seems that RTB is the way now. We got that from a hack of Gravy, the parent company of Ventol. It showed a bunch of apps that were subject to RTB data harvesting. We published a big list of those. We've covered another company called Patterns out of Israel, and we got them kicked out of the Google advertising ecosystem after we showed, yes, they were getting data from RTB. So I think that is most likely how WeBlock is getting its data and by extension how ICE is getting that data.

[A]

Yeah, Joe, So this is obviously all kind of happening in the context of broader crackdowns by ICE and then of course the huge national conversation spurred in the killing of Renee Nicole Goode in Minneapolis last week. You want to talk real quick about how we covered that with this piece? DHS is lying to you.

[B]

Yeah, it's difficult to cover because of course we're not on the ground and we don't feel like we have to cover everything. But of course this case is monumentally important. For me, it was the disconnect between seeing these videos come out of the shooting of Renee Goode first from sort of angles further away, and then actually later after we published the first person perspective of Jonathan Ross, the ICE official themselves. Obviously a huge disconnect between what those videos showed and the statements that dhs. DHS head were putting out the Trump himself as well. And they just simply did not line up. I think people intuitively who probably listen to us and read our website probably agree with that, probably saw the same thing. And subsequently you had Bellingcat come out and do amazing technical forensic analysis. There was New York Times as well. They did their own sort of visual investigation as well. I think those are always good and they're always amazing to see and they're always helpful. Here. You did not need Bellingcat of the New York Times to tell you what was going on. You could watch the videos with your own eyes and that's what we encourage people to do. Provided links to them in the piece, explained what we thought and happened based on those videos. And of course, as the headline says, DHS is lying to you and absolutely believe that based on the statements and the videos themselves. Let's. Sorry, were you going to say something?

[A]

I was just going to say, I mean, this is going to continue to be probably one of, if not the most discussed stories this year. So we're going to stay on it. But yeah, let's leave that there and move to the second half of the show.

[B]

Absolutely. When we come back, Emmanuel and Sam are going to talk us about, talk us through some of their recent GROK and X coverage. We'll be right back after this.

[A]

If you've ever thought about upgrading your window treatments but immediately stopped because you didn't want the hassle, I get it. Window shopping sounds fun, but in practice it's totally brutal waiting around all day just to get a quote, then getting hit with a price that makes no sense. That's why I've been really impressed with blinds.com they completely changed the game. You can shop custom blinds and shades 100% online, build a quote in minutes, or talk to an expert through a virtual design consultation on your schedule, not theirs. If you know what you want, great. If you don't, their design experts actually help you figure it out. They offer professional in home measure and installation services without the pressure or showroom markups. Blinds.com sends samples straight to your door so you can compare colors and materials at home. And it's not just blinds. They've got everything from bamboo shades to shutters and outdoor patio shades. Blinds.com has been around for 29 years and has covered over 25 million windows. And every order is backed by their 100% satisfaction guarantee. Right now, blinds.com is giving our listeners an exclusive $50 off when you spend $500 or more. Just use code 404MEDIA at checkout. Limited time offer rules and restrictions apply. See blinds.com for details on today's podcast.

[D]

We ask, is the litter box the worst thing about having a cat? The answer is of course, but it doesn't have to be. Thanks to Boxee. The boxy in Boxy pro stands for probiotics, which stops the bacteria that causes odors so you Never smell your litter box ever again. Boxy Pro keeps the box continuously odor free. Infinitely. Just remember to scoop. All you do is top off the litter and you never have to dump out the whole box. It has amazing clumping power that makes scooping easy easier. You know, I've got three cats at home. Gus, Lola, Bruce. I love them. But the litter box? It's the worst part. But it doesn't have to be. And that's thanks to Boxy Pro. If you're tired of switching litters looking for the one, get Boxy at box I E C A T.com it's the last litter you'll switch to. Enjoy. 30% off with the code 404 Media at Boxycat. 404 Media. That's B O X I E C A T.com forward slash four hundred and four media.

[A]

We're the generation that watched everything change. Jobs, markets, the rise of the Internet, the economic crash. And somehow we were expected to just figure it out. Now. Throw in the uncertainty of AI, tariffs, global strife. It's hard to have any certainty in this world. Those of us in our 30s and 40s have been forced to take matters into our own hands to protect our financial future and and present. Those of us in our 30s and 40s have been forced to take matters into our own hands to protect our financial present and our future. But if you're like me, maybe you never thought about life insurance. I've always felt invincible. But I got married recently and I had to start taking these things seriously. Which is why I've been shopping for life insurance with select quote to give my family some financial certainty in a world that isn't. I've been surprised that policies are cheaper than I thought. The younger you are, the cheaper it is. Like double the price if you wait to buy. A decade from now, if you're new to life insurance, you're not alone. For over 40 years, SelectQuote has helped more than 2 million Americans understand their options and get the coverage they need. Over $700 billion in coverage and counting. As a broker, their mission is simple. To find you the right insurance policy at the best price. Selectquote takes guesswork out of finding the right life insurance policy. You don't have to sort through dozens of confusing options on your own. Instead, one of their licensed agents will find the right policy at the right price for you. Comparing plans from trusted top rated insurance companies to find a policy that fits your health, your lifestyle and most importantly, your budget. And they work for you for free. Life insurance is never cheaper than it is today. Get the right life insurance for you for less and save more than [e-mail] 404 Media save more than 50% on term life [e-mail] 404 Media today to get started, that's SelectQuote.com 404 Media.

[B]

All right, and we are back. Emmanuel, we'll start with you. The headline of this piece you wrote is Inside the Telegram channel, Jailbreaking Grok over and over again. So on the podcast last time we spoke about Jason's piece and GROK and how it's been using to undress people, put people in bikinis, also some straight up child sexual abuse imagery as well. So how does this Telegram channel differ to what we were sort of talking about last time?

[C]

The short version is that it? The users there and the prompts they use to create images with Grok that it's not supposed to create are much more sophisticated. The prompts that we saw on X were very straightforward. They were simple texts like put her in a bikini, take her clothes off, things like that. And I think we can say now, based on the reaction from Musk and X and mostly the things that they are not doing in response to this, that they consider this to be pretty much a legitimate okay use of Grok on the Telegram channel. The reason it exists, it is part of a larger community where people create non consensual AI generated images of real people. And there are various rooms within the community where people focus on different celebrities, they focus on different types of what they call edits which are using Photoshop, using AI to manipulate images. Then there's one channel that is just dedicated to prompts for different AI image generators and basically how to find loopholes within them in order to produce non consensual images with mainstream tools that are not supposed to do that. We've reported on this thing many times. The most notable example was at 2014 when this image of Taylor Swift went viral. People were using Microsoft's designer tool in order to make that image. And the loophole there was basically misspelling her name and describing sexual acts without using any explicit terms, which is kind of like the first line of defense that AI image generators have to prevent abuse. But it is a primitive, not very good way to stop that. And those filters get better. But this community where that thousands of people participate keep coming up with extremely elaborate, more sophisticated prompts to do that, which can produce far worse things than what we've seen on X. On X We've seen sub nudity images of women in bikinis. And this is straight up hardcore porn images as well as videos.

[B]

Gotcha. Yeah. I'm glad you brought up the Taylor Swift one, because it's interesting. Whereas before with that, you found this telegram group and you sort of drew a fruit line between this group abusing a certain tool and then making those images and then sort of eventually bleeding out publicly onto Twitter. And obviously that was a huge case at the time. I think people continue to think about it. So there is a fruit line there. What's a little bit different here is like, in some ways there didn't need to be a through line because the prompts on X are just, hey, Grok, undress this. Like, it's barely a prompt for the. For sort of the. For the public stuff. But what you're saying is that in the telegram group there are still people who are actually making Grok do way, way worse stuff. It's just that it's not sort of reaching X as the public.

[C]

It's not on the surface yet, which I think is part of the deal. And there's probably two reasons for that. One is it does take time for this stuff to trickle out from these pretty closed groups on fairly obscure corners of the Internet to the mainstream, which is what happened with the Taylor Swift incident. So I think it's only a matter of time until you start seeing this type of thing in your feed. But then also the baseline for GROK is so bad that there's like less of a need.

[A]

Right?

[C]

So it's like you can get Grok to do pretty bad things without trying hard at all, as you said. So I think a lot of people don't even consider the possibility that you can make far worse images if you just went looking for the right prompt, but if you did, you'd find them.

[B]

Yeah, and I'm trying to think how to phrase this, but the fact that it's not the public version of Grok on X, generating straight up hardcore pornography of people works in the technique's favor, in it becoming basically a meme. Like it's abuse and it's horrible, but it's taken on this new character on X where even people, sex workers who are selling their only fans or whatever it may be, they're leveraging it, or other accounts are sort of piling in, like, oh, I wish Grok could unblur this video. And then Grok does it. And of course that's to drive engagement to an OnlyFans page or whatever. If people were using Grok to generate really hardcore stuff, it might not become a meme. It would just be like, oh, this is horrible, horrible abuse. You see what I'm getting at?

[C]

Yeah, yeah, yeah. And I guess just a few more things to note about the group. Like I said, I think we can say, judging on Grok's response, that it's just not taking this problem seriously and maybe doesn't consider it a problem.

[B]

Well, X's response.

[C]

X's response. Yeah, sorry. But you can see in the Telegram channel that GROK X, xai, they, they do have their limits in terms of what they would allow people to generate because the general flow of the chat is somebody comes in with a prompt that they find that works pretty well and is able to generate very graphic images. Everyone gets really excited, everybody starts generating it. I don't know exactly what is happening on the GROK end of things because they will not respond to our request for comment. But I imagine that they start to notice that there's a lot of bad generations and then they do some investigating and they close the loophole. I can tell that the loophole is closed because people start reacting and saying like, this is working less, this is working less now it's not working at all. Then everybody's like, brock is dead, dead game. And then somebody comes in with a new exploit and the whole cycle begins again. The other thing to note is the channel is dedicated to prompts. And in the couple of years that have been in there, people go from app to app. Right. Like we talked about Microsoft Designer, people have tried OpenAI, people have tried Google Gemini, people have tried all these different types of app. I would say for like the past three to six months, the entire channel is devoted to Grok. Like there is no other discussion of any other app because they're having so much success with this, with this one app.

[B]

Very interesting. You answered one of my other questions, so I'll just ask this before we move to Sam's piece, but what are people in the Telegram channel worried about when it comes to almost this wave of focus that people have on sort of the public GROK generations? Are they, are they scared something's going to happen because of that?

[C]

Yeah. I included a quote from the chat in the story where people were worried that all the attention that the images on X were getting were going to negatively impact the community and that either Grok would really lock things down or there would be increased interest in the exploits and somehow there would be like a crackdown on the community. And this is something Sam and I often see when reporting on this stuff and breaking news. I just went to check the community to see if there would be anything new to add to the podcast and it has been dissolved. It's unclear at this point if it has been banned or that they decided to move on on their own. But a few days ago, someone came in, one of the admins came in and they said, hey, follow this Twitter account, follow this subreddit, because we might shut this place down and we'll have to regroup somewhere else. And that has come to pass. The group has dissolved, the Telegram channel is gone. But I'm already in the new one. Right. So the problem continues. And this isn't the first time that this has happened. Like I've seen it happen with this community specifically, several times over.

[B]

Right, yeah. So it's not like they were moving to Reddit as a replacement for Telegram. They were using that as a bridge to get to the new Telegram because of course you need to point people there. That makes sense. Sam, let's just briefly talk about your one, which I think came after Emmanuel's. The headline was Masterful Gambit, which I do love. I hope people get the joke. I'm pretty sure they do, but I saw someone in my mentions being like, masterful Gambit. What you need to be more online. I'm sorry, to digest our material. Masterful Gambit Musk attempts to monetize Grok's wave of sexual abuse imagery. Really, really wild. What did X do here exactly, when it comes to monetization of this stuff?

[E]

Yeah. So all last week, and I'll try not to reiterate what you guys have talked about last week in the pod also. But all last week, this is like the only story that was happening in AI, specifically in any of my timelines, was the AI abuse imagery coming out of Grok. So on Friday when I logged on, I saw a couple outlets. It was the Guardian deadline. A couple others had headlines that were essentially Grok turns off image generator for most users after outcry over sexualized AI imagery. That's the Guardian. That's like Elon Musk restricted Grok. The images aren't able to be made anymore following all of this outcry against the bad uses of it that we just talked about. So I was like, well, that's interesting and probably not right, knowing how this platform operates and how its owner operates. So it turns out that actually what was happening was Grok was giving a reply to people who weren't X premium or like paid X Users saying that image generation and editing were turned off and limited only to paying subscribers to X. And then it would give a link to how to subscribe, which is not turning it off. And it's definitely not some kind of like accountability or mea culpa to the fact that it's been generating these horrific images. It's just pushing people to pay for it. And I would say, I mean, I was looking at like the searches for like, make her and bikini and, you know, clear tape, the things that Emmanuel was just saying on Twitter where people were asking Grok to do that stuff and it was like multiple times a second people were asking for this. So obviously Elon said, hey, here's a convenient way or someone on. I don't. I mean, I say Elon as shorthand for the ex monetization team, who should also be quite ashamed for all of this. But the. The move is to just push people to pay for the service, which is something that Elon's been trying to do specifically ever since he bought the thing is get people to pay for this service, X premium service, which gives you more boosts to replies and things like that. Gives you a little check mark, you know, whatever. So, yeah, that was absurd to me that anyone would report that it was turned off. And then pretty quickly after that, the Verge wrote a really good story about how it's not actually turned off. I was in the middle of writing mine when I saw that. So you can pay $8 a month to still get these images generated in your feed and show them to everyone is kind of a crucial part of this. It's not like only premium subscribers can see the images that are generated by Grok by other paying subscribers. Everyone can see them. They're still in everyone's feed. So what is it really like? It's not limiting any harm. It's not actually doing anything good or useful. It's just making more money for the platform. And as of. I just checked while we were talking, it's not the case anymore. Anybody can make these again. It's not giving that reply anymore. Yeah, I mean, it's. It never went away. Oh, my camera just died. It never went away. It's always been this way and now it's. It's fully back for people who don't pay for the platform. You can ask it to put her in a bikini or whatever other vile shit you want. And then a lot of the worst stuff is obviously on Telegram, like Emmanuel said, and on the app itself, which I've been trying to break for about a week with my own images.

[B]

But.

[E]

Yeah, that's where the really, the really nasty shit is.

[B]

So.

[E]

Yeah, yeah.

[B]

I mean from all the reporting we've seen recently, Musk has a real problem with sort of the blowback that this bikini Grok wave has received. Like he fundamentally seems, doesn't believe in it with his fundamentally misguided ideas of what free speech is and all of that, which we don't really need to get into. He seems to have a kindergarten kid's understanding of what that actually means. But from what you just said, it looks like, well, we'll just quickly move it to paid subscribers as a band aid. But actually no, I don't want that. I want to now go back and UK maybe investigating it. I think there's some ofcom stuff as well which is sort of this government body in the UK investigating. Still in the app stores. Right. I mean just very, very briefly, you pinged Apple and Google, right? I think asking them about this. Did they ever get back to you?

[E]

No, of course not. No, they never will. And these apps, this particular app, Grok is all over. It's really popular on Apple and Google on their app stores and Apple and Google very specifically forbid this type of content. Porn in general, but also non consensual pornographies against the rules. So yeah, I guess it's okay if it's a really popular app, which is usually the ML that we see in the app stores. It's like if it's, if it's making them money, it can stay.

[B]

Yeah. Or owned by, you know, a very, very rich guy and then he'll be allowed to stay.

[C]

Can I just ask the.

[B]

Yes, go ahead, please.

[C]

The incident has pulled me back to X just to look at it in a way that I haven't in a few months. And I shouldn't be surprised and yet I was surprised by, by how. God, like I don't even know how to describe it. It's as if you're swimming in like the outbrain ad gutter ads of the Internet. It just like very extreme, very pornographic and in this case non consensual. And it's like am I the only one who is not looking at X regularly at this point?

[B]

I mean I still check it.

[C]

Yeah, I think it's a, it was a good reminder that it's like I don't want to and it's not useful for me very much because I don't know, I don't know any like interesting people who post interesting things there. But I it was kind of a good reminder to go back and look at it because a lot of people do. And it's just a, obviously a very different picture of the world than what you get on on Blue sky or Mastodon or whatever.

[A]

I mean, I look at it for that reason for like what the is going on in this world reasons like usually for articles. But in my like spare time, I do not like hang out on Twitter in a way that I used to.

[B]

Yeah, absolutely. For the best, I think. All right, we'll leave that there. If you're listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 media subscriber, we're going to talk about a really, really wild story from Jason about Flock and police inadvertently exposing millions of license plates that they were investigating. And of course by extension people, you can subscribe and gain access to that content. 404 Media co. As a reminder, 404 Media is journalist founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the Subscribers only section where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope and Alyssa Midcalf. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really helps us out. This has been 404 Media. We'll see you again next week.