The Smart Glasses That Dox Strangers

Summary

The 404 Media Podcast: Episode Summary Title: The Smart Glasses That Dox Strangers
Release Date: October 9, 2024

---

Introduction

In this episode of The 404 Media Podcast, hosts Joseph, Sam Cole, Emmanuel Mayburg, and guest co-founder Jason delve into two compelling stories that highlight the intersection of technology, privacy, and misinformation. The episode begins with an exploration of modified smart glasses capable of doxing strangers and transitions into the repercussions of Hurricane Helena on a major climate data archive, further examining the role of AI in spreading misinformation.

---

Story 1: The Smart Glasses That Dox Strangers

Overview

Joseph introduces a concerning development where Meta's Ray-Ban smart glasses have been augmented with facial recognition technology to identify individuals and retrieve personal information such as home addresses and phone numbers.

How It Works

Joseph explains, “Somebody has taken the Meta Ray Ban glasses... they are Meta's wearable and they do all sorts of AR things as Meta... they do not perform facial recognition, but what somebody has done is basically tacked that capability on” (00:38-01:05). The modified glasses utilize Pimize, a facial recognition service akin to Clearview AI but accessible to the general public. By scanning a face, the glasses access Pimize’s database, employ large language models (LLMs) to interpret the data, and then retrieve detailed personal information from people search websites.

Development and Demonstration

The technology was developed by two Harvard students who demonstrated its capabilities by testing it on unsuspecting individuals in public spaces, such as the subway. Emmanuel highlights a real-life demonstration where a woman was correctly identified and approached based on scraped online information (01:58-05:19).

Reactions and Ethical Implications

The students initially found the project intriguing but later recognized its potential for misuse. Harvard Student 1 recounts, “...the reactions we got were pretty insane... 'how do you know, like my mom's like phone number or something... pretty weird'” (05:19-06:02). Harvard Student 2 adds, “...people were worried, like, oh, this is for stalker-ish reasons... like, young woman...” (07:39-08:30).

Host's Perspective

Joseph expresses frustration over the lack of responsible handling, stating, “...there are real reactions from real people who are being misled... I don't think that's responsible” (14:09-15:17). Sam Cole echoes these concerns, discussing the unsettling presence of such technology in everyday interactions and the potential for widespread privacy invasion (08:54-13:57).

---

Story 2: Hurricane Helena and the Downfall of a Climate Data Archive

Overview

The podcast shifts focus to the impact of Hurricane Helena on one of the world’s largest climate data repositories, the National Centers for Environmental Information (NCEI) based in Asheville, North Carolina.

Data Center Impact

Sam Cole narrates how Hurricane Helena caused significant disruption by knocking the NCEI data center offline. “It's a generational storm knocks out this huge data center... because they can't inhabit the building until they have water hooked up again” (23:57-31:02). The outage affected access to 60 petabytes of critical environmental data used by various industries, from agriculture to the freight sector.

Investigative Efforts

Sam details the investigative process to confirm the outage, including checking the Wayback Machine and reaching out to NCEI employees via LinkedIn. “...the federal building is closed and it has electricity but not water... the whole thing was shut down until further notice” (29:16-32:24).

AI-Generated Misinformation

Emmanuel discusses an AI-generated image that circulated during the hurricane, depicting a distressing scene with a flooded town and a sad child holding a puppy. This image was amplified by figures like Amy Kremer and Laura Loomer, blurring the lines between fact and fiction. “We're at a point where they don't care and we're just, like, wholly not we” (33:18-38:22).

Societal Impact of AI Misinformation

Emmanuel elaborates on the ease of creating and spreading AI-generated fake images, leading to a saturation of misinformation that diminishes the impact of fact-checking. He notes, “the sheer quantity of bullshit on the Internet is going to make it messy” (35:34-42:23). This diffusion of misinformation erodes public trust and complicates efforts to discern truth from fabrication.

Host's Concerns

Joseph voices his dismay, stating, “I didn't anticipate it. And it's really, really sad” (38:22-39:08), highlighting the troubling evolution of misinformation in the AI era and its implications for society and democratic discourse.

---

Conclusion

The episode concludes with a reflection on the profound implications of emerging technologies on privacy and information integrity. The hosts emphasize the necessity for responsible innovation and the critical role of journalism in uncovering and addressing these challenges. Listeners are encouraged to support 404 Media to bolster independent journalism efforts.

---

Notable Quotes:

• Joseph ([07:39-08:30]): “People were worried, like, oh, this is for stalker-ish reasons... like, young woman... some dude could just find some girl's home address on the train and just follow them home.”

• Harvard Student 1 ([05:19-06:02]): “The reactions we got were pretty insane... 'how do you know, like my mom's like phone number or something... pretty weird...'”

• Sam Cole ([08:54-13:57]): “I find the glasses very unsettling... I don't like interacting with them. And then this is just like another layer of, like, are you running some kind of facial recognition on me right now?”

• Emmanuel Mayburg ([35:34-42:23]): “The generative AI era of the Internet makes it easier to embrace that position and then also easier to just admit it... fact checking doesn't really work in this situation because you're choosing to believe the lie, which is what was true.”

---

This comprehensive summary captures the essence of the episode, highlighting the key discussions on privacy invasion through augmented smart glasses and the broader societal impacts of AI-generated misinformation, all underscored by insightful quotes and thorough analysis.

Transcript

[Joseph]

Hello and welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist founded company and needs your support. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content@404Media co. I am your host, Joseph, and with me of 404 Media, co founders, Sam Cole.

[Sam Cole]

Hello.

[Joseph]

And Emmanuel Mayburg.

[Emmanuel Mayburg]

Hello.

[Joseph]

All right, we have a lot to get through and I'm mad about this story, so I want to jump straight into it. This is one I wrote and the headline is, someone put facial recognition tech onto Meta's smart glasses to instantly dox strangers.

[Emmanuel Mayburg]

Joe, I'm going to ask you questions so you don't have to interview yourself. I appreciate technology. Exactly. And what does it do?

[Joseph]

Yes. So somebody has taken the Meta Ray Ban glasses, which is they are Meta's wearable and they do all sorts of AR things as Meta. Facebook is trying to move more into that world. They do not perform facial recognition, but what somebody has done is basically tacked that capability on and just by looking at somebody, it is able to identify, there's a face, find out who that face belongs to somewhat accurately, then find their home address and their phone number and potentially information about their family members as well.

[Emmanuel Mayburg]

And how does it do that? How does that all work?

[Joseph]

Yes. So under the hood, it is basically a hodgepodge of different technologies sort of strung together. So you'll look at somebody's face and it will then contact the facial recognition service pimize. I think we're all familiar with it. We've covered it in various different contexts. For those who don't know, it's basically a facial recognition service that anybody can use and anybody can access. If you're familiar with Clearview AI, that is a very powerful facial recognition tool used primarily, I wouldn't say exclusively primarily by law enforcement, where the police officer will point a phone at the face or they upload a photo and that will grab the social media profiles that that person belongs to. Because it's built on this massive data set of scraped social media images and other parts of the web as well. Pimize is basically that less powerful, but basically that for anybody who wants to pay 10 bucks a month or whatever it is. So these glasses, they contact that, they then use LLMs to interpret the results generated from that. Because what pimize does is that when you scroll through the results, it shows all these similar faces and it also shows the URL where that face was found. So maybe you look at somebody's face, it then brings up, I don't know, their university profile on the university website or something. You click that, presumably it might have their name. These LLMs will then grab that once it's been scraped and provide sort of a summary of who this person is. Oh, they went to this school, maybe they got this award, all of that sort of thing, and then takes that name, the most important part, and it automatically sends that to a people search website, which I'm sure people have seen these. If you literally Google Find xyz, if you're trying to identify, I don't know, a long lost relative or something, there's all of these websites that sell access to that sort of data. And it then grabs results from that as well. And that can include the home address and the phone number. So it's not very elegant. It's not really, I suppose, highly sophisticated. It is using consumer off the shelf tools and capabilities. But I think that is entirely the pointer. It is entirely the point that the two people who were able to do this were two Harvard students. This was not a big tech company, this was not a surveillance firm. This was two people basically doing it in their own time. And what these two students did was they tested it on ordinary people out in the world and their friends and they got all sorts of different reactions. And here is a voice memo that one of them sent me. Just about some of the reactions they got when they were actually demoing this technology out.

[Harvard Student 1]

As you can see from the video, the reactions we got were pretty insane. To be honest. I actually didn't think it was insanely cool as we were building it. The main thing that made me more inspired to work on this project was that as we made incremental progress on it and we're talking to like other Harvard students, random people on the subway, that type of thing, they'd be like, dude, holy shit, this is like the craziest thing I've ever seen. Like, how do you know, like my mom's like phone number or something like that. Like that's, you know, pretty weird. So yeah, I think the reactions of other people were actually like a large like feedback mechanism for seeing like, oh, this can be like pretty impactful.

[Joseph]

So just to stress what they've done is they've built these facial recognition glasses and then they've uploaded the demo where they go and test this on unsuspecting people on the subway. There's one in particular where one of the students is looking at the results. They identify this woman as Betsy somebody. They BLEEP out her surname. One of the students then goes up to Betsy and says, oh, hey, is this you? I think we met at XYZ using information that's been scraped from her online. And you can see in the video she is obviously receptive to that. It's obviously correct. And I also verified this because with the information in that video, I was very easily able to identify who this person was. She didn't respond to a request for comment. I just wanted to know what's your reaction to this being done to you without your consent? And I guess that sort of brings us to the motivation, which is that the students say that, well, I mean, you'll hear this in a second. But initially they did it because it's interesting and cool. They didn't bring up the dangers of this technology, which I'm sure we'll get into, and I'll ask Sam about that as well. But they brought up just how it was interesting first of all. And only later when they were speaking to other students, did they bring up sort of the context on how this technology might be used. So let's just hear some of that as well.

[Harvard Student 2]

The motivation for this was mainly because we thought it was interesting, it was cool. But as we showed people, a lot of people reacted that, oh, this is obviously really cool. Like, we can use this for networking. I can use this to, like, play pranks on my friends, make funny videos. But also a lot of people were worried, like, oh, this is for stalker ish reason. Like, if you just release this, like, it'll be really bad for certain populations. Like, oh, young woman, right? Like, some dude could just find some girl's home address on the train and just follow them home. So I guess, like, we went on to do the next step of like, making a guide on how to remove yourself from these searches, like fixing your digital footprint.

[Joseph]

Sam, you've covered a lot of stalking when it comes to technology. I mean, you've done a lot of stuff on Apple airtags, that sort of thing. What do you think of sort of that sound bite in. That initially started as a fun project and then sort of the stalking implications came later.

[Sam Cole]

I mean, I think it's very classic. And this is kind of what we see happen every time someone creates something, then gets a bunch of heat for it. People aren't thinking ahead about how it might impact other people other than themselves. People who have a very different threat model just moving around the world than they do. These guys probably don't ever think about being stalked or harassed or doxed when they're just walking around outside. But it's something that, like, especially women have to deal with all the time. Yeah. I don't know. It's very. I find it creepy. And it's also just like, of course. Of course this is. Of course this is what was going to happen with these glasses. I find the glasses very unsettling anyway. It's like, even just like, the stock things that don't have any of the facial recognition stuff in them. When someone's talking to me and they're wearing a pair, I'm very distracted. I'm like, what are you doing? Are you recording this? What's happening? I know that there's a light that comes on that tells you when you're being recorded, but still, it's like, I don't know. I kind of hope that these don't reach, like, some kind of mass saturation with consumers, because I don't like interacting with them. And then this is just like another layer of, like, are you running some kind of facial recognition on me right now?

[Joseph]

That's really interesting because I've never actually seen them in the wild, at least as far as I know when it comes to you. So you've had conversations with people that presumably own them and wearing them. I mean, you don't have to get super specific, but did they tell you, hey, check out these glasses, or did you just realize it was them? And the reason I ask is because I don't think many people even know these exist yet. And I'm just talking about the wider public that could be a victim of this, who are not technology people.

[Sam Cole]

Yeah. So I'm referring to. Because I think a new. A new version just came out. Right. And that's why it was in the news a whole lot. But there's an old version that is just like. They're just Ray Bans. They look like regular Ray Bans that meta released and they have a camera in them. Yeah. And it's. I mean, they're new enough that people are like, oh, my gosh, look at my. Like, check them out. And it's like, if you say something to somebody and you're like, oh, let me see your glasses. They'll. They're excited to show them to you. So it's not like a sneaky thing. No one's trying to, like, pull something over on me when they're talking. To me with these. But, like, it would be like having, like, having a conversation with someone with, like, they're just holding their phone, like, up next to their face and being, like, pointing their camera at you the whole time they're talking at you. Even though it's not on, it's not running, it's like, it's still strange to me, but at the same time, it's like, it is. The technology is kind of cool. Like, I get it, I understand why people are into it, but stuff like this was bound to happen, I guess.

[Joseph]

Yeah. I mean, I was going to bring this up later, but since you mentioned that, it would be like someone just holding a phone right up like this, and that's very obvious. And socially we would find that very, very fucking weird. And that sort of leads to some of the responses I got after publishing this, which is that some people said, well, you could do this with any camera. You could do it with, like, a dslr, you could do it with a phone camera. You could set up a security camera. And I'm like, okay, cool. But they did it with meta Ray Bans, and it was on their face. You don't see people strapping a fucking DSLR to their face and walking because they're like, well, that guy's a freak. He's wearing a camera. But with the Ray Bans, it is more subtle. Even though there is a light there that does indicate that somebody's recording. And apparently if you cover it up, it disables the ability to do that. I don't think there is enough adoption for members of the public to realize that necessarily.

[Sam Cole]

Yeah. And in a place like New York especially, it's like people are wearing and doing all kinds of weird shit all the time. So I might not necessarily on, like, a crowded subway train, notice someone's glasses with a little light on them, but I would definitely notice if someone was pointing a camera at me. Like, that would be very strange and unusual. But I'm not looking directly at people's glasses for a light most of the time. So, yeah, it's a totally different thing. I think people pretending that it's not different are, I don't know, like, playing ignorant on purpose or something. It's very strange to me to even pretend that this is the same at all as a regular camera or a cell phone.

[Joseph]

Yeah. I think they're reducing it to the literal tech when it's like, no, there's a massive social component here.

[Sam Cole]

Yeah. And that's the entire point of the product. Right. To wear it and to blend in and to not have it look like a camera on your face. If that was the case, we would be wearing Google Glass.

[Joseph]

Exactly. And we're not wearing Google Glass because everybody knew it was really, really strange and they looked odd. I mean I actually know some people who hack Google Glass now and that's. I think that is cool.

[Sponsor]

That's cool.

[Joseph]

But yeah, so I'm just going to play one more piece of audio and it is about their view on how they did this responsibly because they did produce a guide that says how you can opt out of facial recognition services like pimize out of the people search sites and they have not released the code necessary to do that. I mean they've described the off the shelf tools that you can do and go and use but they haven't released the code and sort of strings it all together. And there was, I imagine and what it sounds like a significant amount of work involved in that as well. So let me just play this last little bit of audio as well.

[Harvard Student 2]

What we did was the most responsible we could do. Basically we show people that this exists, that this works, it's aware so that everyone is aware of this now and like we're not open sourcing it so that there's some high barrier of friction. Like you have to know how to code, you have to spend like quite a few days like optimizing the system which is what we did.

[Joseph]

I'm going to go on a tiny little bit of a rant, I'm sorry, and I'll try to keep it short but I don't think they fully understand what they built and sort of the social contours around that and the privacy implications of it necessarily. You heard earlier on, it started as a fun, cool, interesting research project and then the implications came later. They could have, if acting responsibly, absolutely done this in a controlled environment. Whenever we read about, I don't know, security researchers who discover a really interesting new attack to get your password and keystrokes by listening to the rotations of the fan in your computer. Ridiculous stuff that would be really, really crazy if it was out there in the real world. It's in a lab environment, they are doing it there so they don't impact anybody. They don't typically release a fully working POC proof of a concept exploit for people to be able to go do it. It's in there. If it's done on anybody, it's in that controlled environment with consent similar to when how we've bought the location of somebody's phone because it was being sold by bounty hunters. We did that with explicit consent of the target. And you're still able to, I think, very well demonstrate what is going on there. What is so unsettling about this is that you watch the video and there are real reactions from real people who are being misled. And, you know, I don't think that's responsible. And I think that is just as much as part of the story. The fact that two students can quite quickly build this, and they are people who do not necessarily understand all the privacy implications, the fact they can build it is the story. It's not even really about the tech. It's actually about who made it and sort of how they handled that. And I mean, just briefly, after we published, one of the students said, you know, they said, quote, we tested on unknowing people to demonstrate this tech actually works. And we did it to strangers in a relatively wholesome way, acting as if we were fans of their work. I don't think you can break consent in a wholesome way. I don't. I don't think that is a responsible way to do privacy security research. And I mean, that was reflecting some of the other coverage as well. I really can't bother to pick out particular outlets, but of course, after we broke this news, various other outlets dropped on it, and some of them framed it as just a research project. They didn't bring up that there was no consent involved. They didn't bring up that it was just initially sort of an interesting, cool project. And I think that's absolutely crucial for understanding. I guess, just the last. I mean, you can tell that I care about this issue a lot. And it's true, and I really think it boils down to this, is that I take quite extreme steps to protect my privacy and security. As listeners will know, I don't have a mobile phone. All of this sort of thing. It's hard with facial recognition because you can't change your face. Well, not trivially. It's hard and expensive, and I don't want to. And I find this sort of activity genuinely concerning. A society shifting for what it might mean for us in the future. Basically, if they can make it in a pretty quick way, what is going to happen when somebody commercializes it, streamlines it, and then everybody is potentially walking around with a device that can pick out people's faces and their real identities, or even just the thought that that is possible, essentially. Anyway, rant over. Let's take a quick break. And we're going to shift gears entirely and talk about the recent hurricane and how that relates to digital archives and AI and all sorts of other sort of implications you may not immediately think of. We will be right back after this.

[Sponsor]

This show is sponsored by BetterHelp. It's Halloween, which is a good time to admit that I'm still irrationally scared of the ghost stories I heard as a kid. Spooky season is a good time to face our fears, especially if they're more real than, let's say, Bloody Mary. Therapy is a great tool for facing your fears and finding ways to overcome them, because sometimes the scariest thing is bottling up our fear and not talking it out. I've been using BetterHelp for the last month to learn strategies to deal with my anxiety and to get at the root cause of it. No matter what you're dealing with, therapy can empower you to be the best version of yourself. If you're thinking of starting therapy, give BetterHelp a try. It's online and really flexible. Just fill out a brief questionnaire to get matched with a licensed therapist and switch therapists at any time for no additional charge. Overcome your fears with BetterHelp. Visit betterhelp.com 404Media today to get 10% off your first month. That's BetterHelp. H E L p.com 404Media this episode is sponsored by Shopify. We've taken a DIY approach to everything at 404Media, which means we didn't want to do dropship merch and wanted to handle everything ourselves. But I didn't really know how to set up an online shop. Then I found Shopify. Shopify makes it easy for me to manage our listings and inventory, sell online or in person, print, shipping, labels, process returns and get paid. All with a beautiful front end for customers and a simple back end for us. It's a one stop solution that gives us everything we need to manage our online store. Upgrade your business and get the same checkout we use with Shopify. Sign up for your $1 per month trial period at shopify.com media. All lowercase go to shopify shopify.com media to upgrade your selling today. Shopify.com media I love a great deal as much as the next guy, but I'm not going to crawl through a bed of hot coals just to save a few bucks. It has to be easy. No hoops, no bs. So when Mint Mobile said it was easy to get wireless for $15 a month with the purchase of a three month plan I called them on it. Turns out it really is that easy to get wireless for $15 a month. The longest part of the process is the time you might have to spend on hold breaking up with your old provider. Thankfully, Mint Mobile's easy to use website makes it super simple to buy and activate your plan using the phone you already have. To get started, go to mintmobile.com 404media Right now, all three month plans are only $15 a month, including the unlimited plan. All plans come with high speed data and unlimited talk and text delivered on the nation's largest 5G network. You can use your own phone with any Mint Mobile plan and bring your phone number along with all of your existing contacts. Find out how easy it is to switch to Mint mobile and get three months of premium wireless service for 15 bucks a month. To get this new customer offer and your new three month premium wireless plan for just 15 bucks a month, go to mint mobile.com 404 Media that's mintmobile.com 404 Media cut your wireless bill to 15 bucks a month at mintmobile.com 404 Media $45 upfront payment required. Equivalent to $15 a month. New customers on first three month plan only. Speed slower above 40 gigabytes on unlimited plan. Additional taxes, fees and restrictions apply. See Mint Mobile for details.

[Joseph]

All right, and we are back with one that Sam wrote. Hurricane Helena knocked one of the world's largest climate data archives offline. How did this first come to your attention, Sam?

[Sam Cole]

Yeah, first of all, Helene, I think.

[Joseph]

I was, I was gonna, during the break, I was gonna stop and be like, wait, can you just check my pronunciation of this? I suck so badly. Okay, thank you for correcting me. I appreciate it.

[Sam Cole]

Yeah, it's fine. I think it's Hurricane Aline. But yeah, so obviously like my whole entire timeline, my whole feed, like all my social media feeds for the last week and a half have been hurricane related. And just like the total chaos and destruction and like tragedy that's happening with hurricane recovery in Florida and the Carolinas and Georgia and just all the kind of like, I guess like unexpected ways that this has impacted lots of different systems. I mean we all saw like the chemical plant that's like spewing plumes of smoke out into the air because of damage, because the hurricane. And then this story which I saw someone who's a climate journalist tweeting about on the 29th where the National Centers for Environmental Information, which is like a really big archive, I think it's one of the world's biggest archives of environmental data is actually based in Asheville, North Carolina, which anyone following anything about this news lately knows that Asheville was one of the hardest hit areas by hurricane. So it's just like, it was very like ironic to me almost that a huge repository and resource for environmental and climate change data was knocked out by like a once in a lifetime hurricane, hopefully. I mean, once in a lifetime and then there's another one coming, like literally right now. But like once in a, like a generational storm knocks out this huge data center, which. Yeah, it's just. And then I went to the website to check if it was up and it was like returning a 404 error, which.

[Joseph]

Well, yeah, we'll get to that in a second. I mean, you just mentioned that it stores this climate data. What exactly is this data? And sort of, you know, where does it come from and what are they measuring? Sort of.

[Sam Cole]

Exactly, yeah. So I mean, it's like, it's honestly, it's literally everything based on my readings about it or what I could read about it through like going through archives. Because it was all down when it wasn't down.

[Joseph]

Yeah, well, it was still down.

[Sam Cole]

Yes, yeah, while it was still down. So I mean it's. I learned that it contains 60 petabytes of data, which is a lot of data. It stores and releases data from I think their tagline or the thing that they repeat the most is it's from the seafloor to the, it's like from the seafloor to the moon or something. It's like space, it's like the depths of the ocean to like space is the range of what they can collect and they make all of it available to people. And it's not just like, oh, let's study like historical data of climate change. It's things like, like farmers use it to determine when and how to plant crops and fishermen use it to determine like where the best places are to fish, like on a commercial level. So it's just, it's this thing that's like very useful on a very everyday scale for a lot of people. It's like, it's something that the freight industry uses. And for it to be down, I think was pretty disruptive. Not just for researchers who are obviously very interested in the current hurricane season, but also how it looks historically. But people just doing normal jobs around the world basically weren't able to access that information. And then this, this climate journalist was, he had downloaded the list of billion dollar disasters from the center before it went down because he was writing about hurricanes, I guess, and published it before it went offline. And now at this point, I don't know what the tally is at this point, but Helene is definitely probably going to be on that list.

[Joseph]

Yeah, it's interesting that there's these side effects, basically where, I mean, maybe if you're in the know, you would think this, but I don't think you'd necessarily assume that, oh, it's going to impact data archive and then that's going to have an impact potentially on industries and all of that sort of thing. I mean, you mentioned that to check it was down. I mean, you went there and the site and it was down. Right. But like, what else did you do? I think you tried to reach out to people who work there and that sort of thing.

[Sam Cole]

Yeah, I mean, I checked the Wayback Machine on Internet Archive to try to figure out when it stopped scanning automatically the site, because that service automatically archives lots of websites all over the Internet and that's one of them, obviously. So it stopped. I saw that it stopped doing that on the 27th and I'm looking at it now, and it only started again on the 2nd of October. So I guess that's probably when it came back up those days. But yeah, and then I was like, well, I need someone to confirm this or at least tell me why, maybe why this is because maybe the building's not flooded. Maybe there's some other reason, maybe the power is out or something. So I was on LinkedIn just looking for people who were working for. What's the abbreviation? Ncei, who are working for the center who weren't based in Asheville, because obviously those people don't have power either. So it's like if their workplaces and like their workplace is underwater, they probably personally are dealing with more important things than my confirmation of their data center. Yeah, just having to go through LinkedIn and like find people who worked in, like they have like a Chicago office. They have a couple of different offices. And finally someone replied and passed my info on to other people and then kind of did a little telephone game from there. And then someone got back to me and was like, this is what's going on?

[Joseph]

I mean. And I guess just the last question is, did they provide an answer on what exactly happened? I mean, obviously hurricane happened, but I mean, did they get any more specific than that?

[Sam Cole]

Yeah, I mean, I heard some theories that I still need to go back and confirm with folks now that they actually are getting back on track. But one of them was someone who works there, but wasn't directly involved in the downtown Asheville location. Was like, well, I think it's because the data. Data center needs water to cool it and they don't have water. And I was like, well, that seems plausible. And then someone who works there in a more direct way, he was Dan Kowal. He's a Standards and Evaluation branch chief at the Chicago office. He told me that the federal building is closed and it has electricity but not water, which goes along with something someone else told me, but it's because they can't inhabit the building until they have water hooked up again. So the whole thing was shut down until further notice. And now, I mean, it's back up now. And if you go on the site, it says NCEI in Asheville has been significantly impacted by Hurricane Helene, but it doesn't say what exactly happened. I assume the data is backed up and safe, but I need to circle back with those folks.

[Joseph]

Yeah, sure. I think I've listened to this name in Google Translate while you were talking. So Hurricane Helena and the fuck it. Era of AI generated slop. I know you're doing a face. I'm continuing. It's what Google said.

[Sam Cole]

All right, I'm ag. There's no way.

[Joseph]

Everyone said. I literally just listened to it in Google. Anyway. Okay, so Jason wrote this story. He's not here. Emmanuel, I think you edited it potentially, but there. And it's a fantastic piece. And I actually, really, really recommend everybody just go read the full thing. Jason's very good at articulating all of this. But, Emmanuel, there's an AI generated image at the center of this story. Why are so many people talking about it? And I guess just what is that image, first of all?

[Emmanuel Mayburg]

So the image is a little girl who looks very sad and is very wet. She's in a life vest. She's holding a puppy who is also very sad and very wet. They're in a boat. They're in a body of water that clearly looks like a flooded part of a small town that looks like America, maybe. This could be North Carolina, where we saw kind of the heaviest rainfall that caused a lot of damage in North Carolina. And the reason. I mean, this is just something that happens. Now, Jason notes this in the piece, but there was some intense flooding in Brazil a few months ago. And our friends at Nucleo, which is a Brazilian tech publication, they did a story about how there was a bunch of AI generated images allegedly from that flood that Facebook didn't remove as being fake and people thought were real. So this happens all the time now. It happened here after Helene. And the reason that people are talking about it, because there are so many of them, like, you could be like, why aren't we talking about all the time? Is that it got boosted by some pretty notable people. So Amy Kremer, who has a job at the RNC and is the co founder of Women for Trump, she tweeted it as if it was real. And then from there, like a bunch of other Trump world weirdos. Laura Loomer, people are heavy into Trump lore, know her from many years ago, but she's. She's back in the fold now. So she tweeted it and yeah, there's several versions of it. A few of them went around, but they all kind of have the same look and feel.

[Joseph]

Yeah. To be honest, I'm sure many people have maybe come across it without even trying. It feels like it's absolutely everywhere. So, as you say, this is happening a lot and it slightly different, but it almost reminds me of that viral image that goes around anytime there's an unfortunate natural disaster, like a flood or hurricane, which shows the sharks inside the shopping mall that comes up. I haven't actually seen that this year.

[Emmanuel Mayburg]

We should go back to that. That's a good. I was going to bring that up as well.

[Joseph]

Yes, okay, we'll come back to that. But then. So why are we saying that this image is what is ushering in basically the fuck it era of AI generated sloth?

[Emmanuel Mayburg]

So the first time I saw the image, and I think I dropped it in Slack when I saw this, there was a tweet going around. It has since been deleted, I think. But someone shared like a text conversation with a family member, I imagine a dad or an uncle or something like this, and they shared the image in the chat and the person said, like, hey, this is AI generated. It's not real. And the person who shared it, their response to that was, who cares? Or I don't care, I think was the exact wording. And then Amy Kremer, who tweeted this, who I just mentioned, people were replying to her tweet and saying the same thing, and her response was similarly like, it doesn't matter that it's fake, because it is speaking to some underlying truth about how this disaster is at the feet of Joe Biden and the Democratic administration. So we're at this point now, like, we talked a lot about AI generated images on Facebook, where this is going around as well, and talking about how to, like, tag the images. And Jason did a ton of Reporting about where they come from and why people do this, and all of it in sort of an effort to show that this is a problem that is getting worse and is polluting the Internet. And I think what we imagined would happen is that it would be hard to distinguish between fact and fiction. But the twist that I don't think we really expected is that it's not that people can tell the difference is that we're at a point where they don't care and we're just, like, wholly not we. But, like, some people are wholly embracing AI generated images and presenting them as if they are real, even when they know they're fake. And we saw some of this, I would argue, a little earlier with Trump and the whole Springfield, Ohio thing, where, you know, he told this lie about Haitian immigrants eating pets, and then a bunch of people made AI generated images around that subject, and he retweeted it. And, like, did he tweet it as if he thought they were real? It's unclear, but it's also signaling this embrace of AI Slop.

[Joseph]

Yeah, I mean, it's depressing, man. Like, when I first saw that, as you say, the tweet about the family member, like, saying they don't care or whatever, it's just like, that really hit at home that we're just entering really a new societal phase of AI. And I really think it's like, the most depressing one yet, to be honest. And I didn't anticipate it. And it's really, really sad. And I don't know if this is why you wanted to bring up the shark as well, or maybe that's something else. But how is this different this time from other sorts of misinformation? Or is it different? Or is it now that we now just don't care? What do you think?

[Emmanuel Mayburg]

I think it is different. So the shark thing, that happens every time there is a storm, and to people who are unfamiliar, it's like. I think it's like a flooded highway and there's a shark in the water, and it's. If you think about it for, like, half a second, it's obviously fake, but it keeps popping up. And I would say the thing about that is that. And this is how the Internet used to be, but it's like somebody sat down and, like, photoshopped that, and it's not, you know, Hollywood quality cgi, but it's like, it's kind of good, and it takes effort. Like, I couldn't do it. You know what I mean? I'm not good At Photoshop, I couldn't sit down and do it.

[Joseph]

Yeah, it was, you know. Yeah.

[Emmanuel Mayburg]

And then I also. And we linked to this story in the piece, but Sam wrote in 2019, and this is when, like, a video was slightly edited to make Nancy Pelosi seem like she's becoming Cena or something was going around. And she wrote this piece about how it's very obviously fake, but it also doesn't matter because people are just going to pick up and run with whatever supports their priors. In this case, Republicans just thinking that she's a bad speaker of the House at the time. And this is definitely on a continuum with both of those things. But the difference, I think, is just like, how easy it is to make. Right? So it's like there's this one image, but. And we all know how AI works by now, but, like, once you have the prompt for this image, you can just keep clicking, generate, and you get other versions of the image. And there's a ton of these. So you can't just like, cling to this one image of a shark in a highway that goes around. It's just like, you can make as many as you want. And it just creates this, like, cloud of bullshit, which I think is maybe where people are worn down that to the point where they can say, like, well, who cares? I don't care if it's AI generated. There's a ton. I'm looking at AI generated stuff all the time. I am just embracing the AI generated image that supports what I already believe. And I just think that the generative AI era of the Internet makes it easier to embrace that position and then also easier to just admit it. Right. It's just like whether you're a politician and you're admitting it to the public, you're admitting it to people who follow you on Twitter and trying to point out that you made, like, a factual error or whether you're just talking to a family member who is sharing something that they saw on Facebook. It's like fact checking doesn't really work in this situation because you're choosing to believe the lie, which is what was true. And like, this is what Sam was saying in her piece in 2019. It's just that, like, a deep fake or unedited video is also harder to generate with this than these AI generated images. So I think it's just like the quantity and like the overall amount of bullshit on the Internet that that brings us to this point, which is kind of what we've been warning about ever since we started covering this stuff. It's just like the sheer quantity of bullshit on the Internet is going to make it messy. And like, here we are.

[Joseph]

Yeah. I mean, they're basically saying, who cares if it's fake, it's real. Which is an insane statement. That's what they're saying.

[Emmanuel Mayburg]

Who cares if it's not a fact? It's. It's like it's how I feel. Right. Yeah, well.

[Joseph]

Or they fit well, I mean, maybe that or maybe who cares if it's AI generated? It gets across the real sentiment or echo of what is happening. And it reminds me of this really good piece I read in the Atlantic called Fact Checking Is Not a Political Strategy. And this isn't about AI really, but it's just more broadly about this shift to fact checking politicians in the run up to the US election and they're spouting all these lies and bullshit. And news media and journalists see it as their role to correct the record. And of course, broadly I agree with that. But it doesn't fucking matter anymore. And I think that's a vital piece of context as well. And I mean, you mentioned the Brazilian case and it'd be great to hear more just as we continue to cover it. Is this happening internationally or is it a U.S. problem? And I'm sure it is happening internationally. I'm just saying there's also a very particular US political context where everybody is so divided now and nobody gives a fuck and it's a massive epistemological mess, basically. All right, we will leave that there. If you are listening to the free version of the podcast, I will now play us out. But if you are a paying 404 media subscriber, we're going to talk about a hacked AI companion site. You can subscribe and gain access to that content at 404 Media co. As a reminder, 404 Media is journalist founded and supported by subscribers. If you wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our article and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really does genuinely help us out. This has been 404 Media. We will see you again next week.