Loading summary
Joseph
Hello and welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist founded company and needs your support. To subscribe, go to 404Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404 Media co. I'm your host, Joseph, and with me are 404 Media co founders Sam Cole. Hello. Emmanuel Mayberg.
Emmanuel Mayberg
Hello.
Joseph
And Jason Kebler.
Jason Kebler
Hi.
Joseph
I'm not gonna lie, I went on complete autopilot while reading out that intro and then you start having the metaphors of wait, I'm still talking, but did I do the line correctly? And then, anyway, time has passed.
Jason Kebler
Perfect. When you, when you've out, when you've been out, I've tried to remember what you say and it's, it's hard to remember.
Joseph
That's why I literally read it one episode.
Jason Kebler
I was just like, hey, what's up? And didn't intro anyone or the podcast. Wait, you've been reading it? I thought you did it from memory to be totally.
Joseph
No, no, no. I read it every single time. There's a Google Doc.
Jason Kebler
Got it.
Joseph
Not that impressive. Let's start with a story from Emmanuel. AI powered buzzfeed ads suggest you buy hat of man who died by suicide. Before we get into what went wrong, Emmanuel, which is obviously quite clear in the headline, but what is this AI based advertisement system that buzzfeed is using?
Emmanuel Mayberg
So I want to explain this by first talking about affiliate links because I think most listeners are probably familiar with this. Affiliate links is when you go to a website, you read a story and whether the story is a review of a product, a bunch of products, or is not a review of any kind, but happens to mention a few products, you can click on a link that takes you to an Amazon store page or some other online retailer and if you end up clicking those links or buying an item via those links, then Amazon makes a sale and the publisher that included the affiliate link gets a very small cut of that sale. And this is something that's become a popular like a, an effective way for publishers to monetize in recent years, very common. I'm sure people have seen it on buzzfeed advice where we used to work other websites, what this company called Trendy, which is based in Australia.
Joseph
What it does spells T R E N D I I, which is an interesting spelling.
Emmanuel Mayberg
Yeah, I gotta do it for the SEO, you know, so this trendy company, they are basically taking the same scheme, but trying to do it for images rather than text. So you'll read a story, the story will have images in it. They have an AI model that recognizes objects in images. It tries to match the, let's say, shirt or shoes in an image to a participating trendy retailer. And you can click on a button on the image that says shop this image. It will bring up a bunch of products that look like the items in the image and send you to the relevant store in order to buy them. And in the same way, if you make a purchase and sometimes if you click, it's enough for the publisher to monetize that content.
Joseph
Yeah. And I mean, that sounds harmless enough, right? I can imagine that would have been very useful for buzzfeed if they had that technology when they did the dress article, you know, is it blue and gold or whatever? Everybody's trying to buy that dress. That being said, this technology is obviously from this year, 2024, and it's been applied to articles that were written a lot earlier. Right. Like, it's not just new stuff that's coming out, it's applied to articles that have been on the website for some time. Is that right?
Emmanuel Mayberg
Yeah, yeah. So, you know, in the past couple of years, a bunch of new media companies have totally collapsed vice that we have survived as one of them. BuzzFeed is another one. I used to read BuzzFeed all the time. They had like a very good, vibrant newsroom, did breaking news, investigative feature, just like really well written, award winning articles. And the way that I found out about this story, the fact that they even implemented this Trendy Technology is 44 Media reader was researching this condition I've never heard of called empty nose syndrome. And basically this is a condition where you feel like your airways are obstructed even though medically they are not. It just feels that way. It kind of feels like you're drowning. It's this really awful condition which in some rare cases have caused people to take their own lives because they like couldn't deal with the symptoms. And he was researching this and he landed on a really well written buzzfeed article about empty nose syndrome. And that article opens with this really sad anecdote about one of these people that took their own life because of this condition. And the article includes a very, you know, sad image of this guy with his young nephew, I think it is. And the shop this image button appeared on that photograph and suggested that he buy a beanie or something that looked like the beanie that he was wearing in a photograph. And he flagged that to me and I went poking around, learned more about this trending company, checked out some other sites that they work with, but really digging into buzzfeed because it has so many articles in such like a huge back catalog of articles and to see how it was implemented. And the short version is that it was implemented basically everywhere, which resulted in some like, really inappropriate monetization of horrible images.
Joseph
Yeah, I don't think this is what Trendy intended. And I mean, we'll get to their statement in a minute. Right? But I guess it's. Well, to be charitable, it's an unforeseen consequence. Right. So when you were going through the buzzfeed archive and seeing how else trendy had sort of been you know, implemented on some of these articles or what Trendy was doing, what were some of the other examples? I think there was one from the Challenger team. Right, and what was that one? And what were some of the other ones?
Emmanuel Mayberg
Yeah, so the, I would say like, least appropriate implementation here is like a classic type of Buzzfeed article. It's titled 17 Creepy, Disturbing and Terrifying Things. I learned about this, learned about this month that I really, really, really, really cannot keep to myself.
Joseph
Classic buzzfeed.
Emmanuel Mayberg
A great, a great buzzfeed article. And this is actually from like the post buzzfeed News Collapse. Right? And it's just like a listicle with a bunch of like disturbing factoids that are like ripped from the news or Wikipedia articles and stuff like this. And yeah, there were some really unfortunate attempts to monetize images of. There was the famous 1986 challenger disaster where the space shuttle exploded shortly after launch and all the astronauts died. So there was an image of that crew and the shop, this image button tried to monetize their blue uniform and match it to like blue puffer jackets that people can buy. My favorite one, which is, I would say less tragic and more funny, is this really early medical illustration of some of the worst symptoms of syphilis that can cause like this rot in your face. And the trendy shop this image button matched that color of the faces to like a shade of Mac lipstick. So it's like if you want to look like you're about to die from Stiffilis, shop this lipstick.
Joseph
Yeah, it's only $63.25 as well, according to the picture, you know, and I.
Emmanuel Mayberg
Went to Australian actually, which is another thing I want to highlight here is the company is Australian and I contacted BuzzFeed and BuzzFeed was, I would say, like apologetic and maybe horrified. But also they were like, actually, this is buzzfeed Australia, which has spun out into its own company, which is something that happens in media. I don't know why, actually, but we.
Joseph
Even had that with Vice, right?
Emmanuel Mayberg
Yeah, it happened to Vice.
Jason Kebler
I think Gopher Gizmodo also runs.
Emmanuel Mayberg
Yeah, Gizmodo has its own, like, Australian thing. Some of the gaming sites that I used to work at had the same arrangement. And that happened to Vice, not to buzzfeed not too long ago. So they were like, we're actually not on top of this. It's its own company. They implemented it in Australia. The ads are geofenced to Australia, which is why I had trouble finding them when I was looking around.
Joseph
Well, what did you do? Did you have to use a VPN or something?
Emmanuel Mayberg
We used the VPN and also the original tipster who had the ads active when he was browsing just like, started sending me more and more articles. And then we verified them by sending them directly to buzzfeed and Trendy, which confirmed that they were implemented indiscriminately across the site.
Joseph
Yeah, obviously the 404 Media reader saw it and sent it to you. You've obviously seen it. Have any of the, like, normal BuzzFeed readers spotted this? I think some left some comments.
Emmanuel Mayberg
Right, yeah. So this is interesting because I wasn't able to see the ads in the U.S. but what I was able to see is comments on those articles from Australian readers, which is how I was able to tell that the ads appeared in some other inappropriate places. So BuzzFeed had this article, for example, on how different celebrities were reacting or not to the war in Gaza. And one of the commenters was like, wow, it's really inappropriate that I'm looking at an image of this completely bombed out street and a kid walking through it. And the shop, this image button is like, how would you like to cop this look of this, you know, Palestinian refugee? So, yeah, that's a. In some cases I didn't see the ads themselves, but I saw commenters as far back as a year ago being like, this is a fucked up ad.
Joseph
This is a little bit speculative or more. More of an open question. And I think this would just be more of our opinion really, but like, did this happen potentially because Trendy doesn't see buzzfeed as sort of a news outlet anymore, even though I don't think that's entirely fair. And same as you, Emmanuel. I thought they actually had the best Investigations team in the business for actually quite a while before he got gutted. But again, a little bit speculative. But do you think Trendy sees this as just like a content platform rather than a website that's, as you say, going to have coverage about the war on Gaza or something like that, or even if we don't know if that's how Trendy feels is that is what is happening here. It's like a tool for content, not a tool for news. Do you see what I mean?
Emmanuel Mayberg
Totally. Yeah. I think that's a good question. I think it says two things. One is it is not clear. And I ask specifically, I asked both BuzzFeed and trendy, like, as the publisher, if you choose to have a relationship with Trendy, how are you choosing? Like, how do you manage? Where does this appear or not? And neither one of them wanted to comment on it, which I would take along with the fact that it was kind of appearing everywhere, that they really weren't controlling for it at all. But that's speculation. But I think it also says something about, like, what BuzzFeed is at this point. It's like this weird artifact of a once great newsroom. And now, I'm sorry, I know that there's people work there, but it's just like lowest common denominator, content factory that just like publishes everything and like a ton of celebrity news. So when I was searching the Internet for other sites that work with Trendy, I came upon this. I don't know how to pronounce this. It's the site is DMarge, maybe DMA, R G E. And it's like a bunch of celebrity news and it's what people are wearing on the red carpet and stuff like this. And in that context, the Trendy implementation, I mean, say what you will, but it's like inoffensive. It's sort of like synergistic, right? It's like you're reading an article about what celebrities are wearing and then the AI tells you, like, hey, do you want to buy something like this? Like, click on that. And buzzfeed has a ton of articles like that where that implementation makes sense. But then it also has like this back catalog of very serious, sometimes dark, sometimes tragic journalism where it doesn't fit at all. And at this point, you know, buzzfeed is just something that they're trying to like squeeze every dollar out of and they threw this AI product on it. And this is what we get.
Jason Kebler
Yeah, I mean, that's what I was going to say is like, it doesn't do news anymore at all, to my knowledge. And it's like there's zero respect there for the work that people did or respect for the archives or the stories that they told. Like, this is a publication that won a Pulitzer and did really, really incredible work. And it's like squeezing every penny out of every corner of that website is like, what they're doing now. Like, I. This seems to me like a Not even a side effect of what they're trying to do, but, like, surely someone either forgot to turn this off on news articles or there's no way of turning it off on news articles. Like, the archives are something to be ransacked for this company. It's not something to be respected in any way.
Joseph
Yeah, I mean, not trying to do Trendy's or buzzfeed's job for them, but theoretically, when we go and publish an article, we can tag it a certain way, right, as news. Or maybe it's a subject like AI or whatever. Presumably you could tag all of your news articles as news and tell Trendy, hey, please don't run fucking close efforts on this very horrible news story. Let me just read out Trendy CEO Aaron Wolff's statement just so we get that out there as well. Quote, unfortunately, this was an oversight, an accident, and obviously not what Trendies intended to do. We have accidentally appeared on images which are clearly not right, and our intention is to continue to evolve our product so we may avoid circumstances like this happening in the future. We truly hope we have not caused any offense to the audience of buzzfeed. And I'm sure that you believe Trendy is all about positive, happy experiences for consumers and better advertising without commoditizing consumer personal data, end quote. Kind of an interesting thing at the end where it's like, well, we're not using your data from browsing. We're just lifting from somebody who has syphilis and recommending makeup that way. I guess just the last question in Emmanuel is you mentioned one site that you saw is like celebrity news. Have you seen anywhere else? And do you. Do you see other companies of media companies trying to do something like this? I just mean, could you see that happening?
Emmanuel Mayberg
Yeah, it has a bunch of very notable partners, and I would say most of those partners make a lot more sense. So you have Vogue and Marie Claire and other fashion brands. I believe Popsugar was in there. I'm not sure all of those are active, but they did work with them at some point. And that seems, I think, fine. I just want to add one more thing. And that is because we are talking about an AI product. I want to make clear that, like, it basically doesn't work. Like, it will. Like. Okay, so I'm looking at an image. I'm on this demarge website right now where the Shop this image button is implemented. And it's a picture of Ryan Gosling from. I don't know, this looks like early 2000s. And he's wearing a T shirt that has, like, the tuxedo look, like, printed on it. He's not wearing a tuxedo. He's wearing a tuxedo T shirt. And the Shop this image button is recommending that I buy a black Allbirds T shirt. And what I mean by it doesn't work is that it's like. And I did the same thing. I was looking at, like, an image of Kanye, who is wearing always, like, these really crazy designer striking clothes. And it recommends that you buy stuff in the image, but it's not what he's actually wearing. It's just doing its best to match something from a participating retailer. Right. Because the reason I'm seeing an Allbirds T shirt is because All Birds decided to make a deal with Trendy and advertise its T shirts this way. So it's not showing you the actual product. It's showing you something that looks like the product. And it's like, if you want to dress like Kanye, you're not going to be able to do it by following whatever Trendy recommends. It's just like sort of putting something in front of you that looks like it. It's not. You're not actually shopping the image. You know what I mean?
Joseph
Yeah. Which I think makes it. Again, I know you said it's not clear if it's active with Vogue or whatever, but let's say it was. And it's like, oh, you can get, like, you know, a Runway look or something that might not be readily available, and it's not going to work then either, necessarily. I mean, again, I don't know if that's a use case or not, but. Yeah, exactly.
Emmanuel Mayberg
Yeah.
Joseph
Yeah. All right, let's leave that there. And then when we come back, we're going to talk about an unprecedented leak out of the phone forensics tech Greeky. We'll be right back after this.
Jason Kebler
We're heading into the best time for deals that most joyous time of year. But I love it most when the deals come directly to me. That's why I'm so excited to tell you about Mint Mobile's 15 bucks a month. Deal with the purchase of a three month plan. Turns out the deal really does just come to you. The longest part of this whole process is the time you'll spend waiting to break up with your old provider. And Mint Mobile's website makes it super easy for you to port your device and your phone number over to Mint Mobile's network without changing your wireless experience. To get started go to mintmobile.com 404media there you'll see that right now all three month plans are only 15 bucks a month, including the unlimited plan. All plans come with high speed data and unlimited talk and text delivered on the nation's largest 5G network. You can use your own phone with any Mint Mobile plan and bring your phone number along with all your existing contacts. Find out how easy it is to switch at Mint mobile and get three months of premium wireless service for 15 bucks a month. To get this new customer offer and your new three month premium wireless plan for just 15 bucks a month, go to mintmobile.com 404media that's mintmobile.com 404media cut your wireless bill to 15 bucks a month at mintmobile.com 404media $45 upfront payment required equivalent to $15 a month new customers on first three month plan only speed slower above 40 gigabytes on unlimited plan. Additional taxes, fees and restrictions apply. See Mint Mobile for details. Black Friday and the holiday season is coming up. It's a big time for any online store, ours included, and it's a good time to make sure that yours is set up in the right way. 404 Media uses Shopify to sell our merch and it's one decision we made that has simplified everything for us. Whenever I need to restock items, offer a new item for sale, which by the way, lots of new items for sale on our Shopify or manage our inventory. Everything I need is just a click or two away in Shopify's simple but really powerful backend. It gives me everything I need to start and manage our store so that I can spend less time researching and more time selling. I can't stress enough how easy it is to use Shopify. You can upgrade your business and get the same checkout that we use with Shopify. Sign up for your $1 per month trial period at shopify.com media all lowercase go to shopify.com media to upgrade your selling today. Shopify.com media this show is sponsored by BetterHelp. It's Thanksgiving, so I wanted to take a moment to thank all of our subscribers and listeners. We couldn't do this without you. This month is all about gratitude, and sometimes I think that we're not patient or grateful for ourselves and what we deal with every day. It's important to remind ourselves that we're trying our best to navigate an increasingly complicated and difficult world. Therapy can help. I found that it's been really helpful to talk with a licensed therapist about my feelings, which has made me a more patient, less anxious, and more balanced person. Therapy has taught me coping skills, how to set boundaries, and how to care about myself, empowering me to be the best version of myself. If you're thinking of starting therapy, give BetterHelp a try. It's entirely online, designed to be convenient, flexible, and suited to your schedule. Just fill out a brief questionnaire to get matched with a licensed therapist and switch therapists at any time for no additional charge. Let the gratitude flow with BetterHelp. Visit betterhelp.com 404Media today to get 10% off your first month. That's BetterHelp help.com 404Media.
Joseph
Hackers and cyber criminals have always held this kind of special fascination. Obviously, I can't tell you too much.
Emmanuel Mayberg
About what I do.
Joseph
It's a game.
Emmanuel Mayberg
Who's the best hacker? And I was like, well, this is child's play.
Joseph
I'm Dina Temple Reston, and on the Click Here podcast you'll meet them and the people trying to stop them. We're not afraid of the attack.
Emmanuel Mayberg
We're afraid of the creativity and the.
Joseph
Intelligence of the human being behind it. Click here. Stories about the people making and breaking our digital world. AI machines, satellite engine ignition. Click here and lift up. Click here every Tuesday and Friday, wherever you get your podcast. All right, and we are back. This is one I wrote called Leaked Documents show what phones secretive tech Gray Key can unlock. And before Jason asks me a few questions, I do just want to stress this is really complicated. It's really, really difficult. It was really hard for me to get my head around. Jason and Emmanuel very carefully edited it, caught stuff that I missed and there are a lot of questions remaining because this is a super secretive company, Gray Key and Magnet, the company that now owns it. But there's some pretty interesting stuff. So what we got is two spreadsheets and they are a granular list of what iPhone and Android devices Gray Key is able to retrieve data from. And before I refer to Jason, I'll say that the top line is basically that Gray Key is only able to retrieve, quote, partial, end quote, data from all modern iPhones, that's the iPhone 12 up to the 16 that runs iOS 18 or 18.0.1. We don't quite know about 18.1, which is the current, most recent version of iOS that was released on October 28th. The documents look like they're from October, but just before October 28th, so we don't have that bit.
Jason Kebler
Okay, so, Joseph, what is the Gray Key?
Joseph
Gray Key is a very small device that can sit on a law enforcement officer's desk, or I think there's a mobile version or maybe they've integrated that in there as well. But it really came on the scene in around 2016, 2017, 2018, like around then post San Bernardino. And everybody will remember that it was really, really hard for the FBI to get into the San Bernardino iPhone. Big caught court case. Azimuth Security eventually hacked the device. We actually touched on that last week. In the wake of that, this company called Greyshift launched this tiny little product called the Grey Key. And it was something like 15 to 30,000. The price has actually changed a little bit over the years and there's sort of an annual subscription as well, with a number of unlocks, that sort of thing. But Forbes, Tom Brewster over at Forbes, he first revealed the existence of it and it basically sent shockwaves through the law enforcement and the forensic communities. After all these years of iPhones being uncrackable, it now appeared that, hey, here's a box that if you give them tens of thousands of dollars, cops will be able to get into there. And it has a little USB C cable on the front or a lightning, depending on what you're trying to do. You plug it in and it does two things, right? The first is that it will try to get the passcode and maybe you'll do that through brute forcing, or maybe it will extract a keychain and maybe there'll be clues in there. I've seen some other documents where it'll extract information like messages. And it will then try to find clues on other passwords or other pins, maybe because, oh, I found something that looks like a birthday. And now maybe that could be a pin code or something like that. So you have the trying to break into the phone, which is one of the most important parts. And then you have, well, it's also extracting or I think very fair, as Manual said when he edited it, it retrieves the data from the device and then makes it, it presents it to law enforcement officer. It's not just a big file.
Jason Kebler
You go, oh, it's like an interface That'll be like, here is messages from a spec or here are photos, like stuff like that. Right. It's a backend that you can kind of browse through.
Joseph
Yeah, it makes it much, much, much easier to dig through iPhone and now Android images of devices. And yeah, as I said, it started with iPhones, but then a few years ago they branched into Android because of course its main competitor is Celebrate, which I'm sure everybody's familiar with.
Jason Kebler
I was going to ask about that, actually. I don't even know the answer or if you know the answer, but do. Like, by and large, are Celebrate and Gray Key like more or less the same thing? As in, like, does one offer things that the other does not to our knowledge, or they both just. Are they like direct competitors offering like essentially overlapping products?
Joseph
Yeah, I would say they're basically direct competitors. You then have other ones that are a bit smaller, like ElcomSoft, I think, a Russian company, and they do look into mobile forensics as well. So there's that sort of thing. You then have these other companies that are more just focused sort of on the visualization side, like, oh, you already have the data and they will interpret it for you. But Gray Key and Celebrate, it's supposed to be an all in one solution, right. You buy the box or you buy the service, you plug the phone in, you unlock it or get what data you can and then interpret it. I think a slight difference with Cellebrite is that they have this advanced program where at least this was the case a few years ago, and the leaked Cellebrite documents, we got touch on this a little bit. But in some cases you send the phone to Celebrate and then they do it. And I think that's in part to look after their capabilities. Hey, if it's on a box, even if that's phoning home, there's a world in which someone could get hold of that and try to reverse engineer it. Right. That could be one reason for it. The other is just that when it comes to mobile phone forensics, there are so many variables that it could be better for someone to do it internally rather than a tool of the cops forensics lab or whatever. And by that I mean more stuff like the screen is broken or the battery got hot and expanded and like completely screwed up the phone or something like that. And that's when you send it to Celebrate or someone to look over.
Jason Kebler
Yeah. Okay, so as far. So I remember Greg Key came on the scene like a few years after Cellebrite and it was this pretty mysterious Start like upstart in the industry. As far as I know, there have been like very few leaks about Greg. Like they're. We've learned about them through like FOIA documents and sometimes court cases and things like that. But in terms of like big leaks about how it works and its capabilities and what phones it can unlock, so on and so forth. I'm not aware of any others. There may be a few at some point, but this is one of the biggest ones I think that has ever happened. So what did you get and what do the documents show?
Joseph
Yeah, I think it's, as I said, unprecedented. We've had similar leaks from Celebrite, the ones we've reported on, the ones that other people have obtained. And then even way back at Motherboard, we covered the hack of Cellebrite as well, where I think I got 500 gigabytes of data from Cellebrite and did what we could there. But yes, Gray Key, it doesn't leak very often. They treat their material very, very carefully where if you do a freedom of information request for emails, they don't send this sort of material, at least ordinarily, as like an attachment to an email, you know. And I foyed tons of material over Gray Key and I've never seen something like this. And this was a leak that somebody gave to me. It wasn't through a foia. But as for what they actually show, basically if you update your iPhone, you're probably pretty good. Again, I'm looking at the table now, which shows Grey Key's capabilities against iOS 18 and 18, 0.1 and everything from the iPhone 12 through to the 16 Pro. Max says if you're running either 18 or 0.1, it can only get partial data. We don't know whether that is after first unlock or before first unlock. Afu, bfu. AFU being that the owner has already unlocked the device at least once since it was powered on and that can make it a little bit easier. Or BFU is obviously the opposite of that and that hasn't been the case. We don't know. But I think it's still very. A massive takeaway is that simply the spreadsheet does not say full. It's a fact that Grey Key just cannot get full data from a modern iPhone. I think that's the main sort of takeaway when you look at Android is obviously way more varied. There are, I don't know, a squillion different Android phones.
Jason Kebler
Squillion.
Joseph
Squillion.
Jason Kebler
I don't think I've heard that one.
Joseph
Before, maybe fact check me on that one. But you know, they're all made by different OEMs, different manufacturers, all these different forks of Androids. Maybe you have a Samsung, maybe you have a Google, some phone made from somewhere else in the world as well. And although the Google Pixel devices, according to the spreadsheet, can have data extracted if there are partial AFU status and they get a bit of data in AFU state, the rest is a massive mix of all data, no data. It's a hodgepodge, I feel.
Jason Kebler
We talked about AFU BFU last week, but I think we should do a very quick reminder of what before first unlock and after first unlock is for people just joining us.
Joseph
Yeah, so I touched on it. But to spell it out a little bit more BFU before first unlock, that would be if, let's say the phone is off, the iPhone is off for whatever reason, the police officer turns it on to forensically extract it. And that obviously means that the actual owner of the device has not entered the passcode, which would decrypt a lot of the information on there. That's a BFU state before first unlock. Afu, the opposite of that. And that's going to be when the user has unlocked the device at some point in time. That is especially important for a number of different reasons. I mean, let's say police officers raid somebody's apartment, I'm just gonna say a drug trafficker for sake of example, and they're on their phone and they want to preserve evidence. The police will probably try to grab that phone while it's on. And so it hasn't been turned off by the user because that would probably be in an AFU state and then easier to unlock. Now, let's say, I don't know, tragically, a child goes missing and they had a phone on them and it runs out of battery and it powers down. That would then be a BFU state. And then that could be more difficult. There are sort of like real world consequences to how and when the phone is seized, which will dictate whether it can be unlocked or not, essentially.
Jason Kebler
I want to hear you go through like increasingly tragic hypotheticals in your mind. Okay, so there's been kind of a lot of leaks lately, and you've gotten, I think, pretty much all of them in this world where, you know, last week you had the new iOS feature about the sort of like reboot after a phone's been idle for a while. A few months ago you had a cellebrite leak. Now you have this gray key leak. What is sort of like the current state of play for iPhone hacking by law enforcement? And I guess what is sort of like the status quo here? Like, it seems to me like it's a super uneven, like, playing field, more or less. I don't know if playing field is the right word, but it's not. It's an uneven situation where some phones, they can break into, like, really quickly, really easily with these tools, others not so much.
Joseph
Yeah, I think that's right. And it's always going to be messy. And I think many members of law enforcement wouldn't want that. They would want a more permanent solution. That's, of course, what we had with Apple versus FBI, where really they weren't trying to get access to one phone. They were trying to develop a capability that would ensure access for future cases as well. Right. And then Azimuth hacks the phone and then Grakie comes along and it seems to be like there's this cycle where Apple will do some sort of security upgrade. I mentioned USB restricted mode, which a few years back that meant that you couldn't plug the phone into a computer and get data from it, essentially because the port would just turn into a charging port rather than a data one that comes along. There's all of this. It feels like the sky is falling down for forensic investigators. And then gray key and or celebrate, find workarounds and then it continues. And then we get, as you say, this really interesting iOS18 reboot where if the phone has been left idle for three full days or turning onto the fourth day, it will reboot and go to a BFU state. Now investigators have to find a way to deal with that as well. That is just how this goes. It's a constant cycle. They find exploits, there are workarounds, that sort of thing. That's not to say it'll be like this forever. In the same way that custom encrypted phones of criminals became such a massive headache for cops that the FBI decided to run its own encrypted phone company. It came to a head there. I think it's absolutely possible that if it becomes too hard to get data out of iPhones or Androids or any devices, and there's a really, really, really important case, of course, it could come up again, where the FBI is like, we need a more permanent solution and it's not just a US thing. Right. The UK has demanded sort of technical capability access as well. Australia does some things as well. Europe is part of the discussion too. But at least right now, it looks like Greyking and celebrate, they're a little bit behind and then a couple of months later they catch up. At least in some way.
Emmanuel Mayberg
It seems to me, speaking only about the iPhone, it seems to me like for a few years now, basically the takeaway from all of these leaks is if you keep updating your phone, as soon as the security updates drop, you're probably okay. And if that is the case, I am wondering what do you think the value of a Gray key is to law enforcement? Is it that they know that enough phones that they're trying to get into are not updated, or are they paying for those short points in the cycle where Gray key is ahead of Apple?
Joseph
I think they're paying for those windows and we don't know like the 18 adoption rate or the 18.1 adoption rate or. We know that Tim Cook. Tim, Apple said that it is going very, very quickly and people are moving up to it. So there is, there are those windows. I do think that, yes, even though they can only get partial data from iPhones, that's not no data. So there is something there. And it's kind of what we were talking about earlier, where it can be visualized, it can be mapped with other data, they can still perform investigations. And obviously, I don't know if this is fully fair because I'm not a police officer, but I would say probably the value proposition has probably gone down if it's not unlocking the phones, which doesn't say it's not worthless, but maybe the value has gone down a little bit.
Jason Kebler
You know what? That raises something that we didn't talk about last week, but kind of plays with your story last week about the new iOS update and the phone rebooting, which is like cops sometimes confiscate phones and they might not break into it right away or they might not try to break into it right away. And I wonder if these are often sitting in evidence like storage lockers for quite some time until Gray Key or Celebrate is able to make a new exploit and then. Or is able to update whatever they're doing. And then there's like a frenzy of, you know, unlocking a bunch of phones. And I wonder if that is, I guess, like when we talked about this last week, I was like, oh, well, they're just going to do it right away now. But in some cases they might not be able to unlock these phones right away. They might be waiting for that window where, you know, there is a period where they're able to unlock these phones before Apple gets a new update. And I don't know. Have you thought about that? Do you even understand what I'm saying? I feel like I'm rambling a little bit.
Joseph
No, no, no, I get it because yeah, I've spoken to people even for my book. When I was talking to people who had to get data from PGP blackberries that were used by criminals. You would sometimes open a device and you would see a certain sticker and you'd go, well, that's a phantom secure device. I'm not getting anything out of that. You basically throw it in the trash because what's the point? I can't get anything from it. But if it was another sort of phone, you would leave it there and would be like on the conveyor belt or in storage and you would come back to it later. And cops absolutely do that. They're just waiting for, oh, okay. Well, Gray Key just needs to update the latest version. When we get that, we'll be able to get into that device and that'll be great. And I think that is why the iOS 18 reboot timer is so such a big deal for law enforcement or other people trying to get into the data like thieves potentially. But I do think the main context in which is introduced is law enforcement. No longer can they just have a phone waiting there for Grey Key to hurry up and push an update. It's like we have four days and then we're screwed basically. So I imagine Grey Key is going to try to find a way somehow and celebrate to keep the phone in that non reboot stage. Maybe there's a way in which the timer going down. There's a way to fuck with that or something. I mean I have no idea. But that's going to be what they're going to try to do.
Jason Kebler
I imagine just need an artificial finger touching the screen sometimes and moving it around. Yeah, you need a mouse jiggler.
Joseph
Yeah, exactly. Or just a guy, you know, just a guy touching all the phones constantly.
Jason Kebler
The phone toucher.
Joseph
Yeah. All right, we will leave that there. If you are listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 media subscriber, we're going to talk a ton about AI and the publishing industry and books. Sam has a couple of really interesting stories about that. You can subscribe and gain access to that content at 404-MEDIA CODE. As a reminder, 404 Media is journalists founded and supported by subscribers. If you wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really helps us out. This has been 404 Media. We will see you again next week.
Podcast Summary: The 404 Media Podcast – "BuzzFeed's AI Ads Are a Disaster"
Released on November 20, 2024
Host: Joseph
Co-Hosts: Sam Cole, Emmanuel Mayberg, Jason Kebler
Podcast: The 404 Media Podcast
Description: 404 Media is a journalist-owned digital media company exploring the intersection of technology and society through investigative reporting, smart blogging, and breaking news.
The episode of The 404 Media Podcast titled "BuzzFeed's AI Ads Are a Disaster" delves into the problematic implementation of AI-powered advertising by BuzzFeed, exploring its unintended consequences and broader implications for digital media. Hosted by Joseph, alongside co-founders Sam Cole, Emmanuel Mayberg, and Jason Kebler, the discussion offers an in-depth analysis of how AI technology can sometimes fail in sensitive contexts, leading to ethical and operational challenges.
Emmanuel Mayberg introduces the core issue: BuzzFeed's use of an AI-based advertisement system developed by an Australian company named Trendy (spelled T-R-E-N-D-I-I for SEO purposes). This system was intended to monetize content by recognizing objects within images and suggesting similar products from affiliate retailers. However, its implementation across BuzzFeed's extensive archive led to inappropriate and insensitive ad placements, including tragic and disturbing imagery.
Affiliate Marketing Explained ([01:46] - [02:58]): Emmanuel explains affiliate links as a monetization method where publishers earn a small commission from sales generated through embedded links. Trendy aimed to extend this model to images, enabling readers to "shop the image" by clicking on items within photographs and redirecting them to retailers.
Inappropriate Ad Placements ([02:58] - [09:16]): The technology was retroactively applied to older articles, including sensitive content. For instance, an article on empty nose syndrome—a rare condition causing severe breathing distress—featured a sad image where the AI suggested purchasing a beanie similar to one worn by a person who died by suicide. Other examples included:
Global Implementation and Geofencing ([09:16] - [10:27]): The Trendy system was geofenced to Australia, complicating detection by users in other regions. Emmanuel and Jason discuss the challenges of uncovering these ads, including the use of VPNs to access Australian content.
BuzzFeed's Response and Trendy's Apology ([12:00] - [15:34]): Trendy issued a statement apologizing for the oversight, stating, "We have accidentally appeared on images which are clearly not right... We truly hope we have not caused any offense to the audience of BuzzFeed." However, both BuzzFeed and Trendy remained reticent about controlling the ad placements, leading to widespread inappropriate ad displays across various articles.
Impact on BuzzFeed's Reputation ([14:43] - [19:21]): The misuse of AI for monetization has tarnished BuzzFeed's image, shifting it from a respected journalism outlet to a content factory prioritizing revenue over quality and sensitivity. Emmanuel highlights the disparity between appropriate implementations on celebrity-focused sites versus serious journalism pieces, underscoring the lack of nuanced application.
Emmanuel Mayberg ([07:03]):
"It was implemented basically everywhere, which resulted in some like, really inappropriate monetization of horrible images."
Emmanuel Mayberg ([19:02]):
"It's just like sort of putting something in front of you that looks like it. It's not."
Trendy CEO Aaron Wolff ([15:34]):
"We have accidentally appeared on images which are clearly not right... We truly hope we have not caused any offense to the audience of BuzzFeed."
Transitioning from BuzzFeed's AI mishap, the podcast shifts focus to a significant leak involving GrayKey, a notorious phone forensic tool used by law enforcement to access data on locked smartphones. Joseph introduces the topic, highlighting the complexity and sensitivity surrounding phone hacking technology and its implications for privacy and law enforcement capabilities.
GrayKey's Functionality ([23:40] - [28:35]): Joseph explains that GrayKey is a device used by law enforcement to unlock smartphones, initially gaining prominence post the 2016 San Bernardino case. The leaked documents reveal GrayKey's limited ability to extract "partial" data from modern iPhones (models 12 through 16 running iOS 18 or 18.0.1). The tool’s effectiveness diminishes with each new security update from Apple.
Comparison with Competitors ([28:35] - [31:51]): Jason Kebler inquires about GrayKey’s competitors, notably Cellebrite and Celebrate. Joseph describes them as direct rivals, each offering similar forensic capabilities with slight variations in service delivery. The competitive landscape emphasizes the constant arms race between phone security measures and forensic tools.
Impact of iOS Updates ([31:51] - [36:17]): The discussion underscores that regular updates to iOS significantly hinder forensic tools’ effectiveness. Emmanuel and Jason elaborate on the "After First Unlock" (AFU) and "Before First Unlock" (BFU) states, explaining how device usage patterns influence the ease of data extraction.
Operational Challenges for Law Enforcement ([36:17] - [43:54]): The hosts delve into the practical difficulties law enforcement faces in accessing data from locked phones. With each new security feature or update, forensic tools like GrayKey must adapt, often trailing behind the latest protections. Emmanuel suggests that the value proposition of GrayKey is diminishing as smartphones become more secure, making comprehensive data extraction increasingly elusive.
Future Implications ([39:37] - [44:10]): The conversation speculates on the future of phone forensics, considering potential regulatory actions and the ongoing technological tug-of-war between privacy advocates and law enforcement agencies. Joseph highlights the possibility of more sophisticated methods being developed or legal mandates forcing companies like Apple to provide backdoors, echoing past conflicts like the FBI vs. Apple case.
Joseph ([23:40]):
"Hackers and cyber criminals have always held this kind of special fascination."
Emmanuel Mayberg ([39:37]):
"If you keep updating your phone, as soon as the security updates drop, you're probably okay."
Jason Kebler ([34:32]):
"Have you thought about that? Do you even understand what I'm saying?"
Joseph ([43:54]):
"Gray Key is going to try to find a way somehow, and Celebrate to keep the phone in that non-reboot stage."
The episode of The 404 Media Podcast offers a critical examination of how AI technology, when improperly implemented, can lead to ethical breaches and operational failures, as seen with BuzzFeed’s Trendy ads. Additionally, the discussion on GrayKey exposes the ongoing challenges in the realm of phone forensics, highlighting the delicate balance between technological advancements and privacy/security. Through detailed analysis and insightful commentary, the hosts underscore the imperative for responsible technology deployment and the continuous evolution required to address emerging digital threats.
Subscribe to 404 Media:
To support journalist-founded 404 Media and gain access to ad-free podcasts and exclusive content, visit 404Media.co.