Loading summary

Transcript73 lines

[00:01]
Jason Kebler
This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50 page restoration block. Or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it,
[00:17]
Emmanuel Mayberg
ready to make anything online make sense.
[00:20]
Jason Kebler
There's no place like Chrome. Check responses set up, required compatibility and availability.
[00:24]
Emmanuel Mayberg
Various 18.
[00:26]
Jason Kebler
Sam Altman earlier this month heard from parents who were generating personalized podcasts to like listen to with their kids on the drive to school. The drive to school is like when you should be actually talking to your kid. How about you try talking to your kid versus having an AI podcast? Do it.
[00:46]
Joseph
Hello and welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404Media is a journalist, founder, company and needs your support. Support. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content@404 media co. Also do remember to subscribe to our YouTube channel where you can watch all of our episodes, including this one. Subscribe@YouTube.com Mediaco I'm your host, Joseph, and with me are the other 404 Media co founders, the first being Sam Cole.
[01:31]
Sam Cole
Hello.
[01:32]
Joseph
Emmanuel Mayberg.
[01:33]
Emmanuel Mayberg
Hey, what's up?
[01:34]
Joseph
And Jason Kebler.
[01:36]
Jason Kebler
Hello. Hello.
[01:37]
Joseph
All right, we're starting with the sporadic maybe regular Mark Zuckerberg manifesto. You know, he'll regularly. Was it every six months or something? Maybe every year he'll write something like this or he'll post some insane new video. A ton of people read this and obviously you had some thoughts about it. Jason. So the headline of your piece, mark Zuckerberg posts deranged 6,500 word essay about giving everyone AI super intelligence. Let me just read the lead, or rather the second part of it because I think it'll sum up what you're trying to say is that he posts this. He's describing a future that he says is for everyone, but but which really sounds less social than ever. There's a lot to get into here, Jason. Don't drill into the specifics yet, but like when you read this, like, what was your media reaction to this deranged essay?
[02:37]
Jason Kebler
Yeah, I mean Mark Zuckerberg again does this every so often. Like he did a huge one for the Metaverse. He's done several different ones for AI. And it's basically like a big, long post on his own Facebook or on, like, Meta's blog. And again, this one is called the Future Is for Everyone. And it is essentially his vision for AI superintelligence, which he has been talking about, maybe even more than Sam Altman and Dario Amadei, like, more than the other kind of, like, AI tech mavens. And I feel like he's been doing this maybe in part because Meta feels like it has fallen behind in AI and so he's like, really leaned into, like, we are pitching a future where AI is going to change your life. There's a lot in this essay about how the other AI companies are building AI for businesses and enterprises, whereas Meta is building personal AI for individual users. And so he's trying to really differentiate himself or Meta's product as being this one that's, like, designed for the common person and how it's going to change your life and all this sort of thing.
[03:53]
Joseph
Yeah, definitely a lot of talk with all of Meta's massive AI hires and then people immediately leaving the company, offering people in the AI industry bucket loads of money, and some of them being like, nah, I'm good, I don't want to go work at Meta. It does seem pretty desperate in a lot of ways. So, more specifically on the agents, what is he saying that Meta is going to offer people? Like, you're going to have your own agent. And so what, like, what is Zuckerberg saying we're going to do here?
[04:29]
Jason Kebler
Yeah, I mean, I think the big thing is that there's a lot going on in this essay. So the broad majority of it, like the beginning talks about personal agents, like super intelligent agents that will work for you, but then he later gets into, like, data centers and why the United States needs to lead the AI revolution. And he talks about China and he talks about open weights, which we can talk about. But, like, I think the core thing to understand is that this essay is all over the place. It's not very coherent. It has tons of platitudes that are about nothing. And so, like, with that kind of backdrop, let's talk about agents. For people who don't know, an AI agent is like a personalized AI that you give access to, like, act on your behalf, more or less. And so you would give it, you know, access to your accounts, your messaging, maybe your bank, like, all sorts of things. And then it would, like, theoretically learn about your life and go on and do things for you. And what Mark Zuckerberg is saying is that, like, you will have this personalized AI agent that will free up all sorts of time for you. And then he sorts of, like, gets into specifics. I think it's best to just, like, read a quote here. So this is how he, like, introduces it. He says, everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about. Your agent will work 247 on your behalf to improve your relationships, health, career, finances, home management, hobbies, and more. It will free up time for the things you enjoy and help you accomplish more than you could otherwise. It will have strong privacy and security options so you can trust to handle all of your personal content, knowing that no one else can access your information. You'll be able to interact with your agent through any device, including your glasses, which is funny to keep you present in the moment with the people you care about. So, I mean, this is, like, the pitch for all sort of, like, agents, where it's like, oh, well, isn't buying a flight so annoying? You can just tell your agent that you want to buy a flight, and it will find the best flight for you and it will buy it for you, or it will just know that you need to go somewhere and it will proactively buy a flight for you. He goes on to sort of, like, explain how his own agent that he's made is, like, developing recipes for him to make with his daughter and things like this, like, kind of outsourcing things that normal humans enjoy doing. Part of the point is to spend time and show that you care. It's like, oh, well, the agent did it for me.
[06:58]
Joseph
Yeah. So he simultaneously says that our AI agents will go on your behalf and they will free up time for what really matters. And presumably that would be something like crafting a recipe to make with your daughter or something like that. But no, no, he's having the AI do that. I mean, yes, he's still doing the cooking, but I don't know, he's outsourced half of it to the AI. I thought the whole point was to do that thing with your family. So, I mean, I don't really know. Does he not enjoy that part? Or, like, is that a theme you keep seeing in this essay?
[07:33]
Jason Kebler
I mean, that's a theme in this essay. It's a theme in, like, you know, Sam Altman last week or earlier this month, said that he had heard from parents who were generating personalized podcasts to, like, listen to with their kids on the drive to school. And everyone's like, well, the drive to school is like, when you should be actually talking to your kid. How about you try talking to your kid versus having an AI agent or, like, an AI podcast? Do it. The real thing here is that, like, Zuckerberg is pitching a future that he says is for everyone, but that sounds a lot like the present in some ways, and then also is not a future that, like, necessarily everyone wants or that really many people at all want. And so he sort of like, talks about AI job loss, for example, or he's like, well, automation is happening. But then he's like, but your personal AI agent will allow you to do your own scientific research and to make your own inventions and all of these sorts of things and to start your own business. And it's like, not everyone wants to become an inventor. Not everyone wants to cure cancer. Not everyone. I mean, people want to cure cancer, but, like, people don't want to.
[08:41]
Joseph
Like, not everybody.
[08:42]
Jason Kebler
Not everyone can do it. It's basically. He's basically, like, everyone will be able to do everything all the time, and, like, your AI agent will be doing it for you. He's talking about how, like, people at Meta are using AI to discover, like, new crystal structures that can go into the Meta glasses. And, like, you may be able to do your own. And it's sort of. It reminds me.
[09:03]
Joseph
But you're making the glasses.
[09:05]
Jason Kebler
Yeah, it reminds me a bit of vibe coding, where it's like, oh, like, you can. With vibe coding, you can, like, code your own email client and your own chat client and your own, like, calendar app and all of this. And it's like. Or if that's not something I'm passionate about, I can just use someone else's. And I shouldn't have to become a coder just because that's what we've found AI is good at doing at the moment.
[09:28]
Joseph
Yeah. Just on the work thing, when Zuckerberg is saying, your AI will do all these things so you can go and make your own business or make that railroad, straightforward. Just the other day, I was listening to an interview with Gary Tan, the CEO of Wai Combinator, on the Verge's decoder podcast, and he said something to the effect that was very much the same, like, AIs are going to mean you can go make your own business. And it's very, very straightforward. Actually, I'm sorry, I think it was the Wall Street Journal. I'll double check that. But he brings up that point, and of course, it's in the discussion of labor. And I think the interviewer says, well, look, people are going to lose their jobs. But Tan just replies, well, yeah, but those people should now go become entrepreneurs. But not everybody wants to do that. And there's that through line, I think in that Tan interview and then in this Zuckerberg essay that, as you say, Jason, it kind of just assumes that this is what people want and probably a ton of them don't, actually.
[10:30]
Jason Kebler
Yeah. And I mean, he also says repeatedly, like, the future is for everyone. This AI future is for everyone. And he then describes what this means and how Meta is working on making free AI AI tools of which, you know, there's a lot of free AI tools out there, at least free for the user. And then he describes like this future system where he was like, it has to be accessible to everyone. And then he explains, quote, for everyone to be part of the future, everyone must have the ability to use super intelligence to improve their lives and shape the world. We will offer free versions that will be accessible to billions of people. For those who want to pay to use more compute, there will be a dynamic auction mechanism that will guarantee that everyone gets the lowest price possible for the intelligence and compute they're using, while also ensuring the capacity is used for whatever people collectively find most valuable. And that's just like the system we already have where you, some of it is free and some of it you have to pay for. And if you pay for it, you get more and better. And if you have more and better, it's going to be a tiered system. It sounds like he's recreated the concept of just paying for AI compute and then there's tons of just like drivel and, and like, like weird thought experiments and, and just like platitudes out there. Like these are real sentences from this essay. Quote, humanity is not a monoculture. People's diverse values represent different trade offs that would make, they would make on important issues. Like that means nothing.
[12:04]
Joseph
Thanks, Matt.
[12:05]
Jason Kebler
People have an infinite demand for new experiences and all and have always found new problems to tackle. It's just like a lot of it is. I don't want to say it was like written by AI because it is so weird that I don't think that it was. But it is basically like this just big manifesto. And I think it's just worth remembering that, you know, this is like a top 10 most rich person in the world. I don't know where he is at the moment, but he's just like wildly wealthy. He is driving this innovation, or rather he's like pushing this product on people. He's, you know, the metaverse was A gigantic failure. And now he's saying, well, like, this is going to be the thing that we do. And it's this kind of like hubris and an ego trip that Silicon Valley CEOs have where they say, like, here is what humanity is going to do and we are going to be the ones to take us there. It's like you see in Mark Andreessen's like, it's time to build. You see it with Elon Musk being like, we're going to colonize Mars, we're going to have driverless cars, we're going to have all this stuff. And some technological progress is quite good. But in order to do all of this, it's like there's a huge capital expenditure. Like all of the products are, you know, initiative in a way that is really annoying and changes how people have like, lived and worked for a long time. In the case of Elon Musk and also Mark Zuckerberg, there's like tons of tax breaks involved, there's lots of government lobbying involved, there's lots of, you know, handouts associated with it. And so, you know, these people are quite successful at getting like government buy in for their vision. But it's not clear that this is like something that the masses are interested in.
[13:57]
Joseph
Yeah, absolutely. So, Sam, you've probably interacted with Meta's AI products, I think, more than anyone here. Well, also the Metaverse as well, because I think you have a quest, right? And I think you went in and saw how garbage that was and then you did a bunch of investigations into Meta's AI chatbot around them pretending to be licensed therapists and that sort of thing. That's a little bit different to the agentic AI they're promising here. A chatbot is not exactly the same as a Gentak AI, blah, blah, blah, whatever. But like, just based on your experience of actually interacting with and using Meta AI tools, like, do you think Meta could pull this off in any sort of convincing way?
[14:46]
Sam Cole
No, definitely not. I mean, like, the assessment that you guys have already discussed is I agree with completely. It's like, I guess it was last year, Zach was like, you guys don't have enough friends, so our chat bots will be the answer to that. He just is so detached from the actual experiences of most people that I think he thinks that the things that he feels is what everyone feels, which is obviously like CEO brain, tech exec brain to a T. And you know, he has a lot of people around him who are like sick fence and are telling him that That's a great idea. And, yes, manning his way to these crazy screeds. But, yeah, I mean, he. This is so classic as. As you guys already said, of the scrambling that he does before something totally shits the bed. It was before the metaverse fell apart and that dream died that he was pushing it the hardest and saying, this is the future that everyone's going to have and we're all going to be in the verse and won't it be wonderful and lovely, and we're going to be floating around with no legs. It's. It's such a clear pattern at this point that I. I don't know why he. I don't know who lets him continue to publish these things. I don't know why someone around him isn't like, hey, maybe let's not post this right now. Maybe let's have an edit of this before we say it completely. But I'm glad no one is stopping him because it's very revelatory of his mindset and what he thinks is happening is where the company goes, whether it works or not.
[16:35]
Joseph
Yeah.
[16:36]
Jason Kebler
I mean, good content.
[16:38]
Sam Cole
I feel bad for this country, but it's fantastic content.
[16:41]
Joseph
So to take it back even further, and I know this is sort of an obvious historical timeline, but, you know, with Facebook, he was like the leader in social media and social networking, and obviously that is how Facebook and their meta became this corporate giant. Right. And then through acquisitions of WhatsApp and Instagram. Okay, we've acquired that. So now we're increasing. We're keeping our relevance and increasing it as well, of course, because those platforms were a threat, but now you can't really acquire stuff in the metaverse because it had to be made. So try to make that. And that fell apart, obviously. Just fell straight flat on his face there. Now, in the AI stuff, you can't just go and acquire anthropic or OpenAI. So you try to do this hiring spree, which is get some people. And as I said, some people quit. And sort of, as you alluded to in the start, Jason, meta is not an OpenAI or an anthropic. They're nowhere near that, it seems. So they do have to start to say, oh, no, we're focused on the thing that comes after super intelligence. And it's like. It's just words like, we haven't seen any results. And that is a pattern with Zuck since Facebook way back when.
[17:57]
Jason Kebler
Yeah, yeah. I mean, we didn't even talk about the ostensible, like, reason for posting this, which I think the reason that he posted this now is that Meta released new open weights AI models and there's a fair part of this that's like open weights AI is the future and that's like what we need to be doing. And for people who don't know, it's like it's just open source. It's not fully open source, but it's basically like you can see what these are AI models where you can see sort of like what it is favoring and this sort of thing. Like it is more open source than OpenAI has right now, than Anthropic has right now. And if I didn't get the specifics of that correct, I'm sorry I cover more like the political side of, of this rather than the technical aspects of it. But in any case this is like him sort of like making the argument for more open AI. Sorry for that, but. And I think a lot of the commentary has been like, he doesn't actually believe this stuff. They're releasing open weights models because they are behind and so they need to differentiate themselves in some way. And by saying like AI should be more open, more accessible, you should be able to see the code, you should be able to see sort of like what it's doing under the hood, like you should be able to fork it, etc. This is Meta's way of differentiating itself from it's competitors that are, that are way far ahead of it. It's like anyone who's used Meta AI and Meta claims it's like billions of people at this point, it's like not a good product. You can't use it for serious work, you can't use it for like, you can use it to make slop. But you know, people are not like turning to Meta AI for their day to day AI use. They're using either Claude or they're using ChatGPT or they're like using you know, Gemini, sort of like, because it's just in all the products they're already using. So to the extent that people are using Meta AI, like maybe they're engaging with it on Facebook or on Instagram, but it's not their like daily AI driver, you know, for the people that are using it. And so this is him trying to just sort of be like, oh, we are an AI company also. Even though, you know, to the extent that Meta is an AI company, it's like done very well with ad targeting and it's done really well with using AI to generate ads on its own platforms. But like it's not on the verge of super intelligence by any stretch. And the other thing that it's, you know, quote unquote, like, good at it is the Meta glasses, the AI glasses. And it's like, that's a widely hated product, if not by the people using it, by society at large, because it's like these creepy surveillance glasses.
[20:58]
Joseph
But they made them and they're out there, you know, so that does differentiate it a little bit from OpenAI or something because they don't have like a wearable. It's just that Meta did make a wearable that some people do buy. It's just that the vast majority of people fucking hate it. But hey, they made something well done, you know.
[21:13]
Jason Kebler
Right. And so, I mean, he leans into it. Like he mentions the glasses several times in this. He's like, oh yeah, we'll be able to use your AI assistant on your glasses. And it's like, okay, that's because you have glasses, that's not because you're selling it.
[21:27]
Joseph
Yeah, you don't hear of anybody personally or in an enterprise talking about token maxing with Meta AI or whoa, guys, we need to clamp down on the Meta AI use inside of our company because you're burning through it because nobody's using it in any serious capacity. Yeah. I think just super briefly before we go to the next session, does he say anything about data centers? Does he touch on that at all?
[21:51]
Jason Kebler
Yeah, I mean, he talks about how we need data centers and he talks about one in Richland Parish, Louisiana, which we've written about a few times. Times we have a freelancer named Roshan Abraham who's covered that. And basically it's like the Manhattan sized data center that Meta is building in Louisiana. And he's basically like, we need to make sure that our data centers are helping the local community, that we're paying taxes, that he's like, we need to own the power infrastructure. Which is like super interesting that he says that, because one, they don't own the power infrastructure there at the moment. And the power infrastructure that they're building is all fossil fuels. It's like quite bad fossil fuels, like in that specific one. And so, I mean, he nods at it, I think, because he just knows that this is like a really hot button political issue. And I do think that Matt has been leaning into this idea that basically teachers in Richland Parish got a pay raise because of the tax money flowing in there. And that's like a political win. Or at least the way that they're painting it, they've they've been like, look, we built a data center here and then teachers got paid. More like data centers are good, actually. We need to reclaim kind of like the argument there. I don't know, your mileage may vary on that one. It's like a lot of the people there don't like it. I think the teachers there have, or some of them have changed their tune because they did get this, like, quite large pay raise. But like that, who knows how long that lasts? The big thing that, I mean, you know, we've talked about it various times, but it's like, yes, there are jobs that come to town that are construction jobs when you're building a data center. But like, how many jobs will be there once this thing is fully up and running and there's no construction needed anymore? So, yeah, he does talk about it.
[23:47]
Joseph
All right, we'll leave that there when we come back after the break. We're going to talk about a company that offers 100% human written, never AI papers, but there's something interesting under the hood. We'll be right back after this.
[24:04]
Jason Kebler
I have a pretty simple test whether a piece of clothing was actually worth buying. When I'm packing for a trip, does it automatically come with me? Because I'm traveling all the time and suitcase space is limited, I'm not bringing something that I just might wear. And lately, the hoodie making the cut every time is my Packa hoodie. That's pronounced Pakka. Pakka makes outdoor and lifestyle clothing using alpaca fiber, one of the world's most sustainable natural fibers. Their best selling hoodie is softer than cashmere, warmer than wool, and breathable, which is a pretty ridiculous combination when you actually wear it. The Pakka hoodie is also thermoregulating and odor resistant. So whether I'm wearing it around the house, traveling, or throwing it on when I'm headed outside, it's incredibly comfortable without feeling bulky. And you might notice I'm not wearing the hoodie. It's been really, really hot in Los Angeles, but I am wearing a Paca T shirt, which I've been wearing all the time around LA in this scorching summer. But every hoodie from Paca is made start to finish in Peru, and each one features an Inca ID hand woven by artisans, honoring generations of knowledge and tradition. More than 250,000 people have already picked one up. So clearly I'm not the only person who's figured this out. Again, I wear the Packa hoodie at night. It does get cold here in Los Angeles in the summer even, but I wore it pretty much all fall and winter as well. And then I've been wearing Paca T shirts and Paca boxer briefs like pretty much every day. So I highly recommend Paca. They make really high quality stuff. To grab your pack a hoodie, go to www.pacaapparel.com that's www.pacaapparel.comp a k a apparel.com we have at this point written an embarrassing number of stories about data brokers. The companies that quietly compile your name, your home address, your phone number, your relatives, your daily habits, and then sell that file to basically anyone with a credit card. Advertisers? Sure. Also spammers, scammers, and the occasional person you'd really prefer didn't have your address, you never opted in. There are hundreds of these brokers and opting out of them one by one is effectively a part time job. That's what Incogni does for you. It acts as your personal privacy advocate, identifying the brokers holding your information and handling the legal legwork of demanding they delete it. Incogni manages the entire request and follow up process on a recurring schedule. Because this data has a notorious way of reappearing, you set it up once and watch the verified removals stack up on a dashboard. The practical result is fewer spam calls, less junk mail, and a much smaller footprint for anyone trying to find or impersonate you. For the most comprehensive protection, we recommend their unlimited plan, which includes custom removals. As journalists, we know that sensitive data doesn't just sit on broker databases. It winds up on public forums, random directories, blogs and news portals. If you find your personal details exposed on a specific site, you simply send Incognito URL and their team of privacy experts will step in to manually handle the corporate takedown request for you. Take control of your data footprint risk free with a 30 day money back guarantee. Go to incogni.com 404 Media and use code 404 Media for 60% off in the annual plan. That's code 404 Media@incogni.com 404 Media it feels like a lot of people are waiting for some kind of signal before they start a business. Almost like an invitation. They think they need more experience, more money, more free time. Somebody telling them, yeah, this is a good idea. At some point you realize nobody's going to give you permission. You just have to build the thing. And Shopify makes that first step a whole lot less intimidating. When we were putting together, the 404 Media storefront. What we appreciated was that Shopify gave us everything we needed to actually get it live and start selling. We were focused on the journalism. We didn't want to spend a lot of time trying to piece together a bunch of different infrastructure before we could figure out whether people even wanted to buy our merchants. Then there's that moment that actually matters, when someone actually wants to buy your products. Shopify checkout makes that part ridiculously smooth. And when customers come back, their details can already be saved. So buying again can be as simple as a tap. That's the sound you want. And as the business grows, Shopify keeps handling the infrastructure so you can focus on the stuff that actually moves the business forward instead of constantly fixing your storefront. That's the biggest reason Shopify powers millions of businesses worldwide, from companies like Mattel and Gymshark to people launching their very first store to 404 Media's merch store. With Shopify, nothing stands between your idea and a real business. So go make it one. If you're ready to stop putting off your business and start hearing, then you're ready for Shopify. Start your free trial@shopify.com media today. You heard that, right. Start your free trial today@shopify.com media. That's shopify.com media.
[29:00]
Joseph
All right. And we are back. The headline of this one that Emmanuel wrote and just published, company offering quote, 100% human written, never AI, end quote. Peer review is entirely AI. I only. I only read this this morning just to prepare for the podcast, and the headline immediately got me. I was like, okay, yeah, we're going to talk about that for sure. So first of all, before we get to the AI stuff, what is research gold and what does it advertise? This is a company, an organization. What's the deal?
[29:37]
Emmanuel Mayberg
Yeah, this is a private company. In order to explain what it does, I think we have to talk a little bit about how new research papers get published in scientific journals and get into the minutiae of that a little bit. So let's say I want to write a research paper about the effect of, I don't know, beans on blood pressure. Right. The first task I would have as a responsible scientist or researcher is to look at all the existing literature and find out what are the previous results out there, make sure I'm not duplicating work, making sure that I am comparing my results against those results. So kind of like doing a review of the literature and then taking all those results and putting them together. In relation to my research question, and this is a very laborious and according to researchers, like a very boring, not rewarding part of the job. It's like the research before you do the research that you actually want to do.
[30:53]
Joseph
But they have to do it, basically.
[30:55]
Emmanuel Mayberg
But that's a responsible thing to do. Yes, that is part of being a rigorous researcher is reviewing the existing research. And this is just like one of the many steps that you don't think about in the publication of new research. And it is one of the things that this company says that you can basically outsource this to. Traditionally, you would either do this yourself or you can hire professional methodologists to work with you and do this work for you. And Research Gold promises to expedite that process, do it for cheap, do it fast, and kind of save you that hassle. And obviously, as the headline implies, they promise that this is all top tier human PhDs and methodologists that are doing this work.
[31:52]
Joseph
It's not even a new service. It's obviously very established in the academic method of doing it. And as you say, you could hire somebody else to do this work before. It's just this company, for some reason is leaning on the. This is all 100% human written. This is never AI. Presumably because at the moment, as you've reported that academic journals are being slammed with AI generated submissions to the point where sometimes we're like, well, we're actually not going to look at meta analyses or literature reviews or anything like that because we're getting so much AI slop churning through. And I presume the marketing here is basically a response to that, right? To reassure people, hey, this is real. You think that's why they're doing it?
[32:38]
Emmanuel Mayberg
Yeah, I mean, it's another field at which AI generated content has really changed how everything works. And I would assume that if you're a serious researcher, you would want to make sure that if you're outsourcing anything, it's not AI generated. It's like you have an actual human PhD looking at the, at the, at the work.
[32:57]
Joseph
Yeah. So what's the problem with Research Gold? This sounds great.
[33:01]
Emmanuel Mayberg
The problem is that none of it is true. It's like quite the opposite. It's all entirely AI. AI generated is the issue.
[33:08]
Joseph
I mean, that's obviously really, really fucking funny and obviously incredibly ironic. So you hear about Research Gate, I think a researcher tells you about it, you then look, Research Gold.
[33:22]
Emmanuel Mayberg
Sorry, I thought you were talking about Research Gold, not Research Gold.
[33:25]
Joseph
Sorry, sorry, my bad. So you're looking at this company. How do you figure out this AI generated, Is it pretty obvious from the jump or like, how do you go about figuring that out?
[33:37]
Emmanuel Mayberg
It's extremely obvious. I learned about it from a researcher who was doing some of this work that I just discussed in preparation to defend his dissertation. And he stumbled upon this website and I was like, oh, this is interesting and potentially useful. And just a few minutes of clicking around, he figured out that it's entirely AI generated. There are a few very obvious signs just from the website. One is if you look at the team of researchers that operate this website, they're completely made up. They are names that have no digital footprint, certainly not one that matches their expertise as claimed on the website. And then they have obviously AI generated profile images. I was doing some more digging and I found that there was like a separate team listed on the site of these professional methodologists, which again is like a profession that exists. Like there are freelance methodologists out there that you can hire to work on your research. And there were about five of them listed on the site. And I reached out to, I searched them like on LinkedIn and other platforms. They were clearly real people, they were clearly in this field. I reached out to all of them and I heard back from one who said, yep, this is me on the site. I have no idea what the site is, I've never worked with them, they're stealing my identity. And then quick, she, she said that she would reach out to them and see what her options are in terms of getting that removed. And quickly after that, the entire page where these human methodologists were listed was removed. Also, just like as a funny side note, all the profile pictures on these real methodologists were the exact same profile pictures that were used on LinkedIn, including the Open to work hashtag. You know, the, you know, when like somebody is looking for work on LinkedIn, they have like a little graphic added to their profile image. So like that was also there, which I think is like not what you would use if you were like advertising your, your, your methodology.
[35:58]
Joseph
Well, no, no, maybe it shows they're really open to work, to work with
[36:02]
Emmanuel Mayberg
people who want to come open to work with you. Researchable client. Yeah, exactly.
[36:07]
Joseph
So there's the first set which is like the main team of people allegedly behind this service and they're completely made up and they have AI generated photos and names that don't correspond to anybody, all of that sort of thing. And then there is another page of another team of people who are real, but they're being ripped off and they have Nothing to do with research gold. I mean you only spoke to one, but presumably they were all ripped off because the entire page was taken down.
[36:32]
Emmanuel Mayberg
Right, right, right, yeah. And I would say like, again, this is just stuff that you would see by clicking around on the site and doing some Googling. I also did a bunch of other stuff like calling them, which is a very funny conversation that I had on the phone with an AI agent that refused to admit it was an AI agent. Yeah, just like I kept saying like, hey, are you human? Can I talk to a human? And this AI agent that was identified as Sarah was like, oh, hahaha, that's real funny. But let's focus on your research and like let's try to get you. Let's try to get you a quote on your research. And I was like, oh, what's your last name? It's like, oh, let's keep it simple. I'm Sarah. Yeah, let's not get confused with, with these complicated questions.
[37:17]
Joseph
Yeah, I don't think you know this, but just to ask it, like, do you have any idea who is actually behind it? Because you didn't get a response when you asked them for comment. You got this stupid. Clearly AI generated Sarah. All of the people that are generated, like any idea who's actually behind this or.
[37:36]
Emmanuel Mayberg
I was not able to figure it out. They do have a physical address listed which goes back to like some office complex in Texas. But I'll be cautious. I'm not claiming that the address is actually affiliated with them. Maybe they just picked a random place on the map. But yeah, no, we don't know who is actually like offering the service.
[37:57]
Joseph
We'll just talk about another couple of things very quickly. Just because I thought they were, you know, somewhat related to AI and faking it and being caught and that sort of thing. So you recently did a piece about how Substack has introduced something to do with AI that flags a piece as AI generated or something like that. What's that feature and how are people reacting to that? Which is a newsletter platform for those who, I presume everybody knows it, but you can use it to send newsletters and write stuff on the Internet, Correct?
[38:34]
Emmanuel Mayberg
Yeah. Can we call it. I'll speak for myself? It's the bad one? I would say it's the bad one.
[38:40]
Joseph
I think that's probably fair. Yes. And we did not go with it for various reasons, including that we never
[38:45]
Emmanuel Mayberg
really even seriously considered it. And it's bad for all the reason everybody knows, like people they're affiliated with in some business Practices and so on. Anyway, doesn't matter with quite like, like. With some pride and fanfare, Substack announced that it has partners with Pangram, which is like an AI detection software. Disclosure. Pancram has advertised with us once in the past and they made it possible for any reader to use Pangram to check whether the content of a newsletter is AI generated. And it does it in the way that some other detectors do it, which is like percentage
[39:27]
Joseph
score, like a confidence rating or I think it's how much
[39:31]
Emmanuel Mayberg
of it was AI generated. Correct me if I'm wrong, but I believe it's like 50% of this newsletter was AI generated, like 80% or something like that. And I think that is in theory, like it's directionally a good thing, I would think, because it is acknowledging. And the announcement from Substack makes this clear that people don't want a ton of AI generated content, especially if it's coming from a platform like Substack, which ostensibly is a place where you choose to follow a specific person and you're following for their human insight and they're delivering the content directly to you. A lot of like the selling points of our website, right? And in that context, I think it is pretty offensive to then send a bunch of ad generated content to people who are paying for your newsletter. And that's sort of the framing that Substack gives. And there are two issues with it from Substack's perspective. The issue is that clearly they have a lot of people who are using AI to generate newsletters on their platform. And, and those people are pissed. They're like, how dare you do this to us? It's my prompt. Like, it's my insight. Just because, like an AI wrote the text, it doesn't mean that my prompt and my, my thoughts are not unique and valuable. And clearly they're valuable because people are subscribing and that's like a marketing and business problem for, for an audience problem for, for Substack. I think the bigger problem that applies to everyone across the Internet and across all attempts to identify AI is that there are false positives. Even, even Pancream, which is considered, you know, one of the better tools for detecting AI generated content. It's not as if it never flags human content as AI and then you're creating like a culture of suspicion and what a lot of bloggers on that platform called like a witch hunt, right? It's like, is this AI generated? Is it not? If you accidentally get flagged, you're forever besmirched. As using AI and so on, which is, you know, I think, again, the intentions, I think, are good, but it's a very real unsolved problem.
[41:43]
Joseph
Yeah. Especially because the AI detectors, frankly, until very, very recently, straight up sucked, and they were basically like snake oil. Oh, yeah. We can use this to detect AI, and we would try them in the course of reporting to not only figure out whether something was AI, but just because we report on those tools as well. And they were garbage a lot of the time. And that's changed, I feel like, in the past 12 months, something like that. So there's that part, and this is a new phenomenon now where maybe these tools are more reliable. But just on the other one where you said that the criticism or the response from some of these people who do use AI to write on substack, they're like, well, it's my prompt, so what's the issue? Where it's like, well, if you're omitting it, then what's the problem? If you're openly using AI, what's the problem of readers knowing that, oh, this was scam with AI? It doesn't seem to gel those two positions together.
[42:39]
Emmanuel Mayberg
It's because of the stigma. I'm sorry, I'm taking this in a completely different direction, and it's not something we plan to talk about, but has everybody heard about Phoenix Flexing in that controversy?
[42:51]
Joseph
Literally no idea what you're talking about.
[42:52]
Emmanuel Mayberg
It's like this very viral rap song and like an 80s style that's become like, a legitimate top 100 hit. And I am not saying this that I'm 100% sure, but there's a lot of reporting or a lot of videos from other YouTubers who claim pretty confidently that it was AI generated and the artist Phoenix Flexing claims that it is not. But, yeah, I don't know. Famously, I'm like the. The. The least musically inclined person on the podcast, and I thought, like, it sounded like a cool song, but. But this is exactly the problem. You know what I mean? It's like the song is a hit, clear that people like it, but now that. That it's credibly accused as being AI generated, it's a huge thing, and people are very angry at the artist.
[43:43]
Joseph
But then I guess some people just won't give a shit. Potentially.
[43:47]
Emmanuel Mayberg
Clearly not, right? Clearly not, because the thing is doing well.
[43:51]
Joseph
Last question before we go to the subscriber section. Not to get into the details of how this will work, because, frankly, I still think we need some answers and we need to see it deployed, but Claude or Avanthropic says Claude is now going to watermark text generated by his AI models released after early August. And this is to comply with some European legislation, but I think it's going to be rolled out worldwide. Basically you might be able to tell if a piece of text was generated by Claude. How do you think people are going to react to that, Emmanuel? Again, we don't know yet because it hasn't quite happened. But do you think they're going to react in a similar way to the substack one or two?
[44:31]
Emmanuel Mayberg
I don't think it's going to be the same controversy as the substack one because again, like that, that personal relationship that people have with newsletters, I think is the issue there. But I will say that while watermarking is important and AI companies should continue to try to watermark AI generated content, it is sort of a fig leaf in terms of addressing the problem of flooding every communication channel with AI generated content. Because it's like, okay, yes, maybe if you have a bunch of AI generated scientific papers and they're watermarked, you can find them, but it still requires you to filter through the mess, which is the problem. If you're getting x1000 submissions to your scientific journal and you have to figure out which ones are ASMR AI generated and which ones are not, the fact that they are watermarked I don't think is a huge help. You know, it's like you still have the slot problem is still. It's still the SLO problem.
[45:36]
Jason Kebler
I agree with that. I also think it's a mix of people who are making AI generated content for slop purposes. Like one, many of them don't care if people know that it's AI generated or not. Sort of like as Emmanuel said, because it's a numbers game, you're trying to just like win an algorithmic lottery. But then the people who do care always find a way to circumvent the watermark. And so, I mean, in some cases it's as simple. Like with images, it's as simple as like cropping it out. Sometimes it's like you open it up in a different video and an image editor and like change things there and with text. Like, I don't understand exactly how this will work. Maybe, maybe Anthropic has explained it, but I don't see what would. Like if you copy it over to Notepad or TextEdit and use something to strip all the metadata from that. I don't know how a watermark could persist across apps and things like this unless there's a technology that I am not aware of. But I think it's something where it's like if you are seriously trying to circumvent the watermark, people will be able to do it. And if you're not, you still have the slop problem.
[46:55]
Joseph
Yeah, makes sense. All right, we will leave that there. If you're listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 media subscriber, we're going to talk about how while a bunch of cities are ditching Flock, they are immediately replacing them with Axon, which is a, you know, infamous law enforcement contractor and Flock rival. You can subscribe and gain access to that content at 404 MediaCo. As a reminder, 404 Media is journalist founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really, really does help us out. This has been 404 Media. We'll see you again next.
[48:03]
Emmanuel Mayberg
Sam.

Summary7 min read

The 404 Media Podcast

Episode: Mark Zuckerberg’s 'Superintelligent' AI Future That No One Wants
Date: August 12, 2026
Hosts: Joseph, Sam Cole, Emmanuel Mayberg, Jason Kebler

Brief Overview

In this episode, the 404 Media team dissects Mark Zuckerberg’s latest 6,500-word manifesto about Meta’s vision for an “AI superintelligent” future built around personal agents. The discussion critically engages with Zuckerberg’s utopian—and, in their view, largely unwanted—concepts of always-on personal AI, the reality (and failures) of Meta’s AI offerings, and broader industry patterns around ambitious tech manifestos. In the second half, the crew explores the rise of fraudulent academic services pretending to reject AI while being AI-powered, the practical pitfalls of AI content detection and watermarking, and public reactions to these developments.

Key Discussion Points & Insights

1. Mark Zuckerberg’s AI Manifesto: Grand Promises, Dubious Connection to Reality

Initial Impressions and Cynicism

The team sees Zuckerberg’s essay—titled The Future Is for Everyone—as a repeat of his Metaverse-era declarations: grandiose, out of touch, and recursive.
Quote - Jason Kebler [03:53]:

"[Zuckerberg] posts this... describing a future that he says is for everyone, but which really sounds less social than ever."
Meta, feeling behind in the AI race, is striving for relevance by pitching personal, consumer-facing AI agents, compared to competitors focused on business and enterprise.

Personal AI Agents: Utopia or Unwanted?

Zuckerberg’s vision: each person will have a “superintelligent” AI agent—an always-on assistant handling health, career, finances, home, and more.
Zuckerberg's pitch, paraphrased by Jason Kebler [04:29]:

"Everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about. Your agent will work 24/7 on your behalf... It will have strong privacy and security options... interact [with you] through any device, including your glasses..."
The team doubts this pitch, viewing it as “outsourcing things normal humans actually enjoy.”
Quote - Jason Kebler [07:33]:

"The real thing here is that Zuckerberg is pitching a future... that also is not a future that really many people at all want."

Critique of Technosolutionism and "Everyone as Entrepreneur"

Zuckerberg (echoing other Silicon Valley leaders) suggests those automated out of jobs should become entrepreneurs or inventors with AI help—a future the panel finds deeply disconnected from most people’s lives.
Quote - Joseph [10:29]:

"Not everybody wants to do that. And there's that through line... that kind of just assumes this is what people want and probably a ton of them don't, actually."

Platitudes and Systemic Issues

The essay is filled with empty, circuitous language, e.g.,
Quote - Jason Kebler [12:04]:

"Humanity is not a monoculture. People's diverse values represent different trade offs..."
The team critiques Silicon Valley's recurring pattern: an ultra-wealthy CEO conjures “solutions” for humanity with little popular demand, often after a previous failure (e.g., the Metaverse).
Quote - Jason Kebler [13:57]:

"It's this kind of hubris and ego trip that Silicon Valley CEOs have... It's not clear that this is like something that the masses are interested in."

2. Reality Check: Can Meta Even Deliver on its Promises?

Track Record of Meta’s AI Products & Culture

Sam Cole, who has tested Meta's AI chatbots and the Metaverse, is skeptical of their ability to execute: Quote - Sam Cole [14:46]:

"No, definitely not... He just is so detached from the actual experiences of most people... It's such a clear pattern at this point... scrambling before something totally shits the bed."

The Open Weights Gambit

Jason explains Meta’s current focus is “open weights” (partially open-source AI models) as a PR move to differentiate from more advanced competitors.
Quote - Jason Kebler [17:57]:

"A lot of the commentary has been like, he doesn't actually believe this stuff. They're releasing open weights models because they are behind..."

AI Glasses and Hardware

Meta touts its AR glasses as proof of innovation, but these are derided by the panel as “creepy surveillance” and widely disliked outside of the company’s product ecosystem.
Quote - Jason Kebler [20:57]:

"...the other thing it's, you know, quote unquote, good at is the Meta glasses... that's a widely hated product, if not by the people using it, by society at large..."

Data Center Politics

The essay also nods to Meta's massive investment in US data centers, using tax incentives as political cover and claiming local economic benefits.
Quote - Jason Kebler [21:50]:

"...Meta is building in Louisiana... [saying] teachers got paid more like data centers are good, actually... The big thing... is how many jobs will be there once this thing is fully up and running?"

3. “100% Human Written—Never AI!”: The Fraud of Research Gold

AI Slop in Academic Publishing [29:36 – 37:36]

Emmanuel reveals “Research Gold,” a company promising human-only, PhD-reviewed academic summaries, is actually entirely AI generated.
- Staff photos are AI fakes.
- Another team page rips off real scientist profiles and uses their LinkedIn photos—complete with “Open to Work” banners.
- Emmanuel directly calls the company, only to get an unconvincing AI phone agent (“Sarah”).
Quote - Emmanuel [33:01]:

"The problem is that none of it is true. It's like quite the opposite. It's all entirely AI generated."

Why Is “Human Written” Such a Selling Point?

Flood of AI content in academia has led journals and researchers to desperately seek “human-created” work, generating paradoxes—and new forms of fraud.

4. AI Content Detection and the Culture of Suspicion

Substack’s Pangram Integration [38:34 – 43:43]

Substack, the newsletter platform, now offers a tool (Pangram) that flags newsletters as AI-generated—providing AI-content percentages.
Many writers using AI are angry; they argue their prompts justify the content's value and resent the stigma of being “outed.”
Quote - Emmanuel [39:30]:

"...it's my prompt and my, my thoughts are not unique and valuable. And clearly they're valuable because people are subscribing..."
Warnings about false positives: AI detectors are still imperfect, and such flagging can drive “witch hunt” behavior and mistrust.

Music and AI Accusations

Touches on the viral “Phoenix Flexing” controversy: a hit rap song accused of being AI-generated, sparking anger and suspicion among listeners.
Quote - Emmanuel [42:52]:

"...it's credibly accused as being AI generated, it's a huge thing, and people are very angry at the artist."

5. AI Watermarking: Is It the Solution?

(Timestamps: 43:51 – 46:54)

Claude (Anthropic) to watermark its AI’s text outputs for compliance, but panel doubts it will stem the flood of “AI slop.”
- Watermarks might help with filtering but don’t address the scale or ease of circumvention.
Quote - Jason Kebler [45:35]:

"...the people who do care [about hiding AI] always find a way to circumvent the watermark..."

Timestamps for Key Segments

Zuckerberg Manifesto Debrief: 01:37 – 13:57
Reality Check: Meta’s AI Track Record: 13:57 – 21:26
Data Center Discussion: 21:50 – 23:47
Research Gold Fraudulent Service: 29:36 – 37:36
AI Detection on Substack: 38:34 – 43:43
AI Watermarking Debate: 43:51 – 46:54

Notable Quotes & Memorable Moments

On Zuckerberg’s AI vision:
"The real thing here is that Zuckerberg is pitching a future... that sounds a lot like the present in some ways, and then also is not a future that... really many people at all want." — Jason Kebler [07:33]
On Meta's pattern of hype and detachment: "He just is so detached from the actual experiences of most people that I think he thinks that the things that he feels is what everyone feels, which is obviously like CEO brain..." — Sam Cole [14:46]
On AI-generated "human" work in academia:
"It's all entirely AI generated is the issue." — Emmanuel Mayberg [33:01]
"...there are false positives... you're creating a culture of suspicion..." — Emmanuel Mayberg [39:26]
On the limits of watermarking:
"In some cases, with images, it's as simple as cropping [the watermark] out... with text... I don't see what would [prevent someone from] copy[ing] it over to Notepad or TextEdit... I think if you are seriously trying to circumvent the watermark, people will be able to do it." — Jason Kebler [45:35]

Tone and Language

The conversation, informed by investigative expertise and a critical orientation, blends sarcasm and skepticism. The hosts use direct language, frequently paraphrase or quote from the Zuckerberg essay, and push back against Silicon Valley platitudes. Humor undercuts much of the AI hype (“floating around with no legs” in the Metaverse, [16:04]; AI phone agents refusing to admit they’re bots [36:31]), but the analysis is rigorous and rooted in real-world impacts.

Summary Flow

The episode starts with a takedown of Zuckerberg’s “future of AI for everyone,” transitions to an assessment of Meta’s actual product pipeline, and then zooms out to the cultural and ethical chaos caused by “AI slop”—from fake academic services to algorithmic skepticism infecting journalism and art. The end of the episode speculates on the technical and societal effectiveness of AI watermarking as a mitigation strategy. Throughout, the panel asks: Who is this AI-tinted future actually for, and who’s benefiting from all this change?

Loading summary

Transcript73 lines

[00:01]
Jason Kebler
This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50 page restoration block. Or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it,
[00:17]
Emmanuel Mayberg
ready to make anything online make sense.
[00:20]
Jason Kebler
There's no place like Chrome. Check responses set up, required compatibility and availability.
[00:24]
Emmanuel Mayberg
Various 18.
[00:26]
Jason Kebler
Sam Altman earlier this month heard from parents who were generating personalized podcasts to like listen to with their kids on the drive to school. The drive to school is like when you should be actually talking to your kid. How about you try talking to your kid versus having an AI podcast? Do it.
[00:46]
Joseph
Hello and welcome to the 404 Media podcast where we bring you unparalleled access to hidden worlds, both online and IRL. 404Media is a journalist, founder, company and needs your support. Support. To subscribe, go to 404 Media Co as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content@404 media co. Also do remember to subscribe to our YouTube channel where you can watch all of our episodes, including this one. Subscribe@YouTube.com Mediaco I'm your host, Joseph, and with me are the other 404 Media co founders, the first being Sam Cole.
[01:31]
Sam Cole
Hello.
[01:32]
Joseph
Emmanuel Mayberg.
[01:33]
Emmanuel Mayberg
Hey, what's up?
[01:34]
Joseph
And Jason Kebler.
[01:36]
Jason Kebler
Hello. Hello.
[01:37]
Joseph
All right, we're starting with the sporadic maybe regular Mark Zuckerberg manifesto. You know, he'll regularly. Was it every six months or something? Maybe every year he'll write something like this or he'll post some insane new video. A ton of people read this and obviously you had some thoughts about it. Jason. So the headline of your piece, mark Zuckerberg posts deranged 6,500 word essay about giving everyone AI super intelligence. Let me just read the lead, or rather the second part of it because I think it'll sum up what you're trying to say is that he posts this. He's describing a future that he says is for everyone, but but which really sounds less social than ever. There's a lot to get into here, Jason. Don't drill into the specifics yet, but like when you read this, like, what was your media reaction to this deranged essay?
[02:37]
Jason Kebler
Yeah, I mean Mark Zuckerberg again does this every so often. Like he did a huge one for the Metaverse. He's done several different ones for AI. And it's basically like a big, long post on his own Facebook or on, like, Meta's blog. And again, this one is called the Future Is for Everyone. And it is essentially his vision for AI superintelligence, which he has been talking about, maybe even more than Sam Altman and Dario Amadei, like, more than the other kind of, like, AI tech mavens. And I feel like he's been doing this maybe in part because Meta feels like it has fallen behind in AI and so he's like, really leaned into, like, we are pitching a future where AI is going to change your life. There's a lot in this essay about how the other AI companies are building AI for businesses and enterprises, whereas Meta is building personal AI for individual users. And so he's trying to really differentiate himself or Meta's product as being this one that's, like, designed for the common person and how it's going to change your life and all this sort of thing.
[03:53]
Joseph
Yeah, definitely a lot of talk with all of Meta's massive AI hires and then people immediately leaving the company, offering people in the AI industry bucket loads of money, and some of them being like, nah, I'm good, I don't want to go work at Meta. It does seem pretty desperate in a lot of ways. So, more specifically on the agents, what is he saying that Meta is going to offer people? Like, you're going to have your own agent. And so what, like, what is Zuckerberg saying we're going to do here?
[04:29]
Jason Kebler
Yeah, I mean, I think the big thing is that there's a lot going on in this essay. So the broad majority of it, like the beginning talks about personal agents, like super intelligent agents that will work for you, but then he later gets into, like, data centers and why the United States needs to lead the AI revolution. And he talks about China and he talks about open weights, which we can talk about. But, like, I think the core thing to understand is that this essay is all over the place. It's not very coherent. It has tons of platitudes that are about nothing. And so, like, with that kind of backdrop, let's talk about agents. For people who don't know, an AI agent is like a personalized AI that you give access to, like, act on your behalf, more or less. And so you would give it, you know, access to your accounts, your messaging, maybe your bank, like, all sorts of things. And then it would, like, theoretically learn about your life and go on and do things for you. And what Mark Zuckerberg is saying is that, like, you will have this personalized AI agent that will free up all sorts of time for you. And then he sorts of, like, gets into specifics. I think it's best to just, like, read a quote here. So this is how he, like, introduces it. He says, everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about. Your agent will work 247 on your behalf to improve your relationships, health, career, finances, home management, hobbies, and more. It will free up time for the things you enjoy and help you accomplish more than you could otherwise. It will have strong privacy and security options so you can trust to handle all of your personal content, knowing that no one else can access your information. You'll be able to interact with your agent through any device, including your glasses, which is funny to keep you present in the moment with the people you care about. So, I mean, this is, like, the pitch for all sort of, like, agents, where it's like, oh, well, isn't buying a flight so annoying? You can just tell your agent that you want to buy a flight, and it will find the best flight for you and it will buy it for you, or it will just know that you need to go somewhere and it will proactively buy a flight for you. He goes on to sort of, like, explain how his own agent that he's made is, like, developing recipes for him to make with his daughter and things like this, like, kind of outsourcing things that normal humans enjoy doing. Part of the point is to spend time and show that you care. It's like, oh, well, the agent did it for me.
[06:58]
Joseph
Yeah. So he simultaneously says that our AI agents will go on your behalf and they will free up time for what really matters. And presumably that would be something like crafting a recipe to make with your daughter or something like that. But no, no, he's having the AI do that. I mean, yes, he's still doing the cooking, but I don't know, he's outsourced half of it to the AI. I thought the whole point was to do that thing with your family. So, I mean, I don't really know. Does he not enjoy that part? Or, like, is that a theme you keep seeing in this essay?
[07:33]
Jason Kebler
I mean, that's a theme in this essay. It's a theme in, like, you know, Sam Altman last week or earlier this month, said that he had heard from parents who were generating personalized podcasts to, like, listen to with their kids on the drive to school. And everyone's like, well, the drive to school is like, when you should be actually talking to your kid. How about you try talking to your kid versus having an AI agent or, like, an AI podcast? Do it. The real thing here is that, like, Zuckerberg is pitching a future that he says is for everyone, but that sounds a lot like the present in some ways, and then also is not a future that, like, necessarily everyone wants or that really many people at all want. And so he sort of like, talks about AI job loss, for example, or he's like, well, automation is happening. But then he's like, but your personal AI agent will allow you to do your own scientific research and to make your own inventions and all of these sorts of things and to start your own business. And it's like, not everyone wants to become an inventor. Not everyone wants to cure cancer. Not everyone. I mean, people want to cure cancer, but, like, people don't want to.
[08:41]
Joseph
Like, not everybody.
[08:42]
Jason Kebler
Not everyone can do it. It's basically. He's basically, like, everyone will be able to do everything all the time, and, like, your AI agent will be doing it for you. He's talking about how, like, people at Meta are using AI to discover, like, new crystal structures that can go into the Meta glasses. And, like, you may be able to do your own. And it's sort of. It reminds me.
[09:03]
Joseph
But you're making the glasses.
[09:05]
Jason Kebler
Yeah, it reminds me a bit of vibe coding, where it's like, oh, like, you can. With vibe coding, you can, like, code your own email client and your own chat client and your own, like, calendar app and all of this. And it's like. Or if that's not something I'm passionate about, I can just use someone else's. And I shouldn't have to become a coder just because that's what we've found AI is good at doing at the moment.
[09:28]
Joseph
Yeah. Just on the work thing, when Zuckerberg is saying, your AI will do all these things so you can go and make your own business or make that railroad, straightforward. Just the other day, I was listening to an interview with Gary Tan, the CEO of Wai Combinator, on the Verge's decoder podcast, and he said something to the effect that was very much the same, like, AIs are going to mean you can go make your own business. And it's very, very straightforward. Actually, I'm sorry, I think it was the Wall Street Journal. I'll double check that. But he brings up that point, and of course, it's in the discussion of labor. And I think the interviewer says, well, look, people are going to lose their jobs. But Tan just replies, well, yeah, but those people should now go become entrepreneurs. But not everybody wants to do that. And there's that through line, I think in that Tan interview and then in this Zuckerberg essay that, as you say, Jason, it kind of just assumes that this is what people want and probably a ton of them don't, actually.
[10:30]
Jason Kebler
Yeah. And I mean, he also says repeatedly, like, the future is for everyone. This AI future is for everyone. And he then describes what this means and how Meta is working on making free AI AI tools of which, you know, there's a lot of free AI tools out there, at least free for the user. And then he describes like this future system where he was like, it has to be accessible to everyone. And then he explains, quote, for everyone to be part of the future, everyone must have the ability to use super intelligence to improve their lives and shape the world. We will offer free versions that will be accessible to billions of people. For those who want to pay to use more compute, there will be a dynamic auction mechanism that will guarantee that everyone gets the lowest price possible for the intelligence and compute they're using, while also ensuring the capacity is used for whatever people collectively find most valuable. And that's just like the system we already have where you, some of it is free and some of it you have to pay for. And if you pay for it, you get more and better. And if you have more and better, it's going to be a tiered system. It sounds like he's recreated the concept of just paying for AI compute and then there's tons of just like drivel and, and like, like weird thought experiments and, and just like platitudes out there. Like these are real sentences from this essay. Quote, humanity is not a monoculture. People's diverse values represent different trade offs that would make, they would make on important issues. Like that means nothing.
[12:04]
Joseph
Thanks, Matt.
[12:05]
Jason Kebler
People have an infinite demand for new experiences and all and have always found new problems to tackle. It's just like a lot of it is. I don't want to say it was like written by AI because it is so weird that I don't think that it was. But it is basically like this just big manifesto. And I think it's just worth remembering that, you know, this is like a top 10 most rich person in the world. I don't know where he is at the moment, but he's just like wildly wealthy. He is driving this innovation, or rather he's like pushing this product on people. He's, you know, the metaverse was A gigantic failure. And now he's saying, well, like, this is going to be the thing that we do. And it's this kind of like hubris and an ego trip that Silicon Valley CEOs have where they say, like, here is what humanity is going to do and we are going to be the ones to take us there. It's like you see in Mark Andreessen's like, it's time to build. You see it with Elon Musk being like, we're going to colonize Mars, we're going to have driverless cars, we're going to have all this stuff. And some technological progress is quite good. But in order to do all of this, it's like there's a huge capital expenditure. Like all of the products are, you know, initiative in a way that is really annoying and changes how people have like, lived and worked for a long time. In the case of Elon Musk and also Mark Zuckerberg, there's like tons of tax breaks involved, there's lots of government lobbying involved, there's lots of, you know, handouts associated with it. And so, you know, these people are quite successful at getting like government buy in for their vision. But it's not clear that this is like something that the masses are interested in.
[13:57]
Joseph
Yeah, absolutely. So, Sam, you've probably interacted with Meta's AI products, I think, more than anyone here. Well, also the Metaverse as well, because I think you have a quest, right? And I think you went in and saw how garbage that was and then you did a bunch of investigations into Meta's AI chatbot around them pretending to be licensed therapists and that sort of thing. That's a little bit different to the agentic AI they're promising here. A chatbot is not exactly the same as a Gentak AI, blah, blah, blah, whatever. But like, just based on your experience of actually interacting with and using Meta AI tools, like, do you think Meta could pull this off in any sort of convincing way?
[14:46]
Sam Cole
No, definitely not. I mean, like, the assessment that you guys have already discussed is I agree with completely. It's like, I guess it was last year, Zach was like, you guys don't have enough friends, so our chat bots will be the answer to that. He just is so detached from the actual experiences of most people that I think he thinks that the things that he feels is what everyone feels, which is obviously like CEO brain, tech exec brain to a T. And you know, he has a lot of people around him who are like sick fence and are telling him that That's a great idea. And, yes, manning his way to these crazy screeds. But, yeah, I mean, he. This is so classic as. As you guys already said, of the scrambling that he does before something totally shits the bed. It was before the metaverse fell apart and that dream died that he was pushing it the hardest and saying, this is the future that everyone's going to have and we're all going to be in the verse and won't it be wonderful and lovely, and we're going to be floating around with no legs. It's. It's such a clear pattern at this point that I. I don't know why he. I don't know who lets him continue to publish these things. I don't know why someone around him isn't like, hey, maybe let's not post this right now. Maybe let's have an edit of this before we say it completely. But I'm glad no one is stopping him because it's very revelatory of his mindset and what he thinks is happening is where the company goes, whether it works or not.
[16:35]
Joseph
Yeah.
[16:36]
Jason Kebler
I mean, good content.
[16:38]
Sam Cole
I feel bad for this country, but it's fantastic content.
[16:41]
Joseph
So to take it back even further, and I know this is sort of an obvious historical timeline, but, you know, with Facebook, he was like the leader in social media and social networking, and obviously that is how Facebook and their meta became this corporate giant. Right. And then through acquisitions of WhatsApp and Instagram. Okay, we've acquired that. So now we're increasing. We're keeping our relevance and increasing it as well, of course, because those platforms were a threat, but now you can't really acquire stuff in the metaverse because it had to be made. So try to make that. And that fell apart, obviously. Just fell straight flat on his face there. Now, in the AI stuff, you can't just go and acquire anthropic or OpenAI. So you try to do this hiring spree, which is get some people. And as I said, some people quit. And sort of, as you alluded to in the start, Jason, meta is not an OpenAI or an anthropic. They're nowhere near that, it seems. So they do have to start to say, oh, no, we're focused on the thing that comes after super intelligence. And it's like. It's just words like, we haven't seen any results. And that is a pattern with Zuck since Facebook way back when.
[17:57]
Jason Kebler
Yeah, yeah. I mean, we didn't even talk about the ostensible, like, reason for posting this, which I think the reason that he posted this now is that Meta released new open weights AI models and there's a fair part of this that's like open weights AI is the future and that's like what we need to be doing. And for people who don't know, it's like it's just open source. It's not fully open source, but it's basically like you can see what these are AI models where you can see sort of like what it is favoring and this sort of thing. Like it is more open source than OpenAI has right now, than Anthropic has right now. And if I didn't get the specifics of that correct, I'm sorry I cover more like the political side of, of this rather than the technical aspects of it. But in any case this is like him sort of like making the argument for more open AI. Sorry for that, but. And I think a lot of the commentary has been like, he doesn't actually believe this stuff. They're releasing open weights models because they are behind and so they need to differentiate themselves in some way. And by saying like AI should be more open, more accessible, you should be able to see the code, you should be able to see sort of like what it's doing under the hood, like you should be able to fork it, etc. This is Meta's way of differentiating itself from it's competitors that are, that are way far ahead of it. It's like anyone who's used Meta AI and Meta claims it's like billions of people at this point, it's like not a good product. You can't use it for serious work, you can't use it for like, you can use it to make slop. But you know, people are not like turning to Meta AI for their day to day AI use. They're using either Claude or they're using ChatGPT or they're like using you know, Gemini, sort of like, because it's just in all the products they're already using. So to the extent that people are using Meta AI, like maybe they're engaging with it on Facebook or on Instagram, but it's not their like daily AI driver, you know, for the people that are using it. And so this is him trying to just sort of be like, oh, we are an AI company also. Even though, you know, to the extent that Meta is an AI company, it's like done very well with ad targeting and it's done really well with using AI to generate ads on its own platforms. But like it's not on the verge of super intelligence by any stretch. And the other thing that it's, you know, quote unquote, like, good at it is the Meta glasses, the AI glasses. And it's like, that's a widely hated product, if not by the people using it, by society at large, because it's like these creepy surveillance glasses.
[20:58]
Joseph
But they made them and they're out there, you know, so that does differentiate it a little bit from OpenAI or something because they don't have like a wearable. It's just that Meta did make a wearable that some people do buy. It's just that the vast majority of people fucking hate it. But hey, they made something well done, you know.
[21:13]
Jason Kebler
Right. And so, I mean, he leans into it. Like he mentions the glasses several times in this. He's like, oh yeah, we'll be able to use your AI assistant on your glasses. And it's like, okay, that's because you have glasses, that's not because you're selling it.
[21:27]
Joseph
Yeah, you don't hear of anybody personally or in an enterprise talking about token maxing with Meta AI or whoa, guys, we need to clamp down on the Meta AI use inside of our company because you're burning through it because nobody's using it in any serious capacity. Yeah. I think just super briefly before we go to the next session, does he say anything about data centers? Does he touch on that at all?
[21:51]
Jason Kebler
Yeah, I mean, he talks about how we need data centers and he talks about one in Richland Parish, Louisiana, which we've written about a few times. Times we have a freelancer named Roshan Abraham who's covered that. And basically it's like the Manhattan sized data center that Meta is building in Louisiana. And he's basically like, we need to make sure that our data centers are helping the local community, that we're paying taxes, that he's like, we need to own the power infrastructure. Which is like super interesting that he says that, because one, they don't own the power infrastructure there at the moment. And the power infrastructure that they're building is all fossil fuels. It's like quite bad fossil fuels, like in that specific one. And so, I mean, he nods at it, I think, because he just knows that this is like a really hot button political issue. And I do think that Matt has been leaning into this idea that basically teachers in Richland Parish got a pay raise because of the tax money flowing in there. And that's like a political win. Or at least the way that they're painting it, they've they've been like, look, we built a data center here and then teachers got paid. More like data centers are good, actually. We need to reclaim kind of like the argument there. I don't know, your mileage may vary on that one. It's like a lot of the people there don't like it. I think the teachers there have, or some of them have changed their tune because they did get this, like, quite large pay raise. But like that, who knows how long that lasts? The big thing that, I mean, you know, we've talked about it various times, but it's like, yes, there are jobs that come to town that are construction jobs when you're building a data center. But like, how many jobs will be there once this thing is fully up and running and there's no construction needed anymore? So, yeah, he does talk about it.
[23:47]
Joseph
All right, we'll leave that there when we come back after the break. We're going to talk about a company that offers 100% human written, never AI papers, but there's something interesting under the hood. We'll be right back after this.
[24:04]
Jason Kebler
I have a pretty simple test whether a piece of clothing was actually worth buying. When I'm packing for a trip, does it automatically come with me? Because I'm traveling all the time and suitcase space is limited, I'm not bringing something that I just might wear. And lately, the hoodie making the cut every time is my Packa hoodie. That's pronounced Pakka. Pakka makes outdoor and lifestyle clothing using alpaca fiber, one of the world's most sustainable natural fibers. Their best selling hoodie is softer than cashmere, warmer than wool, and breathable, which is a pretty ridiculous combination when you actually wear it. The Pakka hoodie is also thermoregulating and odor resistant. So whether I'm wearing it around the house, traveling, or throwing it on when I'm headed outside, it's incredibly comfortable without feeling bulky. And you might notice I'm not wearing the hoodie. It's been really, really hot in Los Angeles, but I am wearing a Paca T shirt, which I've been wearing all the time around LA in this scorching summer. But every hoodie from Paca is made start to finish in Peru, and each one features an Inca ID hand woven by artisans, honoring generations of knowledge and tradition. More than 250,000 people have already picked one up. So clearly I'm not the only person who's figured this out. Again, I wear the Packa hoodie at night. It does get cold here in Los Angeles in the summer even, but I wore it pretty much all fall and winter as well. And then I've been wearing Paca T shirts and Paca boxer briefs like pretty much every day. So I highly recommend Paca. They make really high quality stuff. To grab your pack a hoodie, go to www.pacaapparel.com that's www.pacaapparel.comp a k a apparel.com we have at this point written an embarrassing number of stories about data brokers. The companies that quietly compile your name, your home address, your phone number, your relatives, your daily habits, and then sell that file to basically anyone with a credit card. Advertisers? Sure. Also spammers, scammers, and the occasional person you'd really prefer didn't have your address, you never opted in. There are hundreds of these brokers and opting out of them one by one is effectively a part time job. That's what Incogni does for you. It acts as your personal privacy advocate, identifying the brokers holding your information and handling the legal legwork of demanding they delete it. Incogni manages the entire request and follow up process on a recurring schedule. Because this data has a notorious way of reappearing, you set it up once and watch the verified removals stack up on a dashboard. The practical result is fewer spam calls, less junk mail, and a much smaller footprint for anyone trying to find or impersonate you. For the most comprehensive protection, we recommend their unlimited plan, which includes custom removals. As journalists, we know that sensitive data doesn't just sit on broker databases. It winds up on public forums, random directories, blogs and news portals. If you find your personal details exposed on a specific site, you simply send Incognito URL and their team of privacy experts will step in to manually handle the corporate takedown request for you. Take control of your data footprint risk free with a 30 day money back guarantee. Go to incogni.com 404 Media and use code 404 Media for 60% off in the annual plan. That's code 404 Media@incogni.com 404 Media it feels like a lot of people are waiting for some kind of signal before they start a business. Almost like an invitation. They think they need more experience, more money, more free time. Somebody telling them, yeah, this is a good idea. At some point you realize nobody's going to give you permission. You just have to build the thing. And Shopify makes that first step a whole lot less intimidating. When we were putting together, the 404 Media storefront. What we appreciated was that Shopify gave us everything we needed to actually get it live and start selling. We were focused on the journalism. We didn't want to spend a lot of time trying to piece together a bunch of different infrastructure before we could figure out whether people even wanted to buy our merchants. Then there's that moment that actually matters, when someone actually wants to buy your products. Shopify checkout makes that part ridiculously smooth. And when customers come back, their details can already be saved. So buying again can be as simple as a tap. That's the sound you want. And as the business grows, Shopify keeps handling the infrastructure so you can focus on the stuff that actually moves the business forward instead of constantly fixing your storefront. That's the biggest reason Shopify powers millions of businesses worldwide, from companies like Mattel and Gymshark to people launching their very first store to 404 Media's merch store. With Shopify, nothing stands between your idea and a real business. So go make it one. If you're ready to stop putting off your business and start hearing, then you're ready for Shopify. Start your free trial@shopify.com media today. You heard that, right. Start your free trial today@shopify.com media. That's shopify.com media.
[29:00]
Joseph
All right. And we are back. The headline of this one that Emmanuel wrote and just published, company offering quote, 100% human written, never AI, end quote. Peer review is entirely AI. I only. I only read this this morning just to prepare for the podcast, and the headline immediately got me. I was like, okay, yeah, we're going to talk about that for sure. So first of all, before we get to the AI stuff, what is research gold and what does it advertise? This is a company, an organization. What's the deal?
[29:37]
Emmanuel Mayberg
Yeah, this is a private company. In order to explain what it does, I think we have to talk a little bit about how new research papers get published in scientific journals and get into the minutiae of that a little bit. So let's say I want to write a research paper about the effect of, I don't know, beans on blood pressure. Right. The first task I would have as a responsible scientist or researcher is to look at all the existing literature and find out what are the previous results out there, make sure I'm not duplicating work, making sure that I am comparing my results against those results. So kind of like doing a review of the literature and then taking all those results and putting them together. In relation to my research question, and this is a very laborious and according to researchers, like a very boring, not rewarding part of the job. It's like the research before you do the research that you actually want to do.
[30:53]
Joseph
But they have to do it, basically.
[30:55]
Emmanuel Mayberg
But that's a responsible thing to do. Yes, that is part of being a rigorous researcher is reviewing the existing research. And this is just like one of the many steps that you don't think about in the publication of new research. And it is one of the things that this company says that you can basically outsource this to. Traditionally, you would either do this yourself or you can hire professional methodologists to work with you and do this work for you. And Research Gold promises to expedite that process, do it for cheap, do it fast, and kind of save you that hassle. And obviously, as the headline implies, they promise that this is all top tier human PhDs and methodologists that are doing this work.
[31:52]
Joseph
It's not even a new service. It's obviously very established in the academic method of doing it. And as you say, you could hire somebody else to do this work before. It's just this company, for some reason is leaning on the. This is all 100% human written. This is never AI. Presumably because at the moment, as you've reported that academic journals are being slammed with AI generated submissions to the point where sometimes we're like, well, we're actually not going to look at meta analyses or literature reviews or anything like that because we're getting so much AI slop churning through. And I presume the marketing here is basically a response to that, right? To reassure people, hey, this is real. You think that's why they're doing it?
[32:38]
Emmanuel Mayberg
Yeah, I mean, it's another field at which AI generated content has really changed how everything works. And I would assume that if you're a serious researcher, you would want to make sure that if you're outsourcing anything, it's not AI generated. It's like you have an actual human PhD looking at the, at the, at the work.
[32:57]
Joseph
Yeah. So what's the problem with Research Gold? This sounds great.
[33:01]
Emmanuel Mayberg
The problem is that none of it is true. It's like quite the opposite. It's all entirely AI. AI generated is the issue.
[33:08]
Joseph
I mean, that's obviously really, really fucking funny and obviously incredibly ironic. So you hear about Research Gate, I think a researcher tells you about it, you then look, Research Gold.
[33:22]
Emmanuel Mayberg
Sorry, I thought you were talking about Research Gold, not Research Gold.
[33:25]
Joseph
Sorry, sorry, my bad. So you're looking at this company. How do you figure out this AI generated, Is it pretty obvious from the jump or like, how do you go about figuring that out?
[33:37]
Emmanuel Mayberg
It's extremely obvious. I learned about it from a researcher who was doing some of this work that I just discussed in preparation to defend his dissertation. And he stumbled upon this website and I was like, oh, this is interesting and potentially useful. And just a few minutes of clicking around, he figured out that it's entirely AI generated. There are a few very obvious signs just from the website. One is if you look at the team of researchers that operate this website, they're completely made up. They are names that have no digital footprint, certainly not one that matches their expertise as claimed on the website. And then they have obviously AI generated profile images. I was doing some more digging and I found that there was like a separate team listed on the site of these professional methodologists, which again is like a profession that exists. Like there are freelance methodologists out there that you can hire to work on your research. And there were about five of them listed on the site. And I reached out to, I searched them like on LinkedIn and other platforms. They were clearly real people, they were clearly in this field. I reached out to all of them and I heard back from one who said, yep, this is me on the site. I have no idea what the site is, I've never worked with them, they're stealing my identity. And then quick, she, she said that she would reach out to them and see what her options are in terms of getting that removed. And quickly after that, the entire page where these human methodologists were listed was removed. Also, just like as a funny side note, all the profile pictures on these real methodologists were the exact same profile pictures that were used on LinkedIn, including the Open to work hashtag. You know, the, you know, when like somebody is looking for work on LinkedIn, they have like a little graphic added to their profile image. So like that was also there, which I think is like not what you would use if you were like advertising your, your, your methodology.
[35:58]
Joseph
Well, no, no, maybe it shows they're really open to work, to work with
[36:02]
Emmanuel Mayberg
people who want to come open to work with you. Researchable client. Yeah, exactly.
[36:07]
Joseph
So there's the first set which is like the main team of people allegedly behind this service and they're completely made up and they have AI generated photos and names that don't correspond to anybody, all of that sort of thing. And then there is another page of another team of people who are real, but they're being ripped off and they have Nothing to do with research gold. I mean you only spoke to one, but presumably they were all ripped off because the entire page was taken down.
[36:32]
Emmanuel Mayberg
Right, right, right, yeah. And I would say like, again, this is just stuff that you would see by clicking around on the site and doing some Googling. I also did a bunch of other stuff like calling them, which is a very funny conversation that I had on the phone with an AI agent that refused to admit it was an AI agent. Yeah, just like I kept saying like, hey, are you human? Can I talk to a human? And this AI agent that was identified as Sarah was like, oh, hahaha, that's real funny. But let's focus on your research and like let's try to get you. Let's try to get you a quote on your research. And I was like, oh, what's your last name? It's like, oh, let's keep it simple. I'm Sarah. Yeah, let's not get confused with, with these complicated questions.
[37:17]
Joseph
Yeah, I don't think you know this, but just to ask it, like, do you have any idea who is actually behind it? Because you didn't get a response when you asked them for comment. You got this stupid. Clearly AI generated Sarah. All of the people that are generated, like any idea who's actually behind this or.
[37:36]
Emmanuel Mayberg
I was not able to figure it out. They do have a physical address listed which goes back to like some office complex in Texas. But I'll be cautious. I'm not claiming that the address is actually affiliated with them. Maybe they just picked a random place on the map. But yeah, no, we don't know who is actually like offering the service.
[37:57]
Joseph
We'll just talk about another couple of things very quickly. Just because I thought they were, you know, somewhat related to AI and faking it and being caught and that sort of thing. So you recently did a piece about how Substack has introduced something to do with AI that flags a piece as AI generated or something like that. What's that feature and how are people reacting to that? Which is a newsletter platform for those who, I presume everybody knows it, but you can use it to send newsletters and write stuff on the Internet, Correct?
[38:34]
Emmanuel Mayberg
Yeah. Can we call it. I'll speak for myself? It's the bad one? I would say it's the bad one.
[38:40]
Joseph
I think that's probably fair. Yes. And we did not go with it for various reasons, including that we never
[38:45]
Emmanuel Mayberg
really even seriously considered it. And it's bad for all the reason everybody knows, like people they're affiliated with in some business Practices and so on. Anyway, doesn't matter with quite like, like. With some pride and fanfare, Substack announced that it has partners with Pangram, which is like an AI detection software. Disclosure. Pancram has advertised with us once in the past and they made it possible for any reader to use Pangram to check whether the content of a newsletter is AI generated. And it does it in the way that some other detectors do it, which is like percentage
[39:27]
Joseph
score, like a confidence rating or I think it's how much
[39:31]
Emmanuel Mayberg
of it was AI generated. Correct me if I'm wrong, but I believe it's like 50% of this newsletter was AI generated, like 80% or something like that. And I think that is in theory, like it's directionally a good thing, I would think, because it is acknowledging. And the announcement from Substack makes this clear that people don't want a ton of AI generated content, especially if it's coming from a platform like Substack, which ostensibly is a place where you choose to follow a specific person and you're following for their human insight and they're delivering the content directly to you. A lot of like the selling points of our website, right? And in that context, I think it is pretty offensive to then send a bunch of ad generated content to people who are paying for your newsletter. And that's sort of the framing that Substack gives. And there are two issues with it from Substack's perspective. The issue is that clearly they have a lot of people who are using AI to generate newsletters on their platform. And, and those people are pissed. They're like, how dare you do this to us? It's my prompt. Like, it's my insight. Just because, like an AI wrote the text, it doesn't mean that my prompt and my, my thoughts are not unique and valuable. And clearly they're valuable because people are subscribing and that's like a marketing and business problem for, for an audience problem for, for Substack. I think the bigger problem that applies to everyone across the Internet and across all attempts to identify AI is that there are false positives. Even, even Pancream, which is considered, you know, one of the better tools for detecting AI generated content. It's not as if it never flags human content as AI and then you're creating like a culture of suspicion and what a lot of bloggers on that platform called like a witch hunt, right? It's like, is this AI generated? Is it not? If you accidentally get flagged, you're forever besmirched. As using AI and so on, which is, you know, I think, again, the intentions, I think, are good, but it's a very real unsolved problem.
[41:43]
Joseph
Yeah. Especially because the AI detectors, frankly, until very, very recently, straight up sucked, and they were basically like snake oil. Oh, yeah. We can use this to detect AI, and we would try them in the course of reporting to not only figure out whether something was AI, but just because we report on those tools as well. And they were garbage a lot of the time. And that's changed, I feel like, in the past 12 months, something like that. So there's that part, and this is a new phenomenon now where maybe these tools are more reliable. But just on the other one where you said that the criticism or the response from some of these people who do use AI to write on substack, they're like, well, it's my prompt, so what's the issue? Where it's like, well, if you're omitting it, then what's the problem? If you're openly using AI, what's the problem of readers knowing that, oh, this was scam with AI? It doesn't seem to gel those two positions together.
[42:39]
Emmanuel Mayberg
It's because of the stigma. I'm sorry, I'm taking this in a completely different direction, and it's not something we plan to talk about, but has everybody heard about Phoenix Flexing in that controversy?
[42:51]
Joseph
Literally no idea what you're talking about.
[42:52]
Emmanuel Mayberg
It's like this very viral rap song and like an 80s style that's become like, a legitimate top 100 hit. And I am not saying this that I'm 100% sure, but there's a lot of reporting or a lot of videos from other YouTubers who claim pretty confidently that it was AI generated and the artist Phoenix Flexing claims that it is not. But, yeah, I don't know. Famously, I'm like the. The. The least musically inclined person on the podcast, and I thought, like, it sounded like a cool song, but. But this is exactly the problem. You know what I mean? It's like the song is a hit, clear that people like it, but now that. That it's credibly accused as being AI generated, it's a huge thing, and people are very angry at the artist.
[43:43]
Joseph
But then I guess some people just won't give a shit. Potentially.
[43:47]
Emmanuel Mayberg
Clearly not, right? Clearly not, because the thing is doing well.
[43:51]
Joseph
Last question before we go to the subscriber section. Not to get into the details of how this will work, because, frankly, I still think we need some answers and we need to see it deployed, but Claude or Avanthropic says Claude is now going to watermark text generated by his AI models released after early August. And this is to comply with some European legislation, but I think it's going to be rolled out worldwide. Basically you might be able to tell if a piece of text was generated by Claude. How do you think people are going to react to that, Emmanuel? Again, we don't know yet because it hasn't quite happened. But do you think they're going to react in a similar way to the substack one or two?
[44:31]
Emmanuel Mayberg
I don't think it's going to be the same controversy as the substack one because again, like that, that personal relationship that people have with newsletters, I think is the issue there. But I will say that while watermarking is important and AI companies should continue to try to watermark AI generated content, it is sort of a fig leaf in terms of addressing the problem of flooding every communication channel with AI generated content. Because it's like, okay, yes, maybe if you have a bunch of AI generated scientific papers and they're watermarked, you can find them, but it still requires you to filter through the mess, which is the problem. If you're getting x1000 submissions to your scientific journal and you have to figure out which ones are ASMR AI generated and which ones are not, the fact that they are watermarked I don't think is a huge help. You know, it's like you still have the slot problem is still. It's still the SLO problem.
[45:36]
Jason Kebler
I agree with that. I also think it's a mix of people who are making AI generated content for slop purposes. Like one, many of them don't care if people know that it's AI generated or not. Sort of like as Emmanuel said, because it's a numbers game, you're trying to just like win an algorithmic lottery. But then the people who do care always find a way to circumvent the watermark. And so, I mean, in some cases it's as simple. Like with images, it's as simple as like cropping it out. Sometimes it's like you open it up in a different video and an image editor and like change things there and with text. Like, I don't understand exactly how this will work. Maybe, maybe Anthropic has explained it, but I don't see what would. Like if you copy it over to Notepad or TextEdit and use something to strip all the metadata from that. I don't know how a watermark could persist across apps and things like this unless there's a technology that I am not aware of. But I think it's something where it's like if you are seriously trying to circumvent the watermark, people will be able to do it. And if you're not, you still have the slop problem.
[46:55]
Joseph
Yeah, makes sense. All right, we will leave that there. If you're listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 media subscriber, we're going to talk about how while a bunch of cities are ditching Flock, they are immediately replacing them with Axon, which is a, you know, infamous law enforcement contractor and Flock rival. You can subscribe and gain access to that content at 404 MediaCo. As a reminder, 404 Media is journalist founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404 Media co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really, really does help us out. This has been 404 Media. We'll see you again next.
[48:03]
Emmanuel Mayberg
Sam.