Transcript
Narrator (0:00)
You ever get the feeling the city walls closing in the concrete jungle suffocating your soul? You crave wide open spaces, the chance to connect with nature, maybe chase some elk, fish a private stream. Well, listen up. There's a whole world out there and finding your own piece of it just got easier. Head over to land.com they've got ranches, forest, mountains, you name it. Search by acreage, location, the kind of hunting or fishing you dream of. Land.com it's worthy. Adventure begins.
Bobby Bones (0:33)
Bobby Bones here. Join me on the new Top Shelf Country Cruise. It sails February 2026 aboard the luxury Celebrity Reflection, stopping at St. Kitts and Nevis and also St. Martin. There'll be live music with top tier country artists and I'll be performing as well. Go to topshelf countrycruise.com before January 30th and register for the free Friends of Bobby Presale. No deposits required and you get early access to the best staterooms.
Clorox Advertiser (1:06)
Clorox Scentiva Smells like lavender.
Ed Zitron (1:11)
Cleans like Clorox, and feels like hmm.
Clorox Advertiser (1:22)
All right, that could go on for a while. Experience the long lasting freshness of Clorox Scentiva. Now available in Clorox Scentiva Lavender Scented.
Clorox Advertiser (1:31)
Bleach Use as directed at Ameca Insurance we know it's more than a life policy. It's about the promise and the responsibility that comes with being a new parent, being there day and night and building a plan for tomorrow today for the ones you'll always look out for. Trust Amica Life Insurance Amica Empathy is our best policy.
Ed Zitron (2:05)
Call Zone Media Chosen by God, perfected by Science I'm Ed Zitron. This is better offline, better offline. And as I've written about many, many, many, many times and argued on this very podcast just as often, the large language models run by companies like OpenAI, Anthropic, Google and Meta are unprofitable and unsustainable, and the transformer based architecture they run on has peaked. They're running out of training data, and the actual capabilities of these models were peaking as far back as March 2024. Nevertheless, I'd assumed incorrectly, by the way, that there would be no way to make them more efficient, because I had assumed, also incorrectly, that the Hyperscalers, along with OpenAI and Anthropic, would be constantly looking for ways to bring down the ruinous cost of their services. After all, OpenAI lost $5 billion last year, and that's after $3.7 billion in revenue too, and Anthropic lost just under $3 billion in 2024. And in the last episode I told you a little bit about deep seek. By the way, in this one we're going to get into, well, how fucked things might actually be. But what I didn't wager was that potentially nobody was actually trying to make these models more efficient. My mistake was, if you can believe this, being too generous to the AI companies, assuming that they didn't pursue efficiency because they couldn't and not because they couldn't be bothered. But then, as I just hinted at, a little known Chinese company released a product that was broadly equivalent to OpenAI's latest reasoning models, but cost a fraction of the cost to train and run. And now the conventional understanding of how generative AI should work has been fundamentally upended. You see, the pre deep seat status quo was one where several truths, and I say that in the loosest sense of the word, allowed the party to keep going. So the first one is that these models were incredibly expensive to train. GPT4O cost $100 million in the middle of 2024. And future models, according to Dario Amadevanthropic, might cost as much as 1 billion or more to train. And training future models, by the way, as a result of this, would necessitate spending billions of dollars on both data centers and the GPUs necessary to keep training these bigger, huger models. Now another thing was these models had to be large because making them large, pumping them full of training data and throwing masses of compute at them would unlock new features, such as an AI that helps us accomplish much more than we ever could without AI, which is a Sam Altman quote. And in the words of Sam, again, you'd be able to get a personal AI team full of virtual experts in different areas working together to create almost anything we can imagine. I don't know, mate. You ever try creating a functional fucking business, dipshit. Anyway, here's another one. These models were incredibly expensive to run. They has to be this way. But it was all worth it because making these models powerful was way more important than making them efficient. Because once the price of silicon comes down, and this is a refrain I've heard from multiple different people as a defense of the costs of generative AI, we would then have these powerful models that were cheaper somehow because of silicon. Now you may think, ed, that sounds like a. Not a real argument. That just sounds like something someone said once. And it is, it is something someone said once. Anyone who knows anything about chips know how hard it is to make a new chip and Remember one of the CES episodes when I asked Max Czerny about this? You should go back and listen to him. Anyway, another thing, another part of this was that as a result of this need to make bigger, huger, even bigger models, the most powerful ones, these big beautiful models, we love them, we look at the big beautiful models, we would of course need to keep buying bigger, more powerful GPUs, which would continue the American excellence of burning a bunch of money on nothing. And by following this roadmap, everybody wins. The hyperscalers get the justification they needed to create more sprawling data centers and spend massive amounts of money. And OpenAI and their ilk get to continue building powerful models. And also Nvidia continues to make money selling GPUs. You remember I've said in the past that things were kind of a death cult. This is what this is. It's a capitalist death cult. It runs on plagiarism and hubris and the assumption that being that at some point all of this would turn into something meaningful. Now, I've argued for a while that the latter part of the plan was insane, that there was no profitability for these large language models as I believed, there simply wasn't a way to make these models more efficient. In a way I was right. The current models developed by both the hyperscalers, so Gemini from Google, Llama from Meta, and so on and so forth, and the multi billion doll startups, if you can even fucking call them that, OpenAI and Anthropic, they're horribly inefficient and I just made the mistake of assuming that they tried to make them more efficient and they couldn't. But what we're witnessing right now isn't some sort of weird China situation. This isn't China being Chinese and doing scary Chinese things to us. No, what we're witnessing is the American tech industry's greatest act of hubris. It's a monument to the barely conscious stewards of so called innovation who are incapable of breaking the kayfabe of the fake competition where everybody makes the same products, charges about the same amount of money, and mostly innovates in the same direction. Somehow nobody, not Google, not Microsoft, not OpenAI, not Meta, not Amazon, not Oracle, thought to try or was capable of creating something like Deepseek. Which doesn't mean that Deep Seek's team is particularly remarkable or found anything super new. But that for all the talent, trillions of dollars of market capitalization and supposed expertise in American tech oligarchs, not one bright spark thought to try Things that Deepseek had tried, which appeared to be what if we didn't use as much memory? And what if we tried synthetic data? And because the cost of model development and inference was so astronomical in the case of American models, they never assumed that anyone would try to usurp their position. This is especially bad considering that China's focus on AI as a strategic part of its industrial priority was really no, even if the ways it supported domestic companies kind of is. In the same way that the automotive industry was blindsided by China's EV manufacturers, the same's happening with AI. Fat, happy and lazy, and most of all, oblivious. America's most powerful tech companies sat back and built bigger, Messier models powered by sprawling data centers and billions of dollars of GPUs from Nvidia, a bacchanalia of spending that strains our energy grid and depletes our fucking water reserves without, it appears, much consideration of whether an alternative was possible. I refuse to believe that none of these companies could have done what Deepseek has done. Which means that they either chose not to, or they were so utterly myopic, so excited to burn so much money on so many parts of burning the earth, boiling lakes, and stealing from people in pursuit of further growth, that they didn't think to try. This isn't about China. It's so much fucking easier if we let it be about China. No, no, no, no. It's about how the American tech industry is incurious, lazy, entitled, directionless and irres. OpenAI and Anthropic are the antithesis of Silicon Valley. They're incumbents, public companies wearing startup suits, unwilling to take on real challenges, more focused on optics and marketing than they are on solving actual fucking problems, even the problems that they themselves created with their large language models. By making this about China, we ignore the root of the problem that the American tech industry is no longer interested in making good software that actually helps people. Deep Seek shouldn't be scary to Silicon Valley, because Silicon Valley should have come up with this first. It uses less memory, fewer resources, and uses several kind of quirky workarounds to adapt to the limited compute resources available. All things that you'd previously associate with Silicon Valley. Except now Silicon Valley's only interest, like the rest of the American tech industry, is the rot economy. It only cares about growing, growing at all costs, even if said costs were really, really things you could mitigate, or if the costs themselves were self defeating. To be clear, if the alternative is that all of these companies simply did not come up with this Idea that in and of itself is a damning indictment of the Valley. Was nobody thinking of this stuff? If they were, why didn't Sam Altman or Dario Amadei or Satya Nadella or anyone else put serious resources into efficiency? Was it because there was no reason to? Was it because there was, if we're honest, no real competition between any of these companies? Did anybody try anything other than throwing as much computing training at the model as possible? It's all just so cynical and antithetical to innovation itself. Surely, if any of this shit mattered, if generative AI truly was valid and viable in the eyes of these companies, they would have actively worked to do something like Deepseek has done. Don't get me wrong, it appears Deepseek employed all sorts of weird tricks to make this work, including taking advantage of distinct parts of both CPUs and GPUs to create something called a digital processing unit, essentially redefining how data is communicated within the servers running training and inference. And just to just as a reminder, inference is the thing where when you type something in, it infers the meaning. Just could have specified that earlier. Deepseek had to do things that a company with unrestrained access to capital and equipment wouldn't have to do, and it often used impractical and quirky methods to do so. Nevertheless, OpenAI and Anthropic both have enough money and hiring power to have tried and succeeded in creating a model this efficient and capable of running on older GPUs. Except what they wanted. What they actually wanted was more God damn growth and the chance to build even big, bigger data centers with even more compute that they would own. OpenAI is as much a lazy, cumbersome incumbent as Google or Microsoft, and it's about as innovative too. The launch of its operator agent was a joke. A barely functional product that's allegedly meant to control your computer and take distinct actions like ordering stuff off of Instacart. You know, things you could do with your hands. But just to be clear, it doesn't work. You'll never guess who was really into it, though. His name is Casey Newton. He writes a blog called Platformer, and he he's a man so gratingly credulous that it makes me want to fucking scream. And of course he wrote that Operator, when he used it, was a compelling demonstration that represented an extraordinary technological achievement that also somehow was significantly slower, more frustrating and more expensive than simply doing any of these tasks yourself. Casey, of course not to worry, had some extra thoughts about Deepseek. That There were reasons to be worried, but that American AI labs were still in the lead saying that Deepseek was only optimising technology that OpenAI and others had invented first before saying that Deep sequels only last week that OpenAI made available to Pro Plan users a computer that can use itself. This statement is bordering on factually incorrect. It is fucking insane that Casey is still doing this. I do not want to. I don't know what to do with this guy. This guy just. That's a fucking lie. This. The computer can't use itself. This shit can't. Just to explain what operator is, you're meant to type in something like, hey, order me some milk. Order me some milk off of Instacart. And when Casey tried this, it tried to find milk in Des Moines, Iowa. Just fucking insane. Just, this is how these companies have got big. It's people like Casey. It's people like Casey who are just like anything they showed are like, God damn, that's the most impressive thing I've seen in my life. It's a fucking farce. But let's be frank, these companies aren't building shit. OpenAI and Anthropic are both limply throwing around the idea that agents are possible in an attempt to raise more money to Burn. And after the launch of Deep Seek, I have to wonder what any investor thinks they're investing in, other than certain ones I'll get into in a bit. And to be clear, an agent is meant to be this autonomous thing which you say, hey, go and do this action. Go and sell things for me and go and email people for me. They don't really work. There are some that kind of do that are really expensive. But large language models are not built for this kind of thing. But let's be honest, Deep Seq and as I said in the last episode, they've built a more efficient reasoning model. So like OpenAI's 01. And you'd think, well, okay, couldn't OpenAI simply add on Deep SEQ to its models? Not really. First of all, with the way these models work, you can't just like plug it in. It's just not how it works. They could train a new model using DeepSeek's techniques, but the optics of that aren't brilliant. It would be a concession. It'd be an admitted OpenAI slipped and needs to catch up. And not to its main rival, pretend rival, I mean anthropic, or to like another big tech firm, but to an outgrowth of a hedge fund in China, a company that few had heard of before December, and like really not that many people had heard of before January 25th. It's very embarrassing. And this in turn I think will make any serious investor think twice about writing the company a blank cheque. They're going to have to dip into some very bothersome pockets. And as I've said ad nauseam, this is potentially fatal, as OpenAI needs to continually raise money, more money than any startup has ever raised in the history of anything, and it really doesn't have a path to breaking even, even if they copy what Deepseek did. Because we still Right now though, Deepseek is 30 times cheaper than O1. We don't know if that's profitable. We don't know. We haven't found out. And if OpenAI wants to do its own cheaper, more efficient model, it's likely to have to create it from scratch, like I said. And while it could do distillation to make it kind of more like OpenAI using their own models, by the way, Deepseek taught itself using OpenAI's outputs like I mentioned in the last episode, it's kind of what Deepseek already did. It already has been fed OpenAI bullshit. Even with OpenAI's much larger team and more powerful hardware, it's hard to see how creating a smaller, more efficient and almost as powerful version of 01 benefits them in any way, because said version has, well, already been beaten to market by DeepSeek, and thanks to DeepSeek, will almost certainly have a great deal of competition for a product that to this day lacks any killer apps. Anyway.