← The Artificial Intelligence Show
The Artificial Intelligence Show
#133: DeepSeek, US vs. China AI War, Anthropic CEO: AI Could Surpass Humans by 2027, OpenAI Plans AI for Advanced Coding, & Meta’s 1.3 Million GPUs
01:01:10
Loading summary
Transcript53 lines
- [00:00]
Paul Raitzer
You just showed how to disrupt the US Economy in three days. Like in a very hands off. We had nothing to do with this kind of way. And it may be what we saw with TikTok is US consumers don't care if their private data is going to China. They just want convenience and personalization. So if an app from China offers value and use like US consumers have shown time and again, they're going to use it regardless. Welcome to the Artificial Intelligence show, the podcast that helps your business grow smarter by making AI a approachable and actionable. My name is Paul Raitzer, I'm the Founder and CEO of Marketing AI Institute and I'm your host. Each week I'm joined by my co host and Marketing AI Institute Chief Content Officer Mike Kaput as we break down all the AI news that matters and give you insights and perspectives that you can use to advance your company and your career. Join us as we accelerate AI literacy for all foreign welcome to episode 133 of the Artificial Intelligence Show. I'm your host Paul Raitzer along with my co host Mike Kaput. This is our second episode we are recording on January 27th. So it is Monday morning, 10:50am Eastern Time, January 27th. Nvidia stock continues to go down, which we're going to explain why in a moment. So if you didn't listen to episode 132, just a quick recap. Normally we do one weekly episode that recaps the previous week's news. Last week's news was so crazy that we decided to do this over two episodes. So I guess you could think of episode 3132 as part one. Episode 133 is part two. So we are continuing on with the major news from last week. This episode is brought to us by the AI mastery membership program. This is a our 12 month membership program that includes quarterly classes like Ask Me Anything sessions, Generative AI Mastery classes, Trends briefings and then we announced and if you listen to episode 132 you heard us talk about this, the AI literacy project where we announced some major changes to our AI Mastery Membership program including as of now, the Piloting AI and Scaling AI Course series are bundled into the membership. And then in spring of this year we are going to be launching a new AI powered learning management system and user experience, expanding courses and professional certificates and then a new turnkey AI Academy solution for businesses. And so we're going to launch that AI fundamentals course series which is kind of like AI 101 for all knowledge workers. New piloting AI and scaling AI courses. A new weekly gen AI app course series that we're really excited about. AI for industries, AI for departments. So just a ton coming and it's all going to be built into that same AI Mastery membership. So if you join now, you will get first access to all the new stuff as it's coming online this spring. You can go to SmarterX AI and just click on Education and it's right there. Just look at AI Master membership in the drop down or SmartRx AI slash forward slash AI mastery. We'll put the links in the show notes if you want to. Just click on the link in the show notes and there is a promo code you can use. Pod 100 pod 100. That will get you 100 off the membership. And then also a quick reminder, we have open submission right now to speak at Macon 2025 that is taking place October 14th to the 16th in Cleveland. So you can register too. Registration is open. We're expecting probably north of 1500 people. Last year I think we had. Or next next year. This year I think we had 1100 last year.
- [03:53]
Mike Kaput
Yeah.
- [03:54]
Paul Raitzer
So in Cleveland last year. So this year we're thinking 1500 plus. We'll see. I mean, it's hard to predict these things in the event world, but it's looking like it's going to be another amazing event. This is our sixth year again, so if you want to speak, get those applications in soon. It's a rolling basis, so as phenomenal applications come in, we'll reach out to people and get them out of the agenda. But I think last year we had, I don't know, it was like close to 200 or more submissions to Speak and there obviously aren't that many slots. So get those submissions in early. Just go to Macon AI, that's M A I C O N AI and click on Submit your speaker application. There's a button right there on the homepage. Okay, Mike, the thing that was all the rage, I think it started on Thursday. I don't remember when this started taking over the news cycle, at least in Twitterverse, but all that I saw all weekend was this. I listened to probably three podcasts about it. I've read probably 20 articles about it, and I have been watching Nvidia's stock plummeting this morning as a result of. So let's talk about Deep Seat, which also is the number one app in the App Store, I think as of Sunday night. So it's just, it's everywhere. I've never seen anything quite take off like this?
- [05:08]
Mike Kaput
Yes. So Deep Seek is a Chinese AI lab that is sending shockwaves through Silicon Valley because it's had some breakthroughs that are challenging some fundamental assumptions about AI development. So Deep Seq has actually created AI models that rival or surpass top US based or created systems, while spending a fraction of the time and money on these models and releasing them in an open fashion so others can use and build on them. So one of the company's models, DeepSeek V3, was built using only about, they claim, 2,000 specialized Nvidia chips, which is compared to 16,000 or more chips that major US companies are using. Even more striking is the cost. DeepSea claims it spent just $6 million on computing power to train the model. And the model, they say, is comparable to something like GPT4O. So it challenges the closed models cost only $6 million to train, which is literally 1/10 of of what someone like Meta invested in their latest AI technology. And the system can match the leading chatbots out there, apparently in answering questions, solving logic problems, writing code. I mentioned it's open so developers can freely access and build on it. And as a result, Paul, like you mentioned, Deepseek recently overtook ChatGPT to become the top rated free application in the App Store in the us. Now this also happened because or despite of US government restrictions on sending advanced AI chips to China. So rather than hindering progress, these constraints may have forced Chinese engineers to develop more efficient approaches. Now, on the heels of Deep Seq v3, Deep Seq also released R1, which is an open source competitor to OpenAI's advanced reasoning models, specifically O1, and then eventually O3, which is coming out now. R1 actually costs like 90% less to use than 01. And this is kind of further baffling everybody as to how this company is able to create such powerful AI at such a low cost. So it's raising a bunch of uncomfortable questions, which is why you're hearing about it a lot. So if what they say is true, and that's a big if we'll dive into, that really calls into question how much money is required to actually build AI? How much, how many advanced chips are required to build advanced AI systems? Have all these big labs just lit money on fire and way over engineered this when there's an easier way? And then also the emergence is having people worried about China's AI capabilities. Former Google CEO Eric Schmidt previously estimated China was two to three years behind the US and AI development, but now acknowledges they have caught up. So Paul, a Bunch of different angles here. But let's first talk about what Deepseek says they have achieved. How credible are these claims? How likely is it that they were actually able to achieve this level of performance this fast for this cheap?
- [08:22]
Paul Raitzer
I have no idea. I mean, so I've been trying to track this as closely as possible. Look at all the different angles, listen to the different players involved. There are some who think they just innovated that, that the US reducing the amount of chips they're allowed to have just drove innovation. And they found out they built more efficient algorithms to, to do this training more efficiently. And there are some who are very strongly opinionated that they're lying. And there's no way they actually trained it this efficiently. And they probably have way more illegal Nvidia chips than their saying they have. And they're never going to disclose that they do because that's illegal. And they are certainly a fair amount of people who think they probably stole the data and illegally trained the models on US model outputs. Like, obviously we have no idea. Like we're, we have no inside knowledge on this. There are lots of opinions flying around. Whatever the truth ends up being there. There is a lot of concern in Silicon Valley at the moment and in the US stock market that they may have actually just found more efficient ways to do this. I think there's probably little debate that they leveraged US innovation to do this. That is, that is pretty much guaranteed shortcut their path to success by leveraging what the US had done. Um, but throughout the weekend the app kept climbing the App Store, which is hilarious. Like, I don't even, I don't even know, like people who, like, what, what, what would you know to do with this? Like, I don't know how it climbed. Like, and even that there was some question about whether it was like bots and just paid things that was getting it to the top. And then there was questions about is this like a psychop by the Chinese government to actually like just mess with the US stock market? I don't know. It. It gets really wild, really deep. But we referenced on 132 at start and we referenced at the beginning here. Nvidia stock is like crashing this morning as a result of this. The reason is Nvidia's value is based on the belief that we are going to keep building massive data centers filled with millions of Nvidia chips to not only train more powerful, generally capable models, but to run the inference on these models, when all of us consumers use AI as apps and devices so Nvidia's entire future, at least in the stock market, is based on this belief that we're going to keep building, we're going to keep needing millions of chips. Well, this puts into question do we really need all these data centers and infrastructure that we just got a $500 billion for Stargate and we got trillions more coming. Like is that all going to be necessary? So that's why Nvidia stocks sort of like just drop dropped because stock Wall street doesn't like uncertainty. And this was, this is very uncertain. Like there's way more questions than answers. There's, there are risks, there's massive risks here. So like one these models, if you ask them about, you know, things related to like democracy or Tiananmen Square, things like that, like they just won't answer. Like they're, they're obviously in some ways controlled by, they're from a company within China, so they have to adhere to the policies of the government. The data is stored on Chinese servers, I believe, like that. I saw something last night about like, where's this data going? It's, you're sending personal data, whatever you put in, whatever the inputs and outputs are of the model, like you're, you know, sending that through the app, if you're using the app. But you know, again, like there's differing opinions. So we had Satya at Davos said to see the Deep Seq new model. It's super impressive in terms of both how they have really effectively done an open source model that does this inference time compute and is super compute efficient. We should take the developments out of China very, very seriously. But then Satya also tweeted, I may not mispronounce this. Jevons paradox strikes again. So like this I. So the idea that because something became cheap that we aren't going to need like the data centers. So his exact tweet was jevons paradox strikes again. As AI gets more efficient and accessible through things like Deep sea, we will see its use skyrocket, turning it into a com, a commodity we just can't get enough of. So what he's saying is Nvidia stock should actually be going up. Is the basic like interpretation here that because something was made cheaper that there actually will be an increasing demand of it. So Wikipedia is the correct, the quickest thing I can get in economics. The Jevons paradox or Jevons effect occurs when technological progress increases the efficiency with which a resource is used, reducing the amount necessary for any one use. But the falling cost of use induces increases in demand enough that the resource is use is increased rather than reduced. Meaning we're going to need more Danish setters, we need more Nvidia chips. So the stock market is making the assumption oh, we won't need as many Nvidia chips or data centers. What Satya is saying is no, that's not the case. This is a Jevons paradox. Like it'll actually increase. We'll need more as we go. And then just like to to drill home the significance of this, there's an information article that said Meta scrambles after Chinese AI equals its own upending Silicon Valley. And so I'm just going to read a couple of quick excerpts here because I think this gives the mentality of and again, this is like five days old that just happened like third or whatever. So it said Leaders including AI Infrastructure director Matthew Oldman have told numerous colleagues they are concerned that the next version of Meta's flagship AI Llama won't perform as well as Deep Seq. Now this is why I was kind of surprised Meta stock was up today. This month, Hangzhou based High Flyer Capital Management up the ante by releasing another version of Deep Seek that you had mentioned. App developers can freely download Deepseek or buy access to it through cloud based APIs. Researchers at OpenAI, Meta and other top developers have been scrutinizing the Deep Seat model to see what they could learn from it, including how it manages to run more cheaply and efficiently than some American made models. Noam Brown, who we've talked about many times on the show, he tweeted, deepseek shows you you can get very powerful AI models with relatively little computer compute. The article goes on to say, even more surprising than the quality of Deep Seeks results was High Flyers claim that developing it cost a fraction of the amount American competitors spent on developing similar models, a claim that various researchers have met with skepticism. Underscoring the efficiency of its models, High Flyer also sells a cloud hosted version that is 17 to 27 times cheaper than OpenAI's comparable automatic offerings. The arrival of Deep SEQ is particularly galling to researchers at Meta because like Llama, it is freely available for other developers to use with publicly accessible settings that control the model's behavior, a concept known as open weights. So this is really important because as we talked about in the the podcast last year, Zuckerberg's play was to undercut the market by making a free open source model. Well, he just got undercut by a Chinese company with a model that's better based on this article with than what Meta hasn't even released yet. So like their next model, there's concerns at Meta that this thing is not only more efficient, it's actually better than what they were going to release. So the article says that they have researchers at leading American Hose probably impressed with the results. High Flyer may have taken some shortcuts to mimic already released models, including training its own models answers from 01 and llama. And then they said that managers and engineers from Meta AI Group and Infrastructure team have started four war rooms to learn how Deep Seek works. Two are mobilized to trying to understand how they lowered the cost of training and running Deep Seek. Meta wants to apply whatever they can learn. And then they said managers, engineers from Meta have started. Okay. And then there was like there was two other war rooms dedicated to different elements of Deep Seek. So this is real like the. They again they don't know yet if they're being truthful in, in their research and what they're saying. And maybe there are some like larger issues at play here, but it seems like there's enough to this that it has AI research labs scrambling to figure out what is going on. And then you throw in the fact that it all of a sudden is number one in the app store and now consumers are seeing and using this thing. It's like it just, it took the world by storm. It's really crazy.
- [17:02]
Mike Kaput
And it seems like one way or another, whether they have in fact found some breakthroughs to do this or are hiding something, there's a lot of money and interest in figuring that out. Because all of the investors, all the business models of these major players seem to be under threat from this. Right?
- [17:21]
Paul Raitzer
Yeah, I mean I, I didn't do the math but I mean Nvidia is a $3 trillion company. If they lost 14% market or value this morning, I mean we're talking about a 400 to 500 billion dollar market cap swing in two hours, right? Yeah, it's a, it's a massive economic impact.
- [17:43]
Mike Kaput
So you don't think there's some corporate espionage about to happen around.
- [17:46]
Paul Raitzer
Yeah, that's a hundred percent. So even if there wasn't something more nefarious at work here, you just showed how to disrupt the US economy in three days like in, in a very hands off. We had nothing to do with this kind of way. So even if it didn't come from that, you may now see future copycat things done where. Because what we, and it may be what we saw with Tick Tock is us Consumers don't care if it's from China. Like they don't care if their private data is going to China or it's owned by some holding company in China. They just want convenience and personalization. They want that experience. So if an app from China offers value and use like US consumers have shown time and again, they're going to use it regardless. So I don't know man, it's is heavy stuff for a second episode we're recording in the same day. Like, it hurts my brain to be trying to like process this and I know we're only going to talk more about it in the next topic.
- [18:48]
Mike Kaput
Yeah, for sure. Because the second topic is pretty intimately related to this. It kind of zooms out from just this deep seat drama. And looking at how some US based AI leaders are now speaking up louder than ever about the need for America to win the AI war against China. So the most notable example of this is Alexander Wang, the founder and CEO at Scale AI, which is a major data platform company that's used by a lot of companies building AI, took out a full page ad in the Washington Post the day after President Donald Trump was inaugurated, titled literally, dear President Trump, America must win the AI War. This ad linked to a letter from Wang to Trump that is published on the Scale AI website and it outlines a five point plan to maintain US leadership in AI. And while that is generally, you know, leading all the countries, it's especially focused on China. Wang warns that the Chinese government, quote, outspends the US government by about 10 times on AI implementation and adoption. So his proposed strategy centers on a fundamental restructuring of how the US Government approaches AI development. He says that there's a critical misalignment in current government spending where 90% of investments focus on algorithms. Contrary to what's actually a best practice in the industry, which is allocating resources across three pillars. Compute at about 60%, data at about 30% and algorithms at about 10%. The plan also calls for five specific actions in the administration's first 100 days. Beyond realigning those AI investments, Wang advocates for building an AI ready workforce, with projections suggesting AI could create 50 million new jobs by 2030. He also emphasizes the need to modernize federal agencies AI capabilities by 2027. And he notes that while the US government is the world's largest data producer, it's not effectively leveraging this advantage. And the proposal also addresses two critical infrastructure challenges, energy and regulation. He calls for an aggressive national energy plan to support AI's substantial power demands while simultaneously advocating for A balanced regulatory framework that ensures safety without hampering innovation. So, Paul, it is not news that there's a brewing AI arms race between the US and China, but based on the Deep Seq news, it seems that this is now at the forefront of everyone's mind. Is the US falling behind? How critical is this scenario?
- [21:36]
Paul Raitzer
I don't know. I've seen a lot of charts in the last week or so on. You know how much China's built out in energy and infrastructure. Yeah, so. So you and I both read AI Superpowers by Kai Fu Le years ago. I think anybody wants to understand the dynamics here because they are becoming very important and starting to become reality. So AI Superpowers, the subtitles China, Silicon Valley and the New World Order Kai Fu Lee was the former president of Google China and now he runs a venture fund in China. So he, he, he knows what he's talking about. And so he tweeted just yesterday, I think this was in my book AI Superpowers. I predicted that US will lead breakthroughs, but China will be better and faster in engineering. Many people simplified that to be China will be US and many claimed I was wrong with Gen AI. With the recent Deep Seq releases, I feel vindicated. So it is exactly what he laid out in his book, that US will drive innovation. We will build the data centers, we will build the biggest models, we will have the breakthroughs in memory and reasoning and all these things like that is what we do in America. And China will very quickly follow and improve on them. And that is what has always happened in innovation for decades. And he said AI was going to be no different. And the other thing that China has going for them is they don't have the civil rights around, like privacy and data usage of civilians and things like that. So they're going to use all the data. One of the big question marks was always, would they allow a large language model to exist? Like, would they allow something that could talk about the actual history of Tiananmen Square? Like, would they let something like that exist? And the answer seems to be yes, that, that they will. And, and if they're going to do that, it definitely creates a whole lot of new wrinkles in this. I don't know what else to call. I don't know if we have a better name for like US China War. I don't really like referring it to like that, but it is a, is an AI war for sure. And it's going to be fought on a lot of different levels. And sometimes we're not going to know that that's what's happening. And we may find out years later that that's what things were and how it all played out. But, yeah, I don't know. I mean, people listen to, to Wang Scale. AI is a very important company. They, they work. You can't even like, step back and say, okay, so Alexander works with Sam Altman and Open AI, but he doesn't like Elon Musk. Like, no, they all work with them. Like, yeah, Meta uses them. I'm sure Elon uses them. Like, they're a critical component of the data infrastructure that trains these models. And so people, people do listen to him. And I think that this, I'm sure that this, you know, letter to the President has been seen. And, you know, I think there's elements of it that I certainly agree with and I think it'll be fascinating to see how this all plays out. But this is going to be a major ongoing news thing. This is not going away. This is going to only grow in importance.
- [24:35]
Mike Kaput
Yeah, I think we talked about a bit last year. There were all these scenarios where we could see AI becoming this, like, hot button political issue. And it kind of didn't really hit right away. But now this is certainly one area.
- [24:48]
Paul Raitzer
I had the same thought last night when I was like, scanning through, getting ready for today.
- [24:51]
Mike Kaput
Yeah.
- [24:52]
Paul Raitzer
That we were saying, like, up until November, how, like, AI just didn't play a role in the election. It wasn't really talked about as a campaign item. And then day one, it's all that's talked about. Like, it is like, not all. There's obviously immigration and a bunch of other stuff going on. But it became very obvious, day one, minute one, that AI was fundamental to the administration.
- [25:13]
Mike Kaput
All right, and our third big topic for this episode. So the World Economic Forum had their annual conference in Davos on episode 132. We talked about a few interesting interviews with AI leaders. On this episode, we wanted to deep dive in a more formal way into one of them from Anthropic CEO Dario Amadei, because he made some pretty interesting predictions about AI's trajectory. He suggested that AI could surpass human intelligence by 2027. And of course, people started then quoting and asking questions of other AI leaders to respond to this. He actually revealed during this interview that his confidence about rapid AI advancement increased dramatically in recent months. While he previously maintained uncertainty about the timeline for transformative AI, he now says he is, quote, relatively confident that within the next two to three years we will see AI systems that are, quote, better than us at almost everything. He also talked a bit about Anthropic's growth, their fundraising, and their immediate roadmap indicating significant updates coming to Claude within coming months. But really it's this 2027 prediction, Paul, that kind of got everyone's attention because he also said society is going to need to fundamentally rethink how we organize our economy as AI becomes increasingly capable. He said there are a lot of assumptions we made when humans were the most intelligent species on the planet that are going to be invalidated by what's happening with AI. So, Paul, like, can you maybe walk us through what is he seeing that's leading him generally to make this prediction and then even further accelerate his timeline?
- [27:08]
Paul Raitzer
Yeah, I don't. It's interesting. I don't think he's accelerating his timeline really. Like, okay, it's. So if you go back to when was this we, we talked about the Machines of Loving Grace article he wrote in. Let's say this is October 15, 2024, episode 119 of the podcast. So he had published this Machines of Loving Grace article where he had sort of like radical predictions for AI. And at that time he talked about that, what he calls powerful AI, he doesn't like AGI. He thinks it's kind of like a marketing term, so he refers it as powerful. But he had said then like, he thought as early as 2026. So I don't, I don't know. He doesn't do many interviews. So I think some, in some ways this may just. It may have gotten a lot of run because he was out at Davos World Economic Forum doing, doing these interviews, but he said, like, could be 2026. Purpose of this essay, you know, we're looking at maybe five to 10 years, like anywhere in that realm, basically. So he, he historically tends to be quite vague. Like, he's hard to pin down on exactly what he means by things and when exactly he thinks things are going to happen or why he thinks things are happening the way they're happening? He, more than most, he speaks in pretty broad generalities and he's hard to drill into specific. So I, I found I, I always listen to what Dario has to say, but I think he often presents these outlandish scenarios and then basically says, like, I have no idea what it means. Like, that's his, that's his general answer is like, we don't know. Okay, why are you accelerating if you're so worried about this, why are you accelerating development? Well, we, like, we think, you know, it can be good and we're going to figure it out and we're going to build AI that figures out why, you know, the risks are and things like that. So I, I, it's, it's weird. I get unsettled listening to him, I think is, like, what I'm trying to say. I, I think he's, I think anthropic is probably making breakthroughs. Like I said on episode 132, I think they're holding back right now for whatever the reasons. Maybe it's a safety thing, maybe it's training run didn't work exactly how they wanted it to, but I think they have way more than they're saying they do or that they're currently sharing with us. But I find interviews with him unsettling because he never seems to have answers to, like, what does this mean? And he, more than most people, throws caution to the dangers of what they're doing and never has the answer to, like, what we're going to do about it, other than when we see the risk has emerged, we will solve for it. And so, I don't know, like, I listened to a couple interviews with him last week, and it's a lot of the same stuff, but he, you know, this 2027 timeline, you know, I think it's coming from something because we're now, what, four months removed, three months removed from when he did the Machines of Loving Grace thing. Right. I, I just, I do think that he thinks, and that others think that we are very near significant advancements in AI and I, I believe that. I don't know that he vocalizes it the best, but I, I, I think that he thinks that this is very real.
- [30:28]
Mike Kaput
Yeah. On that note, and we've talked about it in previous episodes, you don't see a lot of the major leaders saying, whoa, whoa, whoa, pump the brakes. This is slowing down a bit.
- [30:39]
Paul Raitzer
No, because they're all racing for the same funding, they're all racing for the same influence. They all think that they're probably best situated to identify and solve for the risks. But I, I do, I don't know, like, I, I, I almost wonder if sometime this year or next year we don't start seeing much more collaboration between these players. Like, I, I think at some point Altman and Amade ob, you know, Amade left OpenAI. I don't know how, what kind of terms Sam and Dario are on these days, But Dario took 10% of the staff with them when he left in 2021. I think at some point we, we really need Demis Asabis and Dario Amade and Sam Altman or whoever the lead engineer is at Open. Like, these people need to be in a room talking about the reality of what if we do get to AGI or superintelligence by 2027. All they all talk about the need for some international council to exist and somebody to, like, figure this out. I think they need to get together and figure this out. Like, they're the ones building the technology and they're just hoping someone else comes along and solves for what happens as a result of the technology that they're all building. And, and so I. I don't know. Like, I don't know if something needs to happen for, for them to then get together. I can't imagine Elon Musk wanting to get in the room with Sam and some of the other guys. But there's like five or six people in the world who are leading companies that are building something that they think changes society within three to five years. And they're not talking to each other that I'm aware of about what to do about that.
- [32:24]
Mike Kaput
All right, let's dive into some rapid fire for this episode. The first up Rapid fire topic is a provocative new benchmark that is called Humanity's Last Exam. And this is highlighting just how quickly AI is advancing and raising concerns in the process about our ability to measure its capabilities. So this Humanity's Last Exam was released this week by researchers at the center for AI Safety and Scale, AI Humanities. Last Exam is basically being billed as the most challenging test ever created for AI systems. It consists of roughly 3,000 questions spanning fields from analytic philosophy to rocket engineering. And each question is crafted by leading experts, some of whom were paid up to $5,000 per accepted submission. And these are just not typical test questions. They are specifically designed to push the boundaries of what AI can achieve, often matching or exceeding the difficulty of PhD level challenges. The creation of this test was spurred by an urgent problem. Existing AI benchmarks are becoming obsolete. And they're becoming obsolete very, very quickly because new models from OpenAI, Google, Anthropic, et cetera, have been consistently mastering graduate level tests. So researchers are kind of stuck trying to figure out even more difficult challenges now. Right now, the most advanced AI models out there are struggling with this test. OpenAI's latest systems have scored the highest among those tested, but still only got at 8.3% accuracy. However, the test creator, Dan Hendricks, predicts these scores could surpass 50% by the end of the year. And that's a threshold that would suggest AI systems have become world class oracles, capable of outperforming human experts across virtually any academic domain. Now Paul, like this is a pretty interesting name. Seems like a little hyped up and wild, but definitely addressing like a real problem we're seeing. Like how closely should we be watching model performance on this particular test?
- [34:42]
Paul Raitzer
Um, so I think these like super advanced tests, like the ARC AGI test, this one, I think they matter to the research labs a lot because they get to benchmark the, you know, the overall potential and power of these models. My, this become like my soapbox thing. I think, like, I want to see the evals by profession. Like, I don't really care. Like, I. So like I, I assume this is going to be achieved in the next one to two years. Like, I just, anytime I see something like this, like, oh, we figured out a way to answer like questions I've never thought of. And it's going to, it's not going to be anywhere in the training data and it's going to be amazing. It's going to be so hard. Humanity's last. And then like 12 months from now somebody will have like done it and it's like, oh, okay, like, well now we're going to do this one. What are we proving? Like, right at the end of the day, I want to know how much of a writer's job can it do? How much of a doctor's job, a consultant's job, a psychologist job? Like I want, I want the same energy doing evals of people's careers because that's what actually matters in the economy is like, when are we going to get to the point where this thing can do 80% of attorney's job? Right now we got a problem. And, and that's like, that's the part where I think we're way closer to the answer to those kinds of questions being yes than most people want to accept. And, and so I want, and maybe this is something like I again, kind of like the literacy project. Part of the reason I did that was because we talked all last year about someone has to step up and do this. Like, we need to drive literacy across America and throughout the world. And it just wasn't happening. So I was like, all right, let's just do it. I almost feel like maybe this needs to like fit under the umbrella of the literacy project. It's like somebody has to start doing this evals at the professional level. At the job level.
- [36:38]
Mike Kaput
Yeah.
- [36:38]
Paul Raitzer
And looking and saying, okay, this job is within 12 to 18 months is going to achieve like 30 to 50% automation. Okay, what are we doing about it? Like, let's be proactive here. Let's not wait until we get to 2027 and these things are, have passed humanity's last exam and they can do 90% of the jobs. Like, I'm not saying that's going to happen. I'm not like, don't quote me on like knives of the jobs by 2027. I'm saying jobs are going to be transformed. These things are going to increasingly do the tasks that make up the jobs and no one's doing anything about it. Like nobody's running evals on that and nobody's like proactively reskilling and upscaling based on that or telling us what all these amazing 50 million new AI jobs are going to create. Create. Like, I don't see it. I don't see 50 million new jobs being created by AI by 2027 or 2030. I, I would love for somebody to tell me what they're going to be or at least like directionally say what they're going to be. Again, it's where Dario and Sam, they talk in these generalities like it's just gonna be okay. It's gonna, it always happens when we have general purpose technologies. Like new jobs come and it's going to be amazing. No, it's not. It's not going to happen that fast. Maybe 10 years from now we'll get there, but not in three years. So until somebody lays out that plan for me, I have a really hard time believing that, that it's just going to work out.
- [37:56]
Mike Kaput
So this next topic is actually quite related to this because it's kind of looking at something like this, what you're talking about in practice, because we got news that OpenAI is reportedly developing a new AI system that aims to match the capabilities of expert software engineers. So this comes from the information and they're reporting that this advanced COD agent is designed to handle complex programming tasks that typically require senior level engineering expertise. So you know, there's existing tools already like ChatGPT, other assistants like Copilot and GitHub that help with programming tasks. But this new agent is being designed to tackle really sophisticated challenges like code refactoring and system architecture. That's work typically performed by like high level, what they would call an L6 or senior staff engineer. So Sam Altman actually views this as crucial to the company's revenue targets. So they want to reach 1 billion daily active ChatGPT users in the next year. They want to generate a hundred billion in revenue by 2029. Those kinds of markets require them going after these types of jobs. So like Paul, I'm not a software engineer, but the reason I mentioned it's related to the previous topic is like this is an example of at least one AI lab explicitly targeting a high level, highly paid knowledge work task. I mean, Zuckerberg's talked about this as well, that AI is going to replace mid level engineers. It seems like the impact on highly skilled jobs is like happening now. But to your point, we're not really preparing for this when they're telling us what the roadmap is.
- [39:35]
Paul Raitzer
Yeah. And so this, you know, it's interesting as you're reading this, it made me think back to the quote I've mentioned many times on the podcast from Automate this by Christopher Steiner, where he talked. And this is the book I read in 2012 that sort of like tipped me into like insane curiosity around AI and the future of the world. His equation was the potential to disrupt plus the reward for disruption. So if you're going to automate jobs, if you're going to apply your ability to build these agents to take actual entire jobs, what is the most valuable job to an AI research firm? Firm. It is an engineer, an AI researcher. So the, the ones we're going to hear about, as you're pointing out, we are now hearing about. We have Meta Zuckerberg telling us by the middle of this year they're going to have mid level Engineer. We have OpenAI telling us they're going to build this. So what are you going to build first? If you're capable of fulfilling an entire job of a human, you're going to, you're going to build an AI researcher, an AI engineer, because the compound value of that engineer is massive. And it can, if it can work with other engineers now, you can build more stuff. So it's not saying the job of engineers goes away. It's saying like we can employ a thousand of these super engineers and we only need 100 or 200 or whatever human engineers to like manage these thousand or million. And you know, AI engineers.
- [40:52]
Mike Kaput
Yeah.
- [40:53]
Paul Raitzer
So this is kind of the canary is canary in the coal mine. Is that the right. Yeah, like this is it. We build once we do this, once we've built the thing that's really complex now, what's stopping us from going and building the next thing that offers massive value? And so you start working from the top down of how much value can be created by building an AI version of a whole job or profession. So it's this, this is my point. We're not going to get to the end of 2025 and have just replaced the need for humans to do all these jobs.
- [41:25]
Mike Kaput
Right.
- [41:25]
Paul Raitzer
But you're going to start to see the highest value jobs where the AI now can do 90 to 95% of the work. Doesn't eliminate the profession, but it does dramatically change what that profession looks like when we have an AI accountant or an AI attorney or whatever it is that can now do the majority of what that high performing human would do. So this is the stuff we're not modeling enough. People aren't talking about this enough. Economists just are ignoring this, which I just cannot comprehend. But they're not thinking about the reality of this and the impact on the economy or the impact on education, like as we think about the jobs for our kids and stuff. So, yeah, it was so funny. Like, I had no intention of, like, this being like the thread of this podcast episode. It just so happened that as you're going through these topics, they're all building on the same concept.
- [42:15]
Mike Kaput
Yeah. All right, so Meta CEO Mark Zuckerberg has announced that the company plans to spend 65, up to $65 billion this year on AI infrastructure, which is nearly double their spending from last year and well above Wall Street's expectations. So this investment includes construction of a new data center with more than 2 gigawatts of computing power. That's enough to cover a significant portion of Manhattan. They also plan to amass an arsenal of over 1.3 million GPUs by year's end. This would cement them as one of the largest buyers of Nvidia chips. So this comes, of course, just days after we have unveiled the 500 billion Stargate Initiative, which is going to benefit OpenAI primarily. And Zuckerberg really just emphasized the strategic importance of the investment. He thinks 2025 is a defining year for AI and wants to expand Meta's AI assistant to serve more than a billion people by year end. So, Paul, like, the big elephant in the room here is that this is like part of this is likely Deep Seek, putting the fear of God into Meta, perhaps. But they also were probably going to be very aggressive with R and D regardless. And an infrastructure investment rather.
- [43:36]
Paul Raitzer
Yeah, I mean, this was happening. It was just such a weirdly timed flex. Like the day when everyone else is like, oh my God, llama 4 just got surpassed and they're gonna have to not release it. And yeah, Zuckerberg's entire strategy was to undercut the market with open source models and now China just undercut him and like he's tweeting a picture of Manhattan with like the size of their forthcoming data centers. Like, it's like, I guess you're just doubling down right or wrong on this whole thing. So yeah, I mean it's been in the works forever and they already had this capex allocated for the year, the 60 some billion. But it was just so bizarre to like see that tweet or the threads or wherever he put it that this is what they're doing when everyone's like, dude, did Llama just get completely undercut?
- [44:17]
Mike Kaput
Right. Yeah. And you're spending more. And like you and I were talking about before the episode, there's no way to tell why some of these stocks are moving the way they do. But I, I don't understand how they're up a bit or.
- [44:28]
Paul Raitzer
Yeah, I'm truly. This is why I don't day trade. This is why I just buy and hold like the stocks I really believe in. Because last week I said I was losing faith in Apple and they're up two and a half percent. I assume Meta would be down more than Nvidia today, which is now down 17%. Geez. Oh man, I am not looking at my retirement portfolio. I want to get off of here. And then Meta's up like 2%. I was like, what? What? Makes no sense.
- [44:54]
Mike Kaput
All right, our next update here this week is that Google has come out with a big Update to Gemini 2.0 flash thinking, their experimental thinking model. They haven't. The new model showcases some remarkable capabilities. It can process up to 1 million tokens of text, which is five times more than OpenAI's O1 model. It also has faster response times. It has achieved unprecedented scores on advanced math and science benchmarks. And what sets this release apart is really how Gemini 2.0 flash thinking goes about doing all these reasoning tasks. It actually explicitly shows its work, which makes its decision making process transparent to users. The model has already claimed the top spot for the time being on the Chatbot arena leaderboard and leads in categories including hard prompts, coding and creative writing. Now, what's interesting is, at least for the time being, it is also free. Google offers the model to anyone during its experimental beta testing phase in the Google AI Studio platform. So Paul, I just look at something like 2.0 flash thinking and I'm like, just have to appreciate how fast things are moving. Like we just quickly got a more transparent thinking model. It's accessible, it's cheap. This Is like such a change from 12 months ago.
- [46:15]
Paul Raitzer
Yeah. I mean things have changed so dramatically. I do think some of what you just said, like, is kind of what I saw people over the weekend talking about with Deep Seek. Like, Yep. One of the most fascinating parts of Deep Seek is to see the reasoning process. And it's almost like an people would say like, it's like listening to a human's internal monologue. Like the challenges it has like, oh, the human wants this. Oh no, I got to do this. And it's like debates with itself. So I think the more we see the underlying reasoning because I think like oh, one from OpenAI, if I'm not mistaken. Like it's, there's almost summarizes the reasoning. It's like, yeah, it does this like Deep Seek, you, you truly see as those like the thoughts inside of a mind. It's like, okay, they're asking me for this, but if I give them this then that's not going to answer their question. So I need to do. And it's like this is what it's doing in like milliseconds. And so I think we're going to learn a lot about how these models work the more exposed we are to the chain of thought that they're going through to, to create the output for us humans. I think it's gonna be a really fascinating part of people actually starting to realize why three years ago you had Google engineers worried that these things were conscious. Like when you start to really see what they do, it feels very human and it's, it's very odd to have to separate yourself and realize that that's not actually what's happening. We don't think.
- [47:35]
Mike Kaput
And interesting too. Just I realize this is just an experimental model. Google obviously charges for a bunch of its AI stuff but we even saw this with their like Google workspace pricing for Gemini. Like they are releasing stuff for not that much money and or free because not. They don't necessarily have to rely on these subscriptions to like power their business like someone like OpenAI might.
- [47:57]
Paul Raitzer
Yep. Which is a very large advantage.
- [48:01]
Mike Kaput
So the hits don't stop here for Google. They're having a great week as well with their Imagen 3 image generation AI system that has now claimed the top spot on lmarina AI's popular AI text to image leaderboard. So this model is now leading all the other image generation models out there from competitors like OpenAI. And it's leading right now by a wide margin. So this leaderboard, which we've talked about a Bunch ranks AI model capabilities based on a number of factors, including which models people actually prefer to use based on human votes. So in this particular leaderboard for these image generation models, the site doesn't just rank overall how good the model is, but also how good it is in specific categories. So there's one category that they have titled User prompts only that basically evaluates how well these models handle real world use cases. Imagen 3 is number one in that area as well. So Paul, with everything going on, especially in news around reasoning models, like we haven't talked too, too much about image models in the last episode or two, but it sounds like innovation has been moving at light speed here too, especially from Google.
- [49:15]
Paul Raitzer
Yeah, I've heard lots of good things about this model. I haven't personally tested it in a little while. Dolly seems to be standing still like OpenAI. I'm not sure what their plans are there, but you just get same of a lot of the same generic outputs you, you know, did a year ago on Dall E. So it seems like, you know, Google's made a lot of progress on not only image generation, but video generation like we talked about with vo. So this is their whole vision of like this multimodal model, you know, multimodalities in trained on, you know, images and videos and text and, and able to output those things. And so I, I, I, I do think that like we're going to see this vision really come together for Google and maybe it is with 2.0 in the spring or you know, before then where you truly have this like really powerful model. I actually saw Logan Kilpatrick, I think it was on Sunday, tweeted like it was, it was a weird tweet in response to like Deep Seek being number one in the App Store. He said if we packaged AI Studio as an app, it would be number one. Like, because I think he was saying like there's so much amazing stuff happening within Google's AI Studio and if you just like made all that super easy to access. Yeah, and it would just crush because people would realize all the value that's sitting here in these different little products. So something interesting to watch.
- [50:32]
Mike Kaput
Yeah, that is interesting to mention because I feel like even among very savvy people in our audience, I think a lot of them forget, like through AI Studio and like a couple other sandbox areas Google has, you can access a bunch of these experimental models.
- [50:46]
Paul Raitzer
Yeah, they're labs. You can go in and test stuff. I think that's where VO2 is, is in.
- [50:51]
Mike Kaput
I think I believe so, yeah, yeah, yeah.
- [50:53]
Paul Raitzer
They all kinds of stuff in in Google labs. It's cool.
- [50:57]
Mike Kaput
Our next topic is about Sam Altman doubling down on his mission to extend human life. So we had referenced in our past episode a startup he funded, Retro Biosciences. They are now launching an ambitious billion dollar fundraising round. So this San Francisco based company was initially seeded by Altman with $180 million to develop AI powered treatments aimed at increasing human lifespan by a decade. So they have now partnered with OpenAI to create a specialized AI model that designs proteins capable of temporarily converting regular cells into to stem cells, potentially reversing the aging process. They plan to begin clinical trials this year, starting with a potential Alzheimer's treatment in Australia. They're also looking to accelerate the traditional drug development timeline. Rather than the typical 10 to 15 years required to bring a drug to market. They are targeting their first drug release by the end of the decade. They're pursuing three main drug candidates right now. One is a pill that restores cells internal recycling processes. So one is a therapy to replace brain cells linked to Alzheimer's and one is a treatment to rejuvenate blood stem cells. So the company is currently for this funding in talks with family offices, venture capitalists, sovereign wealth funds and a major US data center provider to secure the massive compute needed for these AI models. So Paul kind of related to what Demis Hassabis was saying in his interview at Davos, which we covered in the last episode. Definitely seems like we're seeing some big moves in AI for scientific progress.
- [52:44]
Paul Raitzer
Yeah, so I was, I just made a note to myself, it was kind of a joke, but human life extension is like the new rocket company for billionaires. So like, you know, 10 years ago if you were a billionaire, you needed to be building a rocket company like SpaceX or Blue Origin. Now you need to be all in like human life extension. Seems like all, all these guys are talking about this. So I'm back to Dario Amade's machines of loving grace thing. And in, in that October article he talked about biology and health. He said my basic prediction is AI, Naval biology and medicine will allow us to compress the progress that human biologists make but achieve over fifty hundred years in five to ten years. And he talked about doubling the human lifespan. This might seem radical, but life expectancy increased almost two times in the 20th century from 40 to 75 years. And so it's on trend that the compressed 21st would be to double it again to 150. So Dario's, you know In October, talking about living to be 150, in the Demis interview, he talked about biology and life extension. And he said that the current understanding in biology is that 120 seems to be the natural limit, but that he would be very surprised if that is, in fact, the limit on human lifespan that he definitely sees the ability in this generation to have people living commonly past 120. So I think part of it is these AI people see the future and see how it's being applied to advancements in biology like alphafold and chemistry. And they think of aging as a disease that's solvable. Like, they don't see anything that actually prevents the rejuvenation of cells and stuff like that. So, like, to them, it. It's not even that crazy to talk about human life extension. And so it's logical that they would want to play a part in that and live longer themselves and, you know, benefit other people, I guess. So that's a whole nother. Whole nother topic, man.
- [54:39]
Mike Kaput
Yeah, no, that feels. It's. Feels a lot like what you were trying to communicate with the point Demis made about how good Alpha Fold is at this stuff over humans. And, like, it's like a billion years of PhD research. It's like, okay, like, we'll see if the life extension thing is actually proven out. But the point being, these models are already capable of doing things that we couldn't even have dreamt of.
- [55:02]
Paul Raitzer
So, yeah, and what Dem has talked about is, like, their next frontier is they're building virtual cells. So they're actually trying to build a cell. Human cell simulation. And then once you achieve that, which he thinks they can do in the next five years, they took a bunch of the AlphaFold people and put them on cell creation. Then you can actually test drugs in a simulated human cell of how they would solve things. So he thinks within five years, they'll have the ability to simulate human cells. And then once you have that ability, you can now run all these simulations. So he sees within a decade, massive advancements because he's very confident they're going to have the ability to run these simulations, which within five years, it's wild. Like, it's. So it's so crazy to think about, like, the possible outcomes of this stuff.
- [55:47]
Mike Kaput
All right, our last topic today, we wanted to quickly share. We're trying to, you know, share practical AI use cases where we find them either from others or stuff we're doing. So I just wanted to quickly share one use case I found for some of the, like, Google AI Studio tech, those experimental models that we just referenced in a previous topic. So I'll just quickly, Paul, go through this and then we can wrap this up. But really what I was able to jump into at Google AI Studio and use some of their experimental models for was a pretty cool experiment with leveling up my speaking skills. So I do quite a few talks. Paul, you're, you're on the road all the time doing talks. I was in San Francisco last week doing a talk and you know, obviously you prep quite a bit even for short talks. And what I found really helpful is I was able to actually turn AI into my own personal speaking coach. So I used a custom prompt and Google AI Studio and basically recorded myself practicing, uploaded the talk to Google AI Studio, the prompt basically being like, you're a speaking coach, here's how I want you to analyze my talk. And then it went ahead and actually analyzed it for tone, pacing, delivery and more. And the reason I did this with Google AI Studio is because you can actually upload audio and video and like a very large context. But notice I was like doing like a 35 minute talk. And what's really cool is like it caught all this stuff I just never would have even been aware of. Like it timestamped my best moments, it talked about when my energy peaked, it talked about, hey, you nailed this stat or this example and highlighted a ton of other stuff to focus on improve. And then I was able to actually compare across different practice runs. So I'd do one, read the feedback, do it again, try to apply it, so on and so forth. So it was just a really cool, really practical, immediately useful way to use some of these tools.
- [57:44]
Paul Raitzer
Yeah, it's awesome. And I think that's. I know you put it on LinkedIn, which is when I saw it. This is it so perfectly demonstrates what we always talk about. And again, going back to like the importance of AI literacy, understanding what AI is capable of enables you to then figure out ways to use it for the mundane things like the repetitive data driven stuff you don't enjoy doing, but for like the creative, innovative things like, and that that's the potential we all have is to like find ways to, to use this technology in a positive way to just make us better at our job, make us enjoy our job more. It doesn't all have to be just replacing repetitive stuff and driving efficiency. It can be about creativity and innovation assisted by AI. First you got to understand that it's possible. If you didn't even know Google had an AI studio, you would never think to do this. So, and I think each week that's what Mike and I try and bring is like this really foundational knowledge so that hopefully you can go take it and do all kinds of cool, incredible things that Mike and I might not even think to mention on the podcast. And so, yeah, I think these kind of real world examples are awesome. I, I think it's fun for me because I see more and more people in my LinkedIn network who share stuff like this on LinkedIn. Like, hey, I did this cool thing last week with AI. Yeah, I think that's like, it's inspirational for me because you just see people taking knowledge about AI and going and doing cool things, and that's the opportunity. I think once you get in any profession, any business, once you get through the fear of this stuff and the uncertainty we all face, and you just say, like, I'm just going to go use it to best of my ability and we'll see what happens, like, figure the rest out later. You, you really start to get excited about what's possible and you start, you know, showing yourself what you can do to kind of reimagine, reimagine your career. And that's, that's what keeps us optimistic about the future.
- [59:30]
Mike Kaput
Yeah, for sure. Yeah. It's a lot of fun to also just mess around with and be able to discover this stuff for yourself.
- [59:36]
Paul Raitzer
Yeah.
- [59:37]
Mike Kaput
All right, Paul, that is a wrap on our second episode of today, the second episode we're releasing this week. Just a quick note for everyone, if you haven't checked out our newsletter, go to marketing AI institute.com forward/newsletter. It has all the news that we've covered today and everything. We couldn't fit into the episode. And leave us a review if you can, on your podcast platform of choice. We'd really, really appreciate it. Paul, thanks so much. How was that? That was a lot.
- [60:05]
Paul Raitzer
While we've been on this, I got like four messages from people asking me about Deep Seek. I just say, just listen to episode 133. All right. Yeah, thanks, everyone. I, I, who knows what this week is going to bring? I can't imagine it's going to be as wild as last week, but I assume we'll be back to one weekly episode next week. I don't, I don't think we're going to make this a regular practice because this is like our entire Monday has been taken up doing these podcasts, but. All right, hopefully it was helpful for everyone. We will be back next week with our regular weekly episode. Thanks again. Thanks for listening to the AI show. Visit MarketingAI institute.com to continue your AI learning journey and join more than 60,000 professionals and business leaders who have subscribed to the weekly newsletter, downloaded the AI blueprints, attended virtual and in person events, taken our online AI courses, and engaged in the Slack community. Until next time, stay curious and explore AI.