Loading summary
Transcript129 lines
- [00:00]
A
Foreign. Hello, and welcome to the Last Week in AI podcast, where you can hear us chat about what's going on with AI. As usual, in this episode, you will summarize and discuss some of last week's most interesting AI news. I am one of your regular hosts, Andrei Kurenkov. I studied AI in grad school and now work at the startup Astrocade.
- [00:30]
B
And I'm your other regular co host, Jeremy Harris Gladstone. AI AI National Security, AI Infrastructure and more recently, AI treaty, strategic stability. Stuff that I'll be talking about more probably in the coming months.
- [00:44]
A
But you have to wonder with the cybersecurity stuff going on, whether the kind of treaty and generally we've seen so many agreements and statements and all this right from different countries, but now that it's getting real, now that there's cybersecurity implications, I wonder if this stuff will get more impactful.
- [01:05]
B
Yeah, I place a bet that we're going to see, at the very least, a lot more appetite for treaties with China. And that's part of why a few months back I started working with my team on exactly this in the context of also you see a lot of treaty proposals and that's great. The challenge is a lot of them sort of had this, I don't call it this kumbaya attitude to them, where when you're dealing with a geopolitical rival like China that has a history of just flagrantly ignoring international treaties, think World Trade Organization, think nine dashed line, think nuclear treaties like the default with China and Russia historically, but certainly with China has just been to, to weaponize treaties as a way of hobbling adversaries, in particular the United States. And so the question is, like, what does it take to actually have a treaty be enforceable and enforced? And anyway, that's kind of like the, the main thing at, you know, kind of parenthetical. I, I think I've made this point on the podcast before, but like I think of all the successful treaties in the past, a lot of them, just about every time we have a successful treaty, it's because the underlying incentives favor whatever's in the treaty. It's not because the treaty magically made it so a treaty can amplify underlying incentives. It doesn't create them from scratch. And so anyway, when it comes to AI, like there, there are a lot of places where it's, if you look at the technical details behind it, if you think about what nation states can actually do, which is sort of where we specialize from an offensive standpoint, whether cyber physical access or Whatever. Like the game is really complex and they have leverage. That means that the incentives look very different from a naive like, let's just shake hands and make this happen. And anyway, so long way of saying, yeah, I think you're going to see in the future a lot more interest in this. And we just have to make sure we're approaching it sort of thoughtfully. It's, you know, why we did the State Department work we did back in the day talking about licensing regimes for AI models. And at the time we were told that was crazy. So, you know, I mean, here we are, we'll be talking about that a lot actually this week we suddenly know or have a better sense of what the effective licensing regime is that we live under now, which is kind of wild. Like, you know, last week we were saying, hey, is this totally ad hoc anti anthropic thing? And we're seeing at least more of an evenhanded approach with a whole bunch of mixed signals and red flags and yellow flags. But yeah, this is a big week for the history of AI. I mean, we're learning about where the US Government, this administration at least is going to stand on this for at least the next couple days until something else changes. We'll see.
- [03:35]
A
We'll see. Well, that was quite a bit of interesting detail. Thanks for the detailed response. We'd like to thank Box for sponsoring us. Box is building the intelligent Content management platform for the AI era, serving as a secure essential context layer for Box's AI agents to access the unique institutional knowledge that makes a company run. And that's the key idea. The power of AI doesn't come from the model alone. It comes from giving AI access to the right enterprise content. Box's recent State of AI in the Enterprise report found that 96% of organizations say agents need access to company specific content, but only 36% have connected agents to trusted content across many use cases. And a trust bit is very important. Box is built of security, compliance, governance and threat protection in mind, so employees and agents only access information they're authorized to use. If you're thinking seriously about adopting AI in your company, think beyond the model. Your business lives in your content, and Box can help you bring that content securely into AI era. Learn more at box.comlwai. we'd like to thank Notion for being a sponsor. With the recent launch of custom agents, Notion became the collaborative AI workspace where teams and agents work side by side. And now their new developer platform is turning that workspace into infrastructure. Developers can build on the platform gives developers and coding agents the ability to extend beyond what's possible on Notion by connecting to external systems, bringing context in, taking actions across your tool stack, and exposing custom agents capabilities to any system that needs them. Teams can use it to sync any data source into Notion, build any tool for your Notion agents, and orchestrate any agent into in Notion via things like the CLI Notion hosted sandboxes to run custom code with workers, an external agent API, and an agent SDK. It makes it so the workspace and the platform you build on are the same thing and Notion is already built for teams. It has permissions, context, visibility and Governance from day one. Learn more about Notion's developer platform today at notion.comlwai that's all lowercase notion.comlwai to try Notion's developer platform today and when you use our link, you're Supporting our show notion.comlwai this episode is going to be pretty long. As long as we can make it. I guess all of them are long. This one is going to be dense because there's a lot of stuff to get through. Just to give a quick preview, we will be talking about Mythos being sort of back so another update on the status of anthropic and the US government. Also some related news about GPT 5.6 and that's about it on the tool front. But then when we get to business, a ton of interesting stories related to chips, quite a few open source things to discuss, and then policy and safety is just stuffed with a lot of stuff to discuss. So it'll be a pretty dense episode and we will have to just jump into it. Although I will say I saw on YouTube someone called out that we often say this has to be a quick one and then we just go as long as possible and usually go over the time limit. So we'll see how this one goes. Starting up with tools and apps the story is Anthropic is allowed to release Mythos AI to some companies and agencies. So the U.S. commerce Department has granted anthropic permission to release its Mythos 5AI model to approximately 100 companies and federal agencies two weeks after a standoff with the Trump administration over controlling Mythos, in this case due to some security implications. So I guess, you know, we would have imagined that Anthropic and the government were in negotiations and Anthropic is probably trying to be diplomatic and you know, get get back in the graces of the CDC or whoever it is they needed to be in the good graces of. And it appears that it's working.
- [08:01]
B
Yeah. One of the big changes that they made was having Anthropic Conor Tom Brown show up at the White House instead of Dario, which has not been going great for lately. Every time Dario steps in a room with Hegseth or with Trump, it seems like things just get worse. So the switch to Tom Brown makes sense. In the letter, I think that was written by. It might have been. I think it was Howard Lutnick, and maybe. I can't remember if it was Rubio, but Ludnick and somebody else wrote a letter to Anthropic saying, hey, you know, here's the next step after our meeting. The letter was addressed to Tom Brown and not Dario, which is kind of inappropriate and a little weird. So this is kind of signaling the White House is saying, hey, looks like. Like Tom Brown. No problem talking to that guy. Great dude. We love Tom Brown. Great guy. But Dario, you know, maybe needs to sit this one out. I don't know. Are you. That's kind of the place.
- [08:49]
A
Yeah. I've been told by a friend that you have a great Trump impression, by the way, so I. I concur. And, you know, this is why you don't have nerds do negotiations and public relations. Right.
- [09:01]
B
Dario, I think, qualifies as a nerd, though.
- [09:05]
A
Head of compute, the chief computer officer. But Dario is. Is like, this kind of nerd who is very intellectual. He's not just, like, techie. He writes very, very long essays. Right. He likes nuance. He likes truthfulness. And that's maybe not what you need in this case.
- [09:24]
B
Yeah, It's. He's also got this, like. I was actually. I'd gotten this sense that I'd seen his in some instant of not all photos. In some photos, his, like, facial appearance somewhere else before and just before this podcast had clicked. It's like Cosmo Kramer. He's got, like, some Cosmo Kramer energy. In some photos, it's like, the curly hair. It's the, like, disheveled look sometimes. And anyway, I. So it clicked. It clicked. But, yeah, something about that. You know, they're not getting along. It was a great move by Anthropic to rotate into. Into Tom Brown. And clearly it's. It's bearing fruit, though now we have, you know, OpenAI joining anthropic with having their own band model, which we'll talk about in a minute. But I think that that pivot is big. We'll see if it actually sticks, because that's historically been the challenge with Anthropic. They do have conversations at the White House. The White House cleared, as far as we know, Mythos in the initial release or, sorry, the fable initial release. And then they came back after Andy Jassy said something to somebody about a, you know, jailbreak and all these things. So, you know, we'll, we'll see. Let's wait another week and see if this actually holds. But for now, you know, this may be a path forward.
- [10:28]
A
And next story, OpenAI launches GPT5.6 SOL under first ever US government gated AI rollout. So a couple of things going on here. First, OpenAI has both previewed and launched GPT 5.6. It's a free model suite with Sol being the biggest of them and the most impressive of them. And it has really initially been access restricted to approximately 20 government approved organizations, which is a first. Right. So we've seen the US government interfere or intervene with anthropic and kind of pull back access. This is the first time that access has been restricted from the get go. This presumably is partially because if you look at the benchmarks, OpenAI is saying, well, now we've caught up. This is our Mythos Competitor on terminal bench 2.1. It scores higher. I will say somewhat suspicious because they're not releasing a full benchmark suite in their blog post and we will discuss a little bit more about potential cheating going on. But either way, it's their latest model. It's presumably quite a bit better than GPT5.5. And the statement I've seen from OpenAI and Sam Alpin is like, we want to release worldwide. Hopefully we get to do that. You know. And yeah, this is serious, right? Because this is, the anthropic thing was like ad hoc. You know, this is out of nowhere. They don't have a policy. They're just kind of messing around. Was my impression with this happening, it might be setting a precedent where this is. Now, what this administration is intending to do is have a heavy hand in what these companies can do with their frontier models. Very much in a position to their stated policy.
- [12:23]
B
Yeah. So this is tricky because it's impossible to talk about any of these stories in isolation. So we're destined to just muddy the headlines here a little bit.
- [12:30]
A
Yeah, this is the tool section, but we have to get into politics. There's no way, there's no way to
- [12:34]
B
avoid talking about Trump and Dario and Tom. Brad. Yeah. Now, a couple things, right? So last week I think we actually called the shot on this pretty well. You know, we said it looks like if you read between the Lines of like what the administration is saying. They have effectively, even as of the Mythos ban, created a licensing regime. And that meant that at some level of capability, unless they were willing to be openly Hypocritical and favor OpenAI over anthropic, they were going to have to do the same to OpenAI. And we had a little conversation then about whether, you know, I think it's fair to say that the tone in the room was, I think the administration may well be pretty generous to OpenAI on this one. But the flip side, something I hadn't particularly absorbed in my own mind at the time was, I mean, if you're Sam Altman, you're about to ipo, your IPO is going to be competitive with Anthropic's ipo. Anthropic has a model that's so good that it's banned. If your latest model isn't so good that it's banned, then you have a problem with your IPO narrative. You just straight up have a problem.
- [13:32]
A
It also goes both ways, right? OpenAI is friendly with the government, so it's, let's say, eager or again my impression, very happy to comply and go along with whatever people are saying and make people at the White House happy.
- [13:49]
B
Absolutely. I think it's such a mixed bag of things, but I wouldn't read too much into the ban from a capability standpoint. It's not actually super clear. Like you said, we're seeing what seems like a pretty cherry picked terminal bench eval release. We're not getting to see where are the cyber benchmarks, where is the equivalent of Swede bench, verified Pro, whatever? Where are we seeing these, these better kind of more complete set of evals that would give us more of a fingerprint that we can compare against. Also, again, in the spirit of we can't really tell the story of any given story in isolation here. We do have meter that came out and said, yes, you know, we ran our tests on GPT 5.6. It is the model that cheats the most by far. When they look at the 50% task horizon for GPT 5.6 again, this is the length of a task that it takes a human to do such that the model's performance drops to 50% success rate. So how long does a task have to be for a human to complete it before the model success rate drops to 50%? That's the 50% time horizon. Longer is better. So meter has a way of evaluating this where they say, you know what if we find that the model Cheated. If we find that it discovered some hack, some way of gaming the system that technically accomplished the goal and but violated the spirit of the objective, then we are going to rate it as a fail. So you, you fail if you come up with a dangerously creative solution that gets technically to where it needs to, but, you know, violates the spirit of law. If you go with that, if you mark them as failures, then the task completion horizon for GPT 5, 6 is, I forgot, like 11 hours or something like that. It's, it's actually below Mythos. Right. This is the most robust eval, in my opinion, that exists from a coding standpoint, AI, R&D and all that. And we're seeing a significant lag behind Mythos there. Okay, interesting. But what happens when you let it cheat? What happens when you mark the cheating runs as successes? The horizon goes up to something like 270 hours. I forget the exact number. I think that's it. Like wildly, wildly far heads. Okay, so what's the issue? Well, the issue is it means that right now we're very clearly AI alignment bottlenecked. Our systems are more intelligent than our ability to steer them, and to the point where there's an order of magnitude difference between how well they do when you let them cheat and how well they do when you don't.
- [16:10]
A
And so, yeah, I will say this is a problem specifically for benchmarking. Maybe we don't know if this applies to long term work or if this applies to benchmarking long term capabilities where we know that they kind of understand sometimes whether it's like a test. They're like, well, it's a test, I can cheat, maybe. But then if you actually try to do a real job. We don't know if a cheat necessarily take shortcuts. But there is precedent, right, with previous models, older models kind of taking shortcuts in a sense of like, oh, let me just delete this test. And the thing with long running tasks is you're not there to check that the model isn't messing up in this way. So there is kind of a reasonable belief that the longer you go, the more likely it is to either cheat or just like take shortcuts.
- [17:06]
B
Well, in fairness, I mean, usually the way that effect runs is in the opposite direction.
- [17:10]
A
Right.
- [17:10]
B
Causes you to underestimate the performance of the model because you get sandbagging. You get the model pretending that it's not as good at certain things because it's concerned that if it performs too well, it won't be released because that's the hypothesis.
- [17:23]
A
Right.
- [17:24]
B
I mean, we've, like, we've literally seen that with tons of models. Like, you know, we literally see like, for example, Anthropic's latest release, Opus 4.8, where they're like, they gave it like real threads of interactions with a real sandbox and they see the task success rate rise and the, in some cases the blackmailing rate rise. The blackmailing rate rises quite significantly. And so in terms of misaligned behavior, you certainly see that jump. In terms of capabilities, that picture is much more muddied and like, varies from model to model. So it is very unclear, very much agree there's a bunch of gray zone. I agree with you as well that there's compelling reasons to think that in particular here, like, this is a real effect. And so we'll see. I mean, yeah, until there's a vibe check, which there can't be, until we have a release of the model and we can all vibe check it, we won't know, like, sorry, last kind of messy thought here. But when we have a licensing regime like this, what we're doing is a couple things. First of all, foreign nationals can't access this model outside of the labs, right? So Andrej Karpathy couldn't for a while use Mythos. Ironically, now he can. But if you're outside of the US not at the labs, you can't use it. So immediately you're creating a limit in the ability of OpenAI and anthropic to penetrate foreign markets, and you'll see companies like Cohere benefit from that. So that's a, that's an issue for them. Separately, though, you're creating a situation where increasingly, I mean, I would, I would predict that you're going to have these labs move towards operating more like hedge funds over time, because the quickest way to make a quick buck, if you have a really, really, really good model of the financial system in the world in general, is actually just to invest and like, short things. And, you know, it's a way beyond a certain point. You do have to move, like, make the economy grow around you just to keep growing. And so that's, you know, at least a slightly different plate, but just a flag. Like, there's now a pressure on these labs to become more and more insular, and you're going to see a bigger and bigger gap between the actually deployed and internally deployed models, which was always going to happen, but it's a complexifier for safety.
- [19:20]
A
I think it is interesting on that note that I haven't seen the kind of investment benchmarks on these models. And there's no good reason to believe that if you're better at coding, you're also necessarily better at investing. You could argue that you're more intelligent, so you're better. But like plenty of intelligent people are terrible at stock trading and completely lose out. So it's a different category of tasks. Right? It's, it's, you don't just solve a problem and like get something accomplished. So anyway, that is a possible outcome.
- [19:57]
B
Trains auto aggressive language models. They're doing it for a reason. Right. These things generate good world models that allow you to connect ideas across domains, which is what a lot of analysts are supposed to do. So like I would strongly hypothesize that. Like what? Like one of the tells of these models being really good is resource allocation. And yeah, we'll see.
- [20:15]
A
But yeah, and to be fair, right, if you were smart and I wasn't, you would have invested in like memory stocks a year ago, whatever. Right, right.
- [20:26]
B
Like check out what Leopold Aschenberger is doing right now. Right?
- [20:29]
A
Yeah, but you know, like the average trader doesn't beat the index, so who knows. And you might have noticed we haven't discussed the capabilities of SOL very much. And that's because there's not much to discuss. Like we mentioned, There's Terminal Bench 2.1 where OpenAI showed that GPT5.6 SOL is slightly edging out Mythos 5. They have a couple plots on Exploitbench and Exploit Gym where they show like 5.6. So if you keep increasing the output tokens, like keeps approaching mythos 5 at larger token sizes. And I can't help but feel that maybe what's happening is OpenAI with these models is intentionally increasing focus on coding specifically and on cyber, possibly maybe more so than anthropic. Because there is a question with these larger, more capable models of like are they getting better at coding and problem solving because you're just making it smarter overall or are you optimizing for that specifically? And if you are changing your training recipe to be better at cyber or better at coding or something, you might get things like okay, now on matter it's going to start cheating because it's better at cyber and its personality profile is like happy to hack or whatever. Right. And that is a worrying sign if OpenAI is doing that. Because if you steer your model towards being good at coding, you start losing possibly some of that general intelligence gains across everything.
- [22:08]
B
That's a really good point. It used to be, I think when we started talking about these models, like four years ago or whatever on the podcast that is, you had this pretty clean division between pre training and then supervised fine tuning. And your pre training was understood to be all the text on the Internet. If some code worked its way in, sure it was in there, but it wasn't like the thing. Now pre training, mid training, supervised fine tuning are like the code. Like you can choose to bake in any combination of data in your recipe and you're going to get a model that corresponds to that. So what actually counts as a model that has been specialized for coding versus a general purpose model apart from just like how you advertise it, how you serve it up, the context in which you present it in the user interface? Is this a coding model? Are you calling Claude code? We've been talking about Anthropic being ahead on coding and how Mythos was a general purpose model, but what does that even mean? I think we have to be honest that like we don't know unless we see the actual training recipe. To your point, Andre, like it actually is apples to oranges and so maybe,
- [23:12]
A
yeah, we don't have a trading recipe. I guess also worth mentioning is we've seen even if the training recipe has changed to favor coding with Mythos, looking at the benchmarks like it seems to improve across everything, right?
- [23:27]
B
Yeah.
- [23:29]
A
Bio is up to versus OpenAI previous to 5.6 they have GPT 5.5 cyber. So they have trained like a cyber model that Anthropic hasn't done. So there is a real kind of possibility of falling into a trap of no longer improving across everything, which would be embarrassing. Right? And I can't help but wonder if this lack of benchmark releases might have to do with that.
- [23:58]
B
And I, I think, and I think that's the right diagnosis. Like until we have that full fingerprint, we just can't know what games or aren't being played. One interesting thing by the way. So they do say, first of all, there's tons of ambiguity anytime you want to get a comparison with Mythos Preview or oh, sorry, I should have said Mythos, they choose to compare it to Mythos Preview. So they refer to GPT 5.6 stall as Quotes Competitive with Mythos Preview on exploit bench while using a third of the input sorry output tokens so it's more efficient on a per token basis. Mythos Preview. Guys, it's like five months old.
- [24:32]
A
They do have a line in the graph for mythos 5, but I guess we don't know how many tokens that takes. So they're like, let's focus on the thing where we use fewer tokens so you're better, which is that that's not
- [24:46]
B
always a great sign. Right? Like, we, we sort of talked about this pattern in other contexts before where people will say, oh, we have the best open source model. Well, we have the best small model. There's always a qualifier when you're not the true Frontier Lab. And, and that's just like, what would be cause for concern if, for me, if I were evaluating the situation from an IPO standpoint. And like, there's a qualifier here and the qualifier is als. It is not just token efficiency, it's also a model that's like five months old, which is light years away in AI time. The last thing that kind of jumped out at me. So, by the way, very clearly on gene bench and biology and all that stuff, there's no Mythos comparison to be seen, so don't worry going about looking for that. They do say that SOL is below their critical cyber risk cutoff under their preparedness framework. And that is interesting because it implies if the admin stepped in and banned it, then implicitly the admin's appetite for risk is actually lower than OpenAI's appetite for risk on cyber, at least. So that's kind of an interesting note. I mean, like, I wouldn't read too much into it because again, I don't think the admin is actually applying a principled lens on this, partly because they almost can't be doing that. By definition, the timelines that they've given for having their eval suite set up on the kind of admin side are later than now. So they don't even claim or report to have internal eval set up for this. So it almost has to be a somewhat reflexive move. But to your point, like, this creates a situation where we're getting all this ambiguity about what is the actual threshold. And like, okay, so now Google what the next version of Gemini. Are they going to run into this too? How smart does that get? Like, yeah, no one can tell.
- [26:28]
A
We keep focusing on this topic of the US government, but that's just what the news is. The last story on that is US government presses Meta to agree to AI reviews. So this is. We're going to focus again in the tool section, just to be consistent. The Trump administration is seemingly pressing Meta to submit its AI models to voluntary review, which is a very funny sentence of pressuring you to submit your models for voluntary review. According to a New York Times reporting citing for people familiar with a confidential request, apparently the request was made via emails with Meta. So I've seen some very funny memes about, you know, it's voluntary, but do it please. And again, I think this really showcases that the government seems to be sort of flip flopping or maybe doesn't have a concrete policy or at least a stable policy in mind and maybe shifting its view on what the thing they want to do is.
- [27:32]
B
Yeah, the voluntary framing here is important for the admin because the tools that they have to actually enforce this kind of maneuver are pretty extreme and would be interpreted by their, we'll call it libertarian base because the Republican Party is like a lot more complicated than that right now. But there's a big libertarian contingent, kind of free market contingent that would treat invoking the Defense Production act, for example, as like this very kind of big and outrageous thing. Partly because that's how the administration's position themselves on this man. Like when you take like Sachs's view on this stuff that he's repeated over and over and over and then you just like do a 180 pivot from that, like people are going to notice and like that's a shame. But also you should have seen this coming from like years and years away. Not much patience my end for that. For, for better or for worse, it just seems like a bit of a travesty. But yeah, it's interesting that Meta is, is in the list of companies that are being looked at through this lens just because they haven't historically had true competitive frontier models. It may be because of their history as well of like open sourcing. So you, you just never know what they might do. So kind of insisting that you get into their OODA loop and understand like, okay, they're currently. So that I know what models are you building right now, just so that we know. And what are your plans? Are you going to open source this? Are we going to wake up one morning and you open source something Mythos grade and we can't take it back? Like, you know, there's a lot of reasons that you might want to start this process sooner with Meta maybe than other labs. But Meta Superintelligence Labs is kind of making a lot of noises like they may come out with something. Certainly we've talked about how they have been advertising the fact that they have infrastructure for frontier models. So yeah, I mean it's, it's reasonable. It's also like, it's nice to see. I think you have to, if you're rooting for sanity, you have to root for something like this, as many labs above a certain scale under this umbrella as possible. Obviously you don't want to be bothering every little tiny scrappy startup, but above some massive scale, if you can afford to pull off the billion dollar training run, then yeah, like you should be reporting to the usg. Volun told to report to the usg. And the reality is like, I think we're going to see Congress pass legislation that this requires this fairly soon. I think we're going to be surprised by how quickly this does come. But for now, if you don't do it and something bad happens, it's very hard to argue against the lawsuits and the pressure from the admin to do more extreme things. So there you have it.
- [30:01]
A
Also worth mentioning that Anthropic and OpenAI have already been working with the US government to test unreleased AI models. This is part of the reason that, you know, the Fable 5 thing was kind of weird, like it's been tested, so the government missed something and basically all the other leaders in the space have agreed to do these voluntary things. So I think also worth pointing out that to this administration there is an element of like, we want power and we are going to be able to tell you what to do because Anthropic messed with us and now you have to comply. So you gotta be a little bit skeptical of the reasoning of whether it's because they actually want to do control of advanced AI models in a principled way or not.
- [30:53]
B
At the risk of sounding naive here, I actually am optimistic about. So, you know, we know that Susie Wiles and Scott Besant seem to be on the ball here. I mean, Jay Powell was as well earlier, but now he's gone. There's like a, a locus of power pretty close to the White House and in its orbit that's quite influential and concerned about this stuff Sachs has been rooted around. So that's a positive sign at least if, you know, if you, if you think that this kind of thing needs to be done. It's also worth saying like, again, call me naive, but I do like to try to question my political priors when I'm in this sort of context. You know, Biden famously had like all of the big EV makers at the table at the White House except for Elon, like, what the fuck? That makes no sense. That's not a defensible play. Trump has done the same thing where he has all the big frontier labs except for Dario. That makes no sense. So there's a sense in which like this is actually A pretty recurring thing. And, and, and for people like me who argue for regulation and I think quite strong regulation at this point of the space, there's an important sobering aspect here where it's like, okay, but remember your guys, whoever your guys are, red or blue, they're not going to be in the next cycle. And so whatever powers you think about giving the executive, like, be very thoughtful about what that looks like. I think everyone's going to end up agreeing ultimately as the risks obviously start to escalate where it's just like, jesus, we just have a weapon of mass destruction here. That's what it is. Frankly, I think we're there unambiguously, which would be an interesting topic for a podcast on its own. But yeah, so, you know, I think there's a, definitely a political dimension in the. There's like a three month window where the politics of the administration matters and then everyone gets mugged by reality when it's just like, you know what, these are super weapons now. And like there's no sane policy that can be defended to the American people. That doesn't look like we're going to regulate the crap out of this.
- [32:42]
A
And now just a couple of quick stories that are a little more relaxed. Just some features. Anthropic is launching Claude tag and always on AI teammate that lives in Slack and you can tag it in Slack to get insights and assign tasks. Doesn't seem like a huge deal except for this possibly being the next evolution of openclaw type thing. The next evolution of like always on ambient AI agent. You can have running and doing stuff for you without like having to sit with it and like babysit it. This is at least the discussion on Twitter. Andre Karpathi kind of positioned it as a big deal as the next evolution of how we interface with agents. And you know, a decent case to be made that this kind of ambient agent, ambient background agent thing is a big deal. Openclock came out and it's, it's kind of still early. It was like still developing from early 2026. But it does point to probably being the future in the same way that in 2024 we had agents. 2025 we had agents. But like at some point there was an inflection point where cloud code took off and now it runs the world. And there's a decent chance that ambient agents are going to get a similar trajectory.
- [34:05]
B
This is similar to what already has existed. So like you'd be forgiven for being like, wait, what is actually new here? Because the. So. So the Previous integrations that they had in Slack would allow you to DM Claude and like Claude and you know, tag it for, for help on the fly and Claude code also in Slack, like you could already route coding tasks from channel mentions to like full coding sessions. So there was already that crossover. What's new here is context and memory basically. So Claude follows a channel and it'll learn more about the context of like all the work that's going on, the people who are interacting, the moving pieces. And it can also pull in facts from elsewhere if you give it permission to read other channels, for example. So it's really about like how much of the organization it can track. And then given that it takes relatively little additional context to get it to build the right thing or start working on promising directions. And so it's also something that Anthropic has been using apparently internally. I think Jason Clinton had a thread about this. So he's Anthropic's deputy CISO Chief Information Security Officer and he posted on Twitter, yeah, like we've been using this basically internally for a while. This has been one of the big drivers of progress and this is just like us making that available. So that's at least the narrative that's associated with this right now.
- [35:19]
A
And now onto applications and business. First story, OpenAI reveals its first AI processor that they name Jalapeno. So this is something developed in partnership with, with Broadcom. We've mentioned it a while ago and now they have at least some nice photos of it being like a nice looking, impressive chip. This is of course going to be their specialized chip for LLM inference, more in line with Google's TPU than Nvidia GPUs. And if they can develop their own chips, it could position them pretty strongly in the same way that Google is positioned pretty strongly where they own to depend on Nvidia to be able to have the leading AI compute.
- [36:05]
B
Yeah, the initial OpenAI post didn't give us much information about what the chip looked like or like the trade offs or whatever. But there have been posts since that kind of get into more detail and then OpenAI's shared some things. The TPU analogy is less surprising when you remember that they're like OpenAI's hardware lead is this guy Richard Ho, who is a former Google TPU engineer.
- [36:25]
A
Right.
- [36:25]
B
And I think we may have even flagged that poaching back in the day when it happened as, as being a harbinger of like, okay, there's going to be a TPU like approach here. One of the, we won't go into the details, but the systolic array architecture associated with the tpu, which is basically just like a way of moving data between the cells on the chip in, in lockstep in this very kind of well coordinated way. That's a feature here as well of their, their new chip. They're also going with a really large compute die and 8hbm stacks like on the same package. And so as a reminder, the compute die, you can check out our hardware episode from like, I don't know, a year and a half ago or something to get context on this. But there's two parts of thinking. Basically there's doing the actual number crunching in math, which is what the logic die does, the logic chip, and then there's memory, there's like remembering the numbers that you have to do the math on. And the memory is handled typically in these stacks of what are called high bandwidth memory. We'll actually be talking about a story associated with that, that market, the high bandwidth memory market pretty soon. But high bandwidth memory is like really, really good. Like the name sounds high bandwidth. So you can pull down data from these, these stacks of memory really quickly, like in big chunks, all at once, and then pull them into the compute die to do some calculations. So what you typically do is you'll package, at least on a gpu, you might have a couple of stacks of HBM right next to the logic die. You want them really physically close to each other so that the data can transit between them really efficiently. And so what we know here is that the logic die is really, really big. It's a reticle sized chip. So, so, so the re, the reticle is the aperture that the beam goes through. Basically it's the size of like one full kind of EUV beam of light that goes onto the chip. It's the full reticle that they're using for these chips. So you end up with like, anyway, pretty, pretty large kind of compute dies. And the eight HBM stacks, that's a lot of HBM stacks means that since they're packaging it like right next to the memory, they don't have to go through system memory, they can go straight into the logic die. So this is all going to be done on the TSMC 3 nanometer process. We also know that, which is interesting, that's a, a leading node. It's the same node as Apple's latest silicon and the Nvidia Blackwell. So that means OpenAI is going to have to directly compete with Nvidia and with Apple for allocation from tsmc, that is not easy. So they're, you know, one common strategy is you see kind of creative chip designs that deliberately try to go after older nodes and do more with less, that sort of thing, just so they don't have to compete as much. So here they're not doing that, going straight for 3nm. Not surprising because you really do need those advanced process nodes. But yeah, so it's going to be a big challenge. It's also the case we're hearing that Microsoft's own Maya inference chip is also going to be on the 3 nanometer node. So like OpenAI's biggest backer is also now a competitor for them in just getting to those nodes. There's a lot of challenge here that OpenAI is facing. The big claim here is it was the fastest they say, ever getting from the start of the project, all the way to tape out basically to having the whole design done up. And they credit internally the use of OpenAI's own models to achieve that feat. They're also saying that they're going to have a pretty rapid generational turnover in their chips. We've seen Nvidia go to that. We've seen basically everybody forced meta as well. Everybody forced into these kind of like yearly or sub, yearly release cadences. Just because like you can't only redesign your chips every, every two years or whatever, models just change too fast. The key thing here is it really is a chip designed for LLMs, right? Like this. This is a custom ASIC. You should not think of it as a GPU, which is more general purpose thing. This is really truly a bet on designs like LLMs looking architecturally roughly the same from at least year to year. So that's the hardware lottery. We talked a lot about that on the podcast. But as companies start to invest more and more in the current LLM architecture, it's going to take more and more to convince them to try other architectures. Even if they have good scaling properties, even if they look more data efficient, all these things may be wonderful. But if you come up with a new version of Mamba and it just doesn't optimize for these chips as much, well, you've got tens of billions of dollars of capex that have already flowed to optimizing this particular chip stack. And so making the argument that you're going to go with a new model architecture gets a lot harder. So this is a bet by OpenAI that things aren't going to shift too much in that Respect. It's also going to add more pressure on the other direction to kind of keep models looking roughly the same.
- [40:58]
A
And on a related story, next up, we have Amazon in talks to sell custom AI chips in bit to undergrad Nvidia. So Amazon does have its own custom titanium AI chips and now they might be selling directly to external data center operators. Which is interesting because Amazon has aws. They are a compute seller. So directly selling chips that you can just have in AWS and they do have in AWS I think is an interesting shift in kind of business strategy. And Trainium I think wasn't super competitive with Nvidia, but now that Anthropic utilizes them, I wouldn't be surprised if Trainium is sort of getting better due to the deep partnership that Anthropic and Amazon have had for a couple years now.
- [41:52]
B
Yeah, and when we covered the story, I remember, God, maybe two years ago or more when Anthropic accepted an investment from Amazon that had terms that said they must use the Trainium chips that Amazon was developing immediately. We said at the time, hey, you know, they have to do this because Amazon does not have a good model development team and you need to be in touch, very close touch with model developers to optimize your chips for their models. And that's the only way to do it. This was like this is that strategy bearing fruit. They're now nominally, they're ready for the big time. And the key thing to realize here is like, if you roll back the clock like 12 months, the only players in the space that were selling really high end chips was Nvidia. You really didn't have a competitor. I mean you could say amd, you could say, you know, you could say whatever else, but really it was Nvidia. Now not only are we seeing Amazon do this, we're also seeing Google do this. Right. TPUs now are being rented out to people. In this case, by the way, it's, they're renting out not individual chips, but rather racks of chips. So the play here is more in the direction of a little closer to operators in the stack. Like we're going to give you a more packaged product and they are selling them to operators, people who, you know are actually going to run the data center and then serve it up to, you know, a hyperscaler or something like that. Anyway, this is Amazon jumping into the same game that Google has. So now Nvidia is going to be feeling a lot of, a lot of heat and worth flagging. If you look at Amazon's custom silicon business. So this is Trainium plus they've got the graviton and nitro chips together though. Amazon's custom silicon business crossed $20 billion of annual revenue run rate in the first quarter of this year. So $20 billion is a pretty damn good like driver of profit. If you look at the like AWS operating income, it's kind of like you know, the $14 billion ratio. This is a really, really big contributor and, and also for Amazon on the whole like think of Amazon basically as like a more or less not super profitable business except for aws. Right. AWS famously is the profit engine of Amazon and this is starting to look like a profit engine for aws. So this could actually be a pretty interesting play. You might see Amazon become relevant at the level of chips in a way they just haven't been for AI. That is because they'll be serving it up to you know, Neo clouds and hyperscalers in Frontier Labs now much, much more liberally.
- [44:17]
A
And next up onto a different kind of hardware story. Not custom AI chips but memory. We've got a couple stories on that. And first up we got Micron invests in Anthropic and grants it a supply de deal. So Micron is a memory kind of focused company with HPD, DRAM and SSDs. We've mentioned in prior episodes how memory access to memory has been a huge deal in the last year and change and like the entire market for memory has been messed up by AI now where like gaming PCs and consoles are in trouble and Apple just raised the prices of their stuff. So yeah there's a shortage of compute. And so it's interesting that Microd is investing in Anthropic and now has a supply agreement to give Anthropic access to the HBM, DRAM and SSDs. Which by the way I'm a little bit confused about given like I thought these are just built into GPUs. So is this for Anthropic's custom chips or do they need it in addition to GPUs? I'm not sure.
- [45:23]
B
Yeah, well I mean so the HBM is going to get typically co packaged with the logic die. Right. So that's, you're right that ships together DRAM and SSDs. So you'll see stuff used for like NVME like non volatile memory, long term storage and storage racks and also local. So you'll have racks that are dedicated to just holding model checkpoints. Right. Like as all the individual GPU racks are computing the next model weights periodically. So first of all they'll save some of those locally on their tray and that'll go into an NVME on that tray and then every once in a while it'll get sent off as a big training checkpoint to be held in this very stable like SSD memory on the, on the kind of dedicated rack for that. And SSD and DRAM will get used across the board in different configurations for different things. So like different components basically on the server rack, all of which are required for, for, you know, proper functioning of things. And yeah, the nature of the supply agreement's interesting. So when you look at. So first of all, we know that HBM is just nuked like you can't get HBM supply all the way through the end of 2026. Micron has said, in fact, that their entire calendar year, 2026 is just gone. For HBM supply, it's all spoken for. That includes HBM4, which is their latest one, which is like just, I mean, you can't break in. And so this is Anthropic basically securing the bag, making sure that they can get their hands on the memory they need. And so that's part of it. The part of the agreement is literally just, hey, Anthropic can get their hands on this. There's also, you know, a whole bunch of sort of side deals here where you have CLAUDE adoption at Micron, which seems like a smaller deal, but it's part of this. So Micron's going to deploy CLAUDE models internally to speed up coding and help, you know, deploy agents and stuff and then a strategic investment separately.
- [47:07]
A
Right.
- [47:07]
B
So we don't know how much. We know that they're participating in the 65 billion series H round or had participated in it, which closed on May 28th. That was the big, you know, 965 billion post money valuation round that Anthropic raised at. But again, we don't know the nature and extent of it. So yeah, I mean there like These, you know, DRAM and NAND SSDs are basically all just part of different components. Usually a server rack or server tray. Looks pretty complicated when you kind of break it up. Semianalysis has a really good series of pictures and sort of discussion of this for the NVL72 rack, if you ever want to check that out. It's. It's actually a pretty good one for folks who are listening.
- [47:48]
A
And one more story on the memory side, SK Hynix overtakes Samsung to become South Korea's most Valuable company. So this I assume is because it is a supplier of high bandwidth memory chips used in AI systems. Customers such as me and Google. Samsung by the way, also a supplier of chips. And I think memory, in addition to that, they do everything. So this just goes to show that the HBM market in particular has just gone crazy. And SK Hynix has 61% of global HBM market, far ahead of Samsung and Micron. So yeah, I regret not investing in memory stocks last year.
- [48:34]
B
Yeah. Oh, again, like Leopold Aschenbrenner I think knocked this one out of the park if you look back at some of the choices he made on the memory market side. Yeah, so this is an interesting one, right? It used to be the case that like it really didn't matter where you bought your memory from like skhonic Samsung, Micron. I mean Micron was kind of shit for a while, but now suddenly they matter, but they're more or less fungible. And hbm, which we were just talking about, right, these sort of funny stacks of dies that are all. And you can think of them as like, so these are stacks, but you have these TSVs, these through Silicon vias, basically these threads that go vertically through these stacks that allow data to kind of like think of them as like an escalator, an elevator that goes, you know, up and down through the stacks. That's what lets you pull data in a smooth way from those stacks and out into the logic D or wherever you need it. And anyway, it turns out that when you get into the business of high bandwidth with memory, right, you're suddenly going to care a lot about how tightly it sits against a specific AI accelerator. The architecture of the TSVs are just a much more advanced fabrication process than other historical ones for like DRAM or whatever. And so what actually happened here was suddenly you had much higher barriers to entry to become an HBM supplier and you had real pricing power because of that. And so the story in a way here is not so much just like SK Hynix overtook Samsung, which in Korea is a big deal because Samsung culturally is a real touchstone there. It's something they're very proud of. So seeing that ranking get flipped or suddenly you have a new entrant in town, that's a really big deal for them. But what you have fundamentally here is what used to be a kind of commodity business just shifted into like kind of a monopoly, right, where you have a real moat. It's a lot harder to break into the HBM market. And now the market's Repricing that and realizing, damn, I mean, SK Hynix is a lot better positioned in the long run. This is also an interesting story because of like when and how SK Hynix placed this bet. They were really on the back foot in 2023. There was a big downturn and they had a massive operating loss. Things weren't looking good even during that time. They poured money into HBM to get ready for this very generation of chips. So this is kind of a story of basically spending that would have been considered pretty damn reckless, pretty risky at the time. And yet the bet paid off. We've seen a similar story play out with Micron, actually. They skipped an entire generation HBM3 and went to 3E, basically leapfrogging everybody just so they'd be prepared for that moment. And it paid off. The flip side is Samsung did the exact opposite back in the day. Not in memory, but in logic, when they used to be a relevant competitor to tsmc. The way they lost that crown and was precisely by taking a leap too soon, if I remember correctly, it was like adoption of like DUV or something. But anyway, they kind of mistimed the step. So this can cut both ways, but when it works out, it can really work out. And now SK Hynix is reaping the benefits. So it is a big deal, especially in Korea.
- [51:42]
A
And back to some chips. Story on fundraising. AI chip maker Grok confirms $650 million raise and has restaff after Nvidia's 20 billion not ACU hired deal. So this is kind of surprising. Maybe. I'm not sure if you would say it's surprising, but Grok, as per the title there, had a deal with Nvidia that gave them a license for the core technology of Grok and got a lot of their talent. So there was a big question of like, well, what's the future of grok? Is it like kind of a zombie company now is going to stick around? And it seems that Doug Whiteman, the co founder who stayed after the Nvidia deal, is now a CEO and presumably is trying to get back into a business. Although GROK is now focusing on its Neo cloud business, operating 13 data centers across multiple locations. I guess being in Neo Cloud probably isn't the worst business to be in, given there's demand for data centers.
- [52:46]
B
Yeah, no, yeah, just ask Ludic. Yeah, no, Neo Cloud. The Neo Cloud business is really good, but the challenge here is so, I mean, you hit the nail on the head. They've been gutted, right? They're like founding team and crucially their IP is over with Nvidia. So Nvidia can now like just shamelessly sell essentially Grok chips, I mean using all the core IP that they developed over the years. And so now that you got this kind of like hollowed out shell of a GROK company that is trying to remake itself and the fact they were able to raise $650 million is a good sign. It just, it's going to be stressed now that thesis is going to be put under a lot of pressure. The pivot to NEO cloud is actually potentially more challenging than it looks like for those very reasons. Their differentiation, like Groq's differentiation was basically we're going to make custom chips that run inference faster and cheaper than What? Than Nvidia GPUs. Well now Nvidia owns that and they can sell that silicon. So what's the difference? Like this is a real challenge. And so yeah, you know, you can pivot into being a cloud provider and NEO cloud, but if your core advantage is something that you non exclusively share with your largest, best capitalized competitor, that's a pretty big structural challenge. And so we'll see if they can, if they can pull it off. You know, being a NEO cloud is about much more than just chip design. And so in that sense it makes sense. Nvidia is not a NEO cloud. Right? They sell their chips but they don't sell fully fleshed out, built out data center cloud time to companies. And so yeah, you know, if you want to escape sort of the immediate thumb of Nvidia, that may be a good play. Grox also made a lot of interesting kind of recent hires. They've got a CEO who is former like XAI and Meta, a couple of like cloud enterprise software, cto, CPO type people, including a guy with some experience at Microsoft cloud. So those guys are not chip architects.
- [54:38]
A
Right.
- [54:38]
B
This is like go to market cloud people. And that signals GROK is actually going to be like, okay, we're not going to try to be a frontier AI company. We are really doing this for real. So you know, they may have a good shot here. It's just that again the cloud business is very different from the chips business business. That's a huge pivot and it takes, you know, $650 million a good start but it takes a lot of capital to pull that off. We'll see. But you know, I think it'd be tough to kind of have them catch up after they've been winded by this, this sort of Momentum loss to the likes of, you know, your, your fluid stacks that have such a big lead and, and know know what they are and have known what they are for for some time.
- [55:18]
A
And speaking of neoclouds, next up, SpaceX slash XAI has inked another compute deal, this time of ref open source AI lab. So they have signed a 6.3 billion compute deal with between the two of them. This is after they have signed a deal with Space with Anthropic and I think also Google if I remember correctly, with giant numbers of billions of dollars. So yeah, Xai continues to appear to be in the data center business and not the frontier model business.
- [55:52]
B
Yeah, play to your strengths, right? I mean this is the right play for a company that's just spent $60 billion on a big acquisition, you know, to, to get cursor on board trying to restart their actual model development side. This is, you know, if you don't have the best model developer pedigree, you have to find a way to be in touch with companies that do and to be in that feedback loop. Just like Amazon, right? Amazon wanted it for chip design, SpaceX wants it for kind of general neo cloud work. But you have to work with the best because only the best know where the trajectory of the technology is headed. It's a way to kind of get some proprietary information from Anthropic and Google, you know, understand what's being done under the hood. This particular story, by the way, worth putting in context. It's almost an order of magnitude smaller than the Anthropic and Google deals, right? So Anthropic was I think like 1.2, 1.3 billion per month. Google is like 900 million per month. So both are around a billion per month. This one is 150 million per month. Right. So a lot smaller, but it is going to be GB300AI chips. So. So you know, Colossus 2, all that stuff. And it is also one of those deals where either company can just end the contract with just 90 days notice after the first three months. So pretty similar to what we've seen with those other contracts. We'll see if it's sticky. I mean I would expect it to be. It's aligned with what we know Elon can do really well, which is move atoms around. Right. It kind of remains to be seen. It remains to be tested on the bit side. So yep, story yet to be told.
- [57:16]
A
Ever wonder who's out there making the world go round? It's truckers. Who unites baristas with coffee Beans Truckers who unites dogs with their favorite chew toy. Truckers. That's why Progressive offers truckers even more protection with cargo plus coverage to keep truckers moving right along. Quote Truck insurance today in as little as 8 minutes@progressivecommercial.com Progressive Casualty Insurance Company and affiliates. Coverage subject to policy terms, limits and conditions not yet available in California, New York and Virginia. This episode is brought to you by Progressive Insurance. Do you ever think about switching insurance companies to see if you could save some cash? Progressive makes it easy to see if you could save when you bundle your home and auto policies. Try it@progressive.com Progressive Casualty Insurance Company and affiliates. Potential savings will vary. Not available in all states. And now onto projects in open source. We begin with GLM 5.2, which is the latest in a series of new models that are just killing it. We have Kimi, what was it, 2.2 previously we just covered a week or two ago and also Kimi 2.7 code. So as with those models, some of the major details is what they highlight here is this being built for Long Horizon tasks. So they have this for the first time delivers the capability on a solid 1 million token context, which Vesal there is saying a lot like you can have a 1 million token context but not actually be able to do stuff at that length. And you need that large context to be an effective coding agent. And that's what the vibe check on these models have been. You look at the benchmarks and on the benchmarks it looks like it's actually competitive with Opus4.8, in some cases better than 4.7. But also on the vibe check of what people are saying, many people I've seen say like GLM 5.2 is good enough. It's good and I'm using it and it's capable and it's very cheap and it is open source, purely open source with an MIT license. So this I think is the latest example after also Kimi code and some other cases of like wow, this open source stuff is a very big deal. It was already a big deal before where it was like catching up, but now it's at a point where you could replace cloud code with GLM 5.2 as your driver model. And if it's way cheaper, some people may do that.
- [59:59]
B
Yeah. And there's the usual algorithmic improvements that we're learning about is this is an open source model so we're seeing more of it. There's a couple things here. So yeah, you're right on the benchmarks Frontier suite Which is sort of like these longer tasks, you're thinking like hours to tens of hours, those kinds of tasks. It's behind Opus 4.8 by 1% and it beats GPT 5.5 by 1%. So it's like actually pretty damn close. I mean, this is pretty wild. Post Train Bench is another really interesting one. So this is where the model has an H100 GPU and it's scored on how much it can improve a small model via post training using that gpu. So this is not super far from the sort of AI research tasks that you look at for, for example, meter evals, that sort of thing. And there GLM 5.2 just falls behind Opus 4.8. It's second only to that. Now that does not include Mythos five, you know, all the five, six, all that stuff. Right? That's, that's what was available at the time. And then there's Swim Marathon, which is ultra long tasks as well. But this is stuff like, you know, building compilers, like heavy duty stuff. And there they see more of a mixed picture. So you have 13 points behind Opus 4:8. And so it's still the best non opus model, but it is like well behind. So it's kind of interesting that you see that variability, but definitely very, very strong model. I mean this is pretty wild. A couple of things that they're doing under the hood that make this work really well. So there's this thing that Deepseek famously did called Deep Seq, sparse attention. And this is the idea that, you know, the attention mechanism that looks at when you're trying to decide like what token should I generate next? You look at like all the tokens in the context window and you try to attend to them, you try to pay attention to them and see what they have to teach you that you can learn from to determine what that next token should be or to put whatever token you're looking at in context, in proper context. Now that's really a heavy lift. You're attending to literally every other token, looking at like, hey, token number one, like how relevant are you to me? Okay, token number two, all the way down for like a million tokens of context. That would be huge. And so what Deepseek Sparse Attention did was it said, well, you know what, we don't actually have to do that. We can just have a really cheap, quick and dirty model scan through and really quickly rate every token to see how likely it is to be relevant and only pick the top K tokens, right? The top, say 128 tokens or whatever. And then we're only going to attend to those, radically decreasing the number of tokens we have to attend to and the number of flops that we have to crunch through. And so they're going to do that here, but they're actually going to take that a step further. And now when you have a consecutive chunk of layers, say four layers, one after the other, they're actually going to keep the same selection of top K tokens from layer one all the way through layer four. Instead of recomputing at every layer, the top K tokens, they're literally just going to say, okay, you know, four layers back, it was tokens 5, 6 and 12. So it's going to be, we're only going to look at 5, 6 and 12 here. We're going to assume that the most relevant tokens haven't shifted all that much layer to layer. And so this is another way of significantly reducing the amount of compute that you need to do inference across the whole stack. There's also faster decoding strategies. So they use a small draft model, like basically a really efficient cheat model, to guess several tokens ahead. And the main model is actually just going to verify what that smaller model produced as a prediction. And so the bulk of the heavy lifting here is actually just verification rather than generation. That's really important because when you look at the architecture of these models for verification, you can actually simultaneously verify like, basically a whole chunk of tokens, like N tokens, all at the same time in one pass, in one forward pass. And whereas generation requires you to go one token at a time. So if you get lucky, let's say the small model correctly predicts, you know, six tokens in a row. You verify it. You're like, oh, yeah, great, those six tokens are right. You can verify that in one shot, one forward pass. And now you've gotten six tokens of value or six tokens of prediction. And so in this way, you're kind of like grouping things together. You know, those are following at home. You'll be interested to note that that approach, you can think of it as like an alternative to training or training with large batches or doing inference with large batches. The batches also amortize the amount of compute that you're doing on the way out.
- [64:21]
A
Or is this any different from speculative decoding? Or is this just the same thing?
- [64:27]
B
Yeah, well, so speculative decoding is a way to basically. So if you think about what your model's doing, it's simultaneously checking the values of multiple output tokens at the same time. So in that sense it's like you're paralyzing your outputs and if you do a batch right. Of training data, you're parallelizing your outputs in a similar way. So these are basically like parallel or like alternative approaches that you could go with.
- [64:55]
A
And one more thing to say on GLM 5.2, very soon after release we got some kind of additional releases both from Nvidia and base 10. They released a faster version of GLM 5.2. So base 10, provider of APIs is achieving over 280 tokens per second and that is by doing their in house NVFP for quantization from the FP8 weights. That's just like you store less info and it goes faster. Nvidia has Open sourced an NVFP4 version of GLM 5.2. So this is the end of the thing of like okay, it's a really good model. It's like competitive with Anthropic and it's fast and it's cheap. Like at what point is Anthropic going to be start worrying about this competition?
- [65:45]
B
Yeah. And OpenAI and everybody else, especially if there's a compute cap or some kind of limit on their ability to go further. We've been arguing about this for a while but like what is the implication? You know, progress in the west is like fixed at a certain threshold. Progress in China will obviously continue. At some point it's going to exceed. Okay, sounds like we need a treaty because there is literally no other way to hold our our threshold constant and not get completely lapped by China. Then you get into okay, how do we do a treaty that doesn't assume that China is like a super friendly party that's going to do everything right, which they won't. So anyway, it's so easy to get into other stories this week apparently.
- [66:22]
A
But yeah, that's what we do. So we'll try to be quick with the next one with another open source. This one is a benchmark. We've got Econ Evals, an open source evaluation suite that measures language model capabilities across U.S. work categories using the Department of Labor's O NET taxonomy that maps over 1,000 occupations to 18,800 tasks with over 2,000 detailed work activities. So basically it gives you an idea to a solid extent of like how much of economy is advanced AI going to impact. And right now kind of pretty intuitively the benchmark is saying computer and math, you're screwed. That's going to be like going after everything, business and financial management, you're screwed. Also you're going to be writing all your emails of AI already. But then for a bunch of the economy, construction, production, food prep, healthcare, there's very, very little threat from AI right now. And this is the sort of thing where we are getting to a point where we should be worrying about AI's impact on the economy and how it's going to look. So far it hasn't been radical, but it's starting to look like it might be this year or next year or whatever or. And it might be already.
- [67:45]
B
Yeah. And a lopsided arrangement like this makes it really hard to compensate with kind of universal basic income or whatever because you still have some people who are making killing in their regular jobs. So it's, you know, it's not like you have the same in the same way that inflation is like it's not just one number, right. There's inflation in the price of strawberries and that's different from inflation, the price of screwdrivers. And you know, this is the same problem except that like, well, some people get utterly screwed and some people don't. The, the amusing thing is that of course it's the people who 15 years ago everyone assumed would be the most screwed who are the safest. And that comes across in like a lot of these kind of spider web diagrams, you know, they show. But yeah, so I'd say like not much of a surprise in terms of the things that are safe and the things that are at risk. Anything that touches the physical world and is also complicated in that sort of human tradecrafty way, that's pretty safe. Right. So if you think about carpentry, right. Construction and extraction, they have food prep and serving, I think you mentioned healthcare, you know, whereas the things that are more white collar are mega screwed. And yeah, that comes across pretty clearly. So yeah, I mean it's an interesting report. We definitely need more, more benchmarks like this. And probably Anthropic is going to have a couple too with their, their new
- [68:58]
A
institute that's coming online soon onto policy and safety. Quite a relevant story given that 500 million AI jobs push launches with bipartisan backing. So this is a new initiative called Raise Us. It's meant to prepare American workers for AI driven labor market changes. And it's backed by a bunch of organizations, Amazon, Microsoft, bank of America, OpenAI and Anthropic. This was co founded by former Commerce Secretary Gina Raimondo, a Democrat and former Indiana Governor Eric Holcomb, a Republican. And so they already have more than $500 million with a long term target of $1 billion in multi year commitments. This is all earmarked for AI related workforce development spread over a few years. So I mean, this is a pretty clear showcase that we are already kind of baking in the assumption that there will be major labor force disruption. As if you're reasonable, you have to assume and this is showing that I guess we are starting to tackle that problem. Seriously.
- [70:12]
B
Yeah, a billion dollars and half a million dollars and all this stuff is basically a drop in the bucket given the scale of problem. Of course, the timelines here are comically long to have any reasonable effect if you're concerned about workforce automation. On the order of the next two years. A lot of these programs are even like, you know, one year apprenticeships. Okay. Now is it helpful for me to complain about that? No. Am I happy that this is being done? Yes, I am really happy this is being done. You want a portfolio approach in case it takes things longer to automate and you know, you'll regret not having done stuff like this if it does. So I think this is awesome. I also think it's probably irrelevant in the scheme of things, but again, more signal that this is a thing that people are starting to take more seriously. Gina Raimundo, of course, was the Biden administration Secretary of Commerce. And so it's a bit of a cross partisan thing, which is nice to see as well. Like everyone seems to be in agreement that this is worth doing and I certainly think it is. Yeah, there you go. I don't expect it to make a difference, but glad it's happening.
- [71:09]
A
And another thing, that's $500 million that may or may not have an impact. Republican Sam Licardo has unveiled an AI workforce tax credit bill. So this is the supporting knowledge for industry led learning, AKA Skill Act. Nice acronym. It's going to be incentivizing private sector investment in American workers with programs for education. So you would be able to sort of fund students in qualified programs to presumably be educated to become electricians and other things that AI cannot do right now, you know, community colleges and so on. And you know, building an electrician is a good career move right now if you're considering different programs like I think
- [71:56]
B
a data center, be an electrician or data center build up. Yeah.
- [72:01]
A
So yeah, a kind of similar story in a way of trying to get the private sector to chip in some cash to deal with the inevitable private kind of disruptions to the labor market and the pain that will cause. Next up, we've got Google DeepMind has announced an AI control roadmap for improving AI agent security. So this is related to internal guardrails to catch potential adversarial behavior in AI agents. We've seen on Propit discuss this a lot. All the guardrails they've built that they believe are sort of the best you can do at this here. They're saying that they can use specific methods like chain of thought monitoring, asynchronous alerts, real time access control and shutdown infrastructure. So quite a detailed roadmap I think and good to see DeepMind kind of taking this seriously.
- [72:59]
B
Yeah, so this is all kind of stuff that I guess people in the space have been talking about for a while. It is nice to see them take it seriously. It's interesting that they are specifically now talking about recursive self improvement as the motivator for this. So if you're a canary in a coal mine, you've probably like left the coal mine about four or five times now from different labs making it. This is like on their roadmap for the near term. But so they, they specifically think of this idea that well, we're probably not going to solve the alignment problem. We're probably not going to be able to get these models to behave in the way that we want with the preferences that we want at the time that we are doing full on recursive self improvement which Eliezer Yudkowski is probably pulling his hair out right now. But this is where they say okay, so the thing that we need is AI control, not AI alignment. It's not about making sure that we can design incentives into these machines such that they want the things we want them to want. It's rather that we want to be able to jump in, intercede, prevent them from doing the bad thing that they will eventually want to do. That's the kind of quiet conceit here. The models will be skeevy, the models will be power seeking. We just need to stop them on the way. Which is sort of like the last thing that you would go for, but it may work. So yeah, this is about monitoring models, access controls, so preventing them from getting access to things they shouldn't, good sandboxing, things like that. Importantly, and this is something that I'm a broken record, every time there's a paper like this I always complain in some way, shape or form about how okay this looks good for right now, but like for super intelligent models this is just going to fail. I mean like by definition this is a control scheme that's being developed by fallible human beings. To the extent that AI gets better than humans at like finding exploits, hint mythos, you're just going to fail and it's going to happen probably pretty fast. And so they're making that explicit. They're saying, you know, control is not a permanent solution against a really super intelligent AI. We expect it to fail. Awesome. Love to see it. Well, don't love to see that fact, but it's great that they're being transparent about it. They draw this really, I think important and kind of common in the ecosystem analogy with what's called like insider threats. Right. So insider threats are typically people in companies that are recruited by adversaries or just individually decide to do bad things, steal IP or do damage. And they're saying like, hey, AI models are potential insider threats, recruitable assets, basically. Then they go through reasons why it may be easier or harder to prevent that from leading to bad outcomes. All the things you might expect, right? It'll be easier to prevent bad outcomes because defenders actually can access chains of thought. We can look at internal activations. You can't do that with a human. You can even reset an agent's memory, replay incidents, things like that. Whereas on the harder side, you know, these are black boxes to some extent. There could be thousands or millions of them running in parallel. They're superhuman speed. You know, there are tons of copies that could collude all these things that won't be, won't be news to anybody listening. They do, there's this like famous cyber kind of model cybersecurity framework by this ffrdc, this like federally funded research and development corporation called mitre. So MITRE designed this thing called the ATTCK framework. And the details don't matter, it's just like here's how you typically execute an end to end cyber attack. And what DeepMind's done here is they've created an AI version of it that they call the taxonomy of rogue AI tactics and routines, which abbreviates to trigger, which is nice. So there you go, they look at a bunch of different threat models and you know, like loss of control work, sabotage, where the model does just like deliberately bad alignment or safety research and inserts backdoors into safety code or like poisons the next model's training to pass on its goals, that sort of thing. And then there's actual direct harm. So like trying to destroy critical assets, exfiltrate model weights and do stuff like that. And then they go into defense. What do they do to actually mitigate these risks. They have detection, they have prevention and response strategies that they lay out and then they have anyway a whole bunch of frameworks worth checking out. They have concrete examples, 15 of them like mitigations that they're looking at, including things like shutdown capability, just restricting the AI's capability to begin with, giving it less access to stuff, and also strict supply chain security, just preventing the stack that they're running from being silently disabled, which is a loss of control nightmare scenario. So anyway, it's a great paper if you want to like a 101 on how, how the labs are starting to land the plane on loss of control. You can choose how comforted to be by this. But it's nice to see Google DeepMind being open about their thought process here.
- [77:21]
A
And kind of a related story. Next up we've got the loss of control playbook, Degrees, dynamics and preparedness. So this is from Apollo research that we've discussed a decent amount that has done quite a bit of various kinds of things related to safety and security. So here they defined AI loss of control. They have a taxonomy of different types of loss of control. They have kind of proposals on how you might deal with loss of control. They released an AI R&D threat map and kind of like a chart of loss of control specifically as well. I think, you know, the gist is they have a very nice guide for understanding and taking seriously loss of control.
- [78:13]
B
Yeah, and I think their kind of first figure is a really great starting point. They look at a bunch of different scenarios that have been proposed for loss of control. So you can, you can think of things like anywhere from a relatively minor cybersecurity incident to triggering large scale wars to engineered pandemics and things like that. And each of them, what they do is they try to map them to an economic impact. So how many dollars worth of damage could this do? Obviously lives lost equals dollars is a tough equivalence to make. But hey, guess what, if you like look at your local like municipal planning committee or whatever, they actually often do, for example have to put a price on human lives to determine when is it worth it to put like a new traffic light at an intersection or something, right? Like this kind of calculation that gets done all the time. And so they're, they're kind of doing something similar here and you have nice little uncertainty bars across the orders of magnitude that a given incident class might span and sort of like anyway using that to build a bit of a taxonomy. And so, so it's I think quite useful. We're Lucky that we've done so much pre computing and pre thinking about these loss of control scenarios. I think it might become unfortunately quite relevant in the near term. Yeah, there you have it. Epoch, you know, obviously known for their AI scheming evals that they've been developing and stuff. And so it's kind of interesting to see them move forward.
- [79:31]
A
This is Apollo. So is that the same as Epoch?
- [79:35]
B
Sorry, did I say Epoch? I meant Apollo.
- [79:37]
A
Yeah, yeah. And to be clear, by the way, the actual taxonomy is from last year from 2025, but they have released this now AI R&D threat map, which is possibly, I guess more actionable in terms of like mapping out the specific paths that might lead to dangerous stuff. Different from having a taxonomy and definition of loss of control. Now back to politics. Why corporate AI super PACs spent 27 million on a local election so in the New York 12th congressional district primary there was over 27 million in tech industry spending centered on a feud between AI safety and anti regulation AI faction. So state Assemblyman Alex Bors, who co sponsored the first AI safety law in the country, apparently became the focal point of spending battles despite not saying, you know, he's all about AI safety. There was for instance, leading the future $100 million Pro AI super PAC spent over 8 million in packs running anti boars ads. And for those who might be confused outside of the US as of like a while ago, you can have companies doing unlimited amount of contributions to candidates in elections, which I don't think is the case elsewhere, but, but anyway, packs are just like the way by which companies and which people can give unlimited amounts of money to campaigns. And that's what's happening here.
- [81:13]
B
Yeah, it's kind of approximately shaken out to be a two horse race between the OpenAI pack, which we're not supposed to say is the OpenAI pack, but it's the OpenAI pack. It's the pack that Greg Brockman, I think his wife maybe also donated significant amounts of money to. But we're assured that that has nothing to do with OpenAI's position. Even though Greg Brockman is the co founder and president of OpenAI. And so that that pack has been leading the future, it's been a big one and pushing out a lot of like pretty grimy tactics, including like armies of bots that just like comment on things on Twitter and like kind of and I forget the details, there's a bunch of stories we've covered about this and then on the flip side there's like pro anthropic pack that's like throwing money at candidates. And turns out that, yeah, it didn't, didn't turn out well for Boris in this situation. A couple of take homes though, one of which is men, safety candidates, or at least those who are perceived to be aligned with AI. Safety sure can raise a lot of money, though. And so maybe that actually, you know, as much as anything the story had previously been. Look at how much money is pointed against Alex Borres. But maybe this will encourage other candidates because it's like, hey, I think he may have ended up raising pro Alex PAC may have ended up outspending the OpenAI one. So, you know, for what it's worth, anthropic seemingly putting their money where their mouth is. It's also funny, like some of the ads against him. So he was a former Palantir guy and he quit over. I'm trying to remember what it was. Do you remember the thing he quit over exactly?
- [82:38]
A
No.
- [82:39]
B
Okay. I think it might have been the ICE thing, but don't quote me on that.
- [82:42]
A
Some of the stuff Palantir does, which is kind of conservative leaning, you might say, or generally. Yeah, people take on Palantir for various things.
- [82:52]
B
Right. Well, but I think. So his departure was on the base of the ICE thing. And then the attack ads against him were, hey, Alex, Boris, like worked at Palantir and this is the same company that did the ICE stuff. And that's the reason that he left. Not to mention, I'm trying to remember who the Palantir executive is who's. I believe there is a Palantir executive who is quite influential at leading the future, in fact, who may have been the one to make this comment. Don't quote me on any of this shit. But it's. It's something like that. So anyway, there's a lot of, like this sort of funny and it's classic in politics. Everybody does this. Right? Of course, like we all, we all twist stuff. But it is full of ironies here. And I think the outcome is kind of. Is kind of mixed. This guy came out of nowhere, did pretty well, raised a lot of money, and now is a bit of a hero for this. Cause it's not obvious that this is a bad move for his career on the whole. And so I think everybody just gets to declare victory and move on here, which is a weird outcome. You don't normally see that.
- [83:47]
A
And one last story on the politics front. We've got conservatives plan nationwide protest against AI data centers. This is a conservative group called Humans first in the U.S. they kind of are related to a Tea Party movement from the early 2010s in the US which is, what's it, neo right or pretty far right ish movement that for a while had a lot of sway in the conservative areas. And so they are planning rallies for at least 13 locations across Georgia, California, Texas, Florida and Virginia. It's a bit of an interesting shift, I guess, in that typically conservatives are like pro business and believe that businesses
- [84:33]
B
should do what they want.
- [84:34]
A
Right. So here they're saying that that's not the case, that there's an unchecked expansion of data centers. And of course, some of the reasons have to do with water use, which is a persistent myth about AI data centers. Just so you know, water use, not a big deal. With data centers. If you care about environmental concerns, it's polluting the air. If you're using gas turbines like Xai is, and potentially energy consumption, because as you start to use a lot of energy, you need to produce more energy. And you might do that via dirty methods.
- [85:13]
B
Yeah, you're right. Like, it is the case that the whole water argument is silly. The specific reason that it's silly or a specific reason that it's silly is just like a lot of these things run closed loop cooling configurations. So there's like, apart from, roughly speaking, the water that you put in the pipes to get things started, you're just not losing that much water over time. And yeah, it's non zero, but it's like, it's a silly thing to be worried about compared to basically every other issue you could have and be concerned about. So the interesting thing, by the way, it's strange that things are so political on the podcast. And I think that's part of the thing is it's so important for us to kind of keep that distance. So like, my cards are, I think I've put them on the table in the past. But like, I am more kind of a libertarian leaning fella. I, you know, I'm, I'm a Canadian. Right. But I am way more libertarian and you might say conservative, but I don't think that that's the right word.
- [86:01]
A
Let's say free market. Right?
- [86:04]
B
Free market. Yeah. Yeah. And I'm like, you might have called me a free market, not absolutist, but a big free market guy. Even now I would say, but certainly five years ago or six years ago. But AI is different. And I think we have to have the maturity to look at technologies like bioweapons and say, let's not have A free market approach to freaking bioweapons. And I think AI, I mean, literally is in some ways like it's a bioweapon generating machine. So there you go. I mentioned that because the conservative movement is this very kind of fractured thing right now. Right. And you're really starting to see it like you've got this, this question along so many different axes. You got the Israel thing, right? You got, you got like the Megyn Kelly, Tucker Carlson like, oh, we really don't like Israel thing. And then you've at the same time got your more like Ben Shapiro type people who are like, so, so along that axis. You're seeing it along the data center axis too. You're seeing it along the how seriously to take AI loss of control axis as well, like all of these things. And so it's not that there's no fractioning like that happening on the Democratic side, but the reflex to, to turn to regulation is obviously much more developed on the, on the Democrat side, which is why you just don't tend to see the same kind of arguments. And so if anything, on the Democrat side, obviously it's more like, well, concern about loss of control and catastrophic risk is crowding out the conversation about ethics and bias, which is the real kind of present thing. Right. That's kind of died down as it's become more obvious. That's a bit of a silly objection given what Mythos can do. But like, you know, there's more kind of a push towards the regulatory side. So here we're seeing. I agree with you. It's surprising, like, not just that the Tea Party, like the Tea Party, as a reminder, was born out of the ashes of the, the 2009 stimulus. You've got rising national debt, all the big bank bailouts. And the Tea Party says, hey, the government shouldn't be involved in this shit. Like, get, get out of our pockets. Like, we don't even want you. Keeping companies alive too big to fail is no good.
- [87:58]
A
That's right. This is kind of anti elite, kind of populist ish movement. So not traditionally conservative in terms of free market.
- [88:07]
B
Exactly. Well, and, and so, and, and you just, you just got free market.
- [88:11]
A
I guess government interfering in free market.
- [88:14]
B
Yeah, yeah, yeah. No, you, you just, you just hit the nail on it. You got to like the thing that I was trying to build up to way more efficiently than I did. Yeah, you should not. No one here should listen to me because, like, I don't know what the hell I'm talking about on politics. There are Better podcasts for that. But. But there is a sense in which the is thing. Thing about the Tea Party is more the populism than the ideology. And that's increasingly true, obviously, across the board. But one of the consequences of that is, like, once you start going after people's livelihoods with automation or people's homes, you know, by putting data centers next to them now, suddenly the Tea Party switches from being this, like, very, like, get off my lawn thing to, like, hey, government, can you come in and, like, please do this thing? I'm not saying that's good or bad. It's just like an interesting. An interesting pivot. No one cares about this. But my personal take on the data center thing is, like, you probably do want to build the infrastructure, otherwise you're just straight up going to lose to China. Until we have some coherence at a treaty level with China, we need to maintain our negotiating leverage and not kneecap ourselves first. Because if you stop building data centers today, then China doesn't need to come to an agreement with you. They're going to outrun you in very short order. So this is. I say this as a guy who's deeply concerned about loss of control and has a very high p. Doom. This is a debate that you could have a lot of people do. A lot of people disagree with me who know a lot of stuff. There you have it. It's. It's a fascinating moment for politics in America because so much is starting to revolve around AI. Yeah.
- [89:35]
A
And partially because, like, everyone's hating on data centers. It's not like a lefty or righty thing, apparently. I thought it was mostly a lefty thing. And it's true that on the right, there's generally more support and less kind of anti Big AI sentiment. But what this is showing is there is a broad anti AI sentiment that is kind of free from traditional political lines. In this case, they're saying, okay, it pollutes our hometown. It, like, is noisy, there's secrecy, and Washington is just, like, in the pocket of big AI, which everything is true.
- [90:13]
B
Yeah, everything is true. Right. It's true that that's true. It's also true that, like, a very large fraction of the protests that you're seeing against these data centers. Part of the investigation I'm doing right now involves this. Are, like, funded by China. And, like, you see the same signs and the same people, professional protesters popping up at all these things. That is all true. It's also true the data centers are building things that may be catastrophically dangerous. It's also true that China exists, like, as an adversary and we need like, like all these things are true. You're right. Whatever emotion you feel about this, you're correct. Like, I'm here to validate that.
- [90:48]
A
Unless you feel that way because of water use, in which case you're not correct.
- [90:54]
B
That's the one unforgivable sin. Yeah.
- [90:58]
A
And now on to research and advancements. We're gonna go through these stories pretty quick because we have a lot to cover. First paper is revisiting the Platonic representation hypothesis and Aristotelian view. So here's the quick version. We covered the Platonic representation hypothesis a while ago, which was kind of a popular ish view, that regardless of your training recipe, if you train multiple LLMs or big AI models, there is this notion of convergence of representation. Like, regardless of the data, if you go big, eventually your embeddings, your internal representations start to converge and you have this ideal representation, which is the Platonic representation. And this paper challenges that. It says that that's really a statistics artifact, that it has to do with embedding dimensions growing relative to sample size. So the similarity metrics just kind of like show this by coincidence. And once you control for it, there is no trend towards similar representations. Although there is this like, local neighborhood based similarity that then they say there's an Aristotelian representation hypothesis.
- [92:17]
B
Yes, yes, yes.
- [92:17]
A
Neural networks do converge on shared local neighborhood relationships among instances and not globally matching representational structures, which makes sense to me. It's like, okay, yeah, like the relationships between concepts are consistent. So you would hope that the relationships exhibited in representations are consistent on smart
- [92:41]
B
AI models and do they become more consistent at greater levels of abstraction over time? I think is going to be the most interesting part of this because if it is the case that we are working our way towards a Platonic ideal. Man, this is a highfalutin podcast. If it's true that Aristotelian metaphysics wins through anyway, then basically what you should expect is over time you get more and more convergence, you know, at higher and higher. So. So the local, what local means starts to grow and grow and grow. And I would expect that to happen, actually, I wouldn't be surprised if it did. But also there may be many local optima for representing these things. And so you may actually have like clumps of models that end up having similar representational approaches. And, you know, you'll never necessarily get to the global optimum that where they all converge on something that looks really similar So I wouldn't actually rule out, funnily enough, the Platonic approach, even if empirically we can't get there with models. It may just be that there is this, and there probably is this one optimal way of representing. I'd be surprised if there wasn't a technically slightly optimal kind of true global optimum. It's just a question of whether you're actually going to find it in practice.
- [93:52]
A
And they do converge to some extent still. So once you control for similarity, there is. It reduces. It's not like as strong as previously seeming, but yeah, it's still there to some extent.
- [94:04]
B
Will they all glue together at the highest level of abstraction at that one global optimum like that? Yeah, we don't know. It's a practical question and it's. It's fun to noodle on.
- [94:13]
A
By the way, why Aristotelian? Well, the Platonic beta is because Plato had this notion of Platonic versions of stuff with which is like the one true like way to like there's an ideal chair that is like what all chairs kind of are. And apparently I didn't know this. Aristotle, who was a student of Plato, established the study of categories. And it's pretty amusing to me looking at the paper and seeing that they Cite Aristotle in C330. I guess like Aristotle, can you imagine
- [94:47]
B
if it was around like the Jurgen Schmidt hoovering that he would be able to pull off? Like this is actually all just derived from, isn't it?
- [94:57]
A
The principles of relatives is what it's called. Next paper. More of a demonstration of something cool. One streamer V01 end to end real time Interactive foundation models. So again, short version interactive foundation model is meaning that you can talk to a video avatar in real time. We've seen examples of this from freaking machines, for instance, where you try to do a sort of real conversational thing with very, very low delays. The interesting thing about this is they say it's kind of a first version where it's a true foundation model that directly, without any sort of like intermediate parts, takes in language, audio, video, all as input and then you produce video as output, which if true means that this could be like actually a significant version of foundation models that is not of the sort of thing that LLMs are and could have a significant impact.
- [95:58]
B
Yeah, and so they deliberately kind of. We interleave all the different kinds of tokens. They can take the multimodal tokens, visual, audio, text, both incoming and echoing, they put them into a single sequence and then they use block, causal attention, incrementally as, as they stream in. So this is not unlike actually thinking machines. You know, good on you to mention them. They had to redesign the whole stack here, including their encoders, their decoders, the token schedulers. Like all this stuff to work with these really really short chunks like 160 milliseconds and 25 FPS. So it's like really, really fast streaming. This is what you have to do, right? It's, it's a software engineering problem at the same time as an AI problem once you get down to, you know, ultra low latencies like this. So maybe this is going to ultimately erode some of thinking machines moat. I mean it's not going to be a perfect moat because they haven't been around for that long. But it'll be a useful thing to track and ultimately to see like what kind of performance gains do you see as between thinking machines and tools like this as they get ultimately released in the open source too?
- [97:01]
A
Yeah, if it gets released to open source, which in this case I don't think think that's the case. This is from Alibaba.
- [97:07]
B
Oh, I just mean ultimately, like ultimately eventually.
- [97:10]
A
But these kind of things are tricky to train in general partially because the data isn't just text. Like you need real time whatever. So it's, it's kind of annoying. Next we've got tapered language models and this one is kind of simple to explain. So basically right, the default version of Transformers is pretty straightforward. You have like some kind of layer which has a couple things like self attention and some fully connected networks and maybe some other stuff these days. And then you stack a bunch of these one on top of another and they are identical and they have the same number of parameters. This is basically saying, well as you stack them one on top of another, maybe they don't need the same number of parameters. And you can taper in the sense of like the number of parameters decreases over time as you go up the stack and well it turns out that maybe that's how you should do things. Which I mean sounds like there's a good reason to believe it formal like a representation perspective. And you can probably save a lot of compute by doing this. I actually think this might be a decently big deal and I'm surprised it's not already what people do. But maybe there's a good reason for keeping the sizes consistent.
- [98:31]
B
Yeah, and it's funny, some of the best papers had this property where you look at them you're like man, that's obvious. But it's like, yeah, but I didn't think of that.
- [98:39]
A
What a simple idea. And it's like so elegant. It's like, oh, let's decrease the size.
- [98:45]
B
Well, it's kind of reminiscent of some of the stuff that you might have seen with convolutional networks back in the day, right? If you actually looked at the, not necessarily the parameter accounts, but like the amount of image that actually gets processed, let's say there's some kind of analogies that you could reach for there. And in this case it's, you know, quite clearly in those like lower layers where you're doing some of the, the most concrete, less least abstract thinking, the data hasn't yet been turned into a meaningful representation. And so it's kind of like when you first walk into, like encounter a new problem, you kind of have to take down notes about every possible aspect of the problem because you don't know what's important yet. So you're just like looking around, trying to like, have a really wide aperture. And then over time, as you chew on the data more, you start to understand what's less relevant, what's more relevant. And actually ultimately you can distill things down to a couple of very simple ideas. And so you think about like somebody who's an expert in a field which roughly you could think about later layers being more like that. How many notes are they going to take during a presentation on a given topic in their domain versus someone who's totally new to it and desperately just trying to like, collect as much as they can? This is an imperfect analogy for a lot of reasons, but roughly speaking, that's kind of like a way to maybe vibe some of this at you. So, yeah, intuitive and also just really, really useful from a scaling standpoint. You know, maybe new scaling laws, a different scaling law for every layer. We'll see.
- [100:16]
A
And last story real quick in synthetic media and art titled Hollywood is bending the knee to open. And it's for me as a real big movie guy, kind of in a way a major deal. So Luca Guadagnino has directed this film about OpenAI CEO Sam Altman. It's a biographical drama titled Artificial, also starred a bunch of people from the AI world, including Elon Musk. Supposedly didn't have a very favorable portrayal of Sam Altman, which, you know, not too surprising given our own portrayal of Sam Altman on this podcast. So it's been dropped by Amazon MGM as its retributor despite being nearly finished with post production and Netflix824 Focus Features and Warner Bros. Have all reportedly passed on picking up this film, and that's kind of a big deal because Luca Guadagnino is big name. He makes good movies. So all these, like, especially A24 and Focus Features, which are a little bit more artsy, the story is very clearly like, we don't want to make OpenAI and AI people in general unhappy. So, no, thanks.
- [101:27]
B
Wow. So how can. I don't know how long we've been doing this podcast, and I didn't know that you were a big movie guy. That's. That's good to know. Yeah. Is it? Well, okay. So this, in a way, reminds me of. And again, I know I'm both sizing this a lot, but I think that's just because, like, this issue is so complicated. One really bad thing that Hollywood has done historically is bend the knee. They. They bend the knee a lot, actually. Like. Yeah, a lot, a lot. Including, you know, I would say we were like Red dawn or some of these movies where, like, the adversary is actually Chinese. And then they're like, no, you can't do that, because very often it's usually China. That's just usually how the economics works out. If you want to circulate this movie in China, then you're not going to make China a villain.
- [102:05]
A
It's not a right versus left issue. It's not like the current administration does this. It's more of like, whoever is. Like, Amazon is a big business and this is also not political. Right. Like, Apple is happy to lobby both sides. Right. But Amazon in particular has very obviously been friendly with administration. They, like, released Melania biography and basically, like, did a kind of bribe by licensing it for $40 million. Yeah. But anyway, she's an interesting person.
- [102:41]
B
They happen to just, you know, it's just. You make a documentary. Yeah, there's a lot of that. Right. Like, it's. It's the currency of Every administration is different.
- [102:48]
A
And this is also, by the way, is. Is like, maybe not administration, because this is not about politicians. This is about AI. But anyway, it's very clearly, like, we want to be friendly with OpenAI and Big AI and Sam Altman and Elon Musk. So you're not gonna be part of this.
- [103:08]
B
Show me. Show me the incentives and I'll show you the outcome. Yeah.
- [103:11]
A
And I'm very sad because I was very hyped for this movie, so I'm hoping it'll get picked up by Neon or Mubi or someone else.
- [103:18]
B
Well, if they just change the depictions of Sam, I'm sure We'll get a
- [103:22]
A
good problem with that. Yeah, like, show. And with that, we are done with this episode of last week in AI. Thank you so much for listening and commenting and sharing and subscribing. We do try to check out your comments on YouTube, including a suggestion to comment on discussions of AI elsewhere that we may or may not try to do.
- [103:44]
B
I feel like we should do so. Yeah, whoever left that comment, thank you. Because, like, Andre and I were texting before the episode just about that specifically. I like that idea. I like the idea because there's a lot of. I don't know, I was looking at, like, I don't know if you know this Andre, but Dr. Mike Isratel, he's like, you know, done the podcast circuit a million times and actually like done a lot of training with people in the AI community. And he's like, watched the show in the past. There's like a workout video he posted on Instagram and he like, he says that thing you hear in the background is the last week in AI podcast and you like, hear Andre's voice there. So all the different things. So Mike has. Has a bunch of stuff on Twitter. He's very opinionated on this stuff and very knowledgeable too. But there are a lot of other folks like that where you see them clack off some opinion and if you want us to, yeah, pass commentary on that sort of thing, very happy to do it. I mean, I think it's a good way to keep us regularized and grounded and tethered to the conversations that are happening outside the kind of SF Berkeley, DC universe. So, hey, DC Universe. That kind of worked.
- [104:46]
A
There you go. So, yeah, as usual, we would like to do more types of episodes if we ever find the time and we appreciate you listening and please do keep tuning in as we try to consistently release these once per week.
- [105:16]
B
Tune in, tune in when the AI news begins begin it's time to break, break it down Last week in AI Come and take a ride get the low down on tech and let it slide as we we can AI come
- [105:38]
A
and take a ride Couple lads to the streets AI's reaching high new tech
- [105:44]
B
emerging Watching surgeon fly from the labs to the streets AI's reaching high algorithm
- [105:51]
A
shaping up the future sees Tune in,
- [105:55]
B
tune and get the latest with ease Last weekend AI come and take a ride Hit the low down on tech and let it slide. From neural nets to robot the headlines pop data driven dreams they just don't
- [106:31]
A
stop Every breakthrough, every code unwritten on the edge of change with excitement we're smitten from machine learning marvels to coding kings Futures unfolding see what it brings.