Loading summary
Transcript198 lines
- [00:00]
A
Hey everybody. Welcome back to this Week in AI. My name is Alex and we are coming to you during one of the most fascinating and chaotic periods of AI that we've seen yet. Anthropic stable model is still banned. The world is going gaga over Zai's GLM 5.2, a food delivery company just open sourced a near frontier AI model and everyone is trying to use as much AI as they can without losing their hope. To help us make sense of the state of play, I've gathered some of the brightest minds from the world of AI.
- [00:27]
B
Thanks to our friends at PayPal, the exclusive sponsor for for this Week in AI try the payment and growth platform that's trusted by millions of customers worldwide. PayPal Open start growing today@paypalopen.com in one
- [00:40]
A
corner we have Victor Perez, the CEO of Korea AI. Victor, welcome to the show.
- [00:44]
C
Thanks for having me.
- [00:45]
A
My absolute pleasure. We also have Div Garg, the CEO of AGI Inc. A very modestly named company.
- [00:51]
D
Div, thanks. Great to be here.
- [00:53]
A
And then we have Andrew Berman, the CEO of Run Layer. Andrew, how you been?
- [00:57]
B
I've been good, thanks Alex.
- [00:59]
A
Yeah, I'm really glad you guys are here because I feel like we went through a period of slow news and then everything seemed to have gone crazy in the last couple of weeks. Glad we're going to kind of get to the bottom of it. The thing that I want to start with, because I think we can't avoid discussing this as a group, is that after forcing the removal of anthropics, mythos 5 and fable 5 for a bit, the US government forced OpenAI to hold back GPT 5.6's release, which I thought was very disappointing. And it's even deciding which customers can get access, which means maybe startups can't. So I'm curious, Andrew, I want to start with you. Is the US AI policy reasonable and legible at this point or am I incorrect in viewing it as insane?
- [01:35]
B
I think you're not incorrect in viewing it as insane. I think basically what we're doing is we're giving China and other actors the ability to encroach on you, on U.S. technological supremacy. I think open source models are being distilled from our world class models and I think we should be pushing ahead as fast as possible and we should be the dominant player in AI globally.
- [01:59]
A
Does anyone have a disagreement with that before I get more specific, But I want to make sure Victor and Dev can jump in if they want. Yeah.
- [02:05]
D
Also like I agree with Sangal, there's There's definitely risk around, like obviously cybersecurity, which maybe prompted like the initial fable or Mythos, like suspension, but I think it's kind of like overblown to some extent. It's like, it's like these things will happen and we can't really like stop this. AI models to combat and matter because China is in the race and they will figure this out if you don't.
- [02:26]
A
I'm just surprised at how quickly we went from it's all free range to absolutely nothing. And there's a lot of talk about how, you know, maybe Mythos was overblown or overhyped, but it seems to be still treated as this essentially cyber weapon. So maybe this is just our new reality. But Victor Kria has done some training of its own. Models now. Not in the cybersecurity space, not even in the coding space and the creative space. I'm just curious, do you think that government regulation of AI in general is going to eventually get down to your domain as well? Pretty far away from now, but directionally, is that where we're going?
- [02:58]
C
I think that the imaging creative domain itself, it has its own specific risks. I don't think that the risks that it comes to are as heavy as cybersecurity threats, which as far as I know is the main point behind this blockage from these new models. In our case, the most dangerous things that folks are doing with the models are NSFW type images, CZAM type images, which of course are illegal. And this is like the main topic that we fight against when running safely on top of these models. And on the other side there's impersonating people, like recreation of public figures, stuff like that, which I don't think that the US government cares as much as potentially having China running massive cyber attacks versus US companies.
- [03:54]
A
It's funny though, how we were talking about IP as a major sticking point in AI and that, you know, being a thing to regulate or not. And now I feel like it's entirely fallen by the wayside. So does that mean that people working on building models in the creative space have, I don't know, a couple of years of air cover here?
- [04:07]
C
Potentially. Potentially. I wouldn't be surprised if it comes. I wonder if it would come at the model layer or at the inference layer, because I think that these are two of the things that I think quite a lot like some regulations seems to be, or I would say previously in the image models or video models they're used to, people try to enforce regulations on the data that you use for training. But I haven't seen as many regulations on the measures that you run at inference time and how even though the model is capable of producing something that is IP protected, what methods do you put on top of that model in order to either alert the final user that this image that the model has produced may be IP protected and to be careful with how they use it versus just trying to make the model completely unable to produce these assets which unavoidably nerfs the capabilities of the model.
- [05:04]
A
Oh, enormously. If you have to write, you can't go over here, you can't go over there. Pretty soon you have one third the surface area. Now div your company, which I should just explain for folks out there who don't know, which is doing essentially computer use agents for mobile handsets, taking what we know from the PC world desktop world and putting it on your mobile device. You guys have built a large action model?
- [05:25]
D
Yeah, yeah, we actually call it like a tiny action model at this point because it's running on the device.
- [05:30]
A
Okay, I apologize. You're running a tam, not an lam. I sincerely apologize. We'll edit that. Talk to me about. Well, first of all, define that for everyone out there who's curious about the difference between an LIM and LLM. And then also, do you think that there's any potential regulatory pressure coming down the pike on what you're working on?
- [05:48]
D
Yeah, I'll start with the first question. So the biggest difference between an action model and a large language model is. Language model is for chat. It's like, what do you do with ChatGPT? It can hold conversations with you. An action model is made to do tasks for you. It can go and operate my whole computer. It can go and automate all my work. It can also maybe do things for me, act as my ea, order me an Uber to my meetings, get me lunch, order me coffee. And it makes it very powerful. And the technology we're building is how can we run all these things on your device? So we have hired some world class researchers to compress these models to run on npus on the phones. And our vision is like, we feel like if your phone itself becomes AI native, it's agentic, I can just talk to it and it can go and use any app. So the phone itself becomes the AI and we want to live in this world.
- [06:44]
A
So then on the regulatory point, just kind of keeping us on this topic because I just want to get this done and out of the way. Is there any sort of regulatory pressure brewing about what you're building. Or is it again a bit like what we see with Kreia, outside of the current focus and therefore outside of the blast radius of hyperactive government action?
- [07:03]
D
Yeah, it's outside of the focus. At least right now. Until we maybe have some sort of cybersecurity incident, we have been focused on privacy and safety. That's why a lot of this is on your device. And our goal is that we want to free up people's time. It's like, okay, if people have more free time, you can do more things. And purchases is definitely one thing we need to be careful about. We don't want someone to go like an air to go and start trading on your Robinhood or buy you some
- [07:28]
A
crypto or mine crypto on your computer when you don't want to, etc. All right, Andrew, just coming back to you on, on the future point of this. You work with a lot of enterprises, helping people kind of get AI to work for their company. Which means, I presume you have a foot in, you know, major model companies and talking to a lot of major enterprises. Is there real regulatory concern out there or is this just what we're talking about on X? Because none of us want to get real work done during the day.
- [07:53]
B
I, I think from the hyperscale perspective there is and the administration perspective, there's definitely real reg. Concern. We're part of a bunch of the trusted access programs because at the end of the day, while we enable AI, we also do security and control. So we, we focus on giving companies the golden path. So how do I enable employees to manage forms of agents and how do I have visibility, security and control in the background? Is there real concern? Yeah. Do enterprises want to use GLM 5.2 for cost reasons? Yeah. Yes. Do they want to serve it from places like bedrock and base 10 and modal and all the other fireworks together?
- [08:35]
A
AI, et cetera.
- [08:36]
B
Exactly. Thank you. Yes, but that's a cost reason. And so I think at the end of the day, what we're seeing from customers, they want open source more than I was expecting earlier this year. Mostly because their inference and token bills have increased and we have some interesting regulatory things, things going on in the background. At the same time, people want Fable because Fable was incredible for that one couple days and we could ship so much. So it's a double edged sword.
- [09:07]
A
Did you guys actually get Fable into any production environments before it was stolen from our hands?
- [09:12]
B
We were using it for internal development, not production environments.
- [09:15]
A
Okay, and what did you see from it?
- [09:17]
B
It was Incredible.
- [09:18]
A
How much better was it than say Opus 48 or GLM 52 pointed at
- [09:23]
B
a product, somebody pointed it at the product roadmap discussion that we were having, took a picture of the product roadmap and then two hours later had a fully shipped product. Maybe we want to call it a feature that probably would have taken like four days with Opus. So yes, it was a step function. Oh, now what I'd love to get access to is I love to get access to 5, 6 to see where that's going. And we're part of trusted access. But we'll see what happens because now the US government still has to approve it.
- [09:51]
A
Can you explain to people out there what trusted access is, how the program works and also the benefits you get from it? Because I think that's a bit niche. But I think it does matter for founders out there who are listening, who want access to 5, 6 and are looking at you as someone already in the promised land.
- [10:06]
B
So I mean look, I'm in the, I'm in these programs. I don't necessarily, I can't necessarily say I'm in the promised land. So if you look at from anthropics perspective, there's Project Glasswig, it's called like 100 large companies. These are the type of, these are Fortune 500, Fortune 150. Anthropic is helping these companies look for vulnerabilities with Fable and other models before those models come into production. OpenAI has a similar version of this where they're focused on five, six. And to get access to some of these models you have to be part, you typically have to be a cybersecurity company or in that infrastructure layer. And then on top of it you have to also work with the US government. And we're very happy to work with all the stakeholders out there so that we can enable our customers to have the best access and the best enablement with security and control and so that we can bring them an agent control plane.
- [11:02]
A
Yeah. So you. So essentially now the conversation is so multi party. It goes from Washington all the way to San Francisco, spans the entire dentation, all the industries and private and public partnerships.
- [11:11]
B
I don't know if it goes to San Francisco. My guess is those policy people are in Washington. But I get your point.
- [11:17]
A
I was thinking technology on one side, D.C. on the other side.
- [11:20]
B
I think there's lawyers and lobbyists and regulators and all sorts of people who live in dc.
- [11:25]
A
That's what I was kind of curious about because I think about your companies. Each one of Them is doing something really cool. Seems to be going very well. Raising money, hiring people, launching cool stuff. Do you think that you have a moral responsibility to be a voice for kind of the AI startup world in Washington? That may not be currently heard given that it seems just to be Andreessen Horowitz, Dario and Sam. I don't know, Div, are you going to stand up and be like, this is what we're building, this is where we're going. Do not get in our way?
- [11:52]
D
Well, in an ideal world, yes. There's obviously we don't want to. It's a double edged sword. Again, if you hit too much attention for yourself in the government, they might just think you're risky.
- [12:05]
A
Maybe it's better to play possum and just pretend that you're dead. Okay, fair enough. I don't know.
- [12:10]
D
I just want a function of size. Once we are big enough that they notice us and they're like, oh, we think you are doing something that might be may be dangerous to the US economy or something, and we're like, okay, sure, we'll figure this out and we'll take the right model path. But unless we are too big to notice, I think we just want to move fast and not be bloated down by regulations.
- [12:32]
A
No, that makes sense to me. Yeah. Okay. Well one way people are getting around model restrictions and so forth is by turning to open source models. We talked about GLM 5.2, which people are raving about, but Victor, you guys have actually put out not only your own foundation model, but also several different open source variants of it. This is Korea K2, Korea K2 RAW and Korea K2 Turbo. Maybe you should just break down the differences there and then I have a couple of questions for you about that.
- [12:57]
C
There was a missing component in most of the open source models for image generation that have been released pretty much since 2023 when stable diffusion 1.5 was released. And the problem with these open source models is that they were overly post trained. When you post train a model, you give the model an opinion and you make it to forget how to make a wide range of styles or a wide range of things that are considered wrong as you run reinforcement learning techniques or other kind of post training techniques. The problem here, the problem with only having post trained models is that they are very hard to tune. And the coolest part of having access to an open source model is that you are able to tune it and you are able to make it work really good for a specific niche that it's valuable for your business or that it's valuable for the use cases that you have for these. That on the one side and on the other side it's really hard to innovate on post training techniques without having access to a RAW model, because as soon as the model has been post trained, anything that you can do farther than it's just very hard to actually be able to do research. This is something that we internally found as we started to become more serious on training AI models, that some of the techniques that they didn't seem to work as well on the papers actually work really well when you apply them with maybe a little bit more data, but especially when you apply them on top of a RAW model. So on the one side there's the tunability, which is one of the core components of open source, and on the other side is hopefully having having universities getting access to a ROM model also helps develop further post training techniques or gives them an opportunity to innovate in this field which is becoming so important in the space.
- [14:51]
A
All right, so I'm going to phrase this to you like I'm an idiot because that's not too far off the case here. So composer from cursor used Kimike 2.5 and did further post training on it, but it was already slightly post trained. So to your point, they may have been able to do more or better post training if it had been a raw er, model from the start.
- [15:11]
C
I'm not super, super knowledgeable about how Apples to Apples is comparing post training in image models with large language models, but in general I think that from a high level the processes do the same, which is to narrow the probability distribution at which these AI models have access to. So essentially, if you're starting from a post trained model, the places where the model can go are more narrow than if the model is on a RAW state. If it's in a raw state, it may fail more often, but it has access to a much wider set of possibilities on the output that it can give you. Those possibilities are what makes post training to work. Because at the end of the day, what you're doing with post training is, is to make the model to not go to the wrong directions, if that makes sense. But you need the range of directions in order to successfully do post training. So not sure if this compares like Apples to apples with LLMs, but I think that from a high level idea it's pretty, pretty similar.
- [16:16]
A
Okay, so Dev, apply what we're talking about here to start small action models as they relate to your company because I'm curious about the trading conversation and where you guys started, if from scratch or from something else and then also how you managed to improve in tune as you've gone along.
- [16:29]
D
Yeah, initially we were starting with a bunch of open source models like Llama and I think back in the day Deepseek and then we did a lot of force training. We were like, okay, let's go and collect all these interactions on mobile devices where we do a lot of parallel. We actually tell people if you have an Android phone, literally let us record your screen and we'll pay you for that. And that's been pretty successful as a way to get a lot of millions of data points. And now we realize at some point you just have to train your own models. We're starting to train our own things from scratch. We have a collaboration with Qualcomm. The interesting thing, not the nice thing, is when you're working on phone handsets you can't use Cuda because this is not powered by Nvidia chips. You have to really go low level and write your own kernels in C and there's a lot of hidden libraries that you don't even know exist from Qualcomm Dev.
- [17:23]
A
We've actually banned mentioning C on this show, so please don't bring it up again. I have scar tissue from lost semicolons and painful large coding books, but keep going.
- [17:33]
D
So it's just like a very gritty work but once you've done it, it's very interesting. And now once this small action models are actually running on your device, then I think then we can do a lot of post training. So it's similar to what Cursor is doing. Once this is in hands of millions of users, it's very easy to build these data loops where it gets better and better over time. And that's where we think the fun part is.
- [17:55]
A
Isn't it tricky though to get people to use your model before it has been fully baked? Because then I present the experience on device for the person I know it's Android to start and also iPhone now you just announced that, I think it was last week. Is it difficult to get people to show up and be like please finish baking our cake so we can sell it to other people even though it's not fully done yet for you?
- [18:13]
D
Yeah, there's definitely like an adoption curve I think like if you're a major user this is like this usually curve you see like in production, like product adoption. So if you're laggard in the space, you probably will not like using us right now. But if you're like an early adopter and you're like, yeah, I really like working with new track products and try it out. It's like it's like so cool and it'll probably work like 95% of the times on most things it's probably like 5%. We are still like figuring out which you're tuning over the next couple of months. But this is kind of like a thing where like we keep improving and make it better and better.
- [18:41]
A
Andrew has run layer already built support for what AGI Inc. Is doing so companies can roll out this Android computer use agent to their devices. Or is this not yet in the spec?
- [18:51]
B
Well, it's not yet in the spec. We don't focus as much as on device computer on mobile computer use as Div does. I think our customers are very interested in how do I enable any tool, any resource, mostly on desktop today. But I'm excited to see where Div takes this and how he could potentially work together in the future.
- [19:13]
D
Yeah, so I fully believe like in three years. So like people will just be using your phones for everything. You won't even own like a PC or a desktop anymore. And I actually know a lot of my friends who don't have this, especially if you're non technical. It's like there's no reason for me
- [19:26]
A
to own a laptop except for access to the desktop web, video games, better zoom call. I don't know, am I becoming old? Because I feel like kids these days should really stop using iPads and phones and really just get back to real computers with discrete keyboards and gaming mice. I mean it's the best setup we've ever invented for our species. Dev. Why take it away? Why not have it be an also versus an either?
- [19:52]
D
I think in work settings you'll still likely have desktops, but a lot of this will become running on the cloud. It's like how you already have Cloud has this dispatch feature. It can dispatch things in the cloud. You can do the same thing with OpenCloud. You probably have a Mac mini box somewhere, but you don't really need to have it attached to a monitor. It's running automatically on its own and you just delegate things from your phone.
- [20:14]
A
I didn't really mean to turn Andrew into our what does the enterprise think about that person? But apparently that's your role today. Andrew, are corporations moving away from desktop computers to this type of setup? Because I can kind of see why certain people in the field might be more in favor of this, but to me, every time I walk into an office it's still just rows of people with headphones on and MacBook Pros.
- [20:35]
B
I'm going to go if this is not happening in digital, native, AI native or enterprise environments, but I think I could see this maybe for some prosumer use cases. I love my whisper flow, but I don't know that I'm going to be running on device. It's a trade off. Are you going to run on device or are you going to run in the cloud? If you're going to really run on device and you believe in that world. Look, I still need my M5 Pro and I still need 48 or 64 gig. So it's an interesting question then if I trust the cloud and I want to share all my permissions with Anthropic and OpenAI, that's a different ballgame and so I don't know the answer.
- [21:17]
A
I wonder if we're going to see a real bifurcation in the computing setups for consumers and then workers, because on one hand everyone already has a smartphone. If Div is right and AGA Inc is correct, then maybe our ROS gets kind of dissolved down to an interface. Fine, fair enough. But I just can't imagine doing any work in that environment. So Div, do you think people will work using this type of system or is this just like for consumers to order doordash without having to actually move their thumbs while sitting on the couch?
- [21:41]
D
I feel people will work. Even for me when I'm working it's usually just using cloud for 50% of things or even more right now. So it's like if I'm doing all my things from one place anyways and it's able to send emails for me and figure out all the things in the background using mcps, I don't have a reason to go and operate all these softwares myself. I just need some sort of AI that I can talk to and it's going and orchestrating things. It's still useful to maybe see what's happening on a monitor, but down the road it's like if it's fully automated I don't even have to see. I have all these AI employees and AI workers and they are running the whole thing and I don't really care.
- [22:16]
A
This feels like a very human light future you're describing here of people talking or whispering to AI agents that are doing all the work. It doesn't seem like people are that busy, but let's pause because there's Some breaking news that just dropped. This will be a few days late when this show comes out, but anthropic just announced Sonnet 5. Here are the benchmarks. My dear friends, I believe you're seeing these for the first time live, so I'll just read them out for everyone on the audio version. Sonnet 5 has an agentic coding swa bench pro score at 63% better than sonnet 4.6. Not quite as good as opus 4.8. Appears to be stronger in agentic coding, blah, blah, blah. Really strong knowledge work, which is the GDP VAL AAV2 benchmark that everyone knows and on computer use, nearly as good as Opus 4.8. So, first impressions. Let's go. Victor, what do you think about this?
- [23:03]
C
As far as I know, this is meant to be a more performant version than the opus 1.
- [23:08]
A
Less. Less performant. Faster, Cheaper.
- [23:10]
C
Exactly. So faster and cheaper. I think that this is 100% the new wave that is coming in the space. I think that there has been a first wave of capabilities where we are having these models to become really good at certain tasks. And after the models are good at specific tasks, everything that people start carrying is about performance. Actually, the new OpenAI model, like the Five Points, is like the Terra one. I think that it was already like undercutting Anthropic's model, and we're seeing the same thing on the creative space. It's weird because it's not something that you can generalize to every single task that these models can do, but there's specific tasks where once the model reaches the capabilities to solve the task, you really don't care about what model is solving the task. You should care about the task being solved. And that's kind of what we're seeing on the image space right now since Nano Banana got released, which was the first editing model that it was really, really good at editing. And it can give you very realistic images. It pretty much like it can do like 70, 80% of the tasks that most people want to do with image models. The model can pretty much do it. Nano 102 got released. Nano 100 Pro got released. GPT2 image got released. New models got released. People don't, you know, like, sure, like with GPT2 image, you can get like these crazy 4k infographics with all these crazy detail on the letters. But that's not a real use case that people have for these. I think that it seems like we went above a certain threshold, and above this threshold, people start caring about. Okay, but how much am I spending? How fast is it going? So there's a new kind of performance wave that I feel like it's starting and I see that on the LLMs. I think that this Sonnet 5 seems like I need to read more about it, but it seems to be going in that direction. We're seeing the same thing on the image space.
- [25:06]
A
This is actually a good point. So you're talking about essentially minimum viable intelligence, after which there's no point in pouring more compute into it because you're already getting what you need out of it. Andrew I think this is kind of the argument we're seeing in the enterprise space more generally because it feels like a lot of models are good enough and you don't need to run the absolute point of the spear, edge of the envelope frontier models to get what you need. And I guess the question is how much work can be offloaded to models that are one step behind but 10 times cheaper. So amongst your customers, what's kind of the balance look like between where they go for minimum viable intelligence and where they go for the absolute max they can purchase?
- [25:43]
B
So I think everyone in an ideal world wants maximum intelligence, minimum cost. So. Right. I mean, who would have thought? And so if you think about that trade off maximum intelligence, minimum cost, how do you then, how do you then at runtime select the correct model for the end user to do this task? And I think that's an interesting challenge in itself and people have been trying to work through that for the last couple of years. I think the step function that we've seen is November of Last year, Opus 4.5 and then the 5.5 series of models in terms of general purpose reasoning and mostly tool calling, where we get into mcp and that's where you can really, that's where you can start actually using agents and actually doing real work. And so yes, I think everyone wants this trade off. I think it's an emerging, it's an emerging concept that has probably come out over the last two to three months as the pricing models for things like Claude Claude Code and Codex have really changed. And so we're going to see more of this and I think the labs are realizing it and they're pushing out models that have better frontier intelligence at lower price and faster to serve. Yeah, yeah, like GLM is a wake up call to everyone.
- [27:02]
A
That's a great segue because databricks, Yu Chen jin said that GLM 5.2 is the open source cloud moment essentially or the open source, the GPT 3.5 moment. So you rate that as correct?
- [27:15]
B
Essentially, yeah. I think if you look at the benchmarks and you start playing with evals, it's very close. That's the answer. There's going to be pros and cons in different domains. Your average person probably can't tell. And what do you see as your average person doing? They're probably still taking their emails and dropping them into Claude and trying to proofread them with Opus. And that's just reality.
- [27:39]
A
Anthropic, by the way, loves it when you do that. Their investors are enormous fans of you, burning just all of your money on bs. Which, by the way, is what I did with Fable when it came out. I was just playing with it. I used it for completely useless tasks, but it was incredible at them.
- [27:52]
B
Did you build your show notes with Fable?
- [27:54]
A
No. You know what I used for show notes is these fingers. And then I do type, type, type, type, type. I use a lot of AI as a research assistant and a reminder tool, but when it comes to actually doing the core substance of my job, it's actually not that good because doing the work is how I prepare. So by preparing the notes, for example, that's how I put it into my head. And if I had an AI do it, then I would just sit here going, okay, next up. And then I'd be panicking reading. That wouldn't work. So it doesn't really apply to me, I guess.
- [28:23]
B
Andrew, makes sense.
- [28:25]
A
Yeah. Victor, on the point about minimum viable intelligence in your models, then when you do essentially kria 2.5, whenever that comes out, or kria 3, where are you going to focus more on the performance side or more on the intelligence side? Because I feel like you could go either way and still have a moving the ball forward moment.
- [28:44]
C
I think we definitely want to make it more intelligent. There are certain capabilities that we still need to integrate into the model. Right now it's just text to image. We're working on the editing version of it, which editing unlocks most of the value for these models.
- [28:59]
A
Is that hard to do?
- [29:03]
C
It's not hard from an architecture or training perspective. It's hard from a data perspective. There's a lot of tricks that you need to do in order to get high quality editing data. That has been where we have been dedicating a lot of time. And also you want to test on small models, then make sure that the testings on small models feed with medium and then apply to large. And the large trainings, of course, take some time, so it's time consuming. And expensive process. But I think that the place where I see a lot of value in the same way that I was first talking about, first you get the capabilities, then you search for the performance. I think that there's an objective. How to put it? Objective capabilities and subjective capabilities. And I think that those fit very well with what you said. Alex. Around. Hey, I think that what you said, it was more around. Okay. Actually, writing the notes helps me later on run the podcast. But I think that there's the component of an AI writing the thing in a way that you don't like, in a style that you don't like, putting
- [30:12]
A
things in the wrong order, not giving me the facts that I need. For example, I know because I wrote the notes that you spent about $20 million training K2, according to a Fast Company article that I read before we came on. But if it put that at the bottom of the docket. 20 million.
- [30:26]
C
$20 million on training K2. I don't think it was that much.
- [30:28]
A
But the GPU cluster Korea is using for a year over which time it will have trained K2, and two future Korea models will cost companies. Oh, so 20 million for the year. I apologize.
- [30:36]
C
Yeah, more or less. More or less.
- [30:38]
A
Okay.
- [30:38]
B
Yeah.
- [30:39]
A
But I mean, how can AI know my context when it's just like my own personal preferences?
- [30:43]
C
Exactly. And I think that there is something there. And I think that the same thing happens with programmers we have. Everyone in Korea, of course, is using AI for coding. But you want to check the code. You cannot put something in the code base that you don't understand because that's just a ticking bomb. Maybe at some point we trust enough these models in order. I feel like it's still a process of building trust. November last year, we started trusting these things to write code by themselves. I think that there's different levels of trust inside the organization. But what I see a lot is that, you know, maybe the output that the LLM gives you solves the problem. Like, hey, I want to solve this engineering challenge. The model can solve the challenge, but the way how it solves the challenge doesn't fit with the way that we do engineering at kria. So I think that right now there's a lot of opportunity around building some sort of design systems for these models to not just solve the problem, to not just have the capability to solve the problem, but to solve it in the way that KRIA would do it. So it's not just about giving you the information about all the companies from this podcast. It's doing it in the way that you would do it. And I think that this personalization layer, it's very interesting and most of the stuff that we are doing around the future of our models has a lot to do with that subjectivity, with not just solving the task, but doing it in the way that you like it or doing it in the way that you would do it and that is personalized to your preferences.
- [32:14]
A
In the case of Kriya, does that mean that the images that are created from the text to image prompts match my own personal esthetics, or are you talking more about how the process of getting to that point works?
- [32:26]
C
It has a lot to do with like on the one side, yes. Like it matches your aesthetics. Like the more you use crea, the better it gets at generating images for you. There's so many interactions that we can, that we can detect and extract knowledge from those interactions in order to improve the model for your personal taste. But I think that it becomes way more important when you can even be explicit about it. When we give you the opportunity of Share with us your brand guidelines. Share with us. I don't know more information about how you think about your brand, whether you want to express to the world every time that someone sees your logo or any ad that you want to make with Creo, I think that it's a mix between implicit knowledge that we can get from the platform and explicit knowledge that you can provide us that we can use as context so that we can use inside workflows.
- [33:12]
A
Okay, that's actually really helpful. Dev, I'm going to get to you in just a second about small action models, but I need to go to Andrew just really quick here. It feels like if we took what Victor just said about the creative AI model space and applied it to what you're doing in the enterprise, he's describing the equivalent of like corporate ontology and trying to understand the customer in question about how they do work, where they store things, how they approach that. I'm just curious. For Runlayer and getting a company set up to run agents in the way you guys do, how much fitting do you have to do to the individual company versus what's kind of off the shelf and fits into any enterprise that's trying to get their agentic AI usage under control.
- [33:48]
B
Our deployments take five to 10 minutes, so everything is quick. I like to sit there and say, give us an hour and three of the right admins and we'll get this up and running in five to 10 minutes. Now that's integration into your Existing systems, into your idp, into everything else you use. But our goal is to make this as simple as possible. Use case finding is a little bit different than actually enabling people to do work and to manage a swarm of agents. But I think to a large degree, most of my customers, they're using codecs, they're using cloud code, they're using cursor, they've rolled out coworkers, and they're looking to enable more people and they want to do it with security control and watching cost. And that's what we do at Run Layer.
- [34:34]
A
Yeah, yeah. How much is the cost conversation changed in the last couple years?
- [34:37]
B
Oh, I mean, cost conversation has dramatically changed. So one of our biggest customer, not one of our biggest customers, but one of our case study customers is Gusto. And we work very closely with them. They're awesome people, we really like them, and I think we're deployed wall to wall and. And everything is changing. Last year was how do I enable people as fast as possible? This year is how do I think through my token budget? I have another customer out there that was working with a competitor that ran that, their agent, harness, let's call it, that ran a loop all weekend and used 80% of their inference budget for 2026 in a weekend. Whose fault is that?
- [35:21]
A
I said ontology, you said loops, and now we're both in the penalty box.
- [35:27]
B
I don't like loops. I think loops are just in general a bad way to think about the world. I think loops can go off the rails pretty quickly.
- [35:35]
A
Well, you just gave a good example of why that's the case. I mean, why would you let something run over the weekend that just sounds like a recipe for being broke, I
- [35:42]
B
mean, social for it. And I'm classy, so we're not going to say who the competitor was, but it's a pretty big shock when you use 75% of your inference budget in a weekend and you've got half a year left.
- [35:54]
A
Yeah, well. Or you're Uber and you burn through it in Q1 and you have three more quarters to go. All right, Dim, I want to go back just for a second and talk about what Victor was getting at, which is models getting to the point when they're good enough and you don't really need to fight for that next kind of quantum of intelligence as long as they're performant and inexpensive. Now, in the on device case, you are dealing with just much less capable hardware. But what I've noticed as a user of computer use AI on the desktop world, where I have lots of ram, lots of processing power, and a blazing fast Internet connection. It's pretty stupid. And I'm kind of embarrassed at how bad we are at AI actually clicking around. Like, if you've ever used the Codex computer usage, it's bad. So how close are we to the point to which in the mobile context, having the technology so good that were more focusing on performance versus adding more intelligence.
- [36:45]
D
Totally. That's a great question. So I believe it's like, Victor, there's a point. It's like you can imagine there's two different tiers. I don't need full ASI superintelligence to order my coffee. Even if it's a good enough model, as long as it can go and get interfaces, it's fine. As long as it's spatialized. And then you probably need superintelligence for doing fundamental research. I want to do new core research in material sciences or to solve space travel. You probably will need this for hard sciences. So I think we will start seeing this differentiation. You need large data centers for actual superintelligence. That's more than human iq. But ordering a coffee should not require more than human iq, for example. And that's why I believe.
- [37:28]
A
I mean, sometimes there's a lot of options, man. I mean, sometimes I freak out about lactose free milk or oat milk is really. It's a toss up in the morning.
- [37:34]
B
I don't know. That's pretty tough. I'm with Alex on this one. There are a lot of options. If you go into the Starbucks app and you start going deep down there, there's lot of fields with a lot of different overflow menus.
- [37:46]
A
It's crazy. Do it worse. Order something that's simple via like Uber Eats. And then there's like 17 different screens. And then if tip more because it's raining. And then, oh my God, you know, it's. It's brutal. Maybe an example would be something simple like, I don't know, designing a new nuclear power plant. That might be simpler than ordering coffee. Div, just for your future examples.
- [38:02]
B
But keep going in and out where there's only like four choices on the menu.
- [38:05]
A
Yes, yes. I'll take a double. Double animal style. And five. Fuck off. Oh, yes, we're on the show. Bleep that out anyways, Div, back to you. Sorry.
- [38:12]
D
Thanks. It can definitely get complicated. It's not that simple, but it's still. We think this is something that is doable, especially with smaller models that we are specializing. One of the tricks we're doing is we're starting to build something almost like skills, like how you do with Claude on different applications so the AI can automatically learn those interfaces. It knows this is how all the doordash menus look like and this is how you choose them. And it's kind of like learning this layer that's becoming better and better. We call it intelligent caching system where you can go into cache all the particular interfaces in the world automatically and that allows us to run these things on device.
- [38:47]
A
Caching all the interfaces to run this on device. Does that mean that it goes out and memorizes every single mobile application that it might be asked to use and then has that kind of an instant recall?
- [38:56]
D
It's more like building a skill. So it's not creating a heuristic or a script. It's more of like it has learned some of the gotchas in a sense, if I am on Amazon, this is how I should order something, this is how I should navigate. And it has kind of almost like a markdown file of what high level things look like, what to do, what not to do.
- [39:25]
A
And all of that is behind the curtain for the user themselves. This is just how it works when they're actually talking or typing to it.
- [39:32]
D
Yes.
- [39:33]
A
Okay, because I was thinking about your company before we jumped on and if you go back maybe six months, Santia Nadella, CEO of Microsoft, was talking about how he wanted to collapse the entire CRUD app stack and essentially dissolve it and turn it into an agent. And I feel like that's kind of what you're doing for mobile devices. So are consumers demanding this or are we showing them the future? And then also is this going to be something that Google and Apple fight back against? Because they currently make a bajillion dollars on mobile operating systems staying relatively static. And we can see that from iOS not really changing in the last 15 years. It feels like it's just a grid of apps.
- [40:12]
D
The way we think about this is in a sense you're building self driving for your phone. So how you have self driving for Waymos and Teslas, it's a bit in the future where your whole phone is going and navigating itself. But we do see this is kind of what people want. We've talked to so many users, they're like, I actually want CD to work for me and I wanted to just use my voice to talk to my phone. And this has been a supply problem. AI and technology just has never been good enough that I still have to use a tiny keyboard on A mobile interface and we particularly want this. And it's something again. It's new. It felt time for this to be fully ready. So we don't think Google or Apple will do this anytime soon and roll it out to everyone and they have a big brand, they have billions of users. They don't want to roll it to start a billions of users and see what happens. It's easier for us as a startup to go and figure this out. It's kind of like okay, if you're building something like self driving, you don't want it to just have a self driving car literally run on every single road in the US So it's more like you want to do something that might be working in only Palo Alto or California and then do gradual rollouts and which is possible for us.
- [41:19]
A
I mean Waymo started in one city in Arizona in San Francisco and then expanded out from there. So there's merit to that. I'm with you. Okay, let's put aside the technical stuff and talk about the business side of things because there's a lot going on. And Victor Higsfield, AI, I presume you view them as a rival fair.
- [41:37]
C
One of them.
- [41:38]
A
One of them, yes. It's a busy space. It's AI. There's a lot of companies doing things. But there's some news out that the company is in talks to raise 300 to 500 million at a valuation of 5 billion pre and that they hit a $500 million run rate earlier this month, more than doubling from January. Those are insane numbers. In a positive sense. Lots of money, quick valuation gains and simply insane revenue growth. Is Korea on a similar trajectory? I know you guys raised 83 million last year.
- [42:08]
C
I think we're quite a different company than Hicksville. And I would also take. I don't know if you're talking about the information article. Yeah, I saw it yesterday. I would be a little cautious about the numbers that they're sharing here. This is a company that, at least from a product perspective, they have been aggressive to the point of they got blocked off X, which X is a place that is already kind of like Wild west. It's a wild west. So they got blocked from that because of the type of content that they were putting on the releases that they were making. It was like erotic content, touching racism. It was very, very weird that they were just extremely aggressive. Growth at all costs. If video of Pinocchio would grab your attention, we're gonna do it.
- [43:04]
A
That's a brand new sentence on all of this week in Podcast. No one's ever said soft core Pinocchio before, but I'm glad we've now broached that window.
- [43:16]
C
I gotta confess, I was also kind of seeing through the video because I couldn't believe that this was actually the way that they were marketizing their thing. So they are known in the space for being shady in the releases that they make. They announce things in a way that they really deceive users. There's a bunch of. There was this launch that they recently did around announcing their games product or something like that. I think that Fable just got released and they were just like, they are really good at growth. They have an entire team ready for anything that is on the space. And as soon as it's ready, they are going crazy pushing four or five videos every day. They're insane at that. But sometimes they push the limit and they really make videos that they look extremely cool. They make you think that you can do something with the platform and then you go into the platform and it's a complete flop. They would use AI generated videos of 3D video games and they would make you think that you can play this 3D video game and that you can create this video game from a prompt. And then you go into the product and it's completely fake. Everything that you can do. It's kind of sketchy, 2D arcade type games. And they did similar stuff when they announced their products around motion graphics, where I don't know how much of this is true, but like there were like many people saying that they stole presets from envato, that it's like a place where you can find a lot of templates and they just like changed the letters from, from. From these motion graphics and they use it like almost like claiming that you could do all of these with their product, which it was also not true. So they have this reputation in the space. Some of the numbers that I've seen in the information, I don't know. I would believe that they are doing a ton of revenue. I think that.
- [45:05]
A
Can I make a guess? Do you think that perhaps a source close to the company is leaking information to the information to generate a headline that might be to their benefit? Not that any company would ever try to leak positive information about themselves.
- [45:20]
C
I know what they're doing on social media. I know how they're grow. I know the way that they are growing through the, through the product. They are doing massive aggressive paid campaigns. The information article says that they were growing from the product and that they were not growing through Paid marketing, which I thought, like, that's just like a straight lie. I don't know where the information got their sources from. Wouldn't be surprised if it's someone close to the company, but I really have no idea.
- [45:47]
A
I do want to throw in here just because I got to do this for the team. Love the information. A lot of my friends have worked there. Shout out to them. They do incredibly good work, and everyone should at least give them a chance and read them. Yeah, no one gets everything.
- [45:58]
C
Oh, yeah, we love the information, too. We have a great relationship with them. We've been in conversations. They announced a bunch of the releases that we make, and we are in touch with some people in there. They're great. But these numbers, again, some of them look a little. I would double check some of them.
- [46:14]
A
Okay, well, let's double check some more numbers while we're here. Live on air. Last November, Div, rumor has it that you guys were raising 50 at a $500 million valuation. Was that true? Did you guys close that round?
- [46:24]
D
I would say it was mostly rumors. We actually don't know where the leaks came from. To be fair, we did have some investors we were chatting with who were very excited about what we're doing. We showed them some demos, and I feel like somehow it became a rumor. Like, we are doing this thing at this number, and we do have some fundraisers. We close and we have some announcements we're doing, but it's like we are not announcing the numbers yet.
- [46:47]
A
Okay. Blink twice if you're going to announce them before the end of Q3. That's a couple of blinks. All right, good to go. I ask that not just to be a brat, but because I can. But also, there's a lot of money flying around, and each one of you guys have raised in your most recent rounds, either reported or otherwise double digits. And this sounds like a ridiculous thing to ask, but is that enough capital in today's market to compete at the level that you want to? And, Andrew, not to pick on you, but let's pick on you. Your Last round was 30, which, back when I started watching and covering venture capital, would have been a very large. I think it's a Series A in today's world. It seems to be relatively modest. So can you just. For the founders out there who are kind of in this world, talk us through why that was the right amount of money for a company that has just been honest. Such large aspirations in what it's building.
- [47:36]
B
Well, at the end of the day, we hadn't spent, I think we had spent a third or a fourth of our seed when we had raised our Series A. So the numbers are a little bit misleading. So if you think about it, we've raised 42 million. We have tier one enterprise customers. These are people like Gusto, Opendoor, DBT, Decagon, et cetera. And we're able to service them with our product. We also work with a number of Fortune 500 and the financial services sector. And so at the end, for our business, this was the right amount of capital. We don't buy GPUs here, so we don't have very costly inference spend. So it's a little bit different of a business. And also we've just ramped revenue so quickly this year that it was completely the right decision for my business and the right place. And at the time I was talk
- [48:27]
A
to me about revenue ramping this year, how far, how fast, how much, when.
- [48:30]
B
Ah, man, it's been a lot of fun and I'm not going to go into the revenue numbers, but it's been a lot of fun.
- [48:36]
A
Can you give me one of those percentage change numbers? Like it's up like 500% from last Tuesday or whatever?
- [48:41]
B
Yeah, like 8x since we signed the term sheet or something like that.
- [48:45]
A
Wow, that's very impressive. Is that ahead of plan?
- [48:49]
B
Oh, yeah.
- [48:50]
A
What was plan?
- [48:52]
B
Not going to give you that.
- [48:53]
A
I can do this all day. But I take it that enterprise demand is so high because companies are very focused on getting their agentic feature kind of sorted out. I'm curious about who, who are your customers and if it's branching out from the world of technology first companies and into companies that are in sectors that are less tech forward.
- [49:11]
B
We're definitely in sectors that are less tech forward. We focus on a few key industries. We focused on, let's call it AI and digital native. So that's the engineering folks of the world, mostly our friends in Silicon Valley, we focus on folks on financial services and we focus on healthcare. Those are our verticals.
- [49:30]
A
Well, it makes sense that Gusto's in there, by the way. Gusto. You don't know this Andrew, but I'll just do it for them. Longtime partner of Twist. I don't know if they're currently an advertiser, but I just checked and gusto.com twist still works. So if you want to use Gusto, make us look fantastic because we love everyone over there. I actually just talked to Eddie.
- [49:45]
C
So do we.
- [49:45]
A
I just talked to their CTO about their new co founder agent. The other day and I was kind of impressed at how it seemed that they were building something that would be very generalizable if they had even more data into their product over time. They already have, you know, payroll and HR and all that stuff. Do you think that companies like Gusto are going to be able to build more general agents, kind of even that go outside their normal or existing product remit down the line, or are they going to stay constrained to just the things that those companies already do?
- [50:13]
B
Well, Andrew, I think Gusto is, I think Eddie is incredibly intelligent and a great leader. I think Tomer their CPO is also incredible. I think in general the team has moved very quickly to seize the moment and I think we'll see what they ship. But they have a large opportunity ahead of them and people love their product.
- [50:33]
A
Yeah, that's all very good. Now I think one of you mentioned distillation earlier. Victor, I want to go back to you about this. Anthropics been making a lot of noise about distillation attacks. People essentially stealing their hard work is kind of how it's phrased. So one, do you think that anthropic's being reasonable in its complaints? And then also are you worried about anyone taking the open source models that you put out and essentially doing the same thing and stealing a lot of the work that you put in?
- [50:57]
C
So in terms of Anthropic being reasonable about this, I do believe that they are getting distilled. I don't think that this is like, I think that this is a fact and this election, you know, like it's wide across domains. Same thing happens on the image space. A lot of the, and especially from Chinese models. A lot of the Chinese models, you can 100% see where they have been distilled from. I don't know if you remember this orange tint that GPT1 images tend to have, but essentially there was very characteristic traits on some of these images. In a few months we started seeing those traits in some of the open source models too. So they are 100% getting distilled. If we fear about. Oh, look at that.
- [51:47]
A
We have editorial director Lon Harris in the background running the screen shares. He's on point today. Lon.
- [51:52]
D
Well done.
- [51:52]
C
Yeah, that was fast.
- [51:53]
A
If you're not watching the video version of this week's show, you're missing out on a lot of great Lon work. But anyways, keep going Victor.
- [51:59]
C
So distillation is a real thing. It's happening in our case. We are not too scared of the distillation that may happen in our model. We would encourage people to distill it if they want to. I think that the places where you can get through this election are always below the places where you can get through owning the model. So they will always be inferior. But at the same time I can understand how a company like Anthropic or OpenAI or Meta, they can be so concerned about this because the economics of doing pre training up until a certain point and then just getting to very similar stages in benchmarks through the stealing versus doing the whole process yourself through paying experts for super clean data that you can fit into the model in terms of how much it costs to produce each one. The difference is massive. So I can understand how the fear around some of these other models reaching a certain level of capabilities maybe. I guess that I would also have those concerns if I was putting these crazy amounts of resources into training these models and then someone else comes and steals these IP that I have been able to create.
- [53:24]
A
I'm really torn here because on one hand, absolutely. On the other hand I have published millions of words on the Internet that have been ingested into AI models and used for training. So kind of like I feel sorry for you, but I also feel sorry for me. Maybe they should get a check if I get a check. I don't know. Off topic, Div, when it comes to building agents, is there anything similar to distillation that we see at the model level? Can people like watch an agent do something and then learn from that and steal from that? Or is that not something that happens in the market today?
- [53:53]
D
We have not seen too much of that. I think it's possible it's happening behind the scene with like anthropics, computer use and other things, but it's not been that big of a topic, at least compared to LLMs. And I do see this becoming a big thing in 2027 because if everyone has agents and one company's agent is working better than others, then everyone's trying to distill it and steal that behavior. So it's going to be a big topic in 2027 actually.
- [54:19]
A
What are the early indications that that's happening in the world of LLMs? Anthropic just drops a blog post every three months pointing fingers at every single Chinese AI lab they can name off top. So what should we look for to see that kind of agenda distilling in 2027? Are you going to tell us?
- [54:36]
D
Yeah, I think we are definitely building security guardrails. It's kind of like you Want to build some sort of watermark so you know someone is stealing and has the same output as you. It's harder to do that with the agents because it's taking actions on your screen. It's hard to compare, like is this agent just good or is it just like copying someone else?
- [54:57]
A
Going back to our minimum viable intelligence, maybe there's a minimum viable agentic competence scale that we're going to have to eventually come up with. I don't know, maybe that's a benchmark you guys could make. Is there a good benchmark for computer use mobile agents?
- [55:09]
D
There is. There's one called Android world. It's from DeepMind. We're actually number one there for about six months now.
- [55:16]
B
Oh well, congrats Dev.
- [55:19]
D
Thanks.
- [55:19]
A
Who's number two?
- [55:20]
D
I'm pretty sure it's a Chinese lab.
- [55:22]
A
Yeah, that makes sense to me. All right, one more major topic before we jump through our lightning round. Simply that I'm seeing a lot of people talk about the ratio of employee salary cost to token spend and Tomas Tungu's theory ventures previously Red Point. One of my favorite VCs says that over at Anthropic they're spending about 2.3x in tokens per dollar of salary that they're currently paying out. So I'm kind of curious if you guys could give us a rough estimate of your company and what that looks like today and how that's changing. So Andrew, over at Runladder, what's it look like?
- [55:55]
B
We were doing this math recently and I want to say it's on average it's somewhere between three and five thousand a month per employee. We have 40 employees. And look, we did it pretty rough. We didn't look at all the additional coding agents that kind of are running in the background, things like Devin or other stuff like that. This was more direct spend to the labs. And then there's probably another, we have some more spend through API usage. So let's say five to seven per month per employee.
- [56:30]
A
So call it seven, which is I can't do math live on the show because I'm tired, but probably like 0.3 token dollars per salary dollar or all in employee cost somewhere in there.
- [56:43]
B
Something like that. And that was the interesting thing though is when my co founder actually just optimized our agents that we were running and swapped most of them from Opus 4.8 to Sonnet 4. 6 and I think what we saw was 40 grand a month savings. So I mean there's real easy, low hanging fruit optimizations. You can do that we hadn't even bothered to do. When Anthropic is releasing those stats from Anthropic. My assumption there is. It's just all fable all the time. Yeah. And probably some other models that we don't have access to. And that is a very intensive compute model. So I think that's a little bit of a misnomer. I think the average company though has seen dramatic increases in Q1 and Q2 of this year in their token spend. And look, Uber is the case study. But it's not just Uber, it's everyone we speak to. And so I want to enable my employees. I want to see the ROI gains. I want to see my employees managing a swarm of agents. That's the future of the like L5 or equivalent of self driving cars. And how do I get there? And how do I get there with reasonable cost structure that is what every enterprise out there wants.
- [57:58]
A
Can we talk about this is just a segue. But I'm really curious. Agentic personhood and kind of corporate rights. I was talking to someone a few weeks back that built like an agent and it was running its own vending machine and they built an incorporated company for it and they were trying to actually give it to the agent but they couldn't because there's not kind of like the law in place there. I know you guys do some stuff with the genetic security, but how are we doing in terms of making these agents self contained and their own entities, if you will, versus just something that's like a hammer.
- [58:25]
B
Yeah. So I mean one of the things we do is we are run layer. So we end up having the policy engine and the control plane. And so all of our customers when they work with us, they get access to not just a deterministic policy engine, but they get access to our runtime security models that focus on things like alignment and our agent guard models. And that stuff is very important, important to our enterprise customers because every week you see somebody new on Twitter used a managed agent and a deleted prod. We help our customers avoid that future. Now, legal rights of the vending machine agent. The concept that you were talking about, I actually have never thought about it. It's an interesting concept. Can you have an agent own a company? And I'm sure we'll get there in the future. It's definitely coming. It's just not something I've had the chance to ponder.
- [59:19]
A
I think it's going to be fantastic because then they're completely uncontrollable at that point in time off they go into the world of commerce. And if more competition is more good, I welcome our agentic competitors. I'm not even really being sarcastic, I'm being kind of serious.
- [59:33]
B
I hear you.
- [59:34]
A
Yeah.
- [59:34]
B
And I think the control plane and the policy engine are incredibly important aspects of this new world. I mean there's some segment of where we're starting to go towards the movie the Terminator and how do you prevent the extinction of humanity? I mean that's a very far fetched version of this. But how do you actually control what agent has access to what tools on a fine grain basis? Are they mapped to an individual user? Are they a delegated credential? Are they just a service worker? These are very interesting challenges that everyone is trying to think through in 2026. And it's just going to get more and more difficult. And that's what run layer helps our customers do.
- [60:14]
A
I appreciate that you brought up Terminator. I thought that was banned in all AI conversations. But I guess you do need granular permissions and machine guns. One plus one equals three div. Victor, your guys corporate level kind of token spend versus employee spend quickly because we've got a little bit long here. But I'm just curious what that looks like and how the graphs are changing. Victor, let's start with you.
- [60:35]
C
Yeah. So I was actually checking, checking our spending cloud on a month to date. The person that that has spent the most, only this month is already above $7,000 and this is not one of the top engineers. It's like actually like 50% of the salary that this person has. So we're getting to 50% in some cases. It's not an even distribution. I think that we have the mega prompters and we have the ones that take it with a little bit more of care. I thought that I was using Claude a lot, but then I really don't understand how they are using it because my spam is an order of magnitude lower than what these guys.
- [61:27]
B
Could it be that you can go into the admin console and select the default model to be Sonnet 4.6 versus OPIT 4 opus 4.8 and you'll see next month if their spend goes down? Because I think to a large degree what a lot of people do is just click and if you don't actually set a default model, you're ending up on Opus 4.
- [61:45]
D
8.
- [61:45]
A
Okay.
- [61:45]
C
Yeah, we didn't have that.
- [61:47]
B
Yeah, we did it last week.
- [61:49]
A
Yeah, I mean you're talking, Andrew, you're talking about like you know, control planes and you know Fine tuned controls. But it seems like a lot of stuff is just like don't click the opus button. Like I mean to me it's kind of shocking that, I mean we're talking about, you know, computer use, agents for mobile and all this and then that's still broken. Is this just like the jagged pace of development of tools around AI models just not being all at the same level of maturity? Because it feels like sometimes we're talking about science fiction and sometimes we're talking about absolutely basic accounting and spend management, which is not complicated and not something that we haven't figured out before as a species. Like it's driving me nuts.
- [62:24]
B
Look, we're an inning, one of a very, very interesting multi decade transformation that we're going to see happen. We've seen for people. Everyone here has been in the space long enough when we would talk about GPT4 when it first came out and the insane cost and I don't know, I think that in 2024 when that model came, there were eight price cuts that year alone. And so yes, the model's going to go down. The cost of inference is going to go down dramatically. It's a 2026 problem. Everything changes in this space every three months.
- [62:55]
A
I think you should just think of every quarter as a year and then that kind of puts the right time frame around how fast these things are moving. All right, before I let you guys go, I want to do kind of a fun little lightning round here. It does seem that as we've gotten increasingly intelligent models, we've talked about ASI or AGI less. And so I'm kind of curious about that and what your timeline is. So div. What's your AGI timeline and has your P doom changed in the last couple of months?
- [63:19]
D
Yeah, I'll honestly say I think AGI is almost here with most definitions. We are able to do most things with AI that you want to do. We don't have ASI. It's like ASI might be like 2030, but I think it's like based on the definition, I can say we almost have AGI and my PW is actually going down with time because this seems that people are responsible and we have not seen except maybe Fable and methods getting banned. There's not been that much things that have gone wrong with AI and it seems like people are responsible, people have the right mindset on how AI can be an uplifting force for good, for humanity. And there's a lot of things that are kind of like, I'll just Say there's a lot of shitty work people have to do and then now you don't have to do that anymore. So it seems to be like improving people's lives.
- [64:13]
A
It makes my life a lot faster. I can do a lot more work, which I love. Victor. So we've had one kind of positive AGI almost here. Asi four years out and PDOOM going down. What's your perspective?
- [64:24]
C
I still struggle to wrap my head around what how to define AGI. I guess that's something that surpasses human intelligence at any degree or something that by itself it becomes smarter than the human intelligence. I don't know how much of that is going to come from the model itself versus how much of that is going to come from the application layer. And something similar to what's happening with the agents that we were seeing models to be able to do way more with the same raw intelligence. So I think that there's a way how you can see the point of deep around, hey, it's actually almost here because you actually don't need a 10x on intelligence, you need a 10x in terms of what they have access to. How do you build this context and what is the way that you put this intelligence to work? So I really don't know. It can be from one to 10 years. I struggle defining and I struggle also understanding the path, how we're going to get there. And when it comes to pdoom, I think that PDOOM goes up the closer that this technology and the less commoditized that this technology gets. So the more distance that we see between the things that startups like us have access to versus the thing that some elite privileged startups have access to, the lower that the pidum goes in my scale and I think that we are definitely seeing how open source is catching up quite fast to the capabilities of these AI models and we are surpassing these thresholds that we were talking about around the model is good enough for certain tasks. So seeing that over the past six, six months or so, it's very, very promising. So pidum going down. But I will also say non0 like a lot of things can happen. A lot of research innovations can happen. A lot of things can happen on the next few years.
- [66:12]
A
Okay, Andrew, are you going to bring us down or are you going to affirm the positive vibes here as we close?
- [66:18]
B
Oh, look, I'm all about AI enablement and I'm all about like I am a optimistic human being by nature.
- [66:25]
A
Hell yeah.
- [66:26]
B
And I think we've working in this space has never been more fun in my entire career. And I think we are just on the verge of unleashing incredible power to the end user. I fully agree with what Victor said in it that the models have gotten very good now, connecting them to the right context, directing them to the right tools, whether it's McP skills, plugins, APIs, CLIS, whatever you want to do. But giving it access to the context is the key. And my goal is to help people enable this and then giving them visibility, control and security. And if you can do all that well, then Div and Viktor and my job is pretty easy on a daily basis. And so I'm a big believer that we're on the cusp of great things.
- [67:14]
A
So your PDOOM is going down or is already at zero? Hard to tell from your answer.
- [67:18]
B
I mean, I get to use run layer, so my PDOOM is pretty good.
- [67:22]
A
That is the best like native plug I've ever seen someone weave into a different answer. That was fantastic. Can I throw one more at your interview just before we wrap? Because I just thought of something that I'm not sure about.
- [67:32]
B
Sure.
- [67:33]
A
Agent swarms. Can you define that for me and tell people why we should care about them? When I think of swarms, I think of drone swarms, which are not particularly appealing. I think about kind of the Ukrainian battlefield. So talk to me about agentic swarms and when they're coming, if they're coming now, and for whom.
- [67:49]
B
You want to define it for me before I can answer it, because I think this means something different to everyone.
- [67:52]
A
No, I want you to define it. You're Mr. Agent.
- [67:56]
B
I mean, look, I think swarms are multiple agents, whether it's top level agents or sub agents that'll work in parallel to help people solve tasks. I think we see engineers already working in this manner. When they write code, they'll be running a lot of parallel, different agents. It enables an individual user to have superhuman like abilities and to have significantly more productivity. So what is an agent swarm? Multiple agents. What is a drone swarm? A drone swarm. I think to the best of my knowledge that's multiple drones.
- [68:32]
A
Yes, but in your view, an Asian swarm could be as few as two agents working in concert.
- [68:37]
B
I would say it's more than two. I would say a drone swarm is more than two drones. Also, I would say there's probably a definition here of swarm. Let's assume five plus, maybe 10 plus.
- [68:47]
A
Are there any agent swarms out there apart from the coding context that you see used regularly?
- [68:54]
B
I mean, it's coding and coding is the P0 here today and what we're really seeing. But I've seen in our sales, I've seen in some of our go to market teams, they're using swarms of agents. How do they think through and optimize exactly how the selling process is. And so that's where I've seen the most of it. I think design in itself is at least designers in our organization have definitely moved towards the engineering mindset. So I think it's very similar there. I think you have to go function by function. Marketing is definitely probably has moved to a swarm of agents. I think most marketers you interview for companies that have names like AGI Inc. You ask them how many skills they're running and what is their ide of choice and that's a very different conversation for a marketer than two years ago. So we see it across the board.
- [69:52]
A
Well guys, thank you so much for coming on today. Let's go around and drop some website URLs. Victor Perez. Where can people find Korea AI online?
- [69:59]
C
Korea AI love it. As easy as it gets.
- [70:02]
A
As easy as it gets. Div. HK Inc. How do I find that on the great wide Internet?
- [70:06]
D
Yeah, we have this, the AGI company And you can also find us on AGI app.
- [70:13]
A
The AGI Company or AGI app. Love it. And Andrew, last word goes to you. Where can people find run layer runlayer.com fantastic. Well guys, I learned a lot. I hope other people did too. We'd love to have you back in six months to talk about what's going on and by then hopefully we'll all have Fable 6. Until then, this has been the speaking AI. My name is Alex. We'll see you all next time.