← The MAD Podcast with Matt Turck
The MAD Podcast with Matt Turck
The Agent Harness: Building Secure Sandboxes for Autonomous AI Workloads
01:05:15
Loading summary
Transcript105 lines
- [00:00]
Ivan Borozin
We are a part of this like super cycle right now. And the super cycle does not last forever. And so if you're going to pause by the super cycle, you are seeding market, like that is what you are doing. I would ask, can you go fetch the data from our bank? And then it was like, oh yeah, just log in and give me access. I'm like, log in, give me. No, I will not give you access right away. Fundamentally for me, that was like broke the entire thesis of it. So you give it its own machine. When I think about agents, I think of them as digital knowledge workers. And to do anything as a knowledge worker, you do need a computer. My argument is that every agent will need at least one sandbox, sometimes more.
- [00:40]
Matt Turk
Hi, I'm Matt Turk, welcome to the Matt podcast. My guest today is Ivan Borozin, CEO of Daytona, one of the most talked about startups in the agent infrastructure. If you've been hearing the word sandbox in every AI agent conversation and quietly wondering what it actually means and why it suddenly matters, this episode is for you. We go from first principles. Why does an agent need a computer at all? All the way to the deep why Daytona had to throw out Kubernetes and write their own scheduler, and why a global CPU shortage might be coming faster than people think. Ivan also unpacked the full agent stack as he sees it. Models, sandboxes, tools, MCP memory, orchestration and where each piece is heading. Along the way, Ivan shares some really interesting lessons on go to market and distribution for technical founders that he has learned through 16 years building developer tool startups. Please enjoy this great conversation with Ivan from Daytona.
- [01:34]
Interviewer
Hey Ivan, welcome.
- [01:35]
Ivan Borozin
Great to be here.
- [01:36]
Interviewer
So you have said that every agent needs its own computer. What's the simplest way of explaining that idea?
- [01:45]
Ivan Borozin
Well, when I think about agents, I think of them as digital knowledge workers. And to do anything as a knowledge worker, you do need a computer, or I should say anything sophisticated. So you and I can be on a conversation and we can get something done, but we usually need some sort of tools in our world is usually a computer to get higher productivity and to be able to do things. And so I think of that in the same lens. So for me, agents are literally just digital knowledge workers.
- [02:13]
Interviewer
Great. And so it's on computer. That's the whole concept of sandbox. So as an introduction to this whole conversation, what is a sandbox?
- [02:21]
Ivan Borozin
Absolutely. So sandboxes are. Although the term SEMCOX first comes from isolation, so making sure that there's a secure place for in this case an agent to run and do things. But on the other side, it is essentially a computer. It's a full on computer that an agent has the ability to install tools, access the web, run scripts, run code, whatever it needs to get its job done. So the shortest answer is a sandbox is essentially, we call them composable computers for AI agents.
- [02:49]
Interviewer
Yeah. And is in some ways, like this whole OpenClo and Mac mini like a good analogy for what a sandbox is? Like the Mac Mini sort of being the sandbox.
- [03:00]
Ivan Borozin
Exactly. So I think the openclaw Mac Mini thing helped a lot of people understand what we actually do. It's like, oh, I get it now, I get it. Right. So it needs a computer. In this case it was a Mac Mini to be able to do different things. So yes, that helped a lot with awareness for sure.
- [03:18]
Interviewer
Yeah. Because again, to unpack it, like the Open Claw as a frameworking system to help an agent do all sorts of things on your computer, basically you want to be able to kill it if it goes rogue and therefore you can unplug the Mac Mini. The way you could sort of kill a sandbox is that, is that kind of.
- [03:36]
Ivan Borozin
So there's a, there's a couple of things that I personally thought about. And so my openclaw runs on a. Not a physical Mac Mini essentially, but a sandboxed virtual Mac Mini. And the reason why I didn't. So the way people usually run these things and Claude code or openclaw is usually on their own computer because it helps them, you know, organize emails, you know, search whatever documents they have and whatnot. There, there's a one, there's a high security risk there because at one point we were doing our board meeting presentation and I would ask claw, can you go fetch the data from our bank? And then it was like, oh yeah, just log in and give me access. I'm like, log in, give me. No, I will not give you access. Right. And so right away, fundamentally for me that was like broke the ENT thesis of it. So you give it its own machine. I personally gave it its own like Daytona account, gave it its own phone number. And the reason I have to give it its own phone number is because it has to do 2fa to get into the bank. Like there's no other way except for two FA with a phone number for this particular bank. And so it has to have all these things, like a employee, like a digital employee to be able to go and access these things. And so the risk now that, now that it has Its own computer, it has its own account. These accounts have the limitations. So it can only, it can only look at the data for my bank. It can't spend the money from my bank. And so except for the credit card that we did give it, which has, you know, you know, $100 a day or whatever it may be. And so the only risk you essentially have at that point, if it's in the sandbox, is will it take this data and now leak it somewhere? And that's something we can talk about later. But essentially the worst thing can happen is that. And to your point, you can kill the whole machine if you need to.
- [05:17]
Interviewer
Kill machine. Yeah. And there is a fundamental concept of stateful versus stateless. Can you, can you unpack that for.
- [05:25]
Ivan Borozin
So that basically comes when we talk to people about what we are building. They're like, oh, doesn't this already exist in any of the hyperscalers? Right? And so the answer is no. And the answer is no is, is because everything that they were built for, which was deploying apps, they were, they were stateless. Like, if you have, let's, let's pick any of your websites or whatever, we want web apps, you do not want that to change on the fly. You, if you, if you are the company, we'll pick like ebay, because they're in this building. So they came to mind. And so if you're ebay and you're an engineer there and you're like, oh, you have this new, you know, update a button is here, or does this thing, you want that state, you don't want that to be changed on the fly, right? The database might change, information might change, but you don't want the app to change, right? And so with those things in mind, that is how people had built the hyperscalers. Like, that is the fundamental architecture you came into and built on top of that. And the simplest analogy I give people is, let's say you're building a truck, right? You are a factory for a truck. There's like the weight, the type of the engine, the chassis, all of that is made to very slowly but surely securely transport some sort of goods, right? On the other hand, you can have a sports car which is still, you know, it has four wheels, it has an engine, whatever, but it's made for different things. It's made to go very fast. So the way the chassis is built, the way the, you know, engine, the, the weight balances is completely different. And so you can do, you can do a lot of things with both, but they're not fundamentally the same thing. And as a company trying to build out both of those, they're separate platforms completely. Right.
- [07:04]
Interviewer
So sandboxes are fundamentally new, primitive.
- [07:09]
Matt Turk
Is that, is that correct? Is it, is it like a history
- [07:10]
Interviewer
of sandbox or did sandbox exist before?
- [07:13]
Ivan Borozin
So I would argue, I would say that the person that could have nudged, although he said he did and someone else did. But anyway, there was a company called Code Sandbox way back in the day when we used to compete with our company Code Anywhere, which also was in the realm of replit. It's all these cloud based IDs and so they called it Code Sandbox because I believe it sounded cute. It's like a box where your code for ID lived. And they were actually one of the firsts. So the team there that actually used micro VMs, did snapshotting, forking, all these things that we do use today. And so that is sort of. And this is maybe a decade ago, but the utilization or the usage from. Or the value that it was giving human developers was not there. There's this whole article about like the end of local hosts and people have been talking about this for. I've been talking about this for 20 years. And basically developers would say, like, you'll take my local hosts like out of my cold dead body. But now that agents are here, local hosts no longer actually one, you don't want that for a number of reasons. And so we're finally getting to that. So that technology and that thesis around sandboxes originally now seems to be coming to fruition.
- [08:26]
Interviewer
And the big acceleration around sandboxes is agents. Right, that's that thing.
- [08:30]
Ivan Borozin
Yeah, agents, absolutely. When we think about agents. So one, if you, Even if you do run an agent on your computer, like you want to do that, up to you go ahead and do that. There's a bunch of problems. One is, let's say you're on your laptop. There's this whole thing on Twitter now where people are holding their laptops open. Yeah, yeah, the people.
- [08:47]
Interviewer
What is that? Yeah, I saw you tweet the.
- [08:49]
Ivan Borozin
Well, that's because I always hold my laptop open. That's just like my vibe without that. But I've been doing that for a long time. But people do that because they want Claude or open clot to finish. Yeah, because if you close it, it's.
- [08:59]
Interviewer
Yeah, it's terminate.
- [09:00]
Ivan Borozin
You terminate? Yeah, it's. Or pause or stop, whatever. So like the problem is like the continued. The ability to work nonstop is not there as long as your laptop. So that's one thing. The other thing is you can't do concurrency. So you can do multiple to the amount of compute that you have on your laptop, which will be quite limited. And so you might want to spin up ten or twenty or fifty or one hundred or a hundred thousand, whatever that number might be. And so that's very hard. So ideally you actually really want to remove it somewhere remotely so you can start on your laptop, you continue on your phone. It's the same computer and same agent that is doing their thing. Right. And so that is a takeoff of we have now decided that it's absolutely okay that it's no longer on our local host, but again, it is still a local host to that agent. So it's still a computer to that agent, what a laptop is to us.
- [09:46]
Interviewer
Do all agents need a sandbox or is that a specific category?
- [09:51]
Ivan Borozin
My, my argument is that every agent will need at least one sandbox, sometimes more. And we can get to that. Again, there are places where you don't need, and I analogize agents with humans for the most part. And again, we can have, we can do productive work without computers. And so there is, and that was the original sort of like chatbot, where basically you would just talk to an agent and it would inference so it just think and give you value. So I don't know, a lot of people use it for like emotional support or whatever. For the most part it doesn't need a computer. It has enough data from you going back and forth and then it can sort of understand and give you feedback. But that's a smaller subset. If we think of where all productivity gains are among the biggest verticals. Healthcare, financial services, whatever, most of that is done via a computer. And so if you want agents to do all these things, then agents will need these computers to do so.
- [10:44]
Interviewer
If you do tool calls, coding, like any of those actions, then you need a sandbox.
- [10:49]
Ivan Borozin
If you just chat, you probably don't need it. Yeah, so like, even if, if you have a chat that we can get to deal. But if you chat and it has to search the web, it still has to open a browser or it has to use something like, you know, parallel or XR or whatever to get to that, that would be a tool call. So it depends on the use case. But the thing that's quite interesting about the world that we live in is one, let's take a step back. One, I firmly believe that in due time all the tools will be headless. Now will it be inside of a sandbox or outside? That's another thing all of them will be. And that's the most efficient way for an agent to to work. But most of knowledge work is still locked into legacy apps inside of Windows for the vast majority, like the absolute vast majority. So if you want an agent today to do a job end to end, you literally have to give it a computer. And so an example of this is again the board report which is like, oh, can you pull out this report? And for our bank has an API and it can pull it out, but it only has spend on the API. It doesn't have incoming revenue on the API. It's just not exposed or through the MCP tool. Actually it's not exposed. And I'm like, then the agent was like, oh, I can't get to it. I'm like dude, log in, download it. Okay, I'll go to it. And then you can see it opens up a browser, it goes down or whatever. Same thing with, I don't know, with any other data source that we have. If it can pull it from the headless, it'll do it headless. If it cannot, then it sort of logs in and do that. And so if we want to give them power today and get value, we have to enable them to have these tools.
- [12:27]
Interviewer
You had a great tweet the other day where you were talking about the actual use cases that you see as a provider of sandbox of agents. Do you want to go through this? You had a code, command, execution, computer use, browser use and RL environment infra. Do you want to unpack that? Sure.
- [12:46]
Ivan Borozin
Basically I've made it, I think I've structured it better since that tweet which is right now we have two major use cases or two types of customers consuming Daytona and different use cases. And one is on the researcher side. So it'll be, you know, RL evals, benchmarks and the other will be on what we call background agents or long running agents. And so when you think of background agents or long running agents, like the most popular those are where a human is the end consumer. The human talks to a, let's call it app layer service that has an agent in that. And then the agent will call on the sandbox. So things will be, you know, think of like Harvey or Perplexity or whatever as or lovable as these types of background long running agents. And so they can both be sort of headless. So code and command execution and or computer browser use. So depending on what they need to do. And the same thing is on the researcher side where it's like RL and evals and whatnot. You can do RL and evals for, you know, coding and for that it's basically just headless like commands and command execution in there. Or you can actually teach it to do things in the real world. And then it does have to fire up a, you know, Windows, a Mac, a Linux sort of desktop or a browser to go through the thing. So code and command execution and browser computer use are like two ways an agent can work. And then it's to basically the consumption of the sandboxes to do those two different things are different.
- [14:10]
Interviewer
So that's a great sort of sandbox one on one introduction to the concept help us understand where sandboxes fit in the overall picture of this emerging agent stack that I think everybody's trying to figure out at the same time. So there's different components, there's file systems, there's orchestration. What are the different pieces?
- [14:35]
Ivan Borozin
I try to think about everything. To me it's actually quite interesting where. And we'll get this a bit later as well. It's like a lot of this all exists in real life today and so people are like overthinking this. I'm not saying that there's not gonna be new products and solutions and technology to solve it, but it's like it's not a new fundamental way of work. And so when you think about the agent stack itself, it's like, okay, you first have the models and the models are essentially the brain that that is what it is of equivalent to what a human's brain is sort of. So you tell it something, it replies, it understands and whatnot. And then under that it was like, oh, what are the tools it can do to get things done right? And so that can be anything from like any of the MCP or tool calls can do. It could be the sandbox, the computer, like whatever. We as humans, we also have a bunch of tools that we use. Everything from a hammer to a computer and everything around that. Right? So all of those things exist as well. Then there is memory that exists. Can, can an agent remember these things? Similar to like, do you remember these things that. Are there people that talk about orchestration of agents? It is like management. You manage people, I manage people. Managing agents is similar, is not dissimilar to managing humans. Now what tools will you use to manage them? Will you manage them the same way? Just send a slack message and. Or you do linear or whatever it is we will see. Or it's gonna be a net new app we'll figure that out, but it is of that. And of course it's like observability. Can you see what your teammate or colleague has done and verify that that job is good? Right. And so there's. That's sort of how I think about that on a very sort of higher level. And then you can break down to different types of solutions people are creating for this.
- [16:20]
Interviewer
Yeah, let's do some of that.
- [16:23]
Ivan Borozin
The things that I think about, like, there's also like things that are more solved and less solved where like the models, I'm not saying they're solved, that they won't continue to progress. There might be different versions of models, but like it's very clear that there is some sort of brain that is there. And we have the leaders in the frontiers and we have people trying to do different things. And that is all said and done. The tooling. I don't know if I would spend too much time. That's all we're talking about now, which is like the sandbox, the computer, the MCP tools. There's like a bunch of them that, that are being there. The thing that's not solved very well is one. So there's memory itself and so memory itself, like how do you do this? Right now the best solution is just dumping things into MD files and then, you know, sort of having access to that and then, you know, compressing that and how you're going to get that. We talked about this the other day, which was the book why we Sleep. Right. And in that book of why we sleep, it's basically a lot of how you do data retention and things like that. And that's sort of what we're trying to solve for agents, agents themselves into that segment. And so memories there. The thing that is actually quite interesting for me is actually the learning of the model. One thing that I didn't understand internalize is that models actually don't learn right now. So you use a model and if, even if you solve memory, it has memory of things, so it has context of these things. And so it can, oh, here's the context. And so it can have a better answer because it has the context, but it doesn't actually learn on the job that is done yesterday. Right. And so who is solving or how do we solve that? Does that one, does that require something like, you know, constant RL learning post training for this or is there something fundamentally that we change in the model so the models can do that? I don't know, but it's definitely something that will. That's hindering progression today. And it's not absolutely clear how we get to that point because it is sort of weird to work with someone that actually doesn't get smarter.
- [18:25]
Interviewer
Right.
- [18:25]
Ivan Borozin
Like you get a new model but like it's like, oh, it's like a new new one that comes out of. But like you do something five times and it can mess up. It literally screws up the sixth time. Like. Right. So like every single time really. And so you can try prompter better, you can have more context. So it doesn't do it, but doesn't actually learn. I think that is something that's going to be quite interesting to see sort of how that sort of progresses.
- [18:47]
Interviewer
Yeah. And from your vantage point, memory is one of those markdown files or like a combination of those versus database. Seems like markdown files are incredibly elegant and simple, but somehow feel less robust than what a database would offer.
- [19:07]
Ivan Borozin
I mean, not from a personal, but I might change this for a personal. I also would tend to agree with you that like markdown seems quite trivial for the, for the problem that's there and the compression and how you get to all these things inside of that. So the thing that we try to do as a company providing infrastructure for these things is to make sure that we can expose the ability to. For the agent to access these things in a very simple way and add to them. But solving it itself is outside of our, let's call it, mandate.
- [19:38]
Interviewer
And in this whole kind of agent harness, I guess everything that you described kind of falls under the current concept of harness. Where do sandboxes fit long term? Do you think a lot of this gets eventually built into the sandbox or does the sandbox remain the execution layer for it?
- [20:02]
Ivan Borozin
All two things. So if you look, there's like the sandbox itself. And again, restating I'm saying it so many times in this conversation, but if we take like an average worker in Goldman, for example, right, you have the worker, the person which let's call it the model in this space. You have the computer which it logs into. And the harness is a set of the way it shapes that model that it can and cannot do things, interact things. So it's almost like hands sort of to speak of that bit deeper. But basically the sandbox does support it. And so if you think of a computer, again, I'll pick on Goldman, for example. I've never worked at Goldman, but it's my just assumption, I've worked at a bigger company. So my assumption is that when you log into that computer, there is so much software in that computer that logs what you do, restricts what you do, makes sure that you don't leak data and do these things. Again, no prior knowledge of Goldman, just assumptions on these things. And so those are the types of things that we as a sandbox provider will most certainly incorporate into that. But that harness still has its function in that which it guides the model. Like, oh, I now know how to interact with this machine. Oh, I know how to do a tool called, oh, I know how to. This is how I work with memory, this is how I change with a model. And so there is value in that. And so we don't take on the harness. That is something we pretty sure that we never do. But there's things that we do in the sandbox that does help or support that entire system.
- [21:35]
Interviewer
Great. We talked about models. Where do the model providers like the big AI frontier labs fit in this overall picture? OpenAI recently at an agent's SDK announcements. What is that?
- [21:53]
Ivan Borozin
Yeah, so, I mean, they, they have a big push into this, this space where, like, Claude Code and their agent SDK has been. And so it's a focus for them to essentially catch up on that segment of. Of the market there. So let's do things right.
- [22:09]
Interviewer
This OpenAI agent SDK and then Cloud manage agents.
- [22:14]
Ivan Borozin
They're different. Yeah, they're different. Yeah.
- [22:16]
Interviewer
Unpack that for us.
- [22:17]
Ivan Borozin
The anthropic managed agents, it is where you essentially have the model, the harness and the sandbox all wrapped into one, basically. And so you have that all managed for you as a service, that entire stack. So that is there. Whereas if you just have an agent SDK, you can use that and run that into any sandbox provider or your own or any machine that you want to run it. And it can connect to, depending on the licensing or whatever, maybe you might connect it to that model provider that gave you that harness, or you might be able to interchange those things. So basically, the one is just the harness, the other is the model, the harness and the sandbox altogether.
- [22:58]
Interviewer
Okay. And I believe Daytona was a partner to OpenAI for agents as DK, right? Like one of the sandbox providers.
- [23:06]
Ivan Borozin
Exactly.
- [23:07]
Interviewer
That's part of that original framework.
- [23:08]
Matt Turk
Yeah.
- [23:08]
Interviewer
Okay. That's the sort of like, overall landscape around the agent stack. We're going to go much deeper into sandboxes and how that works from a technical standpoint in a minute. But as a quick detour, let's talk about your story and Daytona's story. So this is not your first venture. Talk about the prior Thing that you did, you alluded to some of it like that was a little bit like Replit.
- [23:37]
Matt Turk
Yes.
- [23:37]
Ivan Borozin
So we started the Cloud IDE space basically in 2009, both me and co founder. And so in 2009 for those who that were around, there was like no Docker, no Kubernetes, no VS code. And so we had to build the entire stack which was something that we learned how to do and something that we applied today. So like it was actually very, very, very early. The only I say that we started it because the company that started before us was Heroku, which became Heroku and killed the id, which we probably should have done sooner as well. So. But we were the only one that sort of like started and finished finished in that sense. Later on you had other ones like code sandbox like Replit and like others that that had had joined and stack blitz which is now Bolt and whatnot. And so Replit also now changed and Bolt into their other directions and Replit is doing really well. We decided to go in a completely different direction when we decided to do that next company. So we had learned a lot of things on how to build this entire stack. But when we decided to kick off Daytona V1, which is different than it is today, we decided not to go app layer, but just be infrastructure probably like on the teachings of Heroku, which definitely went into that direction. It's like, oh, we learned how to do all these things on the orchestration level underneath that seems to be very, very valuable. So let's push on that. And that is how we kick that off.
- [25:02]
Interviewer
You tweeted. My first startup taught me exactly what not to do at Daytona and to talk about like selling to developers. No on prem solution like talk about some of the.
- [25:13]
Ivan Borozin
We learned a lot of things. One thing we learned is timing. I never understood like timing of the market. Like you to pick. It's very hard to time the market. But you know, if you're in the right time, like fairly soon, ideally you want to be just before the time. But that's like very, very hard. You definitely don't want to be completely wrong. We were like two decades wrong or decade and a half. So definitely wrong on that one. The other thing is like I did not know the difference between a user and a customer. And so selling to Daytona now is a PLG motion 100%. The code anywhere product was also a plgmotion but the Code Anywhere production had no enterprise use case value. It was all single developer value and signal. Developers do not want to pay for These things. Whereas a product like what we are, it is a plgmotion. It gives value to a single developer, but a single developer probably working inside of a company and so they will pay with their company's card, which is very, very, very different. But you don't know when you start out. So those are definitely things that we understood when we were creating this company. And the difference we didn't get into detail, but the difference between Daytona V1, which was managing it was an infrastructure product for human engineers in very large enterprises, it was an enterprise Play. Whereas Daytona v2, the sandbox is a. We can call it a neo cloud, like it's a new cloud. Although neo clouds are usually for GPU clouds, we did understand that there was an enterprise play to be there. So everything that we had learned of like on prem multi cloud, different ways of managing observability, audit logs, all these things that you need for enterprises, we had already either baked in or understood that that would be needed. So we prepared the product for that originally.
- [26:59]
Interviewer
Great. You're also pretty amazing at just distribution in general and building a brand and developers, which is fascinating, right? Because that's one of the typical issues with very technical ventures. The people tend to be excellent and deeply thoughtful about product and technology and then distribution comes as an afterthought. How did you become good at it? And what are some lessons you can share for technical builders?
- [27:26]
Ivan Borozin
I mean it's all about like I'm probably sucked at all of that. Like a terrible, terrible like I was the word. So let's take. There's so many ways to say how we did. So one thing is I think I very well understand humans at scale. Like my, like what the market wants and or needs and or feels. And so when you do that it's like I did not know that inherently, but you sort of start learning that you understand humans. It's very hard for like single humans, not so much, but more like larger masses. Aggregate humans aggregate humans much better. So that's one thing, but the other thing is I was like super shy, geeky, whatever as a child. And the, the stage fright. Terrible, terrible. Like pitching my first startup. Like I would be so nervous. Like I'd be. I had to like Talk on stage. Five minutes before stage, I'd go to the restroom like 10 times. Like just like other pressure of these things. And the thing that happened quasi randomly is with our first venture code anywhere we did pitch around all these like conferences around the world and whatnot. And it was such an interesting thing that I Decided in like with one of the founders of one of these conferences at the after party, quite, you know, intoxicated probably. He's like, oh, do you want to do one in Croatia, where I was living at the time? And I'm like, fuck yeah, let's go do this. And so I go do, set this up, this conference, it's in Croatia, 250 people come, which was really big for us at the time. And I had hired an emcee that had bailed last minute. And so there was no mc. I have stage frame. You're right, I'm it. I have to go. And so you. I was literally the MC for two days in a row. And you broke. Like, you break. It's like you just have to have to do it. So what I'm trying to say, this is very different from the go to market, but it builds on. It's like, okay, now that I no longer have this stage fright, you start teaching, start understanding what interactions excite people less, excite people, how to bring people and whatnot. And it's also a problem because when we said we're into a conference, I didn't understand what a conference was. And so is like, how do you break down what is a conference? Okay, a conference is. It's entertainment first and foremost. And so you have, you know, the show, the show are the speakers. How do you get the speakers? Then you have the this for the speakers. You have to sell the audience. Who is the audience, who's coming there, how do you get them there, what? And then somehow to pay for everything. The audience does pay for tickets, but the vast majority is under sponsorships. And then how do you sell that to sponsors? And so you break those things down and then you start understanding incentives of all these different parties and you start putting that together. And so our go to market motion when we, and we did conferences for a decade, more or less, so you sort of like you have iteration cycles of a year. Then we started doing two a year and we started doing three a year. And then every conference got better because the iteration cycle was much faster to get. The feedback loop was much faster. And then when we decided to do like Daytona and our whole go to market strategy was originally around in real life events. And so we did dinners, meetups, drink ups, hackathons, but all of them were simpler versions of conferences. And mind you, the conference that we had was like 4,000 people. So it's like, it's a fairly big, it's not a hundred thousand, but still fairly, fairly big. And I remember telling one of my teammates that works on the conference business with me now at Daytona, I'm like, there's no way in hell I'm ever doing a conference. Ever. Not doing it. That's the most stressful job in the world. Never doing it again. Never. But we'll do these little things because it's easy. And then people would come up to us. How do you guys do these events? How do these meetups? Well, when you understand how to do a big conference, like, what are the interested parties? Then you know how to do, like, very small meetup. It's the same things apply. It's just like quite a lot smaller, a lot easier to do. And we ended up doing a bunch of these. They were all. We didn't sell, obviously these events do. On sale, but sold out in the sense of, like, there's no more space in the rooms that we were doing. We now, like, attract like partners that, you know, co sponsor these events for us. Just a great motion for us. And then at some point, my, my, my colleague was like, we have to do a conference. Like, no, we have to do it. And so we ended up doing that in the Chase center, but we ended up doing the Chase center in San Francisco two months ago. I think you were there as well. You helped out and the, the whole.
- [31:37]
Interviewer
Which was incredible, by the way. So when you think about marketing as, you know, an effort to make a company, I mean, create a brand and make a company look, you know, big and powerful, that was, you know, a masterclass in how to do that.
- [31:54]
Ivan Borozin
Thank you. But all of it, all the entire GTM that we have, we have no other things that we do. Like Twitter is a go to market. We don't have. We have zero salespeople and whatnot. But it is definitely. So the way I think about this, and I stole this from, or paraphrasing it from David from Century, which is like, there's basically three ways that people pick your product. And one is awareness. Do they even know you exist? Like, if that doesn't happen, there's no way they can pick you. Two is preference, which is, you know, the, the pricing, the brand, the person, the different features, whatever. Like, it's. It's preference. And the third thing is like, is there a deterministic thing that you offer that no one else has? So the easiest example for the third one is like, do you have Fedramp? Right? Do you have that certificate? If you have, like, I can only. The customer can only use you and no one else. But the Two above are quite interesting, which is like how to make sure that everyone knows who you are and then you work on the, then you work on the preference. And so for me, the all I don't. It's not all I think about. A large part of I think about is if all things were equal. So if our product is equal to everyone else's product, what is the differentiator to that? And so like one can more people know about you than others. Like is the branding there is the, the, the experience there? Experience is a nuance which some people don't think about, some people do think about, but it's like, do you prefer the feel of this product versus that product? And these are all things that have nothing to do with the actual technical capabilities of the product. And so my co founder, Vedran, our cto, like his job is mostly to make sure the product is better than anything else in the market. And my mandate is to make sure that even if his, if our product is equal, that we can supersede that. And so that's how I think about the go to market in general.
- [33:48]
Interviewer
Do you think of customer support and customer service as all of that?
- [33:53]
Ivan Borozin
It's all go to market. All of that is go to market. And we've seen this, we've chatted about this, where we've been getting users and customers just because we are so good at that. And so all of it is an experience. So if you think of any experience as a human, you go to like store, restaurant, whatever, it is, the entire experience, it is from your. The way what is a brand? It's the perception of the. That brand itself is just the perception. And the perception is like if you go into whatever store, pick your brand you want, the smell, the music, the people, the smile, that, whatever you get all of that together is the perception of that brand. And so if you think of that through the lens of a product, which might sound counterintuitive, I don't know, or non obvious to people, like I think about that altogether. Let's say we were selling sneakers, right? Like we don't sell sneakers. But the, the, this sneaker, which we won't, man, won't say other brands, but
- [34:49]
Interviewer
if you do, you can, yeah, you can always pivot to gpu.
- [34:51]
Ivan Borozin
Yeah, exactly. Like if you go to the store here in New York, it's like a beautiful store. The people are very nice. Every everything's aesthetically pleasing and you just like enjoy that entire experience. And it's a good sneaker, right? And so you have to have that all together. And so that's how I think about this as well, which is, you know, you have to have a good product, you have to all these things, but all these other things have to co alide with that. Now the risk is, and we've seen this with a bunch of startups, like there's no sustenance. So the product is not good and you have everything else and that is sort of, that's when you get into trouble. But if the product is good, or at least as good, if it's better, it's amazing. And you have all these things from. For me, my belief is that that is a key driver to continue growth.
- [35:34]
Interviewer
Great. You mentioned Twitter and X a minute ago. To which extent is that part of the whole. The whole.
- [35:43]
Ivan Borozin
Hard to measure.
- [35:44]
Interviewer
Yeah.
- [35:44]
Ivan Borozin
Like I've worked in bigger companies and if someone, if I was working for my former company, they're like, how do we measure that? I have no idea that we can measure that. But what I can say is that I was never active on Twitter. Very. I have an account since 2009, but never very active until like this holiday season where there was like this semi viral tweet that went out.
- [36:05]
Interviewer
What did you say again?
- [36:07]
Ivan Borozin
So the tweet was, if you're taking a break these holidays, you're ngmi, you're not going to make it.
- [36:12]
Interviewer
Yeah.
- [36:13]
Ivan Borozin
Which I honestly believed.
- [36:14]
Interviewer
Yeah.
- [36:15]
Ivan Borozin
Like the tweet was basically, I wasn't even thinking about. There's like a typo in the tweet. Like it's not people like, yo, you're rage baiting. It's like. No, I was. My thought is we are, we are a part of this like super cycle right now. And the super cycle does not last forever. And so if you're going to pause by the super cycle, you are seeding market. Like that is what you are doing. Right. And also it's like it's very much towards, you know, founders and executives and tech leaders. It's less about the individual person that, you know, that can't have an impact on in that like people have to take their breaks. And so people took that in all different ways. Or like two poop two boats, like you should die, you're a capitalist or whatever, all these things. And the reason why it became so popular, like it was kind of popular, which was some person said, dude, you never, I've never heard of you or whatever. And then I said, basically, that's why keep working. And that was perfect. Perfect. And again, not. I was like instant reply on that. And so basically not talk too much about that. Is that from then I had noticed that people, even if they like your tweets or don't or are completely opposed or not, has no bearing on, I should say it has that positive bearing on their thoughts of your company. So we have found that people that even are completely opposed to the posts end up being customers, assuming that they need the product line. Not everyone is there. So it was quite interesting to me that oh, just because someone doesn't agree does not mean that they won't like that because it's all generally awareness. And so that has been something that I've spent a bunch of time trying to catch up to you on the tweeter verse.
- [38:01]
Interviewer
Yeah, no, that's super interesting. Right. For any AI builder or AI founder. So the PLG motion ultimately to like to create inbound is a combination of all the things. So we talk about conferences and meetups. Twitter is important. Is there anything else that people should know? We talk about customer. Customer service. You mentioned no salespeople as of now.
- [38:24]
Ivan Borozin
We have no salespeople now. Yeah. So I think like the how do you experience the product? Right? So like once you've seen the product, how do you experience the product is like your door to your product. Like the website, the login, the whatever. We can fix a lot of these things. To be very clear, I'm not saying we're the best at this, but there's that. And then what are the feature sets are in there? Can I get things easily? Our SDKs are really, really, really good. Like people really like, like the ergonomics of them. So it's like really good. That part is great. And then the thing that. And this is even public on Twitter, all our case studies that we've done and we outsource the case studies to third parties, so we're not part of this is that we reply very, very fast. And so this is a very public thing. And it's something just core to who I am and how I learned to work. Because one of the, let's call it first real jobs I had was a system admin. So my job was like to fix the printer and computer and whatever. And so one thing that I learned and I try to talk to my entire team is like one thing you have to do is like one is the first response very fast. Just the first response very fast. Like people are then calm. They know they have transferred their problem to someone else and someone has acknowledged that. And so when they do that, they feel rest said. Right. And Then you just have to promise that you will solve or get back to them at X amount of time, whatever that is, and they were calm till that moment. And the thing that you have to do is one, either solve the problem until the given time or call them, message them, whatever interacts them before that time and state a new time, that is the solution to support that. Is it? There's nothing more than that. There's obviously there's like key things. If you're actually down and don't work and the person can't work, that's a completely different thing. But every other non critical problem and you have the most happy customers in the world because they don't have to think about their problem anymore. You think about their problem. And so you can like keep that on until you get it done. But the key part is if I said that something's going to be fixed or deployed or done or whatever in two days, a day and a half later, if it's not coming two days, a day, half later, I said, hey, this is going to be delayed two more days and they're fine. You thought about it before, they thought about it. And it's not magical. I mean it's a magical experience, but it's not very. It's not a complex thing. But I found that people don't understand that intuitively. And so that is what I learned back in the day. And that is what we do in the company today. And that's why we have very, very happy users and customers.
- [40:51]
Interviewer
Okay, fascinating. Now let's switch back to the more technical stuff. So we talked about the concept of sandboxes. Just walk us through what a sandbox is technically.
- [41:05]
Ivan Borozin
Yeah, there's a lot of things, there's a lot of things there we can talk about. The way I think about sandbox is the ergonomics of consumption of compute. And so because there's a lot of different layers on that, I basically put into three different layers, which is the infrastructure, the primitive and the tooling. Those three things together. And what I mean by infrastructure is like is, does it spin up very, very fast? Can you spin. So like we spin up in 60 milliseconds. Can you spin up a lot of them at once? So the, the rate of creating them. And so we can spin up 50,000 in 70 seconds. So a little less than a minute and a half and then once they're up, how many can you keep running? Like we can have, we have customers that have billions a day. So those are like the infrastructure parts that are non Trivial.
- [41:54]
Interviewer
Why does it matter how quickly you can initialize a new sandbox and how many?
- [41:59]
Ivan Borozin
Depends on the user. Again, it depends on the user and the use case. So if you're like a long running background agent, again, everyone prefers it to be fast, like no one wants to wait, like you don't want to wait for a reply. Like everyone wants to be fast, to be very clear. So the faster the better. But generally there's an actual reason why you want it very, very, very fast. And that is especially if you. So for. I was gonna say for a background agent, a background agent might work for like 10 minutes or an hour or whatever. So the incremental millisecond might not matter. But I still believe that from a user perspective, even a second of waiting is kind of uncomfortable. You don't want that. And so it's 60 or 90, maybe less so but there you want that one, two seconds for sure under that. But the interesting, the more interesting part, where that is really, really important is for the researchers where when you're doing reinforcement learning, you basically have an allotment of GPUs and you want. The GPU is more expensive than the cpu. So the vast majority of sandboxes are CPU boxes, to be clear. So they're the computers that we all work on. There might be a graphics card in there, but basically it's the compute, the ram, the CPU and the hard disk that's in there. And they are cheaper or less expensive and easier to get, at least for now. We'll see for how long that lasts than GPUs, which means you want your GPUs always to be at maximum utilization. And the CPUs can then idle if need to be idled. But you want them, you don't want the GPUs to idle. And so what that means is you want to make sure that the CPU machines, the sandboxes, spin up so fast because you don't spin them in a training run. You don't spin them all up. You spin them on like depending on how many you have, a thousand at once they do the task and do the next one, the next one, next one. And so between turning off and on the CPUs, you want that time to be as short as it can so that the GPU utilization does not go down. Right? And so that's why that's very, very, very important. And obviously the number that you have concurrent so depends on, so on the background agents, let's say, just for the sake of people know, lovable. So the amount of users that they have is astonishing. And so each user for every task, and they can have multiple tasks, multiple agents. They need a sandbox. And so imagine, I don't know their user number in the millions, whatever. So you have to have millions of sandboxes up and running on the RL side. You can have people, smaller labs that will have concurrent 5,000 or 10,000. But we have a request today for 5 million concurrent. Look, so 5 million. And at 1 point in time, right? So being able to handle those types of things is part of that infrastructure segment. The other two are the primitive and the tooling. And so the primitive is essentially, let's call it the computer itself, the isolation, the vm, the container, the micro vm, the isolate, the whatever. We can get into details what those are, but there's different shapes of them. And also what feature set do these things have, right? And so for the most part, a VM that you would find in, you know, AWS or whatnot is usually much lower to start, much harder to get those spikes. But also when you get a certain size, they're usually that size forever and forever. And then we just want one example, whereas in Daytona, you can define a size, the cpu, the ram, the disk, and then while it's running, you can resize that. So if an agent gets to, you know, use the entire memory or 100% CPU, the sandbox would. Or VM would die. In our case, we can expand that. And there's other things. It's not for us. It's not just a Linux CPU box. It could be a Windows, a Mac, an Android. It can be, you know, they can have a GPU in there. It cannot have a gpu, depending on what you need. And so that's the primitive. And the last thing is the tooling. The tooling is the tooling. Tooling is either there to support the agent, to be better at its job, or to be guardrails on the agent to stop it from doing dumb stuff, basically. So you can think of it, we have a bunch of tools, so headless, you know, terminal and file operations and other things that enable it to use less tokens, get the job done faster. But also guardrails like Secrets Manager and a firewall and all these other things. And so the combination of those three, the infrastructure primitive and tooling, essentially creates what we call a sandbox.
- [46:09]
Interviewer
So ultimately, what's the difference between sandbox or container vm, micro vm?
- [46:16]
Ivan Borozin
All the things, all the joy. So most sandbox providers are Firecracker VM micro VMs. Most.
- [46:26]
Interviewer
Let me remind people what firecracker is.
- [46:29]
Ivan Borozin
Firecracker, it's an isolation privilege. So it is essentially a type of vm, a virtual machine that's very stripped down. It was made by the AWS team, it's used inside of Lambda. And so the idea which are like functions. And so the idea was to have something very, very fast that can spin up and stateless. That was the entire time. And they were very, very ephemeral. And so they were used for, you know, when your website has a large, if it's Black Friday, a bunch of people hitting your website, you can spin up a lot of these very, very fast, like handle the load and enable all these humans to, to look at the website and buy whatever they were buying. Right. And so that technology is there and it's a very stripped down version of a full on vm. So it has less features there. But the things that it does, it does very, very, very, very well. So you can spin them up fast. It does that because it can sort of lazy load things into memory. You can do point in time snapshots, you can do all these nice things. The things that I can't do is it can't run a, call it sandbox now. It can't run a sandbox with a gpu. It just doesn't work. So you can't have a firecracker. So if you want a gpu, you have to do something which is a cloud hypervisor or a QEMU spelled Q E M U which are two different types. Qe. QEMU is almost a full vm. So it has all these different things inside of there. So you can't do that. But let's. For example, if you need to run. We were solving this for one customer earlier today is they need a machine that also has an Android device inside of it. And so the only way we could solve that was inside one of these key moves. Like it can't work in a firecracker, can't work in a container, it can't work in anything else. And so there's different types of there. These are like the micro VMs of the world. There's also containers and so container. Most people know Docker, which is there. Daytona originally started running Docker containers. We do have them. The problem with Docker is that they're much less secure than a vm. The thing that we do in Daytona is we harden that with Sysbox, which is vm, like isolation that wraps around that. It's really good for handling density of these machines, it's very fast. It allows us to run Docker and Docker. There's a bunch of features that are very. That it's very, very good for. And so there's differences there. There's also like isolates, which we've talked about, or like other app containers, which is abstractions of these containers. Just you're running a single app. And so these different things all exist in the world and they can all be theoretically sandboxes with these other two things that are there. And our original idea is to get back to the original conversation. Most sandbox providers started as firecrackers, but as we start seeing that agents have different needs and different use cases, they are going to need all these different shapes consumed through one interface. And so we at Daytona now support containers and all the micro VMs. Depending on your use case, the user doesn't know it's the same ergonomics. It's like, oh, I need a Windows and I'll spin up this one, I need a Linux, I'll spin up that one. And so we will continue to add these things inside of the same infrastructure. Sometimes it'll be slightly faster or slower, but the concurrency will be there, all the tooling will be there. And so everything that you would come to know and like about a Daytona will be there. But it'll be different sizes and speeds and configurations depending on your use case.
- [49:58]
Interviewer
Why are snapshots and forks important?
- [50:02]
Ivan Borozin
There's a lot of reasons. Let's get into the commercial one, which is probably the one that people most think about, which is pausing sandbox. So if you are an app layer company and you have 10 million users, and your 10 million users spin up, let's just say 10 million sandboxes, just to make the math easier. These things cost, right, because they're running, they're using cpu, RAM and hard disk the entire time. The most expensive thing is the cpu. Second, the RAM disk is almost free, it's very inexpensive, and the user sends the agent to do something, the agent does something, and now it's waiting for a reply. It could be waiting for the human, or it could be waiting from a service, depending on what it's trying to do. You ideally don't want that sandbox to run idle, because you are now, regardless if you're a customer or Daytona or running your own. Like, there is a cost to running these things. And so what you want to do is pause that and wait for a reply either from a service or for a human, and then resume and the feeling and experience is that it never turned off. So you feel like it never turned off, but it actually did turn off. And so that one is from a, from a compute management perspective, if not from a cost perspective is probably the first reason. The second, the second thing is you can enable your agent to take multiple paths. So your agent can say, oh, at this point in time I'll take a snapshot and then I will either continue and be able to roll back to this point like a point in time, or at this point in time I will replicate the sandbox and have two sandboxes and I can try two at the same time. Obviously it can be more, but. Right, but those are the reasons why you would have that there.
- [51:40]
Interviewer
You mentioned performance and speed and in connection with that you said that you had to rebuild your own scheduler.
- [51:48]
Ivan Borozin
Yep.
- [51:48]
Interviewer
So what is it? First of all, what does a scheduler do? And then how did you go about it?
- [51:52]
Ivan Borozin
Every cloud that exists today, Neo Cloud, hyperscaler, whatever, they are all built on servers, right? Like metal machines, I don't know, like historically, way back in the day, we actually stacked these data, these data centers, me and my co founder, a long, long time ago. But they're all just like servers, like cpu, ram, disk, they're computers basically. And on top of these computers there is a software stack that everyone has built for their own reason. So AWS has their software stack, cloudflare has theirs, which is very different. Everyone has, has their own on top of that. And so basically what you're trying to do, when you think of these machines, these servers, basically what we do is we cut them up into small little machines and then give you that sandbox. And so you have these small on these big servers, you have these small little machines that can run for a minute, a second, three hours, whatever, and you don't have to worry about this. And so basically the scheduler or the orchestrator is the one that basically says, oh, after you send me a request, I send you to this server and turn on that sandbox, a little computer there and then I turn it off or I kill it or I snapshot it, or it's the management of all these things there. And most of our other companies in the space basically have off the shelf schedulers. So it could be like Kubernetes or Nomad or whatever. And when we decided to build Daytona the sandbox product, we inadvertently, with very naivete, we're like, oh, this stuff doesn't work. Because we had worked with all, we had built our own Then we had used Kubernetes. Like we've seen all these different things and we knew what was good and was bad or what was use case or not. None of these were made for these like super fast stateful, long running machines. And so we're like, we have to, we have to do this again, we have to build it again. Because the way we thought about it, which was not known at the time, is because at the time sandboxes were very ephemeral, similar to a lambda function. And we're like, no, why would your sandbox be FML by default? Like your laptop, you don't want it to die. Like you want it to work until it's done. And it has to be very fast. It has all these things. And so my co founder mostly built this sort of initial version of our scheduler. And that scheduler of ours is the basis of everything that we do. And it gives us a lot of things like outside of just like the performance, it also enables us to do things like have four different isolation products. Like we are the only company right now that you as a user don't know. But like we have, depending on your use case, we will spin up any one of these firecracker, cloud, hypervisor, whatever to get the job done. So whatever is needed, we'll get there. The other thing the benefit is we built this, we have colocation providers, so we have like bare metal machines. So you can think of us as our own cloud, but we're more and more akin to now like the Neo clouds, like the base 10s and like the fireworks. Because for us, and we've got to this question yesterday because we have a large request of number of CPUs is like we can't get enough. And so how do we solve that problem? Well, we've already solved it where our scheduler can be attached to any CPU machine, any server that we think is good enough and it becomes part of our cloud essentially. And so why I say base 10 and fireworks is that both of them run on more than a dozen compute providers, depending on what we're going to call them. And so we can do something similar because we've created this in such a fashion. So that has given us, I believe that's given us sort of a advantage there.
- [55:25]
Interviewer
And is there a fundamental technical difference between ephemeral and long running? So if you have an agent that runs for 24 hours, how does that translate in terms of sandbox requirements?
- [55:38]
Ivan Borozin
The reason most sandbox environments do not run forever or even is because it's a technical problem. If you think about servers underneath, these servers also have to be managed, right and maintained. And so if your sandbox can run forever, that means that you can never reset, you can never reboot the underlining server, you can't update it, you can't patch it, you can't do all these things without turning off all the sandboxes. And so the way you solve that, the easiest way to solve that is having sandboxes that have a termination time. It's like they will only last an hour 24, it doesn't matter. Pick your time. And then if you decide that you have to do something with the underlying machine, you just. Because you just flag that machine as non schedulable. And so at some point in time there's nothing else on that machine you can do what you can fix it, you can like reboot it, you can do whatever you want. Easy peasy done, right? And so because historically most of the workloads were ephemeral, you didn't have to try to solve that problem because you didn't care. Like most workloads, like a lambda function usually runs what, five minutes, ten minutes, like whatever. It's not a problem, right? You never had that restraint or constraint now that you do, to have something that can run forever, you have to be able to live migrate the sandboxes itself between the machines so that you can reboot and manage these machines. And I understand that most people, it's interesting, even technical consumers are like, oh, all compute is elastic. Well yes, your consumption of it from us or from AWS is elastic, but someone actually has to like screw together a server, turn it on, boot it, make sure it works right and have enough of that there. So that is the difference from our perspective and why most start off fmrl. It's an easier way to do it, we believe and we've seen this, I think two and a half percent of our revenue now comes from or two and a half percent of our sandboxes run longer than 24 hours. But it's a large percentage like 20% of revenue. So like it's not an insignificant part to be there just because it can run for a much, much, much longer time. On the user perspective, just to finish on that sometime users actually want it to be ephemeral. One is they don't want any data retention, so they want it to run and die. Like whatever was run inside of that, they got the data they wanted and they want that data dead. Dead. And so they flag it with us as ephemeral. Which means that we won't force it to die, but when they kill it, it's no longer. It goes into the ether. It does not exist anymore.
- [58:10]
Interviewer
Listening to this whole technical, deep dive part of the conversation, the thought crosses my mind that a lot of people seem to think that they can create their own sandbox. And it is not that hard. But again, listening to all of this, I'm like, good luck.
- [58:26]
Ivan Borozin
No. So that we've seen this multiple times, and I'm sure you've seen this across other companies as well, where, you know, everyone's like, oh, I can spin up a firecracker on a machine. Yes, you can spin up one, but spin up a million is a problem. Adding all these features is a problem. Having throughputs, different use cases, they're all different problems. And what we already see is like, companies that we're like, oh, we're due ourselves, come back three months later, six months later, eight months later, it's like, oh, actually, actually now we can't do X and Y. And so there's a bunch of things on technical side which we can, like, address. And I think we could finish with that. Is that the thing that Daytona does is we. If you think of most set most sandboxes, this is a performance thing. Most sandbox in most VMs, actually, they use the CPU RAM from the server and then they use the hard disk from a network drive. You do that for a bunch of reasons. Your life is so much easier if you manage that. Why is it easier? Well, it's easier because if a server dies, then all the data is on a shared drive and I can reboot that server really fast without maybe even the user noticing. And there's no data loss, maybe something in the memory, but nothing there is lost. So it's very, very easy to manage. Whereas what we do is we actually use the CPU RAM and hard disk of the machine, which means we have so much more overhead of if that machine dies, how can we reboot it back? Like, you have to do backups that users don't see. You have to do all these things that happen there. But what that means is that the iops, the speed with which you can move data to the hard disk in which the CPU and RAM interacts with from a network drive just to give people numbers is in the hundreds of thousands. If it's a local drive, it's in the tens of millions. So just like it doesn't matter if you know what the numbers are. It's just like extraordinarily Large. Right. It's very, very different. And so you have use cases that no one really cares about that speed. Which is fine. We have new customers that have a use case. They're like, it has to be that fast. Right. And so when you get to the point, it's like, oh, I can use this, but wait, now I need this performance. Oh, now I can't do that. Right. And so to the complexity of sandboxes, if when people are starting out their use cases, like, oh, I just need to run code. You can run that on anything. It's very easy to do except scale. And then when you get to scale and then you need performance, because now, oh, agents can do a lot of things, then it starts like breaking that sort of mold and security. Yeah. And so we didn't even touch the security part. But yeah, great.
- [61:04]
Interviewer
Zooming out as we close this conversation. You mentioned something that caught my attention. So we have been in a very well documented GPU shortage, but it seems that we might be heading towards a CPU shortage.
- [61:21]
Ivan Borozin
Yeah, it might be happening. So one of your former guests and was at our conference, Dylan Patel, like semi analysis, they wrote the report and I believe it's somewhere like October, we have no more CPUs and people hadn't thought about it. Like it's all GPUs, all GPU intensive, then memory. But basically now that agents need all these computers. Like it is very much needed now that RL is the way that we've gotten most of the progress in models over the last 18, 20 months. You need all the CPU machines to do this. Now that the GPUs are more powerful, they can do in parallel more of these CPUs, CPU boxes. And then you see this. There was a tweet that I put a while back and is that you should definitely invest not financial advice. You should invest in intel and look at their stock price. Right. And so like there's definitely need for that. And it's something that people didn't understand or intuitively that you would need there. So it's something that we think about quite a bit, is like, can we alluding to also how our product is created? How do we make sure that we can match the demand of customers for the CPU that they have? Because I don't know, it goes to the extreme to where GPUs are, because that is very, very, very extreme. But it is quite highly high probability that there will be shortages of CPUs going forward.
- [62:46]
Interviewer
How do you think all this agent stack evolves? Does it feel like we have all the core pieces together. Or do you think, like, in two years, the overall architecture may look very different?
- [63:01]
Ivan Borozin
So we talked about the stack there, and things that have to be. The identity part does need to be solved again. It's basically solved but not solved. So there's a lot of things to do there. I'm actually also wary. Wary. I think about. People think that the models that we have, the technology for, the models that we have is the end state. And I really, really don't think that is true. And that would probably be the most surprising thing, is we have a new type of model that is just better or different than what we have now that either one needs less compute or doesn't use GPUs or doesn't like. I don't know. There's things that we've seen where. We've seen things with wetware where people can have fake human brains and it learns how to play doom. You have the. What was it? The fly brain that was replicated inside of a computer that then works like a fly. Like, can you replicate a human brain? And I say this not to replicate myself, but can you. A generic human brain. And then it's like an. It's an AI that is essentially equal to a human in AI. And so what does that mean for, like, all of technology? I mean, I'm saying. I'm now saying extremities for people. Not get me wrong. I don't think that happens tomorrow or like, that I'm completely crazy. But, like, I don't think that this is the end all of the state. The question is, does that happen sooner or later? Do we keep our Transformers, the vast majority of intelligence, going forward? And if it is, then it continues probably going directionally where it does. But is there something that happens that changes that fundamentally, Something that I think about? Because then all our bets, yours and ours, are very, very different. Right.
- [64:48]
Interviewer
Okay. Well, Ivan, that was fabulous. Thank you so much for spending time with us today.
- [64:53]
Ivan Borozin
Thank you for having me. We're great.
- [64:56]
Matt Turk
Hi, it's Matt Turk again. Thanks for listening to this episode of the MAD podcast.
- [64:59]
Interviewer
If you enjoyed it, we'd be very grateful if you would consider subscribing, if
- [65:03]
Matt Turk
you haven't already, or leaving a positive
- [65:05]
Interviewer
review or comment on whichever platform you're watching this or listening to this episode from.
- [65:10]
Matt Turk
This really helps us build a podcast and get great guests.
- [65:13]
Interviewer
Thanks and see you at the next episode.