← Latent Space: The AI Engineer Podcast![[AIEWF Preview] CloudChef: Your Robot Chef - Michellin-Star food at $12/hr (w/ Kitchen tour!) — Latent Space: The AI Engineer Podcast cover](https://substackcdn.com/feed/podcast/1084089/post/186632792/a7257626fb81c69e296b7ca02f9cfc9c.jpg)
Latent Space: The AI Engineer Podcast
[AIEWF Preview] CloudChef: Your Robot Chef - Michellin-Star food at $12/hr (w/ Kitchen tour!)
00:20:50
Loading summary
Transcript32 lines
- [00:04]
A
Okay. We are in the remote studio with a very special podcast. We actually recorded a while ago, a tour of Cloud Chef's kitchen. But we wanted to record a little bit of an intro in our remote studio so that we at least get a nice audio podcast intro to the company. And we're here with my friend and co host, Vibru Sabra, and as well as Nikhil, who's the founder of Cloud Chef. Welcome Nikhil.
- [00:29]
B
Thanks for having me. Sweet.
- [00:31]
A
Okay, so yeah, welcome back, Vibu. But I think by the time this launches, like people will have heard the anthropic podcast that we did. But yeah, so I think the headline that people will see when people see Cloud Chef is that it is an AI Chef. You had this like pretty viral video on Twitter recently, you know, when you launched and told everybody. But also people don't know that this is a real restaurant. Like you actually run a real restaurant. You can order food on, I think Uber Eats.
- [00:57]
B
Yeah.
- [00:57]
A
And it's really good. Yeah. So like, what is Cloud Chef like, what is the scope of it? How do you pitch the company at.
- [01:03]
B
A very high level? What we're trying to do is we want to make high quality, nutritious food available to everyone. And the reason why it's possible to even think of a future like that is because you can automate practically all non managerial work inside a commercial kitchen with culinary intelligent robots. And culinary intelligent robots is basically just robots that act like human beings, learn like human beings and work like human beings, or work like human chefs. So we actually have our first, the, the video that Sean was talking about is the launch video of our first robot. And basically it's a robot that has a mobile base, has two hands, goes around and does work inside a kitchen. So it's just like how you would hire a human employee or a human chef. You would hire our robot. And the robot will come to your facility, cook and come to your facility, learn recipes from the chefs inside the facility with one single demonstration and cook that dish over and over again or participate in that workflow over and over again like a human employee would. And then you pay the robot an hourly wage like how you would pay a human. And so far our robots are used by like Michelin star chefs. They're used by fresh fast food restaurants, airline caterers, a whole bunch of like commercial facilities use our robots as hourly wage labor as compared to as early wage labor as compared to buying a robot. The thing that makes our robot special is the fact that it can execute at chef level or it the fact that it has culinary understanding better than even the best chefs in any single cuisine. Like what that means is okay, if you, if a robot is going on. If a robot is cooking, it needs to know, okay, how brown the onions are, how far along you are in the cooking process. If you're cooking it in a slightly different appliance, like what state is the recipe in, how much heat do you give it? All this like thermodynamics modeling of cooking, understanding visually what's going on. This is what we call culinary intelligence. And this is something that was not possible until like recently when like multimodal models got good enough. And we basically built out some thermodynamics modeling to aid that. And now the end result of that is a robot that can reason and make decisions in the real world in cooking processes like a chef would. More and more robot foundation models coming up, them getting better. These robots are finally also able to do real actions or real motions inside a kitchen. Right now they're good enough to only do stuff like gross manipulation where you, where you don't. Like if a human requires more than two fingers or three fingers to do a task, the robot's probably not able to do it. But the good part is most tasks inside a kitchen can actually, actually be done with just two fingers. So if you go around any commercial kitchen and if your two fingers had enough strength, you could probably do most tasks inside that kitchen. So we start with line cooking which is like the biggest labor cost for restaurants and restaurants and other food producing facilities. And our robot is able to do line cooking for about 40 to 50% of the world's commercially valuable cuisine. To a point that if we put our robot against an expert chef in that cuisine, our robot is able to consistently make the food better than even the chef whose source recipe it is like it's something that computers just do inherently much better than like the human brain. So that's a quick overview on this. Basically we've trained our in house models to, we've trained our in house models to do like thermodynamic perception and we've, we leverage current VLMs and voice models to do perception and also to enable the robot to do tasks or for the robot to actually talk to human beings, interact with other co workers in the facility to course correct its goals and whatnot. So that's a quick overview on what we are. The high level goal, like I said, is to replace all non managerial work inside commercial kitchens with culinary intelligent robots. And when that plays out, we think we'll all live in a future where we all have access to really high quality food at fast food price points, at McDonald's price points. You should be able to eat the tastiest food that you've ever had in your life. That's the thing that we want to create. And we think that now that the robots have started to work in the real world, we see a future in which we see a soon enough future in which we will make that possible. And to Sean's earlier point, we actually started experimenting with these robots in our own facility. So we basically just built an in house delivery kitchen at our office in Palo Alto. We weren't expecting it to do this well. I mean it just like picked up really well on doordash. We just, and we had moved from like my, me and my co founder, we had moved from India to Palo Alto and we were actually just missing really high quality Indian food here. So we just went to our favorite restaurants in Bombay and Delhi and asked can you record your recipes? We'll serve them in California and we'll give, give you a royalty. But that's not the core business that we' focusing on. It's just something that we use to validate our technology and the fact that it's doing so well and the ratings are so good is just a testament to how the tech is and how good the robot is functioning right now.
- [06:49]
C
I can confirm we tried the food. We'll we'll see later in the video. It's really, really good food. I like the term you use there, Artificial culinary intelligence. ACI has been achieved internally.
- [06:59]
B
Yes.
- [07:00]
C
It's interesting though because when you, when you frame it like that, you guys are doing something pretty different than like robotics. Right. Like you mentioned that the robots are more so off the shel of parts. It's not like specialized robotics. It's actually the software underneath.
- [07:12]
B
Right.
- [07:13]
C
So yeah, we can talk a bit more about that.
- [07:15]
B
Correct. So when we started the company we had one core ideology which is that we will only solve problems that can be modeled as software problems. And culinary intelligence was the first like big like culinary intelligence and decision making was like the first big open problem that we could model in software and solve. But when we had started robots, you know, robots were still like very much an electromechanical problem. Like they weren't really a software problem. And now with robot learning models and robot foundation, robot foundation models, it has gotten to a point where you can now start modeling the physical actions that somebody does also in the software and solve it in Software and have software iteration cycles. And we didn't want to build any hardware. Like we didn't want to be a hardware company because that was not our, like we weren't good at it. So we just didn't want to do hardware and we didn't think that the hardware hydration cycles would be beneficial for a company like this. Very recently it has gotten to a point where you can just take off the shelf parts, put a bunch of like robot intelligence, quote unquote, I'll expand on that later and get the robots to work. And that is a software iteration cycle. You don't have to build your own hardware, you don't have to get into what motors. Like how do we manufacture our motors? How do we manufacture these robots? Are we design like all, all those open questions. We rely on the ecosystem for us to solve and then we basically source general purpose robot parts and, or general purpose robots and these general purpose robot and write software on top of these general purpose robots. So we take general purpose robots, we leverage all the general purpose intelligence like LLMs, LLMs, VLMs, robot foundation models, and then build this proprietary culinary layer on top which has everything to do with like the thermodynamics model, cooking, custom evals for like manipulation, understanding, like understanding through perception what stage of the cooking process you are in. All of those things we've basically built and we are hoping to serve the tide on both the advances in multimodal models and robot foundation models and use general purpose robots as like the vehicle to make that happen. So that's, that, that's in a nutshell, how the, how our approach kind of works. And we are very focused on like, like I said, modeling every part of the workflow as a software process. And, and now that we are able to do it, we are able to basically do full stack work inside a kitchen and not just be a assistant robot that can be prompted by somebody on site or just a guidance system that tells humans what to do. Like it's now able to do full stack work because the entire workflow can be modeled as a software process.
- [10:06]
C
I was gonna say, I know that like, you know, we, we see how the robots work and what they're doing under the hood later. But the one, I guess, like, overall question that I'm sure a lot of people have is like, what's the business model? How do people kind of hear about this? How do you see like, you know, rent a robot for $12 an hour versus hire a chef? How do you, how do you come up to this hourly rental and all this stuff. It's very cool to see and, you know, good to see it works. Yeah. I'm just curious, how does that side of the business.
- [10:32]
B
So from a business perspective, the main thing to keep in mind here is that food prep is the most labor intensive industry of all labor intensive industries here. And again, quantitatively, the way you measure it is how many full time employees do you need per million dollars of revenue generated. Food requires about 13 people per million dollars of revenue generated. And the second most labor intensive industry is hospitals which require four people per million dollars of revenue generated. So food is like three times or more than three labor intensive. As the second most labor intensive industry, labor costs are just like going through the roof. And labor costs have been increasing year over year and depending on what Trump does with illegal immigration could go even higher. And the start turnovers are really high. The average restaurant is operating at like 130% staff turnover. By the end of 10 months, you're practically your entire staff is new. So high turnover, very high cost, and the most labor intensive industry. And because the reason why we landed up at this price point or like this sort of pricing models is because food service is not a very profitable industry. So they don't have free cash just lying around to do experiments. And there are no fixed, there aren't like fixed budgets set out for buying new robots, testing things out. If it doesn't work, it doesn't work. They don't take that sort of an attitude. Whereas there is a very readily available labor budget that we can tap into. Just like when you hire somebody, you don't pay for their college tuition, you just pay them a salary. We thought, like, why should that be any different for robot robotics? And these robots are now cheap enough to a point where you are able to put that business model out there, not lose money every robot you sell. So the robot costs, basically the point is the robot costs have gotten to a point where an hourly labor price pricing model works. And the robots are also good enough to now do the entire chain of work so that it's possible. And at $12, it's like 40% of what loaded human would cost. So our customers get their ROI on day one. The robot starts working from day one. And over time these robots get better. The hope is that at some point they also even start making better, like the food at any given facility, that they're cooking substantially better, not just by cooking the same thing, but then enabling the facility to like recipes that they weren't able to do. Earlier.
- [13:03]
A
Yeah.
- [13:03]
B
Awesome.
- [13:03]
A
I think the last part, you know, we'll cut right into the kitchen walkthrough video later. But the last part I think was, I think there's this general goal of demonstration learning, right? Learning from experts, learning from the Michelin star chefs. How realistic, you know, is this like a marketing promise or do you really just learn from one example? Because obviously food is messy, right? Food needs a lot of different demonstrations. How realistic is it?
- [13:28]
B
So I want to clarify that, I want, I want to clarify two things. One, it is not a marketing thing, it's actually true. Two, the reason why it might feel counterintuitive is because our entire pipeline is not one end to end model. If you had one end to end model and you had to train it to do a new thing, being a one shot learners is a very big deal. But in our case we have many AI subsystems that work with each other. Some of those are end to end neural networks, some of them are hard coded software pathways. So we basically use the best of both worlds to function and sort of architecture choice means that we don't have to like, we're not going in from like pixels of what a chef is doing and text or whatever to directly a generalizable recipe that can be cooked across any robot, any timescale. It's. There are like software workflows and pathways before that that take this chef demonstration, convert that into chef demonstration, convert that into like an intermediate format that is easily digestible by different parts of our system. And yeah, I mean one example for you would be so say if you're like making an omelette and if you want to teach it how to make an omelet, what we are doing is we are not learning a new omelette making skill while we are, while you're showing the robot the sick. If we had that capability, we would be a robot foundation model and we would already be single shot learning it. Is that from a single demonstration? Assuming that we have all the base skills for the robot to do it, it is extracting what kind of decisions the chef is making. Is it visual, is it thermal? What kind of skill the chef would be in, is invoking in themselves? Is it like stir, saute and what are the parameters to those skills? So for us, learning is basically configuring this AI system and not going into a end to end model that's going directly from pixels to robot actions. We wouldn't be to do a single short recipe learning. We would have to have the chef cook the recipe in various different backgrounds, various different sizes, various different appliances. Because we have these engineered midpoint, like engineer engineered midpoints, go from one expert demonstration to form a 2. A recipe form that can then be recreated across different kitchens, different appliances and in the future also different robot morphologies.
- [15:57]
A
Ooh, other, other robot morphologies. It's fun.
- [16:00]
B
Yeah.
- [16:00]
A
Now you only do the two arms, right? Okay, so we'll get, we'll get people to call to action and then we'll, we'll cut to the video. You are going to be at the AI Engineer World's Fair next week. Well, if people want to see the robot live, they can, they can see it there, probably taste some food, although I don't know how much food we can serve.
- [16:15]
B
Yeah, we'll see, we'll see.
- [16:17]
A
And obviously I think part of the reason you're doing this is you're trying to hire, right?
- [16:21]
B
Yes.
- [16:23]
A
This is a immediately applicable use case. Like, you know, what's the pitch for engineers?
- [16:32]
B
There are only a handful of applied robotics companies that have a path to deploy more than 100 robots in the next one year. And now that our robot is working and we have early signs of it being super helpful to customers, you will actually be working on a robot that is in production. There are people in robotics who are working on more complex hardware, more complex hardware problems, more complex software problems. But I think we are at the efficient frontier of value being delivered to the customer using cutting edge techniques and having a rapid scale up pipeline. I think that's not a lot of companies can say that they have all these three things and cooking. Like I said, it's. I think we have a very powerful mission in the sense that today the food that you're eating is fast food. Most cheap food is fast food. Whereas fast forward 10 years you can eat very high quality food. And in fact if you work with us, you can already eat very high quality food in our office. Ashan and Vibu can confirm. But I think coming back to it, I think the mission is very powerful. I think if you are excited by creating value in the real world while also doing it in a way that serves all the current capabilities of the state of the art general purpose models, you're probably one of the maybe two or three companies that are at the intersection of that. And if that excites you, you should come talk to me.
- [17:56]
A
Yeah, I think that's actually a pretty very, very strong pitch. I would say that you can actually even just try the food on the, on Uber Eats. You can just kind of order here it's one of those, like, virtual cloud kitchens. And it looks so good. We've tried it. We'll cut to the video later. But yeah, thanks for jumping on and sharing your journey with us. I think this is very exciting. I think you've somehow found the way towards the most immediate applicable industrial use case of robots. And there's obviously a lot of scope for vision, language, models and solving a lot of hard engineering problems. One thing you didn't say is all this is within very tight engineering parameters, which I think is pretty hard. You have to run it a lot of frames per second and also do a lot of that on device. So. Yeah. Cool. I'm looking forward to see you next week at the. The conference. And we'll cut to the video.
- [18:47]
B
Now, all the culinary decision making is like 100% autonomous and the actions are 90% autonomous. Water has gone off or like the probability of all water going off is more than 90%. And we have safety filters like that. That's what makes it deployable. Otherwise it's not very deployable. So basically how appliances are controlled is we go into, like any. So any appliance in any kitchen is controlled, either using a knob or a touchscreen. So we go in, remove the knobs that control the appliance, and then put these knobs in that can turn themselves.
- [19:43]
C
Oh.
- [19:44]
B
So every. That gives us a actuation surface across all appliances. So we don't need to teach our robot to. For a salary, huh? Yeah. So the. Yes. Yeah. So 12 bucks an hour is what you pay this? No. Capex. Yeah.
- [20:02]
A
Oh, Is there anything else that's going on? Like any other equipment?
- [20:06]
B
I mean, for example, the basic. The other thing is all the ingredients that are required basically get measured in these weighing scales. Okay. And regardless of which kitchen we go to, all kitchens store their ingredients in boxes. We just slap. Yeah, we just slap a bunch of cure it.