A (4:22)

Right.

B (4:23)

And maybe do it at the same time with speaking. And all of a sudden a thing becomes an animated companion and it, to me, it still feels like a bit of magic.

A (4:35)

Do you find? I mean, generally when people talk about, you know, robots, robotics, general, I'm not talking about necessarily you or me or people who are kind of in tech, like in this space, right, with AI and stuff, but generally I'm going to use my mom and dad as an example, think robots are scary and dangerous, like, and they're going to take over the world or whatever. Who knows? They're crazy old people. But is the emotions piece did you do. I'm just curious, did you do any work or like, analysis to say, hey, what will make this seem more lifelike or more friendly or more like real? Is that kind of where the idea of emotions came from is to make it seem more lifelike versus, you know, an inanimate object for Example when you

B (5:16)

get it right and the design is also very important. Just like, like the shape of it, the, the difference it makes. When I do a demo and with Rich Mini it works every time, every time. Like sometimes I had this super impressive by two arms robot capable of, of doing, grasping and, and, and then I do a demo of Rich Mini, wakes up, does a little emotion, react to something and I see ten faces light up and smile. And I think we are social animals and it's very important that the social cues we are used to having, our interactions exists also when interacting with AI and robots. Otherwise it doesn't feel quite right. And yes, there's been a lot of research on this topic. It's a bit complicated, complex because there's obviously a strong element of subjectivity and regionality. The way you interact depends on the country. But we've read books about this, we've read papers and then we've done one iteration of how to generate emotion. I think it works pretty good. But I think we're still at the beginning of this.

A (6:30)

Why don't you go ahead and kind of show an example and kind of walk through, you know, what the Rishi Mini is doing and what you're doing to kind of activate it.

B (6:40)

This is a very simple app. I'm going to share my screen. So here's a wheel of emotions, here's a wheel of behaviors. One that I really like is this one is dying.

A (6:55)

I just can't help but laugh. It's just funny.

B (6:58)

I told you it works every time.

A (6:59)

It's hilarious. It's like banging around. It's hilarious.

B (7:04)

And like if it's angry at you, he'll do this. And if it's happy, sometimes I say it, sometimes she, sometimes he, depending on context, it will dance and sing. And it just works.

A (7:32)

I mean, that's so awesome. So you know, with the emotions, I mean obviously you can go out and do like your research, but how do you actually design the emotion? Because when, like for example, with the dying one, right, like to me it felt kind of real, like that it was banging around like last leg, like, how do you design that? Like, do you know what I mean? How do you. I don't know. I don't even know how to describe it. How do you make it seem so real? Are you like looking for examples and training on like things like last leg things? I don't know. I don't even know how to describe it, but it just seemed so real to me.

B (8:12)

Honestly, I think it's mostly talents from one of Our team, team crew to team members. Like, she will teleoperate the robot and the. The little dance you heard, it was her voice. She was singing. Like, was it Fever Nights or something like that? Well, she will just sing at the same time. Do teleoperation. There's several ways of doing the tele operation on that recording. It was like a VR setup, right. And then you move something at that, something that is tracked, and the robot will mimic that movement. And that's how you record it. And after that, we coded a very simple teleoperation app where the motors shut down and you can hold the head with your hands and just move it and talk. And then it would get the sound by the microphones. And that was a motion that's recorded. I know. I think this is like almost a work of an animator. And Charlotte, who did most of the recordings on all of them, she was in video game before, so I think she had a lot of culture about this subject. So one way of moving forward is just doing more of this. Like more recording, more context. But the problem is if I play fear again, it will do the same exact motion again. So it works really well. But a true character would always behave a little bit differently. So first step would be we should be able to add some randomness into it. And let's say you want to appear shy, then you often do it with your eyes. Like you look at the person, you go like this and you look again and you can prerecord this because if the face of the person move, you need to face track it. Right. So there's a limit to prerecorded emotions. And I'd like us to. To go there. There are many ideas I discussed with someone who wanted to use a multimodal LLM to look at a generated motion and judge if it was good or not and do reinforcement learning in simulation. So, like, you could generate a movement and look at and say, hey, that was a bit happy and that was a bit sad. But do it automatically with a multimodal LLM. So it's. That's a novelty, but it could work. There's also a type of generation with heuristics and a lot of randomness and kind of Luigi Mansion noises is very fun. Yeah. But there's many ways of going forward. But I think something that's very important is context, like being able to play the motion at the right moment. And surprisingly, LLMs are very good at this. So I connected GPT real time to

A (10:54)

this and just works and just played. So it's not just the standalone app itself. Like, for example, I'm going to give you a hypothetical that I gave Santiago that. I mean, I have my Rishi coming in. I think it's this Friday. My daughter and I are going to set it up Sunday. But my use case that I'm going to attempt is our dog is getting a little older. Like, he's pretty old. He's like 15. And he's having some issues where he's, you know, stomach's not settled. Sometimes he like will vomit on the carpet and stuff. But he's kind of like a high class dog where he'll try to hide it and then we'll just randomly find in the closet or like random places. Right. I kind of want to trade Rishi, like to be like following Truman around and to recognize when this is happening so it can like alert us so we know. Right. But like, I think what's interesting is because in that context, that wouldn't be necessarily a happy emoji or, you know, like an emotion. It would be like sad or it could be angry. Like, and I think that that part of the, the context of the emotion is what makes this so super cool. Like, it's just super cool.

B (12:03)

That would work. That would work. Well. Currently, rich Mini cannot move around, but the next iteration of hardware is adding wheels behind it so below it so you can, it can move.

A (12:14)

I could also put it on like, you know, the Roomba and stuff. Like, we could get pretty squirrely with like, what we could do. Like, I'm not super technical, but I'm enough where I could, you know, make a few things happen. I love this. And I think what is so interesting about this is if you think about like, I know it's a bit agentic, it's a bit robotics, but there hasn't been a. Or at least not to my knowing is like you think of like the, you know, humanoid type robots, right? Like the, you know, unit trees, right, that are like $75,000. Like, this is very affordable. And I know that you had a big bang at ces, like you sold a ton of units. But where are people, like, what segments are kind of buying units? Is it like, you know, people who are kind of tinkerers at home that wanted, like, do something cool? Is that more companies do you see, like students buying them that want to learn kind of nuts and bolts of robotics? I'm just curious, like, where you think the biggest impact will be. And then I want to start jumping into kind of a few more technical things about setup Python, you know, stuff like that.

B (13:17)

To answer your question, I don't know, I'm not sure.

A (13:19)

At least that's a Santiago question.

B (13:21)

I think none of us really know. I've been blown away by the speed at which people have started building stuff on top of this robot. Like, we had a group of beta testers and they immediately we're shipping left and right ideas and software. I think this is excellent for AI makers. This is the top thing we market for. So they, I mean, I've been having a blast for the past six months, have ton of ideas. Like, at some point I was using Rich Mini as a music instrument, the theremin instrument, where you have the continuous sounds. Very weird. I call this the Themini and then. But I'm a musician, so it sounds like crap. But there's an infinite amount of creativity you can have with this. I would like to see happen is teaching in two different directions. I think this is great for learning. Like, you want to do a project, taking a project, and by doing it you will learn. But also I think we're getting to a state where AI is mature enough to be a great teacher. I think an early MF is correctly prompted and with grounded in truth can be an excellent teacher. And maybe you could get into a place where you have a companion that's always available, always supportive, but providing a way that's optimized for your learning and not just giving you the answer.

A (14:49)

Yeah. Oh, that's key. That is key.

B (14:50)

Yeah, absolutely. Because currently, I mean, I'm a teacher and honestly, it's just so exhausting to see students stopping their efforts and just asking ChatGPT to provide the answer of the question is just a waste of time for everyone. But I think there's a step beyond that that I wish we'll get into soon.

A (15:11)

I've talked a lot about that. Is like my daughter, right? Like, I try to keep her up on AI things, but it's so easy, right? It's like the pathways, resistance, like ask the question in ChatGPT or whatever. You pick your element of choice doesn't really matter. I want to see a world. And I think this is probably a use case that I would try to do with my daughter is basically use Rishi for homework help. But like in a way where it's not giving an answer. It's more like, hey, before you know, I give you the answer, what do you think it is? Hey, how did you get to that answer? Like, walk me through your steps. Do you know what I mean? To like get the Brain going. And then ultimately, you know, help. Help with the answer if we're not getting there. But, like, really challenge the thought. I think would be super cool. Yeah.

B (15:54)

When you think about school, you've got like 20 students and one teacher, and it's impossible to have the exact speed that will match everyone. If you have a tutor specifically for you, maybe you can maximize learning. But I think we need to be careful with these two because if it's done poorly, it would be terrible. But I have a lot of hope into this approach. I think this could be good.

A (16:17)

I agree. So, you know, let's go into a bit more technical. So, Rishi, believe, correct me if I'm wrong, Windows, Linux, and then it can work on Mac OS as well. Correct. Okay. So I'm kind of checking out the startup guide. It seems pretty interesting in terms of, you know, kind of code base. Right. Is it mostly Python? Like, what's kind of the underlying code base?

B (16:42)

Well, the code base is mostly Python and Rust on some. On some elements. Think there's parts of the team that want to redo everything in Rust. And then on top of it, I think we're going to see More and more JavaScript. I think we're going into a direction where we're realizing that making apps that work everywhere is simpler when we're using web technologies and we want to support both. So currently, when you do an app, it's mostly, well, it's Python with a web graphical interface. But then very soon, maybe tomorrow we'll have full Support for full JavaScript apps. So, yeah, mostly JavaScript and Python. If you're a user, if you're a maintainer, the technologies change a little bit, but not too much.

A (17:27)

No, that makes sense. So, you know, once you kind of, you know, get everything up and running, you know, you're basically installing kind of the robot server we get kind of there. One thing I noticed, which I thought was really interesting, right. Is that I. I'm not. I can do a lot of AI demo stuff, but writing, like, code is not a skill that I ever picked up. Right. And I got Claude because I was like, everyone's like, claude, Claude, Claude, Claude, code, whatever. And I get. I'm a sucker for the Facebook ads with this one guy. I don't even know if you see it in France, but there's this guy with a mustache and they just. I don't know, they do such a great job. They sold me on it, but I promise I don't have a code base. Like, I don't have anything to run off of. And I tell me a little bit about, you know, kind of the agents, you know, like MD kind of in the repo for Rishi that allows people to take and build kind of apps from that. Like kind of walk me through, if you were going to use that, what would you do and how would you do it? Because I think that's the best part of building apps. I love it.

B (18:25)

I have the answer for you. So I, I, I've been doing coding stuff for 15 years, and all of a sudden there's this AI that codes in most cases much better than I did. I want to use that power to make it so that everyone can use it. And well, in general software, I would say these agents are very good. But for our stuff, specifically robotic stuff, and Richie Mini specifically, we still needed to add what's called skills in this context of AI, for example, the motion should be continuous, otherwise the character breaks and many specific technical things. So I went and I poured all of my knowledge into different files and gave the AI, hey, if you want to create an app, you were going to use this program that will create the boilerplate code and you will do this and that, and you will make sure the git is set up. You will make sure that you will clone locally other apps that would be good examples. You will clone the code base of which mean. And one thing that's super good with this project is open source. I think we did a good job with the documentation, but there's no better documentation than the code. So if you have a doubt about anything, a function, how this or that behaves, the agent, the AI agent can just read the code and understand. So what we do in agents MD is a document made for AI agents who want to code is we make them read the documentation, the code, and the example apps that are closest to what you want. And then through a conversation between you and the agent, it will code the app for you. And we tried this last week with someone who's not technical at all, but she had a good idea of an app and the thing just worked like you have to. To me, this is crazy. You just discuss with your computer and then there are a thousand lines of code that appear. You run it and the robot moves and does something and it's, I mean, you can abstract all the technical stuff and just focus on the artistic stuff or the product view of the things, but it's fun to understand. You can, you can ask questions and it will never get tired of you. It can get deep and it will, it will be anchored in the truth of the code base and the documentation. And I think I've almost never seen it say something completely dumb. So I think it's just incredible. If you want to learn something, build something. It's never been easier than today.

A (21:06)

I think it's so cool. And I think that is what is so interesting about this, is that you're, it's. And it, you're. I, I mean, I'm not just saying this because you're a guest on the show. Your documentation is very good. Like, you know, I'll pull stuff from like GitHub or whatever and there's like, you know, like the step between three and four. You're missing like the, the 3.5 piece. Because everyone thinks it should be common knowledge to like, a smart person that knows what they're doing, but not to the average person. Do you know what I mean? Like, and I'm reading the documentation as we speak. Like, it's very, very good. Like, it's very, like, well thought out. So is the idea to really empower, you know, others who maybe have low technical experience to create their own apps? Like, for example, I create an app, how do I get it on to the kind of official, like, I'm not going to call it App Store, but

B (21:56)

like, yeah, we call it App Store. It's super simple because we have a script that does it for you. It creates a bullet plate and you write publish. And it will publish it to a public data set of apps. And then on your dashboard, you can see all of the apps created by the community. We cross a hundred apps last week.

A (22:17)

You've got 105 as of today.

B (22:19)

105. This is great. And then, well, the next problem is there are so many of them. Can you test them all? How do you market your own? But then it becomes like the usual problem with apps. If your thing is good to make it known, you need to remark. But sharing it with the community is super simple. I think it's going to be even simpler with the full JavaScript support.

A (22:43)

That's great. That's fantastic. So, like, from a, like, you know, from an installation standpoint, is, is there any kind of gotchas? I mean, especially if someone's running on Windows, right? Which I will not be. I'll be running on Ubuntu 24.4. But like, if somebody is really set on Windows, is there any kind of gotchas or gaps? Because things just never seem to work as well from an AI development standpoint on Windows.

B (23:08)

I just changed my Mind over the course of the last two months before I would say, hey, read the documentation, we tried it several times. This is the main thing you need to do to debug. And if you have a problem, go on Discord and we'll help you. Nowadays I'm like, if you have access to an AI agent, please discuss with him first or it, I don't know how to call it and just the speed at which you can provide the commands you've done or even let it run the commands, read the input and adapt. And it's excellent at handling all of this. Little difference between like hey, the OS version is not what we thought or there's a driver issue or maybe stuff like that. Generally it works really, really well. So if you're able to like get the agents MD file and get a codecs or cloud code, it will probably help you out even if you're not very technical and even if you have a very technical problem and otherwise I would just hope that it works. I think it's getting more and more robust. No, no, I think, I think it should work. I'd be curious to see if, if your installation works first try or not.

A (24:22)

Yeah, I mean I'm going to try both.

B (24:24)

It should be a PIP install into running something that works. Actually we have two versions. We have the developer version where you clone the repo, you pip or you pip install it and then you run the daemon and then you've got a web browser or a dashboard, which is what we call the old dashboard. And it's ugly, but I mean it's compatible everywhere. And they've got a beautiful desktop app that works on Mac OS and Windows in Lennox very soon and it's much more polished. And I think this is going to be the main point of entry for most people.

A (24:58)

And then you said, I mean, I was reading this too is kind of the setup where you have like the full developer version versus just kind of like the user version. I mean from what I'm seeing, right. I mean obviously the developer, you're modifying code but the basically for the everyone version is just you're pulling down apps and using it versus if you want to actually make your own app, you want to do the developers option.

B (25:21)

Correct, Correct. But I think the difference between developer and a non developer is getting slimmer every day. Like a non developer can create really, really cool stuff nowadays.

A (25:34)

Okay, well when I say developer, I'm saying someone who wants to publish an app or create a new app, basically that path. Or they can do it in the

B (25:42)

Let me give you an example. The other day when I was testing this, this, this agency, I went in and, and, and typed, hey, I want to make a simple chess app where I can play chess against Stockfish, which is a. A AI Chess chess engine. And depending on the quality of my move, I want rich immunity to react with.

A (26:04)

Mm, that's cool.

B (26:05)

I saw that when I do a blunder, Richie go, oh, or it dies

A (26:11)

when you, you die.

B (26:12)

And then it worked in 20 minutes. I didn't have to look to read to understand, to write any code, right? And it's published. You can, you can, you can, you can clone it and play with it.

A (26:23)

I saw it. I don't play chess, but I would love to do it with me and my daughter playing checkers. That would be hilarious because we play checkers all the time. That's amazing. So we're going to get kind of a couple of rapid fire questions for you. So, I mean, obviously we're just starting, like, just starting. What is the, the app? I mean, outside of the emotions, right? But like, what is one app that's on there now that has just blown you away? That's like the. I mean, they're all cool, right? But like, what's the one where you're just like, wow, this one is amazing. Like, I love this.

B (26:54)

There's one. Let's say a fork of our official conversation app is the one with an LLM Reacts that was focused on language learning. And then you can choose the language you want to learn, and then you get the robot that speaks to you and then tries to get you to speak this new language. So I'm French, and I asked the robot to teach me French. So he goes, hey, let's start by a sentence. Let's start to say, bonjour. Comment allez vous? And then I tried to say bonjour incorrectly. I said, nonsense. Well, bonjour, something like that. And he would understand that I didn't pronounce it correctly and focus on the word and say, hey, listen carefully how I said it. And he goes, bonjour. And then I again say something wrong. And again and again with infinite patience, it will correct me every time until I say it properly in the ray. Bravo. Let's continue. And I think it's just a cool idea, a good execution of a useful idea. It's called the language partner, if you want to try it.

A (28:03)

Language partner. Okay, I'm gonna look because my daughter is in Spanish right now in sixth grade, and, you know, first Spanish class. And the problem is My wife and I don't speak Spanish so we can't like necessarily, I mean very limited Spanish. I know, but so it's like we can't reinforce at home. Right. But this is a way in which she would enjoy like talking. Do you know what I mean? And like it's, when we practice it's always. That's not how you say it, dad. And I'm like, I know I can't speak and I'm sorry but I love that, that one, that one is super cool. Next question is. You know, obviously I mean I'm just looking at kind of the app store here and I mean, I mean just since you know, beginning of January, right. Like so many apps. Let's fast forward a year. How many apps are on the app store? And Rishi, a year from now, honestly

B (28:53)

it could be thousands because well for February we expect a few hundred maybe the number we set internally was 150 and I think it really depends on how we want to focus. But currently we really, really want to empower people to create apps. We feel like encouraging this makers philosophy is really healthy and we could even make it so that in the default thing you do when you start up the robot, you create a simple app. But we can find clever tricks where it's an app that's your app. Like you create maybe a welcome message or a specific animation for you that will become the default wake up movement of the robot. Doesn't matter. But I think just doing it by yourself and seeing your modest creation in a public list, I think it kind of empowers you to say maybe you can create something that others could use too. So maybe the answer to the question is more than a thousand.

A (29:53)

I mean, who knows? Like, I mean it's, yeah. So let's say last question. My, my daughter, let's say she's interested in, in robotics, right. Is Rishi a good place for her to start like kind of learning and understanding like to be able to position herself for like maybe a major in college or a, you know, potential role in this. Because I, I, I'm just curious like from what. And I'm limited understanding of robotics. Right. Like I think about unitry and like you know, the Jetsons, the Thors, you know, it's seems more complicated than this and like there's a lot more kind of underneath the hood but like at the same time it's very similar because you're basically creating actions and you've just kind of, I don't know, containerize them in an app application. I'M using air quotes, but you've put them in an application. Right. Would you say it's a great place to get started to learn more robotics or more like the understanding of magenta ki?

B (30:48)

Both, I think. I think it will work wonders. First thing to understand is that robot is extremely large. When you want to create a robotics team, you need mechanical engineers, electrical engineers, you need low level software engineering, high level software. Nowadays, even a small portion of what's really needed in an actual team. When you play with an actual robot and try to do any kind of idea with it, you are forced to understand many, many things. So one of the first things I see is the notion of trajectory having a continuous motion. That's not intuitive. And the first time, if you don't do it correctly, you will get movements just go like that and they don't feel natural. Then you think about it and then either you read the documentation, but you'll make progress and you will get a continuous movement. You can get into kinematics, which is kind of complex. For example, if you look inside of it, this is called a steward platform. It's very weird. Like the. Maybe I can, I can take this one which is turned off. The head doesn't have motors here. These are just passive joints. And inside you've got, you've got six motors. And the combination of the motion of these six motors provide maybe this motion, maybe this motion, maybe this rotation. And this is what we call a parallel articulation. The kinematics of this pretty complex, but pretty well studied. And if you want to get into the kinematics, solving elements of it, this robot works. If you want to get into. Learning to program is very important. And I think it's good that you get a reward early in your learning. And with this routine, like 30 seconds of experimenting, you'll do something that's either cute or funny or people get interested in it. So I think the reward look is very good, I'd say. Yeah, you can get into control algorithms too. When you do, for example, face tracking, you use an AI that detects your face and then you send commands to the motor so that it tracks your face. Next step is being able to physically interact with your environment. And, and this robot mostly cannot do that. The next step is just build something on top of it and putting wheels below it would make it a great maker's project where you have to do some electronics and that stuff. I fully expect the community to do it and we are going to do it too. Honestly, I think this is one of Our next steps. And then you get into navigation. You've got mobile base has to move like the vacuum robot and mapping the. And then next step is adding a little arm and try to grab something. Then you have a pretty wide range of technical skills.

A (33:43)

That's awesome. Yeah, I mean this was great. I'm so excited. It's going to be great on Sunday. Set this thing up. So Remy, we're going to go ahead and close down. But what I want to, I like every episode is take like 30 seconds to a minute and pretend someone just started watching this episode. Give them the key takeaways that they need to know about Rishi Mini.

B (34:04)

Hey, this robot is open source accessible. You can build insane apps even if you're not a software engineer. You should try it if you want to learn something or just build and it's cute and you should look at

A (34:18)

the emotions and the community support is great. The installation structures are great. It's a great way to get started learning about agentic AI and robotics and. And you can go as deep as you want or stay as high level as you want and meet you anywhere in the middle.

B (34:33)

And we are in the crazy world, crazy times.

A (34:36)

I love that. Well, Remy, this was great. Really appreciate the time you took with us today. Where can. And we're going to put links to, you know, Lishi Mini, the installation instructions. We'll do all that. But where can people personally find you on social media? Maybe on LinkedIn or you know, your platform of choice or GitHub or where, where can people find you?

B (34:55)

I've been trying to get more active on social media. I have a LinkedIn, I have a small YouTube where I put all of my experimentations and a little bit of Twitter. But yeah, mostly LinkedIn. If you want I can share the links.

A (35:10)

Okay. Yeah, if you. I mean basically it's just your name, right?

B (35:14)

Yeah, you'll find me.

A (35:15)

Okay, perfect. And we'll put a link. We'll find it. We'll put a link at the bottom. So with that, Remy, really appreciate the time. Thanks for coming on and another great episode on, you know, Rishi Mini Robotics Agenta Ki obviously coming up GTC here in a couple of weeks. Obviously this was prerecorded, but very excited to see. We will have a demo of Rishi as well as the Delpro Max with GV10, you know, both our indoor and outdoor activations at gtc. So stop by and say hi, but in the meantime start building your apps. I'm very excited, love to hear feedback on the ones that I'll do. I don't know. We'll see. We can make fun for the dog.

B (35:51)

Ping me.

A (35:52)

Pick me so I won't pig you.

B (35:54)

Yeah, pick me. If. If it. If it went wrong, team, if it went good.

A (35:58)

Okay, I absolutely will, because I. I'm gonna need some help. And with that, you know, check out Rishi Mini, and. Yeah, until the next time. We'll see you on the next episode. Do what you want. Do what you want. Do what you want. This podcast was produced in partnership with Amaze Media Labs.