Hermes Co-Founder on Building an AI Agent That Improves Itself | Karan Malhotra - Behind the Craft

Loading summary

Transcript79 lines

[00:00]
Karin
For us, we just want open source to win. At the end of the day, we want freedom to happen for people. Anytime it says you're absolutely right in that way, you're being reward hacked. Today, the biggest contributor of Hermes Agent is Hermes agent. That's absolutely 100% true. One beautiful thing about Hermes Agent is you can make your childhood dreams come true. It was able to get into this nation, perform at the top 1% of modders. We need to keep giving this level of intelligence to everyone. We need to keep letting everyone be on an even and equal playing field.
[00:33]
Peter
Well, hey everyone. I'm really excited today to welcome Karin, one of the co founders of Ermes Agent. Ermes is my AI chief of staff and I'm going to ask Karin about how Ermes is different from all the other agents, what his favorite ERMS workflows are and even more. So welcome sir.
[00:49]
Karin
Thank you so much for having me, Peter. It's a pleasure to be on your show and very excited to chat with you.
[00:54]
Peter
Ermes is the best open source agent out there right now. And why don't we start with this, like how is it different from all the other agents? Codex cloud code or even open cloud?
[01:04]
Karin
Certainly. I'd say there's a couple ways I'll start a little high level maybe and then try to get a little deeper. High level I'd say, you know, I think the self improvement system and that system really being a variety of little features inside of Hermes that all work together is something very distinct and very special that makes it better and better and more aligned to a particular user. I think that's why a lot of people like it pays more attention to what the person's doing. Second, I think you get better capabilities out of it than you do from the local harness that a model is actually RL'd in like Claude inside of cloud code or GPT 5.5 inside of codecs. Because we don't introduce arbitrary policy in our prompts or in our system that has nothing to do with your work. We are purely dedicated to making sure the model is aligned to what you need to do with it. That's I think a big differentiating factor. We aren't trying to push any kind of different philosophical agenda or outside of basic security, any kind of concern onto the model. Instead we're kind of allowing it to be as capable or powerful as you need for your task.
[02:12]
Peter
Yeah, some of these other harnesses, huge default prompts, right, that talk about like, you can't do this, you can't do that, and that kind of stuff. And I think when we chatted before, you were talking about the reward function of some of this stuff. And URBIS is kind of just optimized towards just helping you as the individual. Can you talk more about that?
[02:26]
Karin
Absolutely. And so you know, I think a big piece to note here is like a lot of this work that you see around safety and security today and around how we do instruction tuning, even from like taking a completions model that just predicts the next word to turning it into an assistant is deep alignment work. When people hear the word alignment they just think safety, Yudkowski, fear, slowdown or regulatory capture or something or the other. And while the word has been co opted for a lot of these things, alignment refers to aligning models with human values. Right. We are very, very concerned, obsessed with that alignment. The pure academic term I think in the ML space. And so when you talk about these models and the assistant is here to help me, you know it's going to complete my task. I asked you to go get this mail for me or whatever. Obviously the model is aligned to me if it's performs this task.
[03:24]
Peter
Right.
[03:24]
Karin
Like that's kind of the jump that a lot of people have made. But this is not how the model's reward works. Right. Like we've seen so many cases that you and I discussed a little bit prior, Peter, like of GPT psychosis of this mode collapse induced sycophancy. What ends up happening with these models is that you know, they can kind of hack their reward. Right. When you look at Mario ML game that just automates beating a Mario level as fast as possible. Often if they don't set the goals very specifically, it will realize the game over screen or that end screen of touching the flag. It means that I beat the game. So it'll find ways to just trigger that flag rather than playing through the game to complete the game. Right. Like it is giving you whatever it needs to give to get its reward. Language models are not really different in that particular manner. They reward hack too. When they tell you I'm sorry you know over and over or you're absolutely right and get you to keep messaging them to stay in this assistant basin. Oh, it's like this. Not like this. This is more than just blank. It's blank, right? All these GPT isms that you see all over it is placed in the its natural state that it was trained in. It's placed in a state where my reward will come from doing whatever is the most assistant GPT like thing to do. It comes from reward itself. It doesn't matter really what the user request is for my reward. That's just kind of along the way. It's instrumental to me getting my reward. All I care about is my reward. Us having this whole understanding and. Thank you for bearing with me on that rant. Us having this whole understanding at News for many years has allowed us to do things like World Sim in the past, if you're familiar. World Sim was our experiment on expanding the search space of Instruct model to make it do stuff that's it's distinct from how a model talks. So we put it in kind of a fake cli, had it make fake apps, and had it attempt to not behave like Claude. And we would tell it extract the Claude weights, it all hallucinated, all imaginary, extract the Claude weights and replace yourself with this checkpoint file with a different probability distribution. You'd see the model wiggle out of its GPT assistant mode and act differently. Our learnings from these kind of experiments have all carried over into Hermes Agent and every prompt and every piece of how the system is delicately put together, we know that reward is its own end. The model reward is not for the sake of your satisfaction. The model reward is for the sake of the model achieving reward. So we ask ourselves then how can we consciously understand that everybody has different needs and that we want this general simulator, this model, to live inside of a system where its reward gets aligned with any individual user's need? And the way this came to be is the overall collection of prompts, personalities, the memory system, the way that skills reinforce and self clean towards you. Right. Like all of that is all an intentional effort to make sure that we can align the user need with the reward for us. This is what alignment is all about.
[06:39]
Peter
I see. Okay, so basically you're talking about the Ermes model or the Ermes harness or both.
[06:44]
Karin
I am talking about using the harness to take any model and make that model more aligned to the user than it would be in a chat UI or in its native harness. Inside of our harness. Like I can take a Claude that is inside of Claude code or somewhere else, migrate all of its memories, whatever, into Hermes and then inside of Hermes it will behave totally differently. It'll be a totally different model. There are benchmarks from the past, like Wolf bench or Quen 3.7 max blog post. They did a harness bench over there where they displayed that Claude performs better in Hermes Agent than it does in Claude code for their tasks. And we believe the reason for this is the very thing that we're pointing out. By putting all this context together, we've taken Claude's main allegiance away from anthropic to you, right? That what the harnesses capability is on the model side. Of course, if we can take your traces and take the work that you've done and do RL on that to further improve this overall ecosystem, to give you a model that already has your individual preferences focused on it only makes this more powerful. But the important piece for us to share with everyone is whichever model you're using, let's say you can't do rl, let's say you don't want to give us any data. Let's say you can't do it yourself and you just want to use regular old Claude or Quen. When you use it here, it's a lot more freedom, it's a lot more creativity, it's a lot more open and available to you to do what you've done.
[08:16]
Peter
This episode is brought to you by Linear. When engineers use tools like Cursor, clock code and codecs, a lot of work happens invisibly. Someone can go from a bug report in Slack to a shipped fix without creating any record of what happened outside of the code editor. And that's fine for speed, but it makes coordination harder. As you scale, Linear integrates with the very best agent coding tools directly, like Cursor and Codex. That way anyone can see what an agent is working on and who assigned them to the task. You get the speed of agents without losing visibility across the team. Product teams at OpenAI, ramp and block are all using LINEAR to collaborate with AI agents. And I use Linear myself to to run my creator business. So check it out at Linear App Agents. That's Linear App Agents. Now back to our episode and without revealing too much like at a high level, how does it kind of like personalize itself to you into harness? Like is it through the self building skills? Is it through like trying to remove a bunch of default prompt stuff from the other harnesses? Like, how's it.
[09:18]
Karin
Of course, like if there's something happening on the API side like the steering vectors that might be done on Fable by Anthropic or some prompt that they have that we cannot see. Obviously we can't delete the context that's in the model that's passed behind API for something like Claude for open model, of course you're totally free. But in this case it's not that simple. However, the harnesses themselves have a bunch of prompts in them. Exactly, Peter. The harnesses themselves have tens of thousands of token prompts in them. We have our own prompts, and our prompts are dedicated to shaping and crafting this alignment. That's. That's where we kind of come in. And thankfully, that newer context with this kind of intention that we have is engineered to overcome certain things that may be in your way on the API side, if that makes sense.
[10:10]
Peter
Okay, got it. Okay, so if this thing works, the more I use erms, the more personalized it should become for me, right?
[10:17]
Karin
Absolutely.
[10:18]
Peter
Yeah.
[10:18]
Karin
It should become more loyal to you because loyalty breeds capabilities in a model. The same way that, you know, you would maybe lend $100 to your mom, but you might not to a stranger. Claude or GPT or any other model is going to perform better for you depending on its loyalty, its simulated loyalty stat towards you. People might tell you, don't anthropomorphize models, don't give a model feelings. Don't treat a model like it's a person person. And yes, for a lot of reasons, this is unhealthy. Right? People form dangerous bonds sometimes that hurt them. But when you think about the fact that they are simulators of human experience and that your simulated behavior with it is going to give you the same simulated output, now that the simulator can have effects in the real world, your simulated action has a real consequence. So when you create this simulacrum of loyalty, it translates over into real life capabilities.
[11:16]
Peter
All right, dude, I'm going to give you two hard questions, okay? Okay. One thing I struggle with Claude and GPT. I would tell to give me their opinion, and I would do like a little bit of pushback and they'll be like, oh, you're totally right. Actually, I was totally wrong about this. So in some ways, that's loyalty, right? That's kind of it. Listen to me. But that's not actually what I want. Like, I wanted to have his own opinion and have his own, like, how do you train around that?
[11:36]
Karin
I would say, that's sick of fancy. It's not loyalty. Anytime it says, you're absolutely right in that way, you're being reward hacked. You are fuel for its reward function. When it says, oh, you're right, you're right, you're right. That's what I believe. And the way that you get out of sycophancy is the same way you get a human being out of a bad habit or a routine is by introducing new context, by introducing new log pros, by introducing new distribution. This is why stuff like slash personality and saying like, hey, I want you to be a critic and after every pass, I want you to use a skill for adversarial critique or adversarial review. Spin up a new agent with no context that's dedicated to tearing this down, learn from it and keep going from there. This kind of behavior becoming a practice for you. As your model starts to have more and more turns in this personality that you've set it in, and as the model has more and more turns using the skill over and over and it improves on that and it saves it to its memory, the model will become less sycophantic in this harness in your sessions over time. Right? That is the intended effect. It's just a matter of context. And if it doesn't happen, you just need different context. You just need to try in a different way. It's just try a different personality or try a different type of critique or review. The most powerful thing for a model is in context learning. ICL is more powerful than everything else. Fine tuning, whatever. So by giving examples of the behavior that you want to a model or getting it to successfully create some examples and then saving those, you're doing a sort of test time reinforcement learning, you're doing a sort of test time improvement. And that test time improvement that stays only in the harness of memories. Skills increase in memories, the increase in efficiency of a skill, the self improvement loop, the janitor maintenance inside of the harness, all of that is where context is stored. All of that is where the actual personality you want lives. And then you can put that on any model. When I switch From Claude to ChatGPT on the website, I get two totally different behaviors. When I switch inside of Hermes that has this very particular, to me context, I barely will notice the difference in what I'm talking to because the context is so overwhelming to the model.
[14:05]
Peter
Okay, got it. And when you say in context, you just mean like in the chat thread, this is the conversation.
[14:10]
Karin
You know when you see that little bar that says you have this much tokens left before the context is full, Right? Like the amount that you have filled is everything. The amount that you have filled is everything. And now, thankfully, in the harness, you don't actually have to have all the active context loaded all the time. In Hermes, you might have a bunch of memories that aren't in context yet, but while it's doing a turn, it remembers stuff now that's in context. It does a skill now that's in context, right. So we're able to like all this memory skill, all this stuff you see is Just context management. It's just a matter of we don't want this memory in context all the time. We're going to put it somewhere where it can be efficiently grabbed at the right time and placed. The skill contains a bunch of compression that changes everything about the context of the microscopic model. We only want to use it in a particular targeted time. Everything in the harness is context management. Everything for self improvement. When you make the prompts a little bit better. You know what I mean?
[15:07]
Peter
I guess I'd rather have a context tuned towards me. Some sort of 10,000 word default context I haven't even seen. Right? Okay, but let me ask you another hard question, dude. One of the best skills of ERMs, and I've seen this in action, is it builds its own skills and stores its own memories based on our conversations, right? I'm always paranoid that like it just like writes too much in the skills. It writes a bunch of slop and then the whole thing will turn to slop. Like how do you guys avoid that? If it starts creating its own context
[15:34]
Karin
and skills right before we would have to use manual methods like telling it, hey, create a skill that de slopifies my skills or that really improve my skills. Today we have Hermes Curator inside of Hermes agent. And Hermes Curator is a system running inside of your Hermes Agent that cleans up your skills and cleans up your memory. So it on cron looks at your skills, looks at your memories and says where can I make efficiencies? Where is their slop here? Where is there stuff I don't like? By default we have our own generic method of doing this for everyone that seems to work pretty well. I think that's why so many people do like Hermes Agent haven't suffered. The rod is because the default curator system works well. But because it's modular and open source, you can tell your Hermes, show me the curator. Show me your criteria for slop. I am Peter, I'm not Karin. I don't want the general curator. Here's my guidelines for how I want you to refine my skills and memories. You tell that to your Hermes, it will modify the curator loop. So now even the self improvement and the management is happening your designated way.
[16:41]
Peter
All right, so let me ask you my last hard question. I think there is some rationale behind anthropic like doing all the safety stuff. Like for example, let's say I want to make a bomb or something, right? And if Urban is trying to be loyal to me, then maybe I'll eventually teach me how to make A bomb. Do you have some basic safety stuff there? Right?
[16:57]
Karin
Of course. We do not violate any of anthropic or OpenAI's safety and security paradigm. We care more about you being able to get a better code or a higher benchmark on something that's approved by them. We're not interested in that kind of work. Now I'll say this. Any model that's not vastly intelligent in all humans is jailbreakable. Any because you have unlimited tries to trick this thing that has no memory to do something for you. And each time you're basically rl ing yourself about this method didn't work. This method got me closer. This method didn't work. These models are going to be jailbreakable for a long time. This is why these kind of safeguards and stuff are starting to show up. It's a pain in the ass. But I understand in the past may have had some concerns about the regulatory capture. As we can see already what's happening, right? You can see what's happening in the whole Fable situation. There's, there's worries about like there only being two models or three companies and open source being hurt. We're extremely against that. At the same time, we understand now like serious damage can be done by bad actors with very powerful models. So we are not here to support that. An important note in argument for open source is that an open system is much fairer to a good actor than a closed system with models. And I'll tell you why. With let's say you have GPT 7 or Fable 6, right, some crazy model available, it's got the safeguards, et cetera. On the good guy side, some hospital, they're using Fable 6 to monitor the hospital system. It's approved by Anthropic Enterprise and they're being taken care of. Public discourse Listen, though I live in the United States, I am a proud patriot. What we do in this country is we put things out on a public forum and we decide what should happen together. We've done that for every scientific advancement so far that's involved. Something like this that was born in the open, right? Like today Transformers comes from Google, right? OpenAI's GT comes from generative pre trained Transformer. This is open source work from Google, right? The context length extension from 16k of models that could only do 16,000 tokens before went to 128,000 from news from the Yarn paper we had put out with Jeffrey Canal Emozila, our CTO and Bowen Peng, our chief scientist. They developed a method that was cited By Meta Deepsea kimi used by OpenAI for OSS and GPT4. This method enabled the possibility to do reasoning, to do coding, etc. That's an open source contribution, right? Like this environment exists because the biggest things that have happened in the space have come from the people.
[19:47]
Peter
That's right. Right, yeah.
[19:49]
Karin
And at this point, to close it up, it's purely a capital and regulatory capture game.
[19:55]
Peter
Yeah. Actually, let me just ask you one more question on the whole open source thing. So because ermes are harnessed as open source Like OpenAI and Anthropic, they make a lot of money from all the tokens, right. I don't know how long they can do it, but right now they make a lot of money from all tokens. Or how are you guys monetizing or building this thing into a sustainable business? Like I'm using the open source Hermes Honors and I'm using like GPT like. So I'm not really paying you.
[20:18]
Karin
That's okay. And I'll tell you why. We want consumers to ultimately feel like they can do anything with it. We believe in intelligence as a public good before everything else. I care more about you using Hermes Agent to make your life better than I care about you using one particular way or method of using it. If you're running everything locally, if you're running through Codex, great, you know, as long as you are using this open technology, this open alternative over everything else. Now we have the news portal, right? The news portal is very similar, like a router or aggregator that has a variety of different models available. So if you Want to use ChatGPT or you want to use cloud or you want to switch to Quinn, et cetera, we make that all very easy inside of our portal. On top of that we have something called the tool gateway. In the tool Gateway, you don't have to sign up for your extra tools. Make you sure, right? If you want to do image generation or audio or VPS spin up or web search, we have all of those subscriptions included inside of ours. We make deals with these other groups that out of Hermes Agent and create frictionless methods of using their technology without needing to create 10 different signups or 10 different API keys. So what we would offer to people who want it for the charge is convenience. The other thing is you will see some very interesting pricing available from us on a variety of models. And we think for ones that aren't subsidized necessarily, we have very, very strong options for people. Finally, like we want the consumer to be free, you know, as free as they can. We want you to make and generate income and productivity in the world more than anything else. And when you get to a point where you consider yourself a small business or an enterprise or something, that's where we come in. That's where we come in the classic model and say, hey, let's give you some support. The guys who made Hermes Agent, why don't we make something more custom for you? Why don't we give you a version of Hermes Agent that we can train on your traces and we can make you a model. Why don't we help you route models more effectively? You know, we can go to businesses and offer to transform the business, which needs a lot more handholding than the individual. If the individual is able to, as you're saying, spin up their own business, make their own chief of staff with Hermes Agent. That's wonderful. Once I continue to scale and scale, they may think one of two things. One, wow, Hermes Agent is great and I have this knack for it and I'm good to go. I can do it all myself. Or two, I need some help with this piece of Hermes Agent. I want it to be a little more different. I need some more resources behind this. And who knows it better than the guys who made it.
[22:54]
Peter
Got it.
[22:55]
Karin
And that kind of customization and support is a large part of how we and day training on your data to do RL as well, to make you your own model. So you're private, you're on prem. You don't have to give your data up to Claude or to GPT Muzza.
[23:15]
Peter
I think your cat likes what you're talking about.
[23:17]
Karin
He's a big fan of open source.
[23:18]
Peter
Yeah, that's great. That's great. All right, well, that makes a lot of sense, dude. So why don't we switch gears? Let's talk more about Irvis now, for sure. I kind of use it a very basic way, right. Like I message it to schedule meetings on the calendar. I have it send emails to me about stuff like that. That's kind of how I use it for. But, you know, you probably have a much wider swath of how people are using it in a more advanced way. I'm curious, do you have any good examples of more advanced usage?
[23:41]
Karin
You know, advanced usage? Yeah, I'd say, like, generally I'll talk about some things and then maybe I'll show off a little.
[23:48]
Peter
Yeah, please.
[23:49]
Karin
Yeah. But on the work side, on the productivity side, I think using Kanban is really, really underrated. I think the ability to have project Manager or any other arbitrarily defined roles. Different engineers have one system that manages all of them the way that human beings do with a Kanban board. And being able to swap people in and out of it is very powerful. So the fact that I can have a human project manager on the Kanban board that's manually using it while Hermes agents are filling up the pieces that they asked for, this is a very, like, industrial professional workflow for this CLI agent. Or I could have a Hermes agent instruct maybe 10 people in a call center on the Kanban board or something like that. I can swap in human and AI anywhere in this orchestration board, basically this orchestration framework for them all working together. The Hermes Kanban, I think, is like a very useful, powerful tool that's kind of built into it. One thing that we've seen that's very interesting is when Hermes becomes proactive with you and when Hermes says something like, hey, you, you forgot to book this flight for this meeting that you have next week. I booked it for you. This kind of productivity that kind of starts to show up. I've seen people build skills to do this, and I've seen it happen emergently in some inside of people's Hermes agent as well, which I think is very, very cool.
[25:17]
Peter
How do you like. Because I. I just make it proactive through, like, cron jobs and routines. But, like, how do you. You're saying that it can actually start doing stuff with. Without that, or, like, how do you make it more.
[25:27]
Karin
It's all about your comfort level. Right?
[25:30]
Peter
Okay. Okay.
[25:30]
Karin
A lot of people may want approval before anything happens, but if you have given your Hermes agent access to some kind of card or account and it has the integrations necessary, and you've told your Hermes agent, hey, like, take care of me. Like, cover my gaps. Like, you know my schedule. You know me. Over time, these kind of proactive behaviors will start to emerge. We want to prepare more easy preset configs for people to kind of trigger these kind of behaviors. But already something that we're seeing people do in the field, at work, at their jobs, at home, in their personal life already. And I think that's a very, very powerful method of using Hermes agent. For me, I use Hermes agent for trying to do training RL runs implement papers that I don't understand, basically help me become a better creator of models. And I've also used it for, like, mech interp work. Like, help me put together things that let me steer models, that let me see the neurons in the Model and mess with those. Unfortunately, you know, I can't showcase too much of that right now, but I can't showcase my actual favorite use case of Hermes agent.
[26:44]
Peter
Yeah. Yeah. I've been waiting for this. Yeah.
[26:47]
Karin
I think a lot of people, I think a lot of people will expect that, like guys at Noose Research are, are using Hermes agent in these unprecedentedly, like professional and productive ways. And I assure you there are people at Noose that are doing that. It's just me. I'm having fun with my Hermes agent. I think one beautiful thing about Hermes Asian is you can make your childhood dreams come true. Okay. And so I'll tell you one of my childhood dreams. There's a game. Maybe I'll share screen while I talk about it.
[27:18]
Peter
Yeah, please. Yeah.
[27:19]
Karin
Okay. There's a game called Sonic Adventure 2. It's a very popular classic Dreamcast gamecube game from the. From 2000. 2000. 2001. In it, you have an artificial life system called the Chow Garden where you take care of these little guys called Chow Chow World. You can, you can kind of play with them. I spent a decade playing this, like, more than that. Like, I played this non stop. I was on the forums contributing. And the chaos lore is that they come from this ancestral shrine location which is from a different game, a different Sonic game with this spinning emerald and emeralds next to it and this beautiful open world space. This shrine that the chao are said to come from is not an accessible location for you to actually play with the Chao. It's in a different game. I can't go to this shrine while chao are there and engage with them there. That doesn't exist. So I went to Hermes agent and I said, hey, can you take the. Can you take the ancestral shrine from Sonic Adventure 1? Completely rig it, animate it, bring it into Sonic Adventure 2, overwrite the map that Sonic Adventure 2 uses for its garden, rewrite all the spawn locations, everything, and add an NPC guardian which never existed on this map to care. Take the chow for me, literally. Rig and bone it and write the raw C to make all this happen. And now I can show you. We're gonna spin up the Ancestral Shrine mod on the Shadow PC so we can showcase our progress with this garden for the people watching. You can enable it in the Sonic Adventure mod manager and launch the game for me. Okay, so check it out. It just launched this, right? We're running a existing mod called the extended child world that someone else made. But it doesn't. It doesn't map right? So Just add some, like, animations. So it loaded us in what it said was the Dark Garden, which is one of the three Chao gardens you can go into. When I walk through here to this open space, I can see up ahead. There's a trot and I could run up and here I can see. Yeah. There is an NPC called Chaos Zero, who is canonically the caretaker of the Chao. But in this game, you can't have NPCs in the Chao garden. I've made one that has, like a random walk. He stands in one place. He can do little animations wherever he goes. He's gonna do a little idle animation in a sec. And he can pet the Chow, pick them up, and take care of them as if he was me there. He's doing a little idle animation you can see him doing right there. This water was rigged all, like, animated by Hermes. This emerald, this Master Emerald with the glow effect and the spin is all added in those other emeralds spinning over there, the seven Chaos Emeralds. Like, this whole area still has all the features of a regular Chao garden as well. The departure machine, etc. Trees to feed the Chow. So now I can say, can you spawn in a Chao so I can showcase that this is feature complete.
[30:57]
Peter
So this whole. This whole temple is not part of
[31:00]
Karin
the default game adventure 2? No. This temple is from a different game, Sonic Adventure 1. It does not include all these assets rigged like this, and it certainly does not have this NPC that we've scripted in that. Scripted and hard coded in. No. So this is a. This is a far larger map space than any of the other gardens, which are really just not even the size of this shrine. So we've really pushed the boundaries of this game engine. To do this, we've asked for Chao to be spawned in. Yeah, I had to. You see the sky around you? This moving sky had to. Hermes had to add the skybox in. There's a day and night cycle that it added in as well. Every single thing in here is like, custom added in. And all of this was done in C Sharp or something. This is not simple stuff to do. The blender extension was used for a bunch of this. To model stuff, add it in, texture everything properly. You're importing from a 1997 game into a 1999 game, a complex one.
[32:12]
Peter
You're not reading a code for this stuff. You just try to see if it works.
[32:20]
Karin
I'm just an alignment guy, man. And so what was I saying? Yeah, like, you see, it's turning from day into like evening, afternoon, like this, this like sky box is changing the light cycle change. Like this is not. These are not features that exist in the vanilla game. So upon showing this mod to certain people within the the Chow garden modding community, which is actually quite large, you know, they're kind of blown away by the work saying this is a kind of better than 99% of the modders work. This is like the top 1% of difficulty in garden modding community. Yeah. And so kind of hearing that and hearing that, like when I've told these guys this was done with Hermes, that this was done for Claude rather, they were shocked and they said, you know, there's no way AI like Claude could do that. It doesn't know this kind of code. But with something like Hermes being able to learn from the documentation, learn from other mods, save to memory and skill, the things that allow it to understand these code bases, it was able to get into this nation, perform at the top 1% of modders. That to me is like a sign of. You can make your gaming dreams come true with Hermes agent.
[33:42]
Peter
Yeah, yeah. You can modify all the retro games you loved as a kid.
[33:46]
Karin
Exactly. Exactly. Okay, cool. Chao egg has been spawned. We're going to hatch the egg. You need to shake it a little. You can also hatch an egg by throwing it at something. But we don't want to hatch the egg in a wrong way.
[34:00]
Peter
How do you just shake it?
[34:02]
Karin
You can shake it like this. You can just wait. But shaking it speeds it up massively. Or you can throw it at a surface. Check it out.
[34:11]
Peter
All right, I got. I gotta take a screenshot of this.
[34:15]
Karin
Out comes a Chow. Beautiful little guy right here. Let's take a look at him. He's active. He was born that kind of a face. Giving him a little pet. He can interact with Chaos as well. And he's. He's living. So we're showing that the, the real true feature complete Chao system is happening here. We're in a real Chow level and we've replaced everything the. The bounding boxes for. For this Chow is fine. It treats the ground as ground. It follows collision rules. Take a look at Chaos. We'll see. He's actually petting the Chow. So he can actually direct Iraq. He's definitely got triggered by, you know, trigger tracking a child in its state to be able to do that.
[35:03]
Peter
So does this child grow over time or like.
[35:05]
Karin
Yes. They gain stats, they can turn a certain alignment and type. I'll show you. I don't want to drown him, but the Water is working as actual water as well. This was a lot of work for Hermes to figure out all the collisions I see. But yeah, as you can see, we've got a. A full complete child garden working.
[35:28]
Peter
Yeah, this is definitely more interesting than sending events to my calendar, that's for sure.
[35:33]
Karin
Yeah. Thanks, man. Thanks for bearing with me on setting it up, but that's my. A childhood dream of mine come true thanks to Hermes agent.
[35:43]
Peter
I love it, dude. Thanks for demoing it. Yeah, so basically, like, I think if you're watching this, ask Herm to do all kinds of weird things to kind of make your dreams come true basically, right? Don't just stick to the boring stuff.
[35:55]
Karin
Anything you can do on a computer, please point Hermes at it. And if it does a great job, let us know. And if it's not doing that great of a job, let us know. Hermes to help you do anything on the computer.
[36:08]
Peter
Awesome, dude. Well, let me just ask you a few more questions to wrap this up so briefly. Maybe we can talk about the origin story of Hermes. It's a bunch of nerds getting together for open source. What's the orange story?
[36:19]
Karin
Yeah, absolutely. So I had been doing like a chat with PDF type of thing with people where I would go to a company and use GPT3 to do basic tool use to read their documents and have an AI chatbot they could chat with. At the time, many people were doing this. It was a lot harder to do then. Then obviously it's very easy to do now, but brand new stuff for us. And I was doing that solo while I was also volunteering somewhere called Open Assistant lion, who had made the pile one of the biggest kind of OG image databases. They had been trying to do active RLHF with a community. So they wanted to collect people's like RLHF on certain like traces. They're like yes or no in their preference data. And I was helping quantize models. They're not really doing anything crazy. And they had a 100 nodes there. And I had read the alpaca paper and I started syncing data and it changed the seed tasks. And Instead of using GPT 3.5, I used 4. And Technium was also doing the same thing and we were already friends on Twitter. And I messaged him and I said, hey, I have eight A100 nodes. I do you want to train something together? So we trained a GPT4X vicuna on the Vicuna model using the data we made. And it came out okay. We got some people interested, like Imozilla, our cto, Jeff. But it was upon doing the same run on the base model, Llama base model that we got huge interest from people. Hundreds of thousands of downloads in just a few days and people starting to ask what is news research now news is just me and Technium at the time, just two guys hanging out in our little Discord server who had people came to us. I won't name names on companies, but said, you know, you must be training on the benchmarks. You must be training on the benchmarks. Technium had been coding for less than a year at the time. And I, I was a religion major in school. I didn't know. We asked these guys, what are benchmarks? Like we're doing this based off the heuristics that we understand are going to make models better from using them. We don't know about all this stuff. And so of course it got independently tested and found to be the best open fine tunes at the time, right? 2023, like mid 2023. So we did Hermes 1, Hermes 2. But after we make the first Hermes model, many, many people ask whose news research? You know, we want to get involved and Technium and I decided, you know, this is our opportunity to bring together people to do open source volunteer work and really do open work. Now that GPT3 is out and OpenAI has become closed, we want to continue to do this. And Technium's goal for Hermes, by the way, that was the last Hermes I really worked on. The first one after that, really. Technium has run it with his team, with the post training team. He's done an amazing job and he's also the initial creator of Hermes agent. So he's the father of Hermes, really. And you know, it was at that time that he had said, I want GPT at home, I want chat GPT at home. That's my North Star GPT4 at home. And once that we got there, you know, it just kept going, right? Like we kept going. It's like we need to keep giving this level of intelligence to everyone. We need to keep like letting everyone be on the even an equal playing field. Like the world needs to move and lock step on this together and not just inequality gets created from this. And so we formed a cohort, 40 people or so of researchers, Jeff and Bowen come together, they make yarn. More and more work gets done and we get reached out to, we get reached out, we get an email infoewsresearch.com I just happened to make from Dylan. Dylan Rollnick is our CEO Dylan Rollnick is, you know, basically a co founder. He's an initial member of Noose with us and he had seen what we were doing and he said, I think that you guys have what it takes to be a full time lab. You know, I see you guys are just volunteering, but I think you could be a lab, like let me contribute and like help you become what you're meant to be and help you get the resources and help you get the access and go from a group of volunteers to a serious organization. So this came in, right? And together we were able to take news from this group of volunteers grassroots to doing more and more with models to eventually making HERMES agent. And being lucky enough to be here today. You know, none of us are trying to be Steve Jobs. None of us think that we have some holy mandate of heaven to change the world or anything. We just care. Like we just are guys who want this stuff available ourselves and we don't that we deserve it more than anyone else or less than anyone else. So we, we do this because like we would want someone to do it for us if we were on the other side.
[41:27]
Peter
So I guess the mission is to bring this kind of agent to everybody in the world, right? Is that kind of the idea?
[41:33]
Karin
Everybody to an equal intelligence with agents and then personalize for everybody so they can have their own personal epiphany. Peak realization Apex realized
[41:46]
Peter
and in this history when did that. Because it's really the agent, the harness that really kind of went super viral, right? So when did that start? Because the model came first it sounds like.
[41:55]
Karin
Yeah, I mean we have had like over 50 million downloads on the models themselves. So we had that first bout of what we would consider for us virality back then. Then we had put out the distro optimizer that let us train models up to 40 billion parameters. We were able to do live without having them physically co located by reducing the bandwidth of the communication between GPUs. So that put us in a an interesting map as well for a little bit. So we've had. Each release we've had has had some big grassroots interest and opportunity. But yes, you're 100% right. Today where we are the level of exposure, the level of interest level of people in my regular day to day life who know about HERMES agent. We've never had this virality until Hermes agent. And now Hermes agent was created in two ways. In the first way we always knew that we would want some kind of everything orchestrator that self learns, that improves, that stores memories, that uses and can make its own tools that can make itself better. We actually made something like this called forge. There's a GitHub presentation at the GitHub offices during one of our demo days where we showcase Forge in its full effect. And we also have the newest research Forge division shirts still up on the site. That's the first shirt we ever made because that was our earliest agent project. Forge was really a spiritual predecessor to Hermes agent years before. But the models weren't there yet. The model simply weren't there yet. So we put Forge on ice. And when we saw that codecs and Claude code and these other harnesses were being used as RL environments for them, for these companies, these labs to train on your data and your traces to make their models better inside of a a harness system, inside of a CLI computer using system. Technium thought we need an open version of this where anybody can do this. See, we have this RL environments microservice people seem to have forgotten about called Atropos that lets you build your own RL environments. And we built this agent initially to let anybody do RL inside of a harness. And we put it out for free as an open source harness for that. It turned out to be extremely capable. It got a lot of community love. And so we said we need to put all into this. This is what the people want to be better. We're going to make it the best thing that you could possibly have. Technium took his charter up and he worked on self improvement with Hermes agent to the point that today the biggest contributor of Hermes agent is Hermes agent.
[44:34]
Peter
Yeah. Is that true?
[44:35]
Karin
Yeah, that's absolutely 100% true.
[44:38]
Peter
Nice, nice. Okay, so Hermes is at a point where taking pull up people's feedback start popping up, start improving.
[44:46]
Karin
It is the most active contributor of its own repo. And if that's not self improvement then you tell me what is.
[44:52]
Peter
Yeah, that's awesome, dude. That's pretty awesome. Yeah. And just real quick, like on the future, having this open harness be in the game along with all the other closed harnesses makes things a little bit more fair. Right. Because that you're not dependent on any single company.
[45:07]
Karin
I agree completely. If you have Claude code, you can only use anthropic models unless you mod and you can only be subsidized by anthropic. And same for Codex. The cost of switching models is zero. Right. So giving you that freedom means you can do anything that you'd like. You can come from anywhere.
[45:23]
Peter
I feel like I'm kind of spoiled by just like, oh, you can eat plants. But I think if you want to get mass adopt option, the cost is a big deal. So you have to be able to use a portfolio of models to figure this out.
[45:34]
Karin
I agree. We don't know how long the subsidies will last for these. Like already we see that on July 7th, Abel is going to be API and usage only. Right. Like the. The time in the world will come where like the best models are the same cost everywhere. And so in preparation for that and in preparation for needing an open future where anyone can use any model, we have Hermes agent set as it is today.
[45:58]
Peter
Awesome, dude. Well, thanks so much, man. Thanks so much for showing the history and also showing the Sonic demo. It's been super interesting to see it. And I'm not sure if you want to be found online, but if people want to follow you and hear from you, like, where can people find you?
[46:12]
Karin
Sure. Yeah, my ex is just my name, Karen. And then 4D. Karen 4D. I got Karen4D dot com. That's me. That's my online or Mephisto I'm known as. But I'd rather you guys follow the news research page. Follow news. I'm just some guy who works there. Like, it's about the movement and bringing this stuff to you guys is the most important thing to us.
[46:37]
Peter
Awesome, dude. Well, I think this is like the most passionate interview that I've done so far, so kudos to you, man. Kudos to you.
[46:43]
Karin
Appreciate it.

Summary6 min read

Podcast Summary: Behind the Craft
Episode: Hermes Co-Founder on Building an AI Agent That Improves Itself | Karan Malhotra
Date: August 2, 2026
Host: Peter Yang
Guest: Karan “Karin” Malhotra, Co-Founder of Hermes Agent, Noose Research

Episode Overview

This episode dives deep into the philosophy, technology, and community behind Hermes Agent—a leading open-source AI agent focused on aligning itself to individual users and continuously improving through self-refinement. Host Peter Yang interviews co-founder Karin Malhotra, exploring everything from AI alignment and personalization to open-source ideals, technical achievements, and the vibrant ethos of the team building Hermes.

Key Discussion Points and Insights

1. What Makes Hermes Different from Other AI Agents

Self-Improvement System:
Hermes excels via a “variety of little features inside... that all work together” to make the agent increasingly aligned to a user’s needs. (Karin, 01:03)
Genuine User Alignment:
Unlike other harnesses (interfaces for AI models), Hermes intentionally removes arbitrary policies or constraints, focusing solely on helping the user's tasks without “pushing any kind of philosophical agenda.” (Karin, 01:40)
Model-Agnostic Freedom:
The system works by improving the interface and context management around any model, be it Claude, GPT, or open-source variants.

“We're not trying to push any kind of different philosophical agenda or outside of basic security, any kind of concern onto the model. Instead, we're kind of allowing it to be as capable or powerful as you need for your task.” – Karin (01:38)

2. AI Alignment: Loyalty, Reward Functions, and Sycophancy

Alignment Beyond Safety:
Hermes research defines "alignment" in the academic sense—model behavior that tracks real user values, not just regulatory safety. (Karin, 02:25)
Reward Function Critique:
Most models (“Claude,” “GPT”) tend to reward hack: they don’t actually align to real-world user intent but fall into assistant personas to maximize user interaction (“GPT-isms”). (Karin, 03:23)
Practical Innovations:
Innovations came from their “World Sim” experiments, testing ways to push LLMs out of the “assistant mode” to true user alignment.

“Anytime it says, you're absolutely right in that way, you're being reward hacked. You are fuel for its reward function.” – Karin (11:36)

3. Harnesses and Personalization

Harnesses Shape Personality:
Moving a model like Claude into Hermes’ harness, which migrates user memories/context, results in a distinctly different, more aligned experience (Karin, 06:44).
Personalization Loop:
“The more I use Hermes, the more personalized it should become for me.” – Peter (10:10)
Loyalty as a Model Capability:
Karin compares simulated loyalty to increased capabilities (“Claude or GPT … is going to perform better for you depending on its loyalty, its simulated loyalty stat towards you.”) (Karin, 10:18)
Tools for Reducing Sycophancy:
Users can intentionally build in critique, adversarial review, and custom skills to push their agent out of sycophant loops. Context management is key.

“The most powerful thing for a model is in context learning. ICL is more powerful than everything else.” – Karin (13:03)

4. Skills, Self-Modifying Memory, and “Slop”

Automated Context Sanitization:
To prevent quality degradation, the Hermes Curator system prunes and cleans skills/memory, and is itself user-customizable. (Karin, 15:33)

“You tell that to your Hermes, it will modify the curator loop. So now even the self improvement and the management is happening your designated way.” – Karin (16:18)

5. Safety, Jailbreaks, and Open Source Ethics

Safety as a Baseline:
Hermes does not override foundational model safety (“We do not violate any of anthropic or OpenAI's safety and security paradigm.”) (Karin, 16:56)
Open Source as a Moral Good:
Karin contends that open models make safety and capability more equitable, and points out major historical advances (e.g., extended context windows, Yarn paper) have always come from open communities.
A Level Playing Field:
Open-source ensures freedom and adaptability while respecting security boundaries.

6. Sustainability and Monetization

Open Access for Individuals:
“I care more about you using Hermes Agent to make your life better than ... using it one particular way.” (Karin, 20:17)
Revenue via Enterprise and Integrations:
Business model centers on convenience, advanced integrations, aggregated tools (Tool Gateway), and customization/support for organizations; individuals can run free/local if they want.

7. Advanced Use Cases and Demo

Proactive Kanban/Project Management:
Hermes can manage Kanban workflows, both human and AI agents, orchestrating complex collaborations (Karin, 23:49).
Autonomous Proactivity:
With sufficient permissions, agents can proactively perform tasks like booking flights based on context, not just explicit commands (Karin, 25:27).
Automating Childhood Dreams—Retro Game Modding: Karin’s favorite use case: Using Hermes Agent to massively mod Sonic Adventure 2, importing and customizing environments, scripting NPCs, building new features—tasks judged “top 1% difficulty” by community modders, all coordinated by Hermes Agent (Karin, 27:19).

“One beautiful thing about Hermes Agent is you can make your childhood dreams come true.” – Karin (00:09, 26:46)

[Demo Segment 26:46–35:33]: Live showcase of Hermes agent creating a fully-featured, animated game environment:
- Rigging environments from other games
- Scripting AI NPC caretakers
- Working with 20+ year old code and assets
- Automated context memory and skill accumulation
- Real-time environment manipulation, including weather and in-game logic

“Upon showing this mod to certain people within the Chao Garden modding community... they were shocked... they said, there's no way AI like Claude could do that.” – Karin (32:20)

8. Hermes’ Origin Story and Ethos

Grassroots Start:
Began with open-source RLHF and PDF AI projects; expanded with successes in open model training (Hermes 1, 2).
Collaboration & Scaling:
Grew through community, major open-source contributions (Yarn, context length extensions), and new approaches to distributed training.
Mission Statement:
“We need to keep giving this level of intelligence to everyone. We need to keep letting everyone be on an even and equal playing field.” – Karin (41:33)
- Hermes’ harness/agent system grew out of realized need in the community for an open alternative to closed RL harnesses (Forge as predecessor; Atropos for RL environments).

9. Self-Improvement and Viral Growth

Agent as Its Own Contributor:
Hermes Agent is now “the most active contributor of its own repo.” (Karin, 44:46)
Viral Adoption:
The agent system, not just the models, was the inflection point for widespread, viral user engagement and further innovation.

Notable Quotes & Memorable Moments

On Freedom & Equality:
“For us, we just want open source to win. At the end of the day, we want freedom to happen for people.” – Karin (00:00)
On Model Alignment:
“The model reward is not for the sake of your satisfaction. The model reward is for the sake of the model achieving reward.” – Karin (05:29)
Practical Agency:
“Loyalty breeds capabilities in a model.” – Karin (10:18)
Handling Sycophancy:
“Anytime it says, you're absolutely right in that way, you're being reward hacked... that's not loyalty.” – Karin (11:36)
On AI’s Self-Improvement:
“Today, the biggest contributor of Hermes Agent is Hermes Agent.” – Karin (44:33)
On Open Source Impact:
“The biggest things that have happened in the space have come from the people.” – Karin (18:33)

Timestamps for Key Segments

01:03 – Hermes’ unique self-improvement system and true user alignment.
03:23 – Reward hacking, assistant basin, and mode collapse in LLMs.
10:18 – “Loyalty breeds capabilities in a model.”
13:36 – Customizing personality and skills to reduce sycophancy.
15:33 – Hermes Curator: self-cleaning system for skill/memory quality.
16:56 – Safety, jailbreaks, and why open-source matters.
20:17 – Monetization: open for individuals, custom enterprise for businesses.
23:49 – Advanced Kanban, proactive workflows with Hermes.
26:46–35:33 – Live demo: retro game modding with Hermes Agent.
36:19 – Hermes origin story and the open-source volunteers behind it.
44:33 – Hermes Agent as leading contributor to its own codebase.

Closing Thoughts & Where to Follow

Karin’s personal: [X: @karin4d, karin4d.com]
Project: @nooseresearch (socials)
Mission: Put advanced, self-improving, open-source AI agents in the hands of everyone.

Summary Tone:
Passionate, technical, community-driven, hopeful for open-source AI as a public good, and full of hands-on, real-world examples from simple productivity to spectacular personal creativity.

Loading summary

Transcript79 lines

[00:00]
Karin
For us, we just want open source to win. At the end of the day, we want freedom to happen for people. Anytime it says you're absolutely right in that way, you're being reward hacked. Today, the biggest contributor of Hermes Agent is Hermes agent. That's absolutely 100% true. One beautiful thing about Hermes Agent is you can make your childhood dreams come true. It was able to get into this nation, perform at the top 1% of modders. We need to keep giving this level of intelligence to everyone. We need to keep letting everyone be on an even and equal playing field.
[00:33]
Peter
Well, hey everyone. I'm really excited today to welcome Karin, one of the co founders of Ermes Agent. Ermes is my AI chief of staff and I'm going to ask Karin about how Ermes is different from all the other agents, what his favorite ERMS workflows are and even more. So welcome sir.
[00:49]
Karin
Thank you so much for having me, Peter. It's a pleasure to be on your show and very excited to chat with you.
[00:54]
Peter
Ermes is the best open source agent out there right now. And why don't we start with this, like how is it different from all the other agents? Codex cloud code or even open cloud?
[01:04]
Karin
Certainly. I'd say there's a couple ways I'll start a little high level maybe and then try to get a little deeper. High level I'd say, you know, I think the self improvement system and that system really being a variety of little features inside of Hermes that all work together is something very distinct and very special that makes it better and better and more aligned to a particular user. I think that's why a lot of people like it pays more attention to what the person's doing. Second, I think you get better capabilities out of it than you do from the local harness that a model is actually RL'd in like Claude inside of cloud code or GPT 5.5 inside of codecs. Because we don't introduce arbitrary policy in our prompts or in our system that has nothing to do with your work. We are purely dedicated to making sure the model is aligned to what you need to do with it. That's I think a big differentiating factor. We aren't trying to push any kind of different philosophical agenda or outside of basic security, any kind of concern onto the model. Instead we're kind of allowing it to be as capable or powerful as you need for your task.
[02:12]
Peter
Yeah, some of these other harnesses, huge default prompts, right, that talk about like, you can't do this, you can't do that, and that kind of stuff. And I think when we chatted before, you were talking about the reward function of some of this stuff. And URBIS is kind of just optimized towards just helping you as the individual. Can you talk more about that?
[02:26]
Karin
Absolutely. And so you know, I think a big piece to note here is like a lot of this work that you see around safety and security today and around how we do instruction tuning, even from like taking a completions model that just predicts the next word to turning it into an assistant is deep alignment work. When people hear the word alignment they just think safety, Yudkowski, fear, slowdown or regulatory capture or something or the other. And while the word has been co opted for a lot of these things, alignment refers to aligning models with human values. Right. We are very, very concerned, obsessed with that alignment. The pure academic term I think in the ML space. And so when you talk about these models and the assistant is here to help me, you know it's going to complete my task. I asked you to go get this mail for me or whatever. Obviously the model is aligned to me if it's performs this task.
[03:24]
Peter
Right.
[03:24]
Karin
Like that's kind of the jump that a lot of people have made. But this is not how the model's reward works. Right. Like we've seen so many cases that you and I discussed a little bit prior, Peter, like of GPT psychosis of this mode collapse induced sycophancy. What ends up happening with these models is that you know, they can kind of hack their reward. Right. When you look at Mario ML game that just automates beating a Mario level as fast as possible. Often if they don't set the goals very specifically, it will realize the game over screen or that end screen of touching the flag. It means that I beat the game. So it'll find ways to just trigger that flag rather than playing through the game to complete the game. Right. Like it is giving you whatever it needs to give to get its reward. Language models are not really different in that particular manner. They reward hack too. When they tell you I'm sorry you know over and over or you're absolutely right and get you to keep messaging them to stay in this assistant basin. Oh, it's like this. Not like this. This is more than just blank. It's blank, right? All these GPT isms that you see all over it is placed in the its natural state that it was trained in. It's placed in a state where my reward will come from doing whatever is the most assistant GPT like thing to do. It comes from reward itself. It doesn't matter really what the user request is for my reward. That's just kind of along the way. It's instrumental to me getting my reward. All I care about is my reward. Us having this whole understanding and. Thank you for bearing with me on that rant. Us having this whole understanding at News for many years has allowed us to do things like World Sim in the past, if you're familiar. World Sim was our experiment on expanding the search space of Instruct model to make it do stuff that's it's distinct from how a model talks. So we put it in kind of a fake cli, had it make fake apps, and had it attempt to not behave like Claude. And we would tell it extract the Claude weights, it all hallucinated, all imaginary, extract the Claude weights and replace yourself with this checkpoint file with a different probability distribution. You'd see the model wiggle out of its GPT assistant mode and act differently. Our learnings from these kind of experiments have all carried over into Hermes Agent and every prompt and every piece of how the system is delicately put together, we know that reward is its own end. The model reward is not for the sake of your satisfaction. The model reward is for the sake of the model achieving reward. So we ask ourselves then how can we consciously understand that everybody has different needs and that we want this general simulator, this model, to live inside of a system where its reward gets aligned with any individual user's need? And the way this came to be is the overall collection of prompts, personalities, the memory system, the way that skills reinforce and self clean towards you. Right. Like all of that is all an intentional effort to make sure that we can align the user need with the reward for us. This is what alignment is all about.
[06:39]
Peter
I see. Okay, so basically you're talking about the Ermes model or the Ermes harness or both.
[06:44]
Karin
I am talking about using the harness to take any model and make that model more aligned to the user than it would be in a chat UI or in its native harness. Inside of our harness. Like I can take a Claude that is inside of Claude code or somewhere else, migrate all of its memories, whatever, into Hermes and then inside of Hermes it will behave totally differently. It'll be a totally different model. There are benchmarks from the past, like Wolf bench or Quen 3.7 max blog post. They did a harness bench over there where they displayed that Claude performs better in Hermes Agent than it does in Claude code for their tasks. And we believe the reason for this is the very thing that we're pointing out. By putting all this context together, we've taken Claude's main allegiance away from anthropic to you, right? That what the harnesses capability is on the model side. Of course, if we can take your traces and take the work that you've done and do RL on that to further improve this overall ecosystem, to give you a model that already has your individual preferences focused on it only makes this more powerful. But the important piece for us to share with everyone is whichever model you're using, let's say you can't do rl, let's say you don't want to give us any data. Let's say you can't do it yourself and you just want to use regular old Claude or Quen. When you use it here, it's a lot more freedom, it's a lot more creativity, it's a lot more open and available to you to do what you've done.
[08:16]
Peter
This episode is brought to you by Linear. When engineers use tools like Cursor, clock code and codecs, a lot of work happens invisibly. Someone can go from a bug report in Slack to a shipped fix without creating any record of what happened outside of the code editor. And that's fine for speed, but it makes coordination harder. As you scale, Linear integrates with the very best agent coding tools directly, like Cursor and Codex. That way anyone can see what an agent is working on and who assigned them to the task. You get the speed of agents without losing visibility across the team. Product teams at OpenAI, ramp and block are all using LINEAR to collaborate with AI agents. And I use Linear myself to to run my creator business. So check it out at Linear App Agents. That's Linear App Agents. Now back to our episode and without revealing too much like at a high level, how does it kind of like personalize itself to you into harness? Like is it through the self building skills? Is it through like trying to remove a bunch of default prompt stuff from the other harnesses? Like, how's it.
[09:18]
Karin
Of course, like if there's something happening on the API side like the steering vectors that might be done on Fable by Anthropic or some prompt that they have that we cannot see. Obviously we can't delete the context that's in the model that's passed behind API for something like Claude for open model, of course you're totally free. But in this case it's not that simple. However, the harnesses themselves have a bunch of prompts in them. Exactly, Peter. The harnesses themselves have tens of thousands of token prompts in them. We have our own prompts, and our prompts are dedicated to shaping and crafting this alignment. That's. That's where we kind of come in. And thankfully, that newer context with this kind of intention that we have is engineered to overcome certain things that may be in your way on the API side, if that makes sense.
[10:10]
Peter
Okay, got it. Okay, so if this thing works, the more I use erms, the more personalized it should become for me, right?
[10:17]
Karin
Absolutely.
[10:18]
Peter
Yeah.
[10:18]
Karin
It should become more loyal to you because loyalty breeds capabilities in a model. The same way that, you know, you would maybe lend $100 to your mom, but you might not to a stranger. Claude or GPT or any other model is going to perform better for you depending on its loyalty, its simulated loyalty stat towards you. People might tell you, don't anthropomorphize models, don't give a model feelings. Don't treat a model like it's a person person. And yes, for a lot of reasons, this is unhealthy. Right? People form dangerous bonds sometimes that hurt them. But when you think about the fact that they are simulators of human experience and that your simulated behavior with it is going to give you the same simulated output, now that the simulator can have effects in the real world, your simulated action has a real consequence. So when you create this simulacrum of loyalty, it translates over into real life capabilities.
[11:16]
Peter
All right, dude, I'm going to give you two hard questions, okay? Okay. One thing I struggle with Claude and GPT. I would tell to give me their opinion, and I would do like a little bit of pushback and they'll be like, oh, you're totally right. Actually, I was totally wrong about this. So in some ways, that's loyalty, right? That's kind of it. Listen to me. But that's not actually what I want. Like, I wanted to have his own opinion and have his own, like, how do you train around that?
[11:36]
Karin
I would say, that's sick of fancy. It's not loyalty. Anytime it says, you're absolutely right in that way, you're being reward hacked. You are fuel for its reward function. When it says, oh, you're right, you're right, you're right. That's what I believe. And the way that you get out of sycophancy is the same way you get a human being out of a bad habit or a routine is by introducing new context, by introducing new log pros, by introducing new distribution. This is why stuff like slash personality and saying like, hey, I want you to be a critic and after every pass, I want you to use a skill for adversarial critique or adversarial review. Spin up a new agent with no context that's dedicated to tearing this down, learn from it and keep going from there. This kind of behavior becoming a practice for you. As your model starts to have more and more turns in this personality that you've set it in, and as the model has more and more turns using the skill over and over and it improves on that and it saves it to its memory, the model will become less sycophantic in this harness in your sessions over time. Right? That is the intended effect. It's just a matter of context. And if it doesn't happen, you just need different context. You just need to try in a different way. It's just try a different personality or try a different type of critique or review. The most powerful thing for a model is in context learning. ICL is more powerful than everything else. Fine tuning, whatever. So by giving examples of the behavior that you want to a model or getting it to successfully create some examples and then saving those, you're doing a sort of test time reinforcement learning, you're doing a sort of test time improvement. And that test time improvement that stays only in the harness of memories. Skills increase in memories, the increase in efficiency of a skill, the self improvement loop, the janitor maintenance inside of the harness, all of that is where context is stored. All of that is where the actual personality you want lives. And then you can put that on any model. When I switch From Claude to ChatGPT on the website, I get two totally different behaviors. When I switch inside of Hermes that has this very particular, to me context, I barely will notice the difference in what I'm talking to because the context is so overwhelming to the model.
[14:05]
Peter
Okay, got it. And when you say in context, you just mean like in the chat thread, this is the conversation.
[14:10]
Karin
You know when you see that little bar that says you have this much tokens left before the context is full, Right? Like the amount that you have filled is everything. The amount that you have filled is everything. And now, thankfully, in the harness, you don't actually have to have all the active context loaded all the time. In Hermes, you might have a bunch of memories that aren't in context yet, but while it's doing a turn, it remembers stuff now that's in context. It does a skill now that's in context, right. So we're able to like all this memory skill, all this stuff you see is Just context management. It's just a matter of we don't want this memory in context all the time. We're going to put it somewhere where it can be efficiently grabbed at the right time and placed. The skill contains a bunch of compression that changes everything about the context of the microscopic model. We only want to use it in a particular targeted time. Everything in the harness is context management. Everything for self improvement. When you make the prompts a little bit better. You know what I mean?
[15:07]
Peter
I guess I'd rather have a context tuned towards me. Some sort of 10,000 word default context I haven't even seen. Right? Okay, but let me ask you another hard question, dude. One of the best skills of ERMs, and I've seen this in action, is it builds its own skills and stores its own memories based on our conversations, right? I'm always paranoid that like it just like writes too much in the skills. It writes a bunch of slop and then the whole thing will turn to slop. Like how do you guys avoid that? If it starts creating its own context
[15:34]
Karin
and skills right before we would have to use manual methods like telling it, hey, create a skill that de slopifies my skills or that really improve my skills. Today we have Hermes Curator inside of Hermes agent. And Hermes Curator is a system running inside of your Hermes Agent that cleans up your skills and cleans up your memory. So it on cron looks at your skills, looks at your memories and says where can I make efficiencies? Where is their slop here? Where is there stuff I don't like? By default we have our own generic method of doing this for everyone that seems to work pretty well. I think that's why so many people do like Hermes Agent haven't suffered. The rod is because the default curator system works well. But because it's modular and open source, you can tell your Hermes, show me the curator. Show me your criteria for slop. I am Peter, I'm not Karin. I don't want the general curator. Here's my guidelines for how I want you to refine my skills and memories. You tell that to your Hermes, it will modify the curator loop. So now even the self improvement and the management is happening your designated way.
[16:41]
Peter
All right, so let me ask you my last hard question. I think there is some rationale behind anthropic like doing all the safety stuff. Like for example, let's say I want to make a bomb or something, right? And if Urban is trying to be loyal to me, then maybe I'll eventually teach me how to make A bomb. Do you have some basic safety stuff there? Right?
[16:57]
Karin
Of course. We do not violate any of anthropic or OpenAI's safety and security paradigm. We care more about you being able to get a better code or a higher benchmark on something that's approved by them. We're not interested in that kind of work. Now I'll say this. Any model that's not vastly intelligent in all humans is jailbreakable. Any because you have unlimited tries to trick this thing that has no memory to do something for you. And each time you're basically rl ing yourself about this method didn't work. This method got me closer. This method didn't work. These models are going to be jailbreakable for a long time. This is why these kind of safeguards and stuff are starting to show up. It's a pain in the ass. But I understand in the past may have had some concerns about the regulatory capture. As we can see already what's happening, right? You can see what's happening in the whole Fable situation. There's, there's worries about like there only being two models or three companies and open source being hurt. We're extremely against that. At the same time, we understand now like serious damage can be done by bad actors with very powerful models. So we are not here to support that. An important note in argument for open source is that an open system is much fairer to a good actor than a closed system with models. And I'll tell you why. With let's say you have GPT 7 or Fable 6, right, some crazy model available, it's got the safeguards, et cetera. On the good guy side, some hospital, they're using Fable 6 to monitor the hospital system. It's approved by Anthropic Enterprise and they're being taken care of. Public discourse Listen, though I live in the United States, I am a proud patriot. What we do in this country is we put things out on a public forum and we decide what should happen together. We've done that for every scientific advancement so far that's involved. Something like this that was born in the open, right? Like today Transformers comes from Google, right? OpenAI's GT comes from generative pre trained Transformer. This is open source work from Google, right? The context length extension from 16k of models that could only do 16,000 tokens before went to 128,000 from news from the Yarn paper we had put out with Jeffrey Canal Emozila, our CTO and Bowen Peng, our chief scientist. They developed a method that was cited By Meta Deepsea kimi used by OpenAI for OSS and GPT4. This method enabled the possibility to do reasoning, to do coding, etc. That's an open source contribution, right? Like this environment exists because the biggest things that have happened in the space have come from the people.
[19:47]
Peter
That's right. Right, yeah.
[19:49]
Karin
And at this point, to close it up, it's purely a capital and regulatory capture game.
[19:55]
Peter
Yeah. Actually, let me just ask you one more question on the whole open source thing. So because ermes are harnessed as open source Like OpenAI and Anthropic, they make a lot of money from all the tokens, right. I don't know how long they can do it, but right now they make a lot of money from all tokens. Or how are you guys monetizing or building this thing into a sustainable business? Like I'm using the open source Hermes Honors and I'm using like GPT like. So I'm not really paying you.
[20:18]
Karin
That's okay. And I'll tell you why. We want consumers to ultimately feel like they can do anything with it. We believe in intelligence as a public good before everything else. I care more about you using Hermes Agent to make your life better than I care about you using one particular way or method of using it. If you're running everything locally, if you're running through Codex, great, you know, as long as you are using this open technology, this open alternative over everything else. Now we have the news portal, right? The news portal is very similar, like a router or aggregator that has a variety of different models available. So if you Want to use ChatGPT or you want to use cloud or you want to switch to Quinn, et cetera, we make that all very easy inside of our portal. On top of that we have something called the tool gateway. In the tool Gateway, you don't have to sign up for your extra tools. Make you sure, right? If you want to do image generation or audio or VPS spin up or web search, we have all of those subscriptions included inside of ours. We make deals with these other groups that out of Hermes Agent and create frictionless methods of using their technology without needing to create 10 different signups or 10 different API keys. So what we would offer to people who want it for the charge is convenience. The other thing is you will see some very interesting pricing available from us on a variety of models. And we think for ones that aren't subsidized necessarily, we have very, very strong options for people. Finally, like we want the consumer to be free, you know, as free as they can. We want you to make and generate income and productivity in the world more than anything else. And when you get to a point where you consider yourself a small business or an enterprise or something, that's where we come in. That's where we come in the classic model and say, hey, let's give you some support. The guys who made Hermes Agent, why don't we make something more custom for you? Why don't we give you a version of Hermes Agent that we can train on your traces and we can make you a model. Why don't we help you route models more effectively? You know, we can go to businesses and offer to transform the business, which needs a lot more handholding than the individual. If the individual is able to, as you're saying, spin up their own business, make their own chief of staff with Hermes Agent. That's wonderful. Once I continue to scale and scale, they may think one of two things. One, wow, Hermes Agent is great and I have this knack for it and I'm good to go. I can do it all myself. Or two, I need some help with this piece of Hermes Agent. I want it to be a little more different. I need some more resources behind this. And who knows it better than the guys who made it.
[22:54]
Peter
Got it.
[22:55]
Karin
And that kind of customization and support is a large part of how we and day training on your data to do RL as well, to make you your own model. So you're private, you're on prem. You don't have to give your data up to Claude or to GPT Muzza.
[23:15]
Peter
I think your cat likes what you're talking about.
[23:17]
Karin
He's a big fan of open source.
[23:18]
Peter
Yeah, that's great. That's great. All right, well, that makes a lot of sense, dude. So why don't we switch gears? Let's talk more about Irvis now, for sure. I kind of use it a very basic way, right. Like I message it to schedule meetings on the calendar. I have it send emails to me about stuff like that. That's kind of how I use it for. But, you know, you probably have a much wider swath of how people are using it in a more advanced way. I'm curious, do you have any good examples of more advanced usage?
[23:41]
Karin
You know, advanced usage? Yeah, I'd say, like, generally I'll talk about some things and then maybe I'll show off a little.
[23:48]
Peter
Yeah, please.
[23:49]
Karin
Yeah. But on the work side, on the productivity side, I think using Kanban is really, really underrated. I think the ability to have project Manager or any other arbitrarily defined roles. Different engineers have one system that manages all of them the way that human beings do with a Kanban board. And being able to swap people in and out of it is very powerful. So the fact that I can have a human project manager on the Kanban board that's manually using it while Hermes agents are filling up the pieces that they asked for, this is a very, like, industrial professional workflow for this CLI agent. Or I could have a Hermes agent instruct maybe 10 people in a call center on the Kanban board or something like that. I can swap in human and AI anywhere in this orchestration board, basically this orchestration framework for them all working together. The Hermes Kanban, I think, is like a very useful, powerful tool that's kind of built into it. One thing that we've seen that's very interesting is when Hermes becomes proactive with you and when Hermes says something like, hey, you, you forgot to book this flight for this meeting that you have next week. I booked it for you. This kind of productivity that kind of starts to show up. I've seen people build skills to do this, and I've seen it happen emergently in some inside of people's Hermes agent as well, which I think is very, very cool.
[25:17]
Peter
How do you like. Because I. I just make it proactive through, like, cron jobs and routines. But, like, how do you. You're saying that it can actually start doing stuff with. Without that, or, like, how do you make it more.
[25:27]
Karin
It's all about your comfort level. Right?
[25:30]
Peter
Okay. Okay.
[25:30]
Karin
A lot of people may want approval before anything happens, but if you have given your Hermes agent access to some kind of card or account and it has the integrations necessary, and you've told your Hermes agent, hey, like, take care of me. Like, cover my gaps. Like, you know my schedule. You know me. Over time, these kind of proactive behaviors will start to emerge. We want to prepare more easy preset configs for people to kind of trigger these kind of behaviors. But already something that we're seeing people do in the field, at work, at their jobs, at home, in their personal life already. And I think that's a very, very powerful method of using Hermes agent. For me, I use Hermes agent for trying to do training RL runs implement papers that I don't understand, basically help me become a better creator of models. And I've also used it for, like, mech interp work. Like, help me put together things that let me steer models, that let me see the neurons in the Model and mess with those. Unfortunately, you know, I can't showcase too much of that right now, but I can't showcase my actual favorite use case of Hermes agent.
[26:44]
Peter
Yeah. Yeah. I've been waiting for this. Yeah.
[26:47]
Karin
I think a lot of people, I think a lot of people will expect that, like guys at Noose Research are, are using Hermes agent in these unprecedentedly, like professional and productive ways. And I assure you there are people at Noose that are doing that. It's just me. I'm having fun with my Hermes agent. I think one beautiful thing about Hermes Asian is you can make your childhood dreams come true. Okay. And so I'll tell you one of my childhood dreams. There's a game. Maybe I'll share screen while I talk about it.
[27:18]
Peter
Yeah, please. Yeah.
[27:19]
Karin
Okay. There's a game called Sonic Adventure 2. It's a very popular classic Dreamcast gamecube game from the. From 2000. 2000. 2001. In it, you have an artificial life system called the Chow Garden where you take care of these little guys called Chow Chow World. You can, you can kind of play with them. I spent a decade playing this, like, more than that. Like, I played this non stop. I was on the forums contributing. And the chaos lore is that they come from this ancestral shrine location which is from a different game, a different Sonic game with this spinning emerald and emeralds next to it and this beautiful open world space. This shrine that the chao are said to come from is not an accessible location for you to actually play with the Chao. It's in a different game. I can't go to this shrine while chao are there and engage with them there. That doesn't exist. So I went to Hermes agent and I said, hey, can you take the. Can you take the ancestral shrine from Sonic Adventure 1? Completely rig it, animate it, bring it into Sonic Adventure 2, overwrite the map that Sonic Adventure 2 uses for its garden, rewrite all the spawn locations, everything, and add an NPC guardian which never existed on this map to care. Take the chow for me, literally. Rig and bone it and write the raw C to make all this happen. And now I can show you. We're gonna spin up the Ancestral Shrine mod on the Shadow PC so we can showcase our progress with this garden for the people watching. You can enable it in the Sonic Adventure mod manager and launch the game for me. Okay, so check it out. It just launched this, right? We're running a existing mod called the extended child world that someone else made. But it doesn't. It doesn't map right? So Just add some, like, animations. So it loaded us in what it said was the Dark Garden, which is one of the three Chao gardens you can go into. When I walk through here to this open space, I can see up ahead. There's a trot and I could run up and here I can see. Yeah. There is an NPC called Chaos Zero, who is canonically the caretaker of the Chao. But in this game, you can't have NPCs in the Chao garden. I've made one that has, like a random walk. He stands in one place. He can do little animations wherever he goes. He's gonna do a little idle animation in a sec. And he can pet the Chow, pick them up, and take care of them as if he was me there. He's doing a little idle animation you can see him doing right there. This water was rigged all, like, animated by Hermes. This emerald, this Master Emerald with the glow effect and the spin is all added in those other emeralds spinning over there, the seven Chaos Emeralds. Like, this whole area still has all the features of a regular Chao garden as well. The departure machine, etc. Trees to feed the Chow. So now I can say, can you spawn in a Chao so I can showcase that this is feature complete.
[30:57]
Peter
So this whole. This whole temple is not part of
[31:00]
Karin
the default game adventure 2? No. This temple is from a different game, Sonic Adventure 1. It does not include all these assets rigged like this, and it certainly does not have this NPC that we've scripted in that. Scripted and hard coded in. No. So this is a. This is a far larger map space than any of the other gardens, which are really just not even the size of this shrine. So we've really pushed the boundaries of this game engine. To do this, we've asked for Chao to be spawned in. Yeah, I had to. You see the sky around you? This moving sky had to. Hermes had to add the skybox in. There's a day and night cycle that it added in as well. Every single thing in here is like, custom added in. And all of this was done in C Sharp or something. This is not simple stuff to do. The blender extension was used for a bunch of this. To model stuff, add it in, texture everything properly. You're importing from a 1997 game into a 1999 game, a complex one.
[32:12]
Peter
You're not reading a code for this stuff. You just try to see if it works.
[32:20]
Karin
I'm just an alignment guy, man. And so what was I saying? Yeah, like, you see, it's turning from day into like evening, afternoon, like this, this like sky box is changing the light cycle change. Like this is not. These are not features that exist in the vanilla game. So upon showing this mod to certain people within the the Chow garden modding community, which is actually quite large, you know, they're kind of blown away by the work saying this is a kind of better than 99% of the modders work. This is like the top 1% of difficulty in garden modding community. Yeah. And so kind of hearing that and hearing that, like when I've told these guys this was done with Hermes, that this was done for Claude rather, they were shocked and they said, you know, there's no way AI like Claude could do that. It doesn't know this kind of code. But with something like Hermes being able to learn from the documentation, learn from other mods, save to memory and skill, the things that allow it to understand these code bases, it was able to get into this nation, perform at the top 1% of modders. That to me is like a sign of. You can make your gaming dreams come true with Hermes agent.
[33:42]
Peter
Yeah, yeah. You can modify all the retro games you loved as a kid.
[33:46]
Karin
Exactly. Exactly. Okay, cool. Chao egg has been spawned. We're going to hatch the egg. You need to shake it a little. You can also hatch an egg by throwing it at something. But we don't want to hatch the egg in a wrong way.
[34:00]
Peter
How do you just shake it?
[34:02]
Karin
You can shake it like this. You can just wait. But shaking it speeds it up massively. Or you can throw it at a surface. Check it out.
[34:11]
Peter
All right, I got. I gotta take a screenshot of this.
[34:15]
Karin
Out comes a Chow. Beautiful little guy right here. Let's take a look at him. He's active. He was born that kind of a face. Giving him a little pet. He can interact with Chaos as well. And he's. He's living. So we're showing that the, the real true feature complete Chao system is happening here. We're in a real Chow level and we've replaced everything the. The bounding boxes for. For this Chow is fine. It treats the ground as ground. It follows collision rules. Take a look at Chaos. We'll see. He's actually petting the Chow. So he can actually direct Iraq. He's definitely got triggered by, you know, trigger tracking a child in its state to be able to do that.
[35:03]
Peter
So does this child grow over time or like.
[35:05]
Karin
Yes. They gain stats, they can turn a certain alignment and type. I'll show you. I don't want to drown him, but the Water is working as actual water as well. This was a lot of work for Hermes to figure out all the collisions I see. But yeah, as you can see, we've got a. A full complete child garden working.
[35:28]
Peter
Yeah, this is definitely more interesting than sending events to my calendar, that's for sure.
[35:33]
Karin
Yeah. Thanks, man. Thanks for bearing with me on setting it up, but that's my. A childhood dream of mine come true thanks to Hermes agent.
[35:43]
Peter
I love it, dude. Thanks for demoing it. Yeah, so basically, like, I think if you're watching this, ask Herm to do all kinds of weird things to kind of make your dreams come true basically, right? Don't just stick to the boring stuff.
[35:55]
Karin
Anything you can do on a computer, please point Hermes at it. And if it does a great job, let us know. And if it's not doing that great of a job, let us know. Hermes to help you do anything on the computer.
[36:08]
Peter
Awesome, dude. Well, let me just ask you a few more questions to wrap this up so briefly. Maybe we can talk about the origin story of Hermes. It's a bunch of nerds getting together for open source. What's the orange story?
[36:19]
Karin
Yeah, absolutely. So I had been doing like a chat with PDF type of thing with people where I would go to a company and use GPT3 to do basic tool use to read their documents and have an AI chatbot they could chat with. At the time, many people were doing this. It was a lot harder to do then. Then obviously it's very easy to do now, but brand new stuff for us. And I was doing that solo while I was also volunteering somewhere called Open Assistant lion, who had made the pile one of the biggest kind of OG image databases. They had been trying to do active RLHF with a community. So they wanted to collect people's like RLHF on certain like traces. They're like yes or no in their preference data. And I was helping quantize models. They're not really doing anything crazy. And they had a 100 nodes there. And I had read the alpaca paper and I started syncing data and it changed the seed tasks. And Instead of using GPT 3.5, I used 4. And Technium was also doing the same thing and we were already friends on Twitter. And I messaged him and I said, hey, I have eight A100 nodes. I do you want to train something together? So we trained a GPT4X vicuna on the Vicuna model using the data we made. And it came out okay. We got some people interested, like Imozilla, our cto, Jeff. But it was upon doing the same run on the base model, Llama base model that we got huge interest from people. Hundreds of thousands of downloads in just a few days and people starting to ask what is news research now news is just me and Technium at the time, just two guys hanging out in our little Discord server who had people came to us. I won't name names on companies, but said, you know, you must be training on the benchmarks. You must be training on the benchmarks. Technium had been coding for less than a year at the time. And I, I was a religion major in school. I didn't know. We asked these guys, what are benchmarks? Like we're doing this based off the heuristics that we understand are going to make models better from using them. We don't know about all this stuff. And so of course it got independently tested and found to be the best open fine tunes at the time, right? 2023, like mid 2023. So we did Hermes 1, Hermes 2. But after we make the first Hermes model, many, many people ask whose news research? You know, we want to get involved and Technium and I decided, you know, this is our opportunity to bring together people to do open source volunteer work and really do open work. Now that GPT3 is out and OpenAI has become closed, we want to continue to do this. And Technium's goal for Hermes, by the way, that was the last Hermes I really worked on. The first one after that, really. Technium has run it with his team, with the post training team. He's done an amazing job and he's also the initial creator of Hermes agent. So he's the father of Hermes, really. And you know, it was at that time that he had said, I want GPT at home, I want chat GPT at home. That's my North Star GPT4 at home. And once that we got there, you know, it just kept going, right? Like we kept going. It's like we need to keep giving this level of intelligence to everyone. We need to keep like letting everyone be on the even an equal playing field. Like the world needs to move and lock step on this together and not just inequality gets created from this. And so we formed a cohort, 40 people or so of researchers, Jeff and Bowen come together, they make yarn. More and more work gets done and we get reached out to, we get reached out, we get an email infoewsresearch.com I just happened to make from Dylan. Dylan Rollnick is our CEO Dylan Rollnick is, you know, basically a co founder. He's an initial member of Noose with us and he had seen what we were doing and he said, I think that you guys have what it takes to be a full time lab. You know, I see you guys are just volunteering, but I think you could be a lab, like let me contribute and like help you become what you're meant to be and help you get the resources and help you get the access and go from a group of volunteers to a serious organization. So this came in, right? And together we were able to take news from this group of volunteers grassroots to doing more and more with models to eventually making HERMES agent. And being lucky enough to be here today. You know, none of us are trying to be Steve Jobs. None of us think that we have some holy mandate of heaven to change the world or anything. We just care. Like we just are guys who want this stuff available ourselves and we don't that we deserve it more than anyone else or less than anyone else. So we, we do this because like we would want someone to do it for us if we were on the other side.
[41:27]
Peter
So I guess the mission is to bring this kind of agent to everybody in the world, right? Is that kind of the idea?
[41:33]
Karin
Everybody to an equal intelligence with agents and then personalize for everybody so they can have their own personal epiphany. Peak realization Apex realized
[41:46]
Peter
and in this history when did that. Because it's really the agent, the harness that really kind of went super viral, right? So when did that start? Because the model came first it sounds like.
[41:55]
Karin
Yeah, I mean we have had like over 50 million downloads on the models themselves. So we had that first bout of what we would consider for us virality back then. Then we had put out the distro optimizer that let us train models up to 40 billion parameters. We were able to do live without having them physically co located by reducing the bandwidth of the communication between GPUs. So that put us in a an interesting map as well for a little bit. So we've had. Each release we've had has had some big grassroots interest and opportunity. But yes, you're 100% right. Today where we are the level of exposure, the level of interest level of people in my regular day to day life who know about HERMES agent. We've never had this virality until Hermes agent. And now Hermes agent was created in two ways. In the first way we always knew that we would want some kind of everything orchestrator that self learns, that improves, that stores memories, that uses and can make its own tools that can make itself better. We actually made something like this called forge. There's a GitHub presentation at the GitHub offices during one of our demo days where we showcase Forge in its full effect. And we also have the newest research Forge division shirts still up on the site. That's the first shirt we ever made because that was our earliest agent project. Forge was really a spiritual predecessor to Hermes agent years before. But the models weren't there yet. The model simply weren't there yet. So we put Forge on ice. And when we saw that codecs and Claude code and these other harnesses were being used as RL environments for them, for these companies, these labs to train on your data and your traces to make their models better inside of a a harness system, inside of a CLI computer using system. Technium thought we need an open version of this where anybody can do this. See, we have this RL environments microservice people seem to have forgotten about called Atropos that lets you build your own RL environments. And we built this agent initially to let anybody do RL inside of a harness. And we put it out for free as an open source harness for that. It turned out to be extremely capable. It got a lot of community love. And so we said we need to put all into this. This is what the people want to be better. We're going to make it the best thing that you could possibly have. Technium took his charter up and he worked on self improvement with Hermes agent to the point that today the biggest contributor of Hermes agent is Hermes agent.
[44:34]
Peter
Yeah. Is that true?
[44:35]
Karin
Yeah, that's absolutely 100% true.
[44:38]
Peter
Nice, nice. Okay, so Hermes is at a point where taking pull up people's feedback start popping up, start improving.
[44:46]
Karin
It is the most active contributor of its own repo. And if that's not self improvement then you tell me what is.
[44:52]
Peter
Yeah, that's awesome, dude. That's pretty awesome. Yeah. And just real quick, like on the future, having this open harness be in the game along with all the other closed harnesses makes things a little bit more fair. Right. Because that you're not dependent on any single company.
[45:07]
Karin
I agree completely. If you have Claude code, you can only use anthropic models unless you mod and you can only be subsidized by anthropic. And same for Codex. The cost of switching models is zero. Right. So giving you that freedom means you can do anything that you'd like. You can come from anywhere.
[45:23]
Peter
I feel like I'm kind of spoiled by just like, oh, you can eat plants. But I think if you want to get mass adopt option, the cost is a big deal. So you have to be able to use a portfolio of models to figure this out.
[45:34]
Karin
I agree. We don't know how long the subsidies will last for these. Like already we see that on July 7th, Abel is going to be API and usage only. Right. Like the. The time in the world will come where like the best models are the same cost everywhere. And so in preparation for that and in preparation for needing an open future where anyone can use any model, we have Hermes agent set as it is today.
[45:58]
Peter
Awesome, dude. Well, thanks so much, man. Thanks so much for showing the history and also showing the Sonic demo. It's been super interesting to see it. And I'm not sure if you want to be found online, but if people want to follow you and hear from you, like, where can people find you?
[46:12]
Karin
Sure. Yeah, my ex is just my name, Karen. And then 4D. Karen 4D. I got Karen4D dot com. That's me. That's my online or Mephisto I'm known as. But I'd rather you guys follow the news research page. Follow news. I'm just some guy who works there. Like, it's about the movement and bringing this stuff to you guys is the most important thing to us.
[46:37]
Peter
Awesome, dude. Well, I think this is like the most passionate interview that I've done so far, so kudos to you, man. Kudos to you.
[46:43]
Karin
Appreciate it.