Loading summary
Transcript29 lines
- [00:00]
Guillermo Roush
Welcome.
- [00:01]
Naval Ravikant
You're listening to the Naval Podcast, your authoritative source for new knowledge. We're trying something new today. I have three frontier founders with us. Three good looking guys, actually, and a fourth good looking guy. Naval. And let me just introduce everybody. Guillermo the G Roush. He's building Vercel into an AI cloud for the world of agents and whatever comes after that.
- [00:25]
Guillermo Roush
Good to be here.
- [00:26]
Naval Ravikant
Blake Shaw, he's building supersonic aircraft in his own factory and jet engines as well. Blake's company, Boom. Supersonic. And then Max Hodak from science. He's building a biohybrid brain interface that grows living neurons on silicon to restore sensory functions like sight, but then eventually to explore new parts of the brain and new senses. All three of these guys are not composing their products with off the shelf parts. They're building their own factories. And, you know, we don't care as much about what they're building exactly as we do about what they're learning about how they're building, what's the new knowledge they're generating, what's their alpha, what principles are they discovering that other founders can learn from? What are they trying to figure out right now? And also, what are the cutting edge or crazy ideas that they haven't even talked about yet and they're still forming in their brains? Naval. Do you have any reactions to any of that before I jump into Guillermo?
- [01:23]
Blake Shaw
Yeah, let's just have fun.
- [01:26]
Naval Ravikant
Yeah, you guys should just jump in.
- [01:28]
Guillermo Roush
Yeah. So I can't remember my exact quote, by the way, but I've been really pilled with this idea of software factories and the job of the engineer being something that you just show up to work, used to ship the output directly, and everything inside the company was, you know, how good is person A at shipping output B? And now what's happening is the way that I'm judging you as an engineer is like, are you producing the factory that will produce multiplicative outputs B through Z? Right. And that's a pretty significant change because basically we used to believe, and it should be somewhat controversial, that there's 10x engineers, like now, clearly there's 100x or a thousandx engineers, and the world hasn't fully adjusted to this.
- [02:12]
Blake Shaw
I used to get flamed on Twitter for saying they're 10x engineers. Yeah, it flies in the face of so much like equality philosophy that everyone's equal. But the reality is, when you're operating in idea domains, when you're operating intellectual domains and virtual digital domains, it's not even 10x, it's 100x or a thousand X, it always has been Satoshi Notch, you know, the guy who invented JavaScript, the Brendan Ike's of the world, John Carmack. I mean these are thousand X programmers. Not to even mention if you choose the right thing to work on versus the wrong thing to work on, that's an infinity difference. And it could just be not necessarily a better programmer, just one who had a better judgment on what to work on in the first place. And now obviously it's less controversial because of AI leverage.
- [02:52]
Guillermo Roush
What's controversial is that the token leaderboards, right, like people are still getting a little confused because now they think, well I have a bunch of 100x engineers, look at all these tokens that I'm paying for. I'm curious if you guys have seen the same, like how do you measure roi?
- [03:08]
Max Hodak
It's like the old measuring lines of code, you know, token consumption lines of code feel like similarly not direct paradigms. I mean my observation has been that Claude or ChatGPT or GPT is about, is basically as good as you are in a domain. And so if you're, if you're a really capable developer, then these things are really powerful. And if you're a junior developer, then you'll kind of find it to be like more of a junior developer. Like on the one hand these models are incredibly capable. On the other hand, the feedback that you give them sporadically seems to be incredibly important. And these little updates seem to totally determine the types of performance you get out of them.
- [03:46]
Guillermo Roush
There's a new kind of support that I give which is you come to me and you didn't get good output out of the model and I tell you what to prompt the model with. So the idea of the quality of the re prompting, which I think you're alluding to, is extremely important.
- [04:01]
Max Hodak
But I mean, and to be clear, I think that this will become less important over time. Like as the models get much, much smarter, then you'll be able to put in less and get more out. But at least at this stage it really seems to kind of reflect back the judgment that the user brings in.
- [04:14]
Blake Shaw
In my experience I've kind of resisted learning all the tricks and tips. Like, you know, there was, oh, use Ralph Wiggum, use openclaw, use Hermes, use this prompt engine, use this scaffolding plug in this piece, you know, always use plan mode. I just ignored all of that. I just assumed the model is just going to get better faster than I would figure out how to use it. It would figure out how to use me faster than I would figure out how to use it. And so I've just been completely ham fisted with them and I get frustrated at them and just sort of, I found myself typing less and less information and doing less and less work as time goes on with the models because I just assume I can brute force my way through it and I'll throw Codex, Claude and Gemini at the same problem over and over and just waste tokens to save time. And I think no matter how expensive these models might seem, they're still way cheaper than a human. So I would say just waste tokens, save time, don't look at the tokens either as inputs or outputs, just look at your time and look at the final output. And even if they're writing low quality code, which I know in many cases they are, it's not necessarily production quality or scalable code. When the time comes and I want to ship it to production, I'll just throw more tokens at it. I'll say, okay, now go through, look at it, rewrite it. And they're just going to get better every generation. So yeah, I don't see where this necessarily stops. As long as we have verifiable domains and solve problems, they're going to resolve those problems. Now it's in the unsolved problems domain where maybe you're Terrence Tao, you're the cutting edge of creativity that you need to be working very collaboratively and carefully and closely with the model. But I'm not in that, I'm not at that level in software engineering. And Guillermo, you're probably the most extreme software engineer in the team, right? Like out of this set you're probably the one who most hardcore came up from a software background. Like, how are you finding these models at the edge of their capability?
- [06:01]
Guillermo Roush
Well, there's one thing that's happened recently that what you're saying resonates strongly with, which is it used to be that you would give a prompt to the model and it kind of does the classic next token prediction thing and it runs away with your idea. And models now have been doing this intuitive planning mode without to your point, not even having to plan, where it comes back to you and says, look what you're asking me for. There's these three routes we can take, there's this set of trade offs that we're going to go down. That's a moment where people do the whole thing on X. Oh, now we have a PhD level engineer model. Like that's very clear that the models at some point graduated. They Used to be junior engineers, now they're principal engineers because they come back to you with a set of trade offs and obviously sometimes they bullshit, which is hilarious. It tells you this one is going to take three weeks. And this many talk it has made really bad predictions. But clearly it's now this. Like I respect the models a lot more as a peer like that I'm going back and forth intellectually with. But there are a lot of gaps still. So like if you're a really, really proficient engineer or architect, I think you're still extracting more juice. So the question sort of that Max was positing of like if you're junior, do you get junior back? Well, clearly not because a junior gets more advanced knowledge in code. They would have never been able to write by themselves. But doesn't an experienced architect get 10x whereas a junior engineer gets 2x? That's what I'm kind of trying to figure out.
- [07:36]
Max Hodak
Yeah, but I mean I think there's architectural decisions. So when you think about the development, I'm seeing this now with some of our, the junior software engineers of the team of like what is the next step in their career progression? It's going from like writing implementation for a feature to picking technologies like choosing between postgres versus some other database or picking between ZMQ versus some other message queue or like some other queuing system. And those, I mean the models can suggest them but that's the thing where like you'll see it and you'll be like no, no, no, I want to use this other thing. That's the type of little feedback that I'm saying really matters and the types of output that you seem to get at this point.
- [08:08]
Blake Shaw
Taste and judgment.
- [08:09]
Max Hodak
Right, Taste and judgment.
- [08:10]
Blake Shaw
That said, you can ask them which one should I use and why? And they know everything. They'll give you really good trade offs.
- [08:16]
Guillermo Roush
That's the change that I was saying has happened recently where you would say hey, go and put this super high cardinality telemetry data into postgres. And it's like no bro, like we don't put that kind of data into postgres. Like you should consider Clickhouse or Athena or whatever. Like that's happened to me a lot, which is really impressive. But the thing I'm still kind of struggling with is clearly the human is still completing the model at one point. Is it the other way about the human is the one sort of getting the instructions back on go get me this API key. Because it's something that only you can do or get me this amount of capital for my next set of investments that I need to make. You just watch like, clearly we're still not there yet.
- [09:04]
Blake Shaw
That's a temporary aberration. Pretty soon every good SaaS, company or hosting provider will have a CLI and API interface that the models can be directly. They don't even necessarily need an API. Like, as long as it's like text based, UNIX based, the agent can hack its own API. And then the money part, you insert crypto tokens, you know, put in bitcoin, put in whatever, and the model goes and just pays for whatever it needs. And I think like, you know, there are people working on this, but the thing I am now thinking through is, is pure software dead like, is pure software engineering like an obsolete thing. It's like saying, speaking English, right? The models now speak English. We had to learn code to communicate with the models. Now the models speak English and they speak fuzzy, sloppy English like a human and they understand things. So where's the moat? Like for a founder, hardware, it's a boon, you know, like now you had to build hardware. It was hard to build a software company alongside. Like Patrick Collison says, software is art and it's hard to hire artists. So now as a hardware founder, great. You can have really good software develop fairly quickly if you're creating models. Maybe that's the new software engineering training models and tweaking models and post training and fine tuning models. But classic software engineering, is that dead? Is pure software investable? Is pure software something. And organize a company, a team around and try to get some leverage.
- [10:24]
Guillermo Roush
Did you guys see the. There was an article on X by Mitchell Hashimoto called the Block economy, or the building block economy, something like that. His argument is that the most useful thing for agents to have now is really powerful reusable building blocks. Because to Max's example, you wouldn't expect your clanker to reinvent a queue infrastructure system every time it needs to send an email. It needs to bring in the right building block. That's right size for the task that you're asking for and then say, okay, for this one, it's bold. Mq. I challenge the notion that I would want the agent to reinvent the entire universe and first principles in a way that's incompatible with the rest of society and civilization. It's almost like reinventing highways, laws, policies, et cetera, just for you. Even if there is a potential for extra optimization, extra juice that you can get out of it, there's still a sort of like cooperation at large scale value of saying we're Both depending on Postgres 13.2. And so that's still really, really, really valuable. I would say the category of infrastructure software and building blocks that these agents are going to use, obviously in bias it's this world we're building seems extremely valuable and I don't see the agent anytime soon. And by the way, you could even another metaphor I've been using is like anything that's already been created that the models can reuse, it's like a token cache because you don't want to churn through a trillion tokens to reproduce what's already existing. And so there's always starting points that the model can fork off from, but it's going to change things quite profoundly.
- [11:58]
Blake Shaw
So these are like libraries and dependencies, but for models?
- [12:01]
Guillermo Roush
Yes, for agents.
- [12:03]
Max Hodak
Specifically to Naval's question though, I mean, I learned a program when I was really little and I like that was the thing that through all of being a teenager and in my 20s, I get sucked into it and just code for like 20 hours. And it was super fun and I knew all this stuff about programming languages. I haven't written a single line of code in quite a while now. And I mean, partly that's because my job is different, but also since December I've built a huge amount of software that I now use every day. There's all these projects that I've kind of fantasized about for years that now I'm like using that I've actually built and I didn't write any of that. And I just can't imagine going back to like actually writing code by hand anytime. Like, I mean, I'm unlikely to do that anyway. But just like in general I see that. I have a hard time seeing that as part of the future.
- [12:46]
Guillermo Roush
Yeah, there's something really cool is that you understand how the pieces click together. Like I feel like anyone that understands what an API is and how data flows, inputs and outputs performance because you kind of, you have to orient the model around. This is a certain level of expectation that I have out of this operation that's always been infinitely more useful than writing code. I feel like a really good, proficient engineering leader has been vibe coding through people on Slack or one on Ones because you're transmitting your will, your intent, your experience, and you're letting others run with it. It's just that now we do the same, but with agents. And so I think that's why you've been successful with it. But I don't know that Everyone sees the same level of success.
- [13:37]
Blake Shaw
I mean, I went from not having written code in 20 years to I'm coding all the time now, but through agents, and I'm building tons of software. And it turns out that just understanding the basic principles of software engineering and algorithms actually gets you a long ways. Because the reason I stopped coding was because I didn't have time to, to figure out the latest language, latest architecture infrastructure pieces to plug into. And I know Vercel makes it a lot easier, but even then, just getting started was a bare, like just plugging pieces together, assembling infrastructure was just so annoying.
- [14:06]
Max Hodak
The thing that really changed is, I mean, it used to be that you could build a lot like, like there's a lot that was straightforward, but then you would hit some random thing, and then you could spend kind of some indefinite period of time debugging some narrow thing. And now with the agents, what happens is you just don't get stuck anymore, which is pretty amazing.
- [14:24]
Blake Shaw
Or they get stuck, it's removed.
- [14:25]
Max Hodak
Well, no, I mean, like, relatively quickly. They can find like the right way to do things. And it used to be that, like, I remember when their friends would learn to program and be like, nope, it's just like, intrinsically frustrating. Like if, like that's part of the deal, that's how you learn, and that just isn't true anymore.