← Big Technology Podcast
Loading summary
Transcript136 lines
- [00:00]
Amjad Massad
Don't go into this thinking you can just have prompt and have an application pop up at the other end. At least set an afternoon to give it some good effort and try to get like your first app in. And once you do that, you just get evicted.
- [00:15]
Unknown Host
I have the stat here that repl.it has multiplied by its revenue by 10x in less than 6 months to 100 million in annual recurring revenue. Is that growth Vibe coding or is that growth AI coding?
- [00:29]
Amjad Massad
Vibe coding?
- [00:29]
Unknown Host
Is AI coding just a hobby or the beginning of a technological revolution that empowers everyone to build? Our guest today, Amjad Massad, the CEO of Replit, has some answers and we're here in Replit headquarters in Foster City to speak with him. Amjad, great to see you. Welcome to the show.
- [00:46]
Amjad Massad
Thank you. I'm excited to be on the show.
- [00:48]
Unknown Host
So we're going to talk today about Vibe coding and AI coding, which are two similar but different things. I, I first wanted to speak with you about Vibe coding, which is effectively you write a prompt and then the AI goes ahead and builds software for you. This is something that Replit enables. This is something I've tried. What are some of the use cases that you're finding people are actually having effective approaches with this? Like, where are the places where people are doing this?
- [01:14]
Amjad Massad
Well, there's like, broadly three use cases. One is personal life, family life. So, you know, for example, like, a lot of people like to do health tracking. I'm going to track my sleep, I'm going to pull in data from my Fitbit. I'm going to like have the AI sort of process that data and I'm going to have this app on my phone that I use every day. Or I'm going to build an educational app for my kid to learn math or reading. Or we're going to have like someone built like a chore hero for their family to like, you know, have an iPad on the wall and like, here's who's doing the most chores and gamifying their family life. You'd be surprised how popular this use case is. And so, you know, in the, in the niche that I've always been in, which is like creator tools, there's always been this idea of personal software, malleable software. And by the way, this goes to the early computing history. So, you know, for example, like Apple had this piece of software called HyperCard. HyperCard allowed anyone to make personal software. You know, there's Vigil Basic. It's been attempted so many times, but for the first time now, anyone can make software. So there's a lot of, there's a class of personal software. We have a mobile app and you can use that to make software. It's the most fun thing to do is sit down with your kids, 5, 6, 7 years old and just brainstorm games and make games with them. So that's one bucket.
- [02:45]
Unknown Host
Wait, before we go to the next bucket, I want to ask you a question. So does this say something about the software industry? That the software industry just hasn't served so many use cases or are these use cases non economic? Or is it possible that people will build things for their family and the next thing you know they can serve that mass market and it becomes a business?
- [03:05]
Amjad Massad
It is certainly, you know, there's certainly a market there and you can certainly make a lot of money from that.
- [03:12]
Unknown Host
Okay, because when I think about like this concept and we're going to get to Jobs, but this concept that AI is going to take our Jobs, to me it's like, wait, there's so much left to build. If you think just about what we have today and maintaining that, maybe it will. But there's so much that software has not yet touched that it seems to me that there's more opportunity out there than people are.
- [03:32]
Amjad Massad
But just to touch on your earlier question and you tell me how deep we want to go, because I can talk about this for hours. But in the early computing pioneers, they all had this idea that computers are this. The thing that makes computers special is this idea of program programmability. Right the moment we had a program programmable machine that was first invented by Von Neumann and it's the same architecture that we use today. The thinking was, oh, anyone can use a computer to program, to solve problems, to, to build applications and all of that. It, it did, didn't get mass consumer adoption. And the reason is because coding is hard. And so you had the Xerox PARC as the research and Palo Alto, Palo Alto Research center, they developed gui. One day they invite this up and coming entrepreneur called Steve Jobs. Steve Jobs looks at desktops, menus, items and he's like, he has the Apple ii, obviously Apple II is also still command line, you can write some basic. And he's like, okay, this is the key to get mass consumer adoption of computers. And so he copies what Xerox had and he builds it into the Mac and obviously later Windows and Microsoft copies ui. And then suddenly your computers are usable by anyone. And this is amazing. Now like billions of people use computers and now we have phones based on the same idea. What we lost is this idea that anyone can program a computer. So that's something I've been passionate about all my life, is like computers should fundamentally be programmable. And there's been a lot of different iterations with visual programming. We had the no code low code revolution that happened like maybe 10 years ago. I would say it never reached the full potential.
- [05:25]
Unknown Host
It was more of a buzzword than reality. But now I think it is a.
- [05:30]
Amjad Massad
Multi billion dollar market for sure. But it's not a trillion dollar market. And I think this idea of anyone can make software is such a massive market.
- [05:38]
Unknown Host
Okay, so bucket number two, bucket number.
- [05:42]
Amjad Massad
Two tends to be entrepreneurs. And so everyone in the world has ideas. People build so much domain knowledge about whatever their, their field of work. Right. I was hearing a story today of an Uber driver that is starting to make an app with replit and the app is about logistics. He was a truck driver before and so he has domain knowledge about how to manage fleets for example. But he never was able to make it into software because he didn't have the skill. Maybe he didn't have the capital to go commission a contractor to do it and suddenly he can do it. So pick anyone on the street and they all in whatever industry they're in, they realize that there's a need for a piece of software or technology that no one has built because they don't have that deep domain knowledge. So we see entrepreneurs from all walks of life. One of our favorite one, we talked about it publicly on our replace social media channels. A doctor from the UK that he's like, there's all these apps around managing doctor patient relationships but they never, it's not fully integrated. So you know, you have zocdoc, you can go make an appointment but you know, how do you manage your prescriptions? Can I track my patient over time their progress? Can I get information from their wi fi connected scale from their Fitbit from, you know and so he built this comprehensive platform. He got quoted by an agency £100,000 and he belted less than 200, you know, British pounds, not 2000-002002-00200 pounds.
- [07:31]
Unknown Host
So this is stuff that's being vibe coded effectively prompt in I want to build the software and then replit will go build it.
- [07:37]
Amjad Massad
Yeah. And this is now a startup and we've had startups start on Replit multi million dollar revenue run rate. Some of them have raised at like half a billion dollar valuation. And so we have all the way from small entrepreneurs to startup venture scale entrepreneurs. But this is this gets me really excited because America has always been about entrepreneurship and this is really what attracted me to this country. But actually, if you look at the stats, entrepreneurship over time, although we hear about what's happening in, in the Bay Area and Silicon Valley, there's all startups every day, but the rest of the country actually, you know, new firm creation has been going down over the past 100 years. There was an uptick during COVID where everyone's sitting at home is like, I.
- [08:16]
Unknown Host
Haven'T started my business.
- [08:17]
Amjad Massad
Right, exactly. Which is great. But that actually we had a regression to the mean and I think with AI we're going to see that explode again. So that's the second bucket entrepreneurs, one more bucket. Third one is people at companies like this one. So actually I'll give you a story from our HR department. We have a small HR department. Replit is kind of a lean team. We're 80 people. And so we have a lot of these SaaS tools. We pay tens, hundreds of thousands of dollars to do every specific kind of function. And sometimes they don't really fit our use case. We think they're too expensive. So this HR person had a need for an org chart software that can visualize the org chart, that can add, remove people, maintain a history, can look back and see what happened, what changes it. It did and went on the market and saw that none of the software captured the exact bespoke use case where she wanted to connect it to our kind of more other HRIS systems or databases. And they were, they all needed, you know, they were all very expensive and needed a lot of IT support. So she went into replit and built it Vibe, coded it in three days. And so that meant that we have a system that exactly fits our use case. And that also meant that we're not paying 10, 20, $30,000 a year for a piece of SaaS software. And that's happening across the board. We see companies saving hundreds of thousands of dollars replacing SaaS software with built in with internally built software.
- [09:58]
Unknown Host
Now, do you need to be someone with some technical background or some technical know how to be able to do this? Well, because I'll give you an example I mentioned to you before we start recording, I opened a replit account this week. I wanted to build a simple choose your own adventure game. I think it's called History Havoc where you can work your way through different history scenarios. But it just didn't get to the point where I wanted it.
- [10:22]
Amjad Massad
How long did you work on it?
- [10:23]
Unknown Host
So I spent about an hour on it not a lot of time. And I also full disclosure just on your starter plan. I'm not paying yet, but I couldn't get it to work. I also tried to build this story tracker and it wasn't able to crawl the web the way that I hoped it would. So it still seems like this to a lot of people that this is something that is helpful. If you're technical, you want to make a prototype. But these use cases that you're giving seem to be full blown companies or working pieces of software. So explain that disconnect.
- [10:56]
Amjad Massad
I think it requires grit. Obviously there's like stochasticity in the machine learning model.
- [11:01]
Unknown Host
So explain what that is.
- [11:04]
Amjad Massad
The same prompts can put you on a path of success based on randomness that's happening inside the GPUs. There's this parameter in large language models called temperature. And temperature is literally how random is the sampling of the words coming out of the of the LLM. So the LLM, the way it works, you give it a piece of text and it tries to complete the next word, the next token as we call it. And the way it happens, it generates a lot of candidates. So the red fox, jumped, slept, whatever. But jumped is the top one. It's the highest probability one the model have seen it occur after the sentence and millions of of cases. But you know, you have the sampler and could be randomizing what it picks and that that randomization makes it more creative. There's also inherent random randomization inside the like the Nvidia chips or the GPUs. So this style of software is unlike the software, the classic software where everything is discrete, input, output. Machine learning models have inherent randomness and that's a feature, not a bug that creates creativity. Right. So some people sometimes get on a bad luck with a replay. We're obviously trying to mitigate a lot of these problems, but I would say it also requires grit. Like the game you just described. Professional programmers coding might take them a two days thing on replit you can do it in two, three, four hours. But it would require a little bit of grit. So it's not magic. And the skills you were talking about, the technical skills, although they're not required, you can build them up over time. And our environment kind of shows some of these features as you're working with it. And so I would suggest to people that don't go into this thinking you can just have prompt and have an application pop up at the other end. I would say at least set an afternoon to give it some Good effort and try to get like your first app in. And once you do that you just get addicted.
- [13:27]
Unknown Host
So there's Vibe coding, which is again prompt and then you make an app and then you can refine it with more English. And then there's AI coding where you could basically have AI, you know, complete your code, big autocomplete. So what do you think the opportunity is in Vibe coding versus AI coding? And where do you think the energy energy is in the AI industry today?
- [13:52]
Amjad Massad
I gave the analogy of the history of computing and I think it's a very suitable analogy for a lot of what we're talking about. Early on in computing we had the mainframes. So the mainframes really big room sized computers. IBM used to make them. Large corporations and governments use them in universities, but everyday people didn't have access to them until Apple created Apple ii and that was the first mass consumer market computer. And since then we've had Windows and all these devices. The mainframe was already serving the professional's needs, but it wasn't serving the consumer needs. Now if you look at the market for PCs versus the professional workstations, Sun Microsystems, all of that we which used to be the case, the PC not only was a much bigger market, eventually it subsumed the more professional grade software. And this is called the disruption theory. A lot of your audience that might be into business history or theory, Clay Christiansen used to be, I think a Harvard Business School professor and he wrote this book called the Innovator's Dilemma. And the idea is that a lot of technology start at the lower end and because their mass market appeal, they onboard a lot more users and customers and over time they reach certain economies of scale and they subsume even the upper end of the market. Currently the upper end of the market is what you were talking about with AI coding tools, right? So there's like 30 million developers all over the world, maybe a little more. Now those are professional developers that went to computer science classes in college. They were trained for four or five years and now they're working at companies. If you make those developers 20, 30, 40% more productive you get depending on if you're a company of the size of Google, it's like billions of dollars worth of productivity, right? So the market is really obvious there. You can go apply it and get, get. But it's a zero sum market. If you look at Copilot, which is Microsoft's product, which was the first market versus Cursor, which is the more modern kind of AI coding, IDE as cursor is eating market share. You can see it is almost exactly proportional to copilot declining in usage. So that's a sign of a zero sum market. It is very lucrative and there's a lot more growth to be had there. But it is not this fundamentally revolution that we can be going through where it's anyone can make software.
- [16:49]
Unknown Host
Let me ask it this way. I have the stat here that Replit has multiplied by its revenue by 10x in less than six months to 100 million in annual recurring revenue. So is that growth vibe coding or is that growth AI coding?
- [17:05]
Amjad Massad
Vibe coding, really? Yeah.
- [17:08]
Unknown Host
And is, are these vibe coding programs or these bespoke programs that people are building with prompts, are they in production or are they mostly hobbies that people fool around with?
- [17:19]
Amjad Massad
Depends on first bucket is more hobby, personal life. Second bucket entrepreneurs. As you know, most startups die, so most startup ideas don't make it to fruition. The 10% of startups that are small businesses that get off the ground, they get the most value out of replit and some of them are in production now. You know, I've talked about a lot of these stories, but you know, for example, we have this creator, his name is John Chaney, he's a serial entrepreneur. Used to take him many months and hundreds of thousands of dollars to build applications and now he can spin up a business and get to million dollar run rates in the matter of weeks. Obviously he has experience like he knows the formula of what it means to be an entrepreneur. But people can learn that over time. And in terms of the enterprise we have, for example, Zillow, the CEO of Zillow, recently on New York Times DealBook talked about how everyone at Zillow is using Replit to accelerate product innovation. Because product innovation no longer depends on engineers. You can have product managers do the entire iteration getting user feedback even without going to the engineers. So it just like increases it. We have duolingo a bunch of these customers that are really focused on innovating, building their second third product that are now using Replit for, for a lot of these use cases.
- [18:42]
Unknown Host
So is the use case that you build like a prototype and then you get some feedback and then if everything works out well, then you build into the product with your like core engineers.
- [18:51]
Amjad Massad
That's one, that's one use case.
- [18:53]
Unknown Host
Okay, that's interesting.
- [18:54]
Amjad Massad
Yeah, that's one use case. It's really great. It rapidly improves the time to market. The second use case is operations and internal tools. So for example, like Sears Home Services really old company employs people that go and fix homes. And they had an operations team that wanted to build a lot of AI tools and software for their field workers to be able to manage their work and their earnings and all of that. But their software was like this hundred year old Cobalt programs and the engineers were kind of busy kind of migrating that and improving that. So the operations team started using Replit to spin up these AI applications that are deployed, used in productions by those field workers every day to manage their day and kind of design the optimal routes to, to how to maximize their earnings per day. So the operations type use cases tend to be deployed running in production.
- [19:56]
Unknown Host
Okay, so just so I'm clear, are you also facilitating AI coding or is it mostly that you turn Replit into a vibe coding company?
- [20:05]
Amjad Massad
My mission has always been about how do you enable people to do this magical thing that is creating software. It's one of the most magical, exciting experiences you would ever have. And I was a founding engineer at Codecademy and before that I built open source tools to do that. Codecademy taught millions and millions of people how to code and we changed, you know, a lot of lives. So the DNA of replit has always been about how do you make programming more accessible. It was, it had like a more developer bent at some point. But because Replit is sort of batteries included platform, we give you the database, we give you the authentication, we give you the, the deployment, we give you the scalability review all of that out of the box. You don't have to go anywhere else to do any of that. It always meant that the people that are getting the most out of it tend to be, they're not, they're not, not professional programmers, although professional programs do use it. I would say like that's 20% of the use cases.
- [21:09]
Unknown Host
And the question is then do the people using repl.it then come for the people who are those professional programmers? There's a funny thing that happened. I watched you have a talk at the Semaphore Tech event in San Francisco a couple months ago and I tweeted something that you said that in one year or 18 months, companies might be able to run themselves without engineers. And then somebody responded to me with this meme where they said founders in public. AI is writing 99% of our code. In six months we won't need any engineers. Founders in the DMs. Does anyone know a good React developer? $30,000 bonus. And I will name my firstborn son after you. So can you explain that? Disconnect between this view that engineers are going away and this still like very intense demand for engineers in the market.
- [21:59]
Amjad Massad
I never made the point that engineers would go away. I make the point that entrepreneurs can start businesses without needing engineers and that we already see. That we already see. You know, I meet YC companies and Y Combinator is the most prestigious startup accelerator in the, in the world Bay Area. And in the past, Y Combinator would encourage you to go get a technical co founder. But like we said, there's so many people with amazing ideas that don't have a technical co founder. And so they're starting to get into YC and what they tell us is we're just going to build this thing on replit. We're going to see how far we can get. And they often get really, really far. Now if you're building a venture scale company and you want to get to hundreds of millions of dollars of revenue and you want to become billion, 10 billion, $100 billion company, you're going to have to hire engineers. But if you're trying to build a company that creates a really great living for you, even you know, you can potentially get rich from it. You I think we're almost there where you can do it on your own without any developers. And so when I'm talking, I'm talking to our audience, right? As opposed to I'm not, I'm not talking to Microsoft or Facebook. They're not going to replace developers anymore. My view on developer productivity is that developers are much more impactful than they used to be because a single developer can be so highly leveraged these days. And so yes, you want to find the best developers and we're expanding the team. But the scale that replit is at today would be 10x the number of people if we're a SaaS company. Five years ago, wow. To reach $100 million in run rate, you know, five years ago on average we'd have like 500. A lot of companies will have thousand people.
- [23:59]
Unknown Host
How many do you have?
- [24:00]
Amjad Massad
80.
- [24:02]
Unknown Host
Wow, okay. You know, it just makes me wonder that as companies grow like this, what the future is going to look like from the technical side? And I'm curious, do the folks who have technical abilities, you know, let's say the economy expands like this and everyone and their grandma can build, literally can build a company using AI tools. Do the technical people then come in and sort of clean up the problems? Are they your like cleanup crew? I was reading this funny article and publication called Futurism. It says companies that tried to Save money with AI, are now spending a fortune hiring people to fix its mistakes. And it was about, it wasn't about Vibe coding, it was actually about content like content marketing or like your, your content marketing plan is just filled with this like you know, kind of bland chatgpt generated copy. And half the time it says as an AI assistant, this is the message.
- [24:57]
Amjad Massad
But I would use. You'll see so many hits. Yeah.
- [25:00]
Unknown Host
So I am curious to hear your perspective on does, does the technical field end up becoming cleanup crews for Vibe coding gone wrong?
- [25:09]
Amjad Massad
Let me just tell you where I think technical folks have a job security today. So I think if you're writing software for my Tesla, I don't want you to be Vibe coding. I want you to write low level verifiable code. If you're writing code for space shuttle, you're writing low level verifiable code. But also even, I mean those are life or death situations. So I think we don't need Vibe coding there. We need more precision. But even sort of large scale platforms, if you're building core cloud component, the storage or virtual machine components on AWS or Google Cloud or Azure, you want systems engineers that understand distributed systems, understand how to create fail safe systems at scale. So I think engineers there have job security for the foreseeable future.
- [26:12]
Unknown Host
Right.
- [26:13]
Amjad Massad
Because of the problem of stochasticity of these models and, and all of that you need, you need every line of code to be reviewed and managed very carefully. Now where I think AI is going to have the most impact is on product and people building products, they want to iterate on it really quickly. They want to internal tools. People want to replace all the of the SaaS software that we have today. So I think that's happening now in terms of the cleanup. I mean depends on where you think AI is headed. Do you think that AI is good at making software but bad at maintaining it and it's going to stay bad maintaining it for the foreseeable future? If it's good at making software, it must also be good at refactoring software or testing software. Actually right now it's pretty bad at testing software because there's this thing called reward hacking. So when you do reinforcement learning over large language models, you're giving it a reward every time it does the right thing. Reward hacking is the way to. So the models become incredibly goal focused. They want to get that done right. That's what RL does. And oftentimes what we see when we try to get the models to test things, it will start Being corrupt in a way, it will change the test to fit the mistakes it made or sometimes delete the tests. It's really fascinating behavior that actually Anthropic published research on. But do you believe that's going to be the case forever? Obviously not. I think over the, the next three or six months I think we're going to see machine learning models being able to test and verify their work.
- [28:01]
Unknown Host
Okay. So one of the biggest things that this moment depends on is affordable large language models coming from the foundational companies. And that means, you know, in layman speak, if you're going to want to build with AI code, you have to actually have the ability to bring in models from an OpenAI or anthropic that are going to generate that code and not break the bank as you do it. And we're still in this VC funded or investment private market investment moment where we don't really know the true cost of these models.
- [28:42]
Amjad Massad
Meaning the foundation model companies might be losing money on those.
- [28:46]
Unknown Host
Do you think they are?
- [28:46]
Amjad Massad
And the application companies? I don't think they. On the gross margin basis, I don't think they are.
- [28:50]
Unknown Host
Right. But they're also training and that's of money. They're not, they're not profitable. They're losing billions a year.
- [28:56]
Amjad Massad
Of course. Yeah, yeah.
- [28:57]
Unknown Host
And there's been this thing that's happened recently with I just want to run a bayou with both replit and Cursor where I think end users have seen pricing gone up. Zitron wrote about this and I think it's a pretty good piece talking about effort based pricing within replit and that is effectively a different pricing structure. We've seen replit users talk about the fact that their actually paying a lot more for the same services than they were previously. And his theory is that OpenAI and Anthropic found quiet ways to jack up their prices for startups and we're beginning to see the consequences because Cursor had a similar. Who happened at Zitron?
- [29:41]
Amjad Massad
Oh, okay.
- [29:42]
Unknown Host
Is that what's, is that what's going on?
- [29:44]
Amjad Massad
No, the prices haven't gone down and that's a problem. So we used to see these, you know, we've seen Token prices come down 99 since, since ChatGPT and we've seen token prices come down year over year. The thing that's a little disturbing right now is that token prices are not coming down. You better believe that the unit economics of the labs are getting better because of economies of scale, because these models are getting easier to optimize. But they're actually not reducing prices. And so the concerning thing, are we reaching a steady state? Is there price collusion? Is there now oligopoly of few model companies that are able to create these state of the art models and there's no downward pricing pressure. Right. Are there investors starting to demand better business fundamentals? I don't know exactly what's happening. We should talk about the Chinese open source models in a second because I think that will introduce an interesting mix to this. But it certainly is the case that we're not seeing token prices go down. The main reason we went to effort based pricing is. Let me explain about effort based pricing. So when we released Replit Agent V1, version one of Replit Agent would work for like two minutes at a time. You would give it a message, it will go try to do something for two minutes, either succeeds or fails, gives you a checkpoint, commits the source code and charges you 25 cents. And the reason it only worked for two minutes is because the capabilities of the models meant that it can only work for that long now. Models got better and we knew that models are going to get better and, and they're going to be able to work for 10, 15 minutes. And so with version 2 of ReCapt Agent, started in beta in February, came out of beta in April, the model would work for 10 minutes. And so we can't charge 25 cents for like a 10 minutes. So what we started to do is came up with the heuristics. Every nine tool calls will do a checkpoint. And so as it's working, you'll see it make a checkpoint, checkpoint, checkpoint. That's a hack, right? It often means that if you make a small change that cost us 5 cents or whatever, you still get 25 cents. But also if you make a big change, you might be costing us a lot more than what we charge you. So it was really out of whack. Now that was a hack and we need to move to a place where we're charging the user proportional to how much the model's working and the cost on us. And we think that's the best way to create a long term sustainable business. And when those two things are aligned, also opens up new opportunities where when we do optimizations, we're always optimizing. We actually had like 20% optimization on cost recently. We pass it straight to the user because now cost and price are tracking with each other. What happened with our community. The first thing that happened is there was a sticker shock. So you're used to seeing 25 cents every 10 tool calls and suddenly you're seeing one and a half dollars or two dollars after 15 minutes of work. So that's one, two. It's true for some users who are really advanced, the cost has gone up for them because the projects contact size is bigger, their workloads are bigger. But early on in the project it's actually cheaper. You mentioned that you worked for an hour, you didn't have to sign up for the core. We give free users $3. So you work for an hour on $3.
- [33:53]
Unknown Host
Not bad.
- [33:54]
Amjad Massad
Yes.
- [33:55]
Unknown Host
It's cheaper than a developer.
- [33:56]
Amjad Massad
It's cheaper than a developer for sure. And so that being said, we, we recognize that on advanced users it is now it's almost, there's attacks as, as you go on. So we're trying to optimize the context window and make sure that advanced users are not getting, you know, more expensive experience. The other thing that happened is we introduced thinking mode, reasoning mode and we introduced like high power mode and people are enabling those and sometimes they forget them enabled. And now we actually started hide it under advanced, like don't enable this unless you know what you're doing and you want. And there's like a 5x multiplier on it. So a lot of people are enabling those, getting these large checkpoints and we're like, we put out content, we put out a video, we put out some documentation, a blog post. Here's when to use reasoning mode and you shouldn't always have it on. So just describing all of that that's happening, there's a macro trend in the application space where a lot of companies were subsidizing the cost of, of like a lot of companies were paying money, more money on topic and OpenAI than they, they were making. Was that, are you doing that we on V1? No. On V2, yes. Because the pricing model was out of whack with how we're charging. Actually the median cost per checkpoint kind of went up only a little bit. So on the lower end we're charging user less right now. But it used to be that on the lower end we're charging users more, on the upper end we're charging users less. So now it's more proportional, more fair for both. And so now we have solid business fundamentals that allows us to grow. And I've been talking about how replit has been my mission, my passion for 8, 9 years as a company, 15 years as a side project and a vision. And we're not trying to rapidly expand revenue while losing money. In order to flip this company to sell it. We've seen all these acquisitions or raised the next big round. We're really trying to build a business for the long term. And replit is made of all these different components. So we have cost, not just on AI, we have cost of traditional compute, CPUs, storage, databases, all of that stuff. So kind of to summarize, you know, I've talked a lot about what was happening specifically in replit. I don't know what's happening in Cursor. I think for sure that their situation is like a little different because they, they, their dynamics is. I think they actually did raise prices for the. You should talk to them. But, but I think it's like a little different dynamic than, than what happened to replit. To summarize, there is a concerning trend where token prices are not going down. Is that going to be the case for the future? Because that sucks. Because we want to be able to use more tokens to create more intelligence, to be able to create better applications for users. Is that going to be the trend forever? Are we reaching a steady state in cloud, for example? We kind of reach a steady state when you have a monopoly, there's no pricing pressure. But when you also have an oligopoly, they not intentionally, without talking, start colluding, you know, because it's, it's like a market dynamic where it's like if you don't lower a real price, I'm not going to lower our price. It's not in our incentive as a whole because we own 25% each of the market. Right.
- [37:37]
Unknown Host
Okay. I do want to ask you about something that you didn't mention when you looked at the different factors for why prices might not be going down. There might be investor pressure, there might have been this equilibrium reached. Or is it possible that these models have just gotten so big and expensive to run that the fundamental economics of AI are just not working? So explain why.
- [38:03]
Amjad Massad
You can surmise the bigness of the models based on speed, token throughput. It's not perfect, but if you remember, GPT 4.5. GPT 4.5 was an experimental model from OpenAI. It was the idea, let's train a trailing parameter dense model, meaning it is not sparse, meaning all the neurons are activated on every request. And it was so slow. It's really hard to run these things. The new models, even when they're big, they're sparse models that are called moe, mixture of experts. So in every request there's a router layer that takes it to the expert part of the circuit in order to answer that question. So, you know, there are models with trillion parameters, but any given request is 32 billion active. And that's like a kind of small model. And what we're seeing based on speed and things like that is actually probably the models are getting more efficient. I mean, Deep Seq showed that the models are getting more efficient. And if Deep Seq open source was able to make it, you better believe that the labs are also became more efficient.
- [39:15]
Unknown Host
Okay, I do want to speak with you about Deepseek and Kimi K2 and other Chinese models. Let's do that when we come back from the break right after this. Hey everyone, let me tell you about the Hustle Daily Show, a podcast filled with business, tech news and original stories to keep you in the loop on what's trending. More than 2 million professionals read the Hustle's daily email for its irreverent and informative takes on business and tech news. Now they have a daily podcast called the Hustle Daily show where their team of writers break down the biggest business headlines in 15 minutes or less and explain why you should care about them. So search for the Hustle Daily show and your favorite podcast app like the one you're using right now. And we're back here on Big Technology Podcast with Amjad Massad, the CEO of Replit, talking about all things AI code, vibe coding, and now let's talk about these Chinese models. So this episode will air a couple weeks after the emergence of Kimi K2, but we're talking about Kimike 2, which is another Chinese model. And of course, this Deep Seq moment was a big moment where we found out that this seeming small hedge fund in China with some GPUs was able to engineer a more efficient model. That story will be debated about what actually happened for a long time. But let me ask you one influence of Deep Sea question and then we'll get into the others and Kimike too. So you mentioned before the break that Western models have taken after Deep Seq. So do you think they learned what Deep Seq did and sort of put those new innovations into play in their own models, or was that coming anyway.
- [40:48]
Amjad Massad
From what we've seen from the Twitter sphere is that it seemed like there were some surprises because researchers just talk a lot. It seems like there were some fundamental innovations from the Deep Seq models that weren't known in the West.
- [41:04]
Unknown Host
But have they implemented those now? And that's probably why we're getting more models.
- [41:10]
Amjad Massad
Yes, I'm Sure. The models are getting more powerful without getting slower.
- [41:15]
Unknown Host
All right, so tell me about Kimmy K2.
- [41:17]
Amjad Massad
When Anthropic came out with Sonnet Claude 3.5 that was a fundamental shift in the industry where the models got a lot better at coding. And suddenly instead of making small snippets of change, Sonnet could, could generate entire files and enabled things like Cursor Composer, where it was a start of Vibe coding where you can put in a prompt and generate entire files and all of that or generate large edits. Then Sonnet 3.5 V2 was the first model. It was a computer use model, was the first model where you could sense that there's agentic, true agentic behavior. I don't know what they did. They cracked rl. Whatever happened there. You can give a model of VM and it can give it a virtual machine. Virtual machine. You can give it an objective and it can sleuth around in the virtual machine, look at the files, do run some commands and, and then write a program, test it and, and then solve, solve a problem, that experience. There's a benchmark called Suite Bench, Software Engineering Bench and you start seeing the score going up dramatically. I don't know, I think we're at like 10% last year and now we're at like 70% and 80%. 80% world class coding in such a. It's the interesting thing about Sweetbench is not just coding because there are other benchmarks that, that just do like the code generation, right. Sweetbench. I think the harder thing about it is the agentic workflow is writing the code, testing it, running commands, finding files, understanding files. And this, this stuff was like a huge jump that happened with, with Sonnet 3.5 V2, then 3.7, then 4.0. And they've, you know, kudos to Anthropic. They've been able to make create a lead that hasn't been bridged by the other labs. Gemini is getting there on the Agentix stuff, but I would say OpenAI kind of lagged behind O3 has some interesting agente capabilities, especially around deep research, but it hasn't been as good as the other models on this agentic stuff. I mean they did some interesting stuff with Codex. I don't know if those models are in the API, but everyone is using Claude for the agentic coding experience. The interesting thing about kimik2 I would say is they caught up. Not to clotsonic 4.0 perhaps clotsonic 3.7. At least that's the Vibes right now before the other labs.
- [44:06]
Unknown Host
Wow.
- [44:08]
Amjad Massad
You know, I think that's really underreported again. This is Vibes. Everyone's starting trying to figure it out, but it looks like it has a really good sweep bench. It is doing 65 on sweep bench. Sonnet is 7 to 72. If you do sampling, which is you. For every step you ask the model to generate n number of solutions, you can get up to 72%. It can be competitive with Sonnet and.
- [44:32]
Unknown Host
This is with export controls.
- [44:34]
Amjad Massad
Yes. And I think in the paper they talk about the solution is scaling reinforcement learning. We also saw that with Grok 4. Grok 4 spent as much on reinforcement learning as they spent on pre training, which is unheard of.
- [44:53]
Unknown Host
But even, but that's an important point because with that big spend on reinforcement learning, GROK is a competitive model, but they spent billions, billions, hundreds of millions on rl. I don't know which is this goal setting form of training. And it's not like it's a new category. So it shows there's some limits.
- [45:12]
Amjad Massad
XAI is an amazing team and they've been able to achieve so much in so little time. But it's also well known in the industry that they're compute inefficient. They're so compute rich that they're throwing computer. The other problem in many ways.
- [45:27]
Unknown Host
Yeah. So what is the significance that Kimik2 is now as good as some of these anthropic models?
- [45:35]
Amjad Massad
A small research lab, I think the rumor is like the 200 people again, there's export controls as well, was able to figure out how to catch up to near state of the art agentic coding models before big western labs that are highly capitalized a lot, a lot more researchers was able to.
- [45:58]
Unknown Host
And does that mean then that they can undercut them on price or.
- [46:03]
Amjad Massad
So let's see.
- [46:05]
Unknown Host
Right, so let's see. Are you going to integrate Kim and K2?
- [46:08]
Amjad Massad
We're looking at it. We're looking at it.
- [46:09]
Unknown Host
There's a lot of.
- [46:11]
Amjad Massad
So far we're impressed.
- [46:12]
Unknown Host
Okay.
- [46:12]
Amjad Massad
So far we're very impressed. So I mean look, these things, sometimes they overfit to certain things and I would say it's like requires a month from the entire community to kind of like really have consensus over like whether the model is really great. And similarly with Grok 4, I think a lot of people are playing with it, but my sense is that it is good enough. And again the economics are so good that you can expend more tokens to get more intelligence. So it is not at the frontier, but it is near frontier. But given that it's cheap and fast enough, you can spend more tokens that, that creates some more interesting potential for us to create new capabilities in our platform because it is cheap and fast.
- [47:09]
Unknown Host
How much cheaper is it than the Anthropic models?
- [47:10]
Amjad Massad
And I am bad at this. But like I would say I don't. One fourth maybe. Oh yeah, that's, that's on the official API. Perhaps more even. I forgot. Maybe you can look it up after this.
- [47:24]
Unknown Host
We're gonna have to. This, this show is gonna come. It will come a couple weeks after we record but we'll have to release this segment early because that's yeah. Astonishing. I want about Anthropic. I can vibe code in Claude. I do it all the time. And they also have this Claude code product where people are, you know, writing prompts, getting code. Are they your competitor long term or how do you see them on that front? Because that's the question is eventually do the labs just subsume everything else that's built on top of it?
- [47:56]
Amjad Massad
I think the question is for them. Right, like you should ask. I know you're going to talk to Dario. You should ask him the question.
- [48:03]
Unknown Host
Listeners, viewers, this will air a week after Dario, but I'm about to after this go in and speak with them. So you might see this question a week earlier.
- [48:11]
Amjad Massad
Yeah, so look, we're, we're committed to our relationship with Anthropic. They're a great company to work with. We have a great partnership and it's not like we, we didn't anticipate them wanting to build products in addition to, to the models. Every model company is building products right now. The thing that they're going to have to manage is their, their pricing. If they're, if they're going to compete by undercutting everyone on price, they're going to destroy the ecosystem. Right. I think Replit right now is, has the advantage of this platform that we built over eight years that it's going to take a lot of blood, sweat and tears to build. And also the user experience that is focused on, on, on that sort of non technical user and like we really care about this, this idea of empowerment right now. Cloud code is, is used by developers and loved by developers and I think they're competing head to head with Cursor, Windsurf and those kind of products whether they're going to move into our space again. You should, you should ask them about that. But I think a more interesting question. How, how, how do they want to nurture the ecosystem versus just go, go. And because they can compete on price, they can steamroll everyone.
- [49:38]
Unknown Host
Right. I mean, Claude, code is. The max package is 200amonth and you see developers getting thousands of dollars of API value out of that.
- [49:48]
Amjad Massad
This is not good for.
- [49:48]
Unknown Host
You must notice this.
- [49:49]
Amjad Massad
Yeah, yeah.
- [49:50]
Unknown Host
Not good for the ecosystem?
- [49:51]
Amjad Massad
I don't think so.
- [49:52]
Unknown Host
Why?
- [49:53]
Amjad Massad
Because again, you're competing on price, not how good the product is. And there's a price at which maybe the quality doesn't matter as much as how many tokens I'm getting. Although cloud code is a really good product. But then cursor, no matter how good they make the product, they're still going to be more expensive and a disadvantage. And people are like, well, you know, I really like cursor, but like I can get 10x more value out of cloud code. And so the marginal gain in product quality will not matter as much.
- [50:33]
Unknown Host
Right.
- [50:33]
Amjad Massad
And that will, that will destroy the ecosystem.
- [50:36]
Unknown Host
Fascinating. I mean, I think that this question is just one small question or one version of a big question we're going to be asking as these AI models get bigger and better and more intelligent. So I want to, I want to spend the rest of our time talking about some philosophical questions, if that's okay with you.
- [50:53]
Amjad Massad
Sure.
- [50:55]
Unknown Host
There's this idea that the AI research houses want to use the code that they generate to sort of or these coding applications to speed up the development of the next model and compress the time it takes to get better models. People call it an intelligence explosion or things of that nature. Do you see that as feasible? And is that something we should want?
- [51:19]
Amjad Massad
So you should think about what are the limiting factors to the next version of a model? What are the bottlenecks? Where does that invasion needs to happen? I can think of a few areas. One is research. So this is algorithmic research, like figuring out the next algorithm, next improvement in training, algorithm and inference algorithm, whatever it is, and then systems engineering. These training runs are massive. That requires a lot of interesting distributed systems engineering. Will AI coding help with AI research on the margins? Perhaps they can spin up Python notebooks faster. I don't think it's that impactful. Like the models can't do AI research, can come up with ideas and test them really quickly. Will it help with distributed systems? Perhaps it is not as impactful right now on writing Rust code or C or Go, whatever as it is on JavaScript and Python and higher level languages. And like I said, it requires a little more precision and better system design to and that the bottleneck to really good distributed systems is, is design and not like the. The amount of number of codes you can generate, which is more true on the product side. On the product side, you're just. You need to generate tons of CSS and JavaScript and try a lot of things and delete a lot of things and iterate and do AB tests and all of that stuff. So like, volume of code is important there. I would say on the, on the back end, distributed systems, I don't think volumes of code is. So I'm reasoning in real time now.
- [53:16]
Unknown Host
Right.
- [53:16]
Amjad Massad
And I guess my answer would be I don't think it's going to have anything more than marginal improvement on speed to the next model.
- [53:30]
Unknown Host
All right, I guess that makes me rest a little easier then. By the way, just on a. You speak with a lot of people in the AI industry. Of all the economic activity in the AI industry today, how much of it do you think is code?
- [53:43]
Amjad Massad
Someone actually made that, that slide that's been going around. I think it was something like 1.1 billion of ARRs and the AI coding and vibe coding space.
- [53:53]
Unknown Host
Okay. So it's actually kind of small compared to like the total. Well, so revenue.
- [53:57]
Amjad Massad
Yeah. So anthropic has four. $4 billion, right? Let's say. Yeah, $4 billion ARR. Let's say they have also have their own products, their own coding products. I don't know. Let's say 1.5 billion is off of that is AI coding. It's substantial, but it is not the entire thing. But then you have $10 billion of ARR on OpenAI side and that's more consumer.
- [54:22]
Unknown Host
Now, on the rush to artificial general intelligence, which we've talked a little bit about, do you think Silicon Valley is the one that should sort of possess this or be the one that controls it? I mean, it's an interesting place. There's a lot of kooky ideas here. And it seems like if this is possible, it's going to be something that's controlled by or owned by one or more of the labs here. Is that good?
- [54:49]
Amjad Massad
Assuming it'll happen, and assuming one company will reach their first and have some kind of advantage or monopoly over AGI, which I'm not entirely sure I agree with these assumptions. But if you want to make, if you want me to make these assumptions and then answer the question, I'd be happy to, but I just want to make it clear that.
- [55:10]
Unknown Host
Yeah, let's make those assumptions.
- [55:11]
Amjad Massad
Okay.
- [55:11]
Unknown Host
I know there's a lot of things that need to happen in order to get there.
- [55:13]
Amjad Massad
Yeah, I might have some fundamental disagreement with this assumption.
- [55:16]
Unknown Host
But let's talk through the disagreement.
- [55:20]
Amjad Massad
I don't think AGI is any point in time for one. And I think there's going to be. Right now the distance between any lab is just an order of few months on anything that really matters. You know, between 01 Preview and Deep Seek was like two or three months between, I mean the biggest one was, was this Kimi K2 one that we just talked about that, that was like maybe nine months or something like that. But it's still sub one year. And so whomever reaches AGI first, they're not going to go into intelligence explosion and just like suddenly, you know, superintelligence gets born. People, you know, other labs will catch up really quickly and, and then, you know, there's going to be a lot, a lot of models. I don't think it's going to look that different from the ecosystem that we have today. And if you assume that AGI will actually have an impact on model development through research and speed of development, then everyone will get the benefit of that as well. And so actually you might get even more competition once you, once you have AGI. So I don't think it's going to be a monolith.
- [56:27]
Unknown Host
Okay, but if it is.
- [56:29]
Amjad Massad
Okay, if it is, is it would I want Silicon Valley? And I guess it's like a moral.
- [56:38]
Unknown Host
Yeah, philosophical question.
- [56:40]
Amjad Massad
I wouldn't want any human being to. We're all fallible. That's why markets work. That's why, that's how a human society evolved over time. It is, you know, Darwinian evolution and free market capitalism. It's all based on competition and, and, and, and the idea that like one system would be this, this monolith controlled by one human being. We've seen disasters and massive human suffering happen when there is this top down sort of leviathan type thing. Whether it is in Soviet Russia with all the deaths that happened there, or in China or whatever. And oftentimes as I understand it, in the Soviet era they had this kooky idea about evolution. I think. What was it called?
- [57:51]
Unknown Host
I'm not familiar, but I'd love to hear the explanation.
- [57:53]
Amjad Massad
Yeah, so basically they had, they thought that evolution is this bourgeois idea. You know, communism has this, this idea is like anything that's, you know, high class bourgeois is like wrong. And so this had this ideological view on how evolution works or should work that led them to do agriculture in the wrong way and led to famine and, and that sort of thing and like, and so oftentimes they do kill people and cause mass suffering, mass poverty. Even if they don't intend, even if like outside of the Gulags and all the other oppressive, explicitly oppressive system, those systems are inefficient because they have these wrong ideas and there's no competitive pressure to have better ideas. And so that's fundamentally broken static system that doesn't improve like competitive systems. And I think if we were have a super intelligent monolith controlled by single company or single human being, it's bad. It's fundamentally really bad.
- [59:04]
Unknown Host
I agree. All right, last question for you. We're seeing a lot more AI love bots come out. Is that a good thing or a bad thing that people are going to fall in love with AI more often?
- [59:14]
Amjad Massad
It's a bad thing like a priori bad thing. The reason humanity grew and flourished and all of that is because we have babies. And anything that takes away from that, especially given the fertility rate is so low right now, is, is, is what will, will potentially lead to really massive problems. Especially since capitalism is based on large middle class consumerism like the, the, the, the, the, the current of how the economy work requires, that requires taxpayers to fund Social Security and like elder care and all of that. The welfare state is based on this large young population. And when that starts to collapse, you're going to have massive instability in these systems. So even if humanity doesn't go extinct like Elon would say, although Elon is the first person to create a really interesting mass market companion. I think right now interesting is a.
- [60:27]
Unknown Host
Fun word for.
- [60:30]
Amjad Massad
Looks like it's really compelling. I see people, you know, right now talking about it on, on X so.
- [60:34]
Unknown Host
Much and looks it's got some work. Yeah, but it is, these type of things are going to definitely become real partners to people. People when this technology has been bad or hardly workable have gotten married to them right before LLMs. So it's going to happen again and in greater numbers.
- [60:58]
Amjad Massad
The question is, I wrote this, I used to do like more creative, creative writing. I wrote this essay on the hyper real. So I think it is like French post modernist theorists like Baudiard wrote about this concept of the hyper real. And the idea is like we have reality like you and I are interacting right now and then you have media created realities. And the reason sometimes it is hyper real, it is more intense than reality itself and more enticing than reality itself. So you know, even in real things, you know, for example, when you, when you get a, when you eat like A, I don't know, Twinkie or something like that, like fatty, salty, sweetie kind of snack. It is like, it is not like a piece of chicken or beef or whatever. It is this hyper real thing. It like hyper engages your senses and it makes you addicted to it. And similarly, social media is hyper real in a sense that I can go get, go there and get a lot of social interaction, tweet something, get hundreds of likes and it's like much easier than going out in the wild and finding 100 people that could like me. Yeah, right. And so we have these technologies that are, and the market around it that, that is bootstrapped to make us addicted because they're so, so much more enticing and low effort than the reality that we know and experience day to day. And I think that is a huge danger for the existence and evolution and longevity of human civilization. And I think it is. I talked about how good free markets are, how important, how competition is important. This is one thing that capitalism is so adversarial to humans at. Right. And, and so I don't have a solution for it. I think in the past the solution was religion. For example, in like Islam you can't depict humans or animals in art. That's why in Islam the, the art became more geometric and if you go, go, you know, visits like the mosques or whatever, they have like all this geometry that, or like calligraphy. That's really interesting. And I think part of the idea there is, I think the hyper real, like if the ultimate expression of something so enticing is a virtual being like we're seeing right now. And I'm not saying like, you know, Islam had like the foresight or whatever, but I think it's, you know, religions used to have this built in mechanism to protect against these predatory sort of consumer products. And I wouldn't know how to solve it in the future, but perhaps it is potentially societal, maybe governmental. I'm always kind of skeptical of that or religious kind of protection.
- [64:32]
Unknown Host
We're going to need something.
- [64:33]
Amjad Massad
Yeah.
- [64:33]
Unknown Host
So Lord help us. Amjad, great to see you. Thanks so much for coming on the show.
- [64:37]
Amjad Massad
My pleasure.
- [64:38]
Unknown Host
All right, everybody, thank you so much for listening and watching. We'll be back on Friday to break down the week's news. Until then, we'll see you next time on Big Technology Podcast.