Baseten CEO Tuhin Srivastava on the AI Inference Crunch, Custom Models, and Building the Inference Cloud

Summary

---

No Priors Podcast Episode Summary

Baseten CEO Tuhin Srivastava on the AI Inference Crunch, Custom Models, and Building the Inference Cloud

Released: May 1, 2026
Hosts: Sarah Guo & Elad Gil
Guest: Tuhin Srivastava (Founder & CEO, Baseten)

---

Episode Overview

In this episode, Sarah Guo and Elad Gil sit down with Tuhin Srivastava, CEO of Baseten, to discuss the explosive growth and challenges in AI inference, the rapid evolution of custom and open-source models, the realities of compute supply, and building for the future of intelligent applications. Baseten has experienced 30x growth in the last year and is on track for $1B+ in revenue, highlighting both the scale and urgency of the AI inference market. The conversation spans from the strategic importance of inference and post-training, to geopolitical implications of model origins, to the nitty-gritty challenges of scaling infrastructure and talent.

---

Key Discussion Points & Insights

1. The Inference Explosion & Market Dynamics

• Scale & Demand: Baseten's 30x growth is fueled by an increasing realization that AI can be embedded everywhere, across both open and closed-source models. The “long tail” of custom/specialized models is materializing as customers in-house intelligence and the application layer expands. (00:49)
  • "Everyone is realizing that you can put AI everywhere… the long tail models coming true, customers in-housing a lot of that intelligence themselves." — Tuhin, (00:49)
• Application vs. Lab Layer: Tuhin defends the continued existence of an independent application layer, arguing that companies’ unique workflows and user signals can’t simply be absorbed by foundational labs.
  • "The application layer will exist… because what is valuable to a company is the user signal only they can gather." — Tuhin, (02:17)

2. Who's Adopting AI—and How?

• New AI-Native Companies vs. Enterprise In-Housing: The majority of inference is still coming from new application companies, but enterprise adoption is just ramping up. Once enterprises become comfortable with closed-source APIs, custom models will follow.
  • "Most enterprise adoption is well ahead of us. That’s one of the very exciting things about AI—there’s just so much still to come and people are underestimating that." — Elad, (05:25)
• Learning from AI-Natives: Building with frontier customers (Abridge, Open Evidence, Decagon, etc.) allows Baseten to pre-empt enterprise needs, given these companies sell into regulated, demanding sectors like healthcare. (06:21)

3. Open Source vs. Closed Source, & Geopolitical Tensions

• Model Choices: Leading AI companies generally choose models for capability, optimizing later for cost. There’s robust use of both Western and Chinese-origin open-source models. (08:16)
• Security & Geopolitics: Security concerns around Chinese models exist, but Tuhin downplays evidence of direct risk while stressing the vital need for a strong US open-source ecosystem.
  • "If we don’t have access to that intelligence in that form, it’s just a massive loss… we won’t be able to innovate as fast…" — Tuhin, (12:00)
  • "There are five labs in China creating open source models and we’re struggling to get one set up…" — Tuhin, (09:46)

4. Custom Models & Post-Training: Now the Norm

• Workload Breakdown: Over 95% of tokens served on Baseten are from custom, modified models—very few are vanilla open-source. (13:20)
  • "No one is just running the vanilla open source weights… almost all are modified for their own use case." — Tuhin, (13:36)
• Investment in Post-Training: Baseten acquired a research team to build post-training expertise, directly tying infrastructure to customers’ ability to continually improve and specialize their models. There’s a virtuous loop between post-training and inference. (14:34, 17:10)
  • "Inference creates data, you do evals, you can now post-train on that reward function… it’s the entire loop." — Tuhin, (16:23)

5. Inference Capacity Crunch: Realities & Strategies

• Severity of Supply Shortage: The compute supply crunch is worse than many realize; there’s very little slack globally. Baseten has built technology and operations for extreme flexibility—deploying across 18 clouds and 90 clusters to secure compute. (18:47, 21:48)
  • "No matter as much as we hear about it, I don't think people realize how bad it really is… very little slack compute available." — Tuhin, (18:47)
• Supply, Suppliers & Contracts: Not only is GPU supply constrained, but there’s also a shortage of reliable, operationally competent cloud providers. Locking in capacity now requires multi-year contracts and significant prepay—driving unique working capital and financing needs. (21:54, 23:20)
• Strategic Advantage: In a constrained compute world, actually owning compute is a core differentiator. The software layer (custom, sticky inference APIs) further locks in customers, far more than commodity GPU leases. (24:45)
  • "Inference with the software layer included is incredibly sticky. None of our top 30 customers have ever churned…" — Tuhin, (24:45)

6. Chips, Multi-Vendor Ecosystem, and Technical Frontiers

• Multi-Chip Future: While diversification is expected (Nvidia, decode/inference-specific chips), Nvidia’s scale, supply chain, and CUDA developer ecosystem makes it hard to unseat them soon. (26:56)
  • "People really underestimate NVIDIA’s supply chain… the ability to move fast, given their scale, it’s hard to see anyone compete soon." — Tuhin, (26:56)
• Runtime, Workload Trends: Continued focus on optimizing runtimes (diffusion, agents, sandboxes, async batch inference) and deepening the inference–post-training loop. (28:42)
• Scale Surprises: Edge cases at scale manifest as systems and kernel-level incidents, rarely just LLM-specific quirks. (31:44)
  • "You start seeing limitations with kernel panics, log overflows… the craziest stuff is these runtimes are pretty immature." — Tuhin, (31:44)

7. Scaling Talent & Culture at Hypergrowth

• Leadership Lessons: Baseten was flat, engineer-heavy until recently. The imperative now is leaders who can solve whole problems, clarity on hiring philosophy (first principles, kindness, low-ego). Result? Retention is high. (34:19)
  • "If you feel like you need to be involved in everything, it's probably a cop out… you probably don't have the right people." — Tuhin, (35:06)
• Operations Culture: True infrastructure companies live an “on-call” culture; pagers and alerts become part of daily life, and it quickly screens for fit. (36:58)
  • "Inference can't go down… my cofounder's 7-year-old asks, 'Is that a P0?'" — Tuhin, (37:13)

8. Jevons Paradox & Demand Elasticity

• Lower Costs = More Inference: Lowering inference cost just increases consumption—people want more/better intelligence, not less.
  • "If you make it cheaper, they'll insert more intelligence anyway… agents are just longer running now." — Tuhin, (38:54)
  • "Inference going down just begets more. It is the last market." — Tuhin, (40:12)

9. The Next Few Years: The Inference Cloud as a New Paradigm

• Future Vision: Intelligence embedded everywhere—“units of cognition” sold as a utility; companies must embrace the shift or become obsolete; personalized agents for every aspect of life—AI-powered “concierges” for health, education, personal management. (41:01)
  • "Everything is smarter… you get better care, more software, more things built." — Tuhin, (41:01)
  • "Concierge is everything for everyone." — Elad & Tuhin, (41:41)

---

Notable Quotes & Memorable Moments

• Hypergrowth Reality Check:
  • "We've grown a ton over the last 12 months… the answer is always just go bigger, go faster… but the big one is compute." — Tuhin, (32:59)
• On Build vs. Buy—What Makes a Winner:
  • "GPUs as a service is not sticky. Inference with the software layer included is incredibly sticky. That's been seen." — Tuhin, (24:45)
• Cultural Ritual:
  • "When we have P0s—everyone on the call, may as well be a siren that goes off in the office." — Tuhin, (38:09)
• The Inference Cloud Thesis:
  • "To me, this is what an Inference Cloud looks like: you are very good at inference, then do all the things tangential, loop into inference and partner where necessary." — Tuhin, (30:39)

---

Timestamped Topics & Highlights

| Time | Topic | |-----------|-----------------------------------------------------------------------------------------------| | 00:49 | Baseten’s scale: 30x growth, AI everywhere, open source, post-training goes mainstream | | 02:07 | Will the application layer survive vs. the AI “labs”? | | 04:34 | AI-native startups vs. enterprise AI adoption: who drives volume? | | 06:21 | Learning from AI-native customers to pre-empt enterprise needs | | 07:55 | Open source adoption evolution: Mistral, Llama, Chinese origin models traversing frontiers | | 09:46 | Security/geopolitics: Should the U.S. worry about Chinese models? | | 13:20 | Custom vs. vanilla models: tokens served, how everyone is customizing | | 14:34 | Post-training expertise and acquisition rationale | | 17:42 | When customers should start customizing/post-training | | 18:47 | Severity of the compute supply crunch—no slack, global multi-cloud, operational diligence | | 21:54 | Securing capacity: contracts, term lengths, prepayment, working capital considerations | | 24:45 | What makes an inference player “sticky”? Software glue, not GPU commodity | | 26:56 | Multi-chip world? Why Nvidia’s ecosystem is dominant “for now” | | 28:42 | Technical roadmap: runtimes, sandboxes, prefill, decode, async batch, evals | | 31:44 | What breaks at extreme scale—kernel panics, log overflows, runtime immaturity | | 32:56 | What keeps Tuhin up at night: Capacity & the pressure to go even bigger | | 34:19 | Scaling philosophy: moving from flat org to empowered leaders | | 36:58 | Operations culture—inference outages, alerts, fit & retention | | 38:54 | Jevons Paradox: Lower inference cost drives consumption even higher | | 41:01 | Future state: The era of “cognition as a utility,” ubiquitous agents/concierge experiences |

---

Conclusion

This episode delivers a front-row perspective on how AI inference is rapidly maturing into one of technology’s largest, most competitive markets, with Baseten at the forefront. Key takeaways include the crucial role of custom/post-trained models, extreme compute constraints dictating business strategy, sticky value at the software layer, and a future shaped by infinite loops of learning and intelligent automation. The Baseten story is both a real-world playbook and an early window into AI’s infrastructure future.

---

Transcript

[Host 1]

Foreign.

[Host 2]

Hi listeners today a lot and I are here with Tuhim Srivastava, the founder and CEO of Base10, the AI inference cloud. We're here to talk about capacity constraints for AI compute, why inference is the last market, how the workload is changing, the open source and perhaps multi chip Future and what 30x scale in a year looks like. Tuhen, welcome back.

[Host 1]

Hi, great to see you.

[Tuhim Srivastava]

Thanks for having me.

[Host 2]

All right. You are in one of the craziest markets, AI inference, it's very important. There's a lot going on. You guys have grown 30x over the last year and I think I can say you're expecting to do more than a billion dollars in revenue this year. What's going on? Tell us about scale.

[Tuhim Srivastava]

Yeah, no, it's been, it's been nuts. I think what's happened over the last, honestly at 24 months but just kind of keeps getting bigger and bigger is that I think everyone is realizing that you can put AI everywhere. You have all these great options available from closed source to open source models. The open source models have crossed some sort of chasm in terms of their baseline capability and then I think RL techniques and post training is for specialized models has become mainstream enough and there's enough examples of it working. The customer's realizing they can kind of own their inference more and more and what that's meant for us is more the long tail models coming true, customers in housing, a lot of that intelligence themselves. And as the application layer just gets bigger and bigger and bigger and that's growing when we are just someone index on that and we've been around to be able to collect the demand.

[Host 2]

There's an existential question in here that I think everybody is continually asking of does the independent application layer get to exist at all versus the labs? How do you, you have to believe this. Why do you believe it?

[Tuhim Srivastava]

Yeah, look, I think it'd be a sad thing if it didn't exist in general and I think that's like my. But sadness is fine

[Host 1]

all the time.

[Tuhim Srivastava]

Sadness is fine, but that's not the reason why I think the application layer will exist. I think the application layer will exist for a number of reasons. One is because I think this idea that what is valuable to a company is the, the user signal that they can gather, that only they can gather and to the extent that that is encoded in a model, I think a lot of their business will be at risk. But to the extent that it is encoded in workflows, that is where they will be able to develop Modes. So a good example of that is say a company like Abridge, where the clinicians edits of the notes and what they do with those notes after the fact. And the. The thing that happens inside the EMR three steps down, that becomes a workflow that only.

[Host 2]

Can you explain what Abridge does?

[Tuhim Srivastava]

Sorry. Abridge is an ambient scribe that is used by physicians in almost all hospitals in the U.S. i think Larson investor Shiv's amazing. Great company, great team, great product. And they've basically got this very, very deep integration into hospitals, into clinician workflows. And my argument would be here is that actually it's very, very hard for a frontier model company to be able to eat away at that because they just don't have access to that user signal. And what will happen over time is the folks who have access to that user signal can start to post train models on that reward signal and start to get long horizon agentic models running that. And I think to the extent that that is possible and that signal is differentiated and unique and is somewhat rare to get access to, there will be an application layer. And I think support companies is another example of that where a support task isn't one shotted. Usually at a company like Base 10, when a ticket comes in, there's like what, like 1, 2, 10, 20 actions that get taken and that is where, you know, someone can develop a specialized model.

[Host 1]

So there's almost two versions of this. Then there's new companies like Abridged or Decagon or some of these other things that you mentioned that are doing these new types of applications that are using AI and they sell it to customers. The other is enterprises building things in house or building their own models. What proportion of the market today do you think is these new application companies versus enterprises just adopting AI? And how do you think that looks in a couple years?

[Tuhim Srivastava]

Yeah, I think. I think you asked me the same question two years ago.

[Host 1]

I had to be repetitive.

[Tuhim Srivastava]

It is crazy. The answer is just that it's crazy that the answers do. I think. I think if you look by inference count, it'd be 99% the fall. I think that kind of represents the scope of the opportunity here is that the majority of the market hasn't come online and added AI into the market.

[Host 1]

Yeah, most of enterprise adoption is well ahead of us. And I think that's one of the very exciting things about AI. There's just so much still to come and people are underestimating that, I think 100%.

[Tuhim Srivastava]

But what's cool is that we're seeing the transition happen before it was like hey, are they using AI tools? I don't think that was immediately obvious two years ago. I think that's obvious now that yes they are. Are they using closed source model APIs? I think they're starting to get there. I think once you do that and then you kind of see what is possible, then comes the whole customer model adoption. I think that is all that is ahead of us today.

[Host 2]

So if the majority of your customer base today is as you described, the former application companies, AI natives, the fast growing, I mean some of them are at considerable scale now like the abridged cursor open evidences of the world. What you know, what do they teach you? What does that push the company to do? How do you think about serving them versus evolving for the enterprise?

[Tuhim Srivastava]

Yeah, I think firstly like you just learn a lot by building with the companies greatest scale doing the most interesting things. We think of it two ways. I think there's like the most obvious way which is just build for the highest scale the customers that will push you the most from technologically and everything kind of will fall into play. I think the Stripe evolution as a company showed that which is like Stripe now serves so many enterprises but 12 years ago that wasn't the case but they just built for the frontier and kind of went with them. I think the second way we think about this is to just think about building for companies that are serving enterprises. So yes we don't serve the enterprise but our customers serve enterprises. Abridged open evidence decagon all these writer gamma all these companies serve enterprises en masse and what we actually get is like a translation of the requirements from them which is like they're like hey, we need this sort of data retention, we need these web models need to be deployed. This is the types of GPUs level agencies they're okay with. They there's the model requirements from a transparency perspective that they care about. And so I think that is actually the more nuanced answer is that if you listen to what their needs are, we actually get a full translation of what the enterprise will require. I would say that by serving companies like Abridge and Open Evidence we're probably pretty well suited to go serve the healthcare system given that they are selling and latent health, given that they are selling to them.

[Host 1]

How much of a shift are you seeing in terms of the types of open source models that are being used? And so I think we've seen an evolution where two, three years ago I think the main Thing was kind of mistral and then a few other things and then Meta kind of came along with Llama and then it kind of really shifted in terms of the most performant models or of Chinese origin in different ways. Do you see that sort of mix reflected in terms of what's being used by our customers?

[Tuhim Srivastava]

Yeah, I think customers, at least the customers we are serving are very and these are like the fastest growing AI companies in the world that are very forward thinking, they want to use the best model and they are optimizing. I think there's a subset of tasks which I think is small today where people really start to start with cost but everyone comes for capability first because that's really where the economic growth is being unlocked, where the value is being delivered and then they optimize. And I think that's actually been. And so with that in mind, you name, you name it. Everything from GPT oss all the way to Moonshot model to deep seqs to Orpheus, which is like really good text to speech models. Customers generally want to use whatever's at the frontier. And I think the difference has just been I think we have a lot more visibility into how to run these and how to run these really well and secondly that they're good.

[Host 1]

Now there have been a number of different concerns raised about the use of Chinese models in particular security or is there something embedded in the models or Trojan horses or other things? A, do you think there's any real concern there? And B, people often talk about how there should be US counterweights to this from a geopolitical perspective. Do you think that's something that's legitimate or something we should be worried about? Or how do you think about the origins of these models versus their uses?

[Tuhim Srivastava]

Yeah, look, I think these models firstly are fantastic. They're amazing. We work with these teams. They're truly awesome. I'd say look, it is hard for me, it's hard for me to see and I could be wrong, but if I network bound these models that they're not magically going to be able to cross those network boundaries and to Data Zetter and I don't and I've never seen any real evidence except from some very early models that I think people picked up on very quickly that there is some agenda or bias built into these. I do think that to some extent is. I think there is importance to the US that we develop our own models. I think that would be a massive loss if that. There are five companies, five different labs in China that are creating open Source models and we're struggling to get one set up, so it's necessary. I also think it's inevitable. The Deep Seek moment. A year ago I remember someone saying to me, and I thought it was very well said, which is, and the world's changed a lot. But they said, hey, we should just forget that this is a Chinese model. We should just act like this came from Meta and build with that in mind. It's like, I think you're kind of missing the forest from the trees. There's two scenarios, right? Either America does not ever come up with good open source models. I think there's probably a fundamental problem there, or we will get there and we need to be ready for that world.

[Host 1]

Yeah, that makes sense. It's interesting because, you know, like you, I think it's very important for the US to have a strong open source footprint here, at least for now. It looks like effectively the Chinese government is subsidizing at least a large subset of these models and that subsidy or surplus is effectively just being passed on to US enterprises who are adopting these models. In other words, it's a way for the Chinese government to effectively subsidize US enterprise in an indirect manner. And I think that's a little bit lost right now. But you know, it's always interesting to weigh that against some of the other concerns that are raised. I appreciate your comments on this and

[Tuhim Srivastava]

I think the concern also just there just becomes like what happened if we aren't able to, if it is fun. Like I think if you think about the economics here, which is Deep Seek by most, Deep Seq is a very good model. You know, like, and like you can argue whether it's at the absolute frontier or not. But like let's, let's go back three months and it's there and so think about everything. And we were doing a whole lot of things three months ago. And so let's just think about that. Well, you know, if it, you could run deepseek like 20% of the cost of running open anthropic models in production with comparable better latency, probably better reliability if we don't have access to that intelligence in that form. I think it's just a massive loss. And as a country we won't be able to innovate as fast because the cost of intelligence going down in control of intelligence, what we have seen just means more intelligence, intelligence being embedded in more places. Yeah.

[Host 1]

An important note here that we didn't mention explicitly is that the state of the art models, the ones that are Most far ahead on the frontier are actually still the closed source, anthropic, OpenAI, Google, et cetera.

[Host 2]

What has been actually maybe you can just characterize workload, a little bit of tokens being served on base 10. How many of them are from custom models of some kind versus vanilla open source.

[Tuhim Srivastava]

Today it is all custom. It's basically okay, so like 95% plus 95% and I think that's really cool to be honest. Look, we have two businesses, we have three businesses right now.

[Host 2]

Should we help you count?

[Tuhim Srivastava]

No, no. So we have like dedicated inference, which is basically custom auto inference. Your SLA is your sla. Then we have shared inference, which is shared inference, endpoint shared SLAs and then we have a training business. I'd say 95% of the tokens today are on the first business. And almost all of them, there's probably for almost all of them, the customer is making some modifications to the model with their own data specialized for the use case. And I think what's even more important is they might be compiling in different ways. No one is just running the vanilla open source weights. Like you might be customizing it for quality, but you also might be customizing it for performance.

[Host 2]

You made an acquisition of a research team a few months ago. You've mentioned post training customization. What was the rationale behind the acquisition? What is that team doing today?

[Tuhim Srivastava]

Yeah, so the rationale around the acquisition was we are infrastructure and product people. We have product people and now are really good infrastructure people. And we didn't have much of a research capability ourselves. And what we saw was the market moving heavily and heavily that we could accelerate the market itself with post training resources either productized or even just as resources for that market. So parsed was a company that was a base 10 customer. So there were post training models and running them on base 10. And I think what they realized was that they would eventually need to become an inference company. And what we realized was like, hey, we really needed that expertise because it represents a way for us to get closer to the customer earlier and be able to support them more. And it just made sense as pairing them together. And just as I said in the opening statement here, which is as more and more post train models have come up, we've realized that the demand for people to either for software loops to do post training or for post training expertise is very high. And we're really, really investing in that. There are also a bunch of Australians. I like to think that we had a bit of alpha there, but yeah, that's been fantastic. They're working with all sorts of customers. And it's also very interesting when you start, we were doing a lot of research on the performance side and less so on the post training side. It's interesting as we've started to do a lot more research on the post training side. You start to see how linked inference and post training are. And even when you think about stuff like quantization and when you should do that and. How you train the model affects how you need to quantize for inference and how paired these problems are has become very apparent. And more and more we rely on the post training. Inference are kind of both sides of the same problem because inference ideally will beget more post training where inference creates data. You do evals, you can now post train on that reward function that you found with those evals and hopefully just set up the entire look.

[Host 2]

Plenty of folks from Ant and OpenAI, Sam, Greg, et cetera have said in recent months that inference is super strategic. Inference talent is strategic, capacity is strategic. So between that and post training, these are very difficult to gather capabilities. I imagine that lots of your customers go to you guys for advice on how to do this progression of moving to custom models. What do you tell people about the life cycle and when they should invest in that?

[Tuhim Srivastava]

Yeah, I think it's, hey, go prove to yourself with the best in class model that you have something worth optimizing. And I think a lot of you know, if a customer comes to us was that meme which was like, it was like two years ago. It feels like no GPUs pre product market fit. It's like no post training pre product market fit is what, is what I'd

[Host 2]

say the people that you're working with here are very at scale first.

[Tuhim Srivastava]

Yeah, they have a user signal that they know how to optimize and they've shown that they can serve customer value and that they have something special around that value. And once you have that value, it's like, okay, now how can I do that better, faster and cheaper? With the idea being that, hey, if you need to be very good at customer support, you maybe don't need to be that good at coding and that a specialized model might be a better fit for that problem and you can do it better, faster, cheaper.

[Host 2]

What about the capacity side? You started with unifying capacity across all the clouds and neo clouds. How do you think about this when everybody keeps talking about a supply crunch and a multiyear supply crunch?

[Tuhim Srivastava]

I think there's so much narrative around the supply crunch. And no matter as much as we hear about it, I don't think people realize how bad it really is. There is very, very little slack compute available. We run pretty large clusters ourselves and we run them at uncomfortably high utilization. You know when I'm saying we're like mid-90s utilization most of the time there is, we have made, we, we have, we sit in 18 different clouds now. We have 90 clusters around the world across 18 different clouds. And like, you know, initially we started, we like built this technology to be able to like kind of create one runtime fabric that spans all these different clouds and try to abstract that away from our customers as a way to think about reliability, latency, failover, all these things that we think are going to be very important for very mission critical use cases. That same technology, like just our ability to get compute wherever humanly possible has been really, really helpful in our ability to get supply. And what I mean by that is we can be introduced to a new provider in a different country and have it up and running with the whole base 10 inference stack as part of the fabric. Part of the fabric in half a day, maybe less. And that gives us enormous flexibility. Even for us, it is hard for us to grow. We have a, I think it's, yeah, we have a 4pm standing meeting for the company where we basically like how do we manage capacity for the demand right now? I think the second part, which people don't really, the second part that people don't really understand is that there are also a lot of suppliers right now that it's kind of grifty. I think, you know, they haven't run data centers before. You know, they don't understand SLAs especially for inference. And so even when there is capacity available, there's a lot of diligence. We run a lot more than this and we have redundancy, so it's fine. But if you, there's probably like a dozen good clouds and I probably like put like three or four of them in the gold tier. And I think that just means that not only are we supply crunched, we're supplier and operationally crunched onto people who can run these data centers as well.

[Host 1]

How far ahead can you actually buy capacity right now? In other words, is there any slack in the market if you buy two years ahead or five years?

[Tuhim Srivastava]

You mean contract length or actually like hey, I want this in January 28th?

[Host 1]

Either one.

[Tuhim Srivastava]

Yeah, yeah.

[Host 1]

I mean it's more the. I want this in January 28th or at least I have Some visibility into my future supply.

[Tuhim Srivastava]

Yeah, you could buy that. But you got to also remember how quickly the market is, how quickly the market is moving and like, you know, that gets balanced somewhat off. Like the fact that the H100 is such a great chip and like, and then, you know, it's crazy. It's four years, four and a half years old. The price is going up still. Yeah. Maybe at the useful life, nine years. Yeah. So, you know, that's, that's good. But at the same time, at the same time, you know, yes, you can do that, but you know, you're making a lot, like you're making a lot of bets. Yeah. As part of that. And then in terms of, I think that's the big thing that's changed over the last six months is that the term length that people want has just gone up. So if you, if you wanted a thousand, ten, 24B, two hundreds, which is from a good cloud, right now you're not getting that less than a three to five year contract right now with probably a 20 to 30% TCV prepay. So actually what becomes important when acquiring capacity is you need to have enough demand to supply it to server, but then you also need like a low cost of capital which is, which is actually changing the dynamic pretty significantly.

[Host 1]

Does that, does that impact how you think about going public as a company? Because arguably.

[Tuhim Srivastava]

Yeah, I think you'd go sooner. Yeah, exactly. Yeah, I think you need like, I think the, and I think there was demand for that, but I think, you know, the pull the IT also, you know, one of our, one of the, one of the realizations that we had recently and with software people and so we don't think like this all the time, is that our business has very interesting working capital requirements. And I think even. And that as a result of that it has very interesting financing requirements. And we're not, at least right now we're not even going down to the debt.

[Host 1]

There's also things you could do in terms of debt or other structures.

[Tuhim Srivastava]

Learned a lot about debt recently given the supply crunch.

[Host 2]

Inference being one of the top couple markets you've been going after, you have plenty of people who understand this problem and therefore some competition. How do you think about what are the factors that create a dominant player here or a winning player? Is it, as you mentioned, cost of capital? Is it access to supply, Is it software? Is it demand? Yeah, just being excellent at everything.

[Tuhim Srivastava]

Yeah, look, I think what's so interesting about inference is GPU.

[Host 2]

Is it operations?

[Tuhim Srivastava]

I guess, yeah, I think GPUs as a service is not sticky. I think that's been seen. Customers generally just see that as commodity. Inference with the software layer included is incredibly sticky. Just none of our top 30 customers have ever churned. We're talking 400% annual NDR around our business. And so it's very, very sticky. So I think that software layer is very important. The optimist in me is like, oh, there's so much value in the software. And we will build the best software layer for inference that exists. I think, as I think is becoming clear now, access to inference compute is a strategic advantage. And I think that is the strategy that even the labs are going after which is like if we have all the compute, good luck running inference.

[Host 1]

Yeah. In a world of constrained compute, the number one thing to own is compute. And so just owning it in and of itself as an asset. And I think people underappreciate that.

[Tuhim Srivastava]

Yeah, you can't make a good hot chocolate without milk. And you know, unless you're a vegan, unless you're vegan. I don't want the vegan inference.

[Host 2]

Well, I gotta ask you, people might want, they might want alternative milk. Right. So like when you. The H100 is a great chip, people, you know, want a B200, they want GB200, they want of course, tons and tons of Nvidia. When you think about making a bet, you know, several years in the future, do you believe that there's a like multi chip world? Like what do you think happens from a compute perspective on chip side?

[Tuhim Srivastava]

Yeah, I think diversification everywhere is a same way. I want to water many models, I think we want to water many most things.

[Host 2]

And I think you'd be sad if it didn't happen.

[Tuhim Srivastava]

Yeah, I think everyone would be sad. I will say to some extent which is. Yeah. And I think there will be inference specific chips. I think you have like decode specific chips. I think. And we're looking at.

[Host 2]

And Nvidia said this.

[Tuhim Srivastava]

Yeah, yeah. I mean that was a whole Grok LP thing. It's like, you know, I think that is very straightforward and makes sense. I think people really, really, really underestimate supply chain stuff with Nvidia. Like how good they are at that cuda. How good CUDA is the developer ecosystem around it. And you know, we. The ability like to me like one of the most important things as an infrastructure company in this moment is how fast you can move and you can move fastest with Nvidia today. And I think that is the reality. And just given the scale that they operate at. Given the scale that they operate at, It's hard to see. I'm not saying it won't happen the short term in the next couple years how anyone's going to be able to compete for that, especially with so much of the other players. What you need to be able to compete here is the ecosystem to form around you. And if you tie up all your supply with one buyer, which a bunch of the other chip providers have done, it's actually hard for that ecosystem to form. If you think about if you're a big lab and you have a proprietary deal with one chip type where you get 90% of the supply, it's actually in your best interest to make sure you get 95% of supply and everything that's built for you. No one else can ever use it.

[Host 2]

When you think about reacting to the market, what do you think is happening with the actual workloads that you have to go invest in? Obviously code agents and long horizon agents over time have become a big deal. People talk a lot more about CPU compute, video inference is different. I don't know if it's that sandboxes, what's important for you guys to invest in now?

[Tuhim Srivastava]

Yeah, look, I think for us all the runtime stuff is obviously very important and what that means is what chips we run on, how we run, what kind of workloads we support. Do we get very good at diffusion transformers? Yes. Coding agents need sandboxes. We should code sandboxes. There's all sorts of new speculation techniques to get faster inference. We need to do that. Even stuff like kvcache away routing and that stuff's a bit old now, but continuing to be very good at that and somewhat disentangling pre fill and decode and starting to treat them as separate problems. I think that's, you know, something we are very focused on and we're seeing massive gains there. That's at the runtime level I'd say. Beyond that, everything we think about is how to create more of that loop between inference post training because we think that just begets more inference. And so we will build a partner in almost everything there. So we're going to work with the best evals companies in the world to make sure that's very well integrated like BrainTrust into and around base 10. We will partner with all on the sandboxes side, build the best sandboxes experience that will exist and then we'll create the best training APIs to make it so continual learning becomes somewhat of a solved problem. It's not just like a discrete thing. That's I think the core base 10 product thesis. It's like how do we build that loop and then everything around that becomes how do we make sure that we can do everything we can to ensure that gets as big as possible? That's access to compute. That's an infrastructure. Make sure we can get compute anywhere, make sure we have access to our own compute. And then I think it's all the primitives that come after that that just become incredibly marginal creative, both for us and our customers, which is stuff like sandboxes and the async batch inference, like how do we drive utilization by having a first class batch inference experience. To me this is like what an inference cloud looks like. It's that you are very good at inference and then you start to do all the things tangential, all that loop into inference and partner where necessary and build when necessary. But we really do want to own start with our core inference story and then go down to unblock, supply, accrete margin and go up the stack to unlock value.

[Host 2]

What would surprise people about some of the issues you discover only at scale? I'll give you an example. I was surprised when you guys ran into scale limitations, like fundamental limitations with some of the hyperscaler products that you were consuming. And because I kind of think of, you know, the AWS GCPs of the world is supporting infinite scale.

[Tuhim Srivastava]

Yeah, I mean I think you just. And like again, like, I think very, very large companies that run services big scale is probably the same stuff is that all the edge cases just become.

[Host 2]

You actually experience them.

[Tuhim Srivastava]

You experience them and like, you know, and I'll give you a few examples here. Like you see, you know, you start seeing yesterday we had, for the first time ever, we saw some kernel panic that only happened because some fluent bit worker was creating too many logs and the scale was too big and it was all into one node and it was happening at the same time by two different workers. So you see all the systems level and kernel level problems. But then you start to see, I think the craziest stuff is that you start to see with LLMs that these runtimes are pretty immature. Even how we use kvcache is probably a little less sophisticated than most people see. And we are starting to see the limitations of the current and the next set of primitives that need to be built from a scale security performance perspective. But I think it's really at the runtime level and the systems level. But the edge cases are, I'd say a Lot more systems level than they are. LLM specific.

[Host 2]

What are the things that keep you up at night?

[Tuhim Srivastava]

Capacity.

[Host 2]

Quick answer.

[Tuhim Srivastava]

Yeah, I think capacity. I think the other one is probably just this market's so big and it represents a moment when you should be as aggressive as possible. And really we've grown a ton obviously over the last 12 months, the last few months. But the answer is always just go bigger, go faster. And I think that's really, really fun. It's also a little exhausting and it's also like we are all in somewhat uncharted territory in terms of how fast and how big you can go and how things can get. But I think the big one is compute. I think there's no world in which there's enough computer to get the amount of value that we want to get out of LMS in the next five to 10 years.

[Host 2]

Or we have to invent a lot of new stuff. Yeah, maybe if we just talk a little bit about what you're learning. Scaling 30x is an aggressive thing to go through as a company. You've brought in a lot of really amazing talent. Danny and Samir and Stephen Day, folks on both the technical and the go to market side. What do you think is working about how you are recruiting and scaling or what's your philosophy on that?

[Tuhim Srivastava]

We were very, very flat until I know, 12 to 18 months ago. I remember I went on a walk with a lot actually and a lot were just like, you just need leaders and it's actually so contrary to everything. You know, as engineers you're like, oh, everything is overhead, everything is overhead.

[Host 2]

You once told me, I think that you're like, hey Sarah, Sarah, what about we just have engineers instead of salespeople?

[Tuhim Srivastava]

Yeah, yeah, bad.

[Host 1]

Everybody learns it.

[Tuhim Srivastava]

But I remember like, you know, you said it so clearly at the time a lot and I think that's what we're notice, which is like actually having a leadership team that you can trust, that you can trust is, is so important. I think the, the two or three things that I'll say is like you want people where you can give them whole problems. And so like, you know, if, if you are, if you feel like you are micromanaging, if you feel like you need, if you feel like, you know, you have to be involved in everything. I think that's a bit of a cop out as a founder because you're just like, I just need to be involved in everything. It's like, no, you probably don't have the right people. I think the second thing is be Very, very clear what you're optimizing for. Because I think when you're very, very clear what you're optimizing for the people. And if it's something generic, we want the smartest, hardworking people. You can't do much with that. With us, what we cared about was, hey, actually we don't care about a lot of people who have done this before. We care about people who think from first principles. Work has to be a high priority, but they also have to be very kind and nice and care about the collaborative environment. We don't have a hero culture, very low ego. And if you need a manager, it's probably not the right place to be. But I think when you have that clear rubric, the people become very apparent that will fit into it. And the people that don't fit into it also become very apparent. I think what's more like we've hired amazing people like you mentioned, but I think what's a lot more interesting is I think we haven't had a ton of turnover there unnecessarily. People tend to work because we are very clear on what we want. It took us a while to get there though.

[Host 2]

What about the idea of like an operations culture? And we were talking to Alyssa and Henry about this and she's like, well, the hard thing about Cloud is actually just operations. I slept with a pager under my pillow for a decade. I don't think I've seen you detached from your Slack channel.

[Tuhim Srivastava]

Yeah, my phone is buzzing right now. I'm here.

[Host 2]

I hope that's not a step one.

[Tuhim Srivastava]

I'm getting anxious.

[Host 2]

And you've been concerned before, like, do people get it? What is distinctive about that?

[Tuhim Srivastava]

I think one, I think if you've worked at an infrastructure company. We were once in a meeting with a bunch of AWS execs and this was very senior AWS folks. All their pages went off multiple times during our 45 minute. I think it's very much just a cultural thing. But yeah, inference can't go down and you learn to like, you know, I think Amir, my co founder, when his pager goes off, his 7 year old said, Is that a P0? Is that a P0? And so, you know, I think that is you just have to get used to. And that's the culture you live in and it just changes the speed. But also it's, you know, becomes like a cultural thing. I think it's very, very. It rejects people that don't fit into

[Host 2]

it very, very quickly, like engineers who avoid patriotism.

[Tuhim Srivastava]

Yeah. You know, when we, when we have P zeros, we're like everyone on the call, like, you know, like there's been a joke that there may as well be a siren that goes off in the office when there's an incident. So.

[Host 2]

So people have been talking ad nauseam in the AI community about Jevons Paradox.

[Tuhim Srivastava]

Yeah.

[Host 2]

Where if you decrease the cost of. It's a. It's really a question around price elasticity and availability. If you decrease the cost of a good, say intelligence as a good people actually consume more of it, like the personal or business ROI of it, the demand for it goes up, not down. Do you see this and are you working against yourself trying to make these models more efficient? Do people just use them more or less?

[Tuhim Srivastava]

Yeah. I think you think about this from a developer's perspective and a consumer perspective. I think consumers just want the best answers and the best experience that's somewhat governed by more intelligence to some extent. I think when you go to the developers, from the developer's perspective, they would insert more intelligence. If you make it cheaper, they will insert more intelligence anyway. But if you make it more cheaper, they'll insert a hell of a lot more intelligence. And you see this with agents is that agents are just longer running now. And I think that's what we have seen with the cost of inference going down, which is folks are just like, okay, we can run this for longer or we can make it do a bit more work and we'll get to a larger end. I think compute scales from an inference perspective as well. And I think we are seeing that with almost all our customers, which is they either start with, this is the quality events I need to get to, and this is the amount of inference I need to do to get there, or this is the base level model that I can start with, that I can work with to get there. And I think the more we drive down the costs, what they realize is more intelligence just means better user experience.

[Host 2]

I just want a better answer.

[Tuhim Srivastava]

Better answers, better experiences, more dollars, more actions, more dollars, more revenue. So, yeah, I think inference going down just begets more. It is truly, I think we kind of in a world that is, you know, it is the last market. Right. Like, even if there's AGI, all that's left is inference. Yeah.

[Host 2]

So you do not see in your customers a, like, this answer is enough and this action is enough dynamic.

[Tuhim Srivastava]

No.

[Host 1]

Yeah. It's going to keep going for a long time, it looks like.

[Host 2]

Yeah.

[Host 1]

How do you view all this kind of evolving towards the future? So basically this is one of the. It seems like it's going to be one of the biggest markets of all times. We have this massive shift where we're moving from software and seats and digitization into actual intelligence. Selling units of cognition, selling agentic workflows. What does this all look like in a couple years? What is your view of this future world?

[Tuhim Srivastava]

I think for consumers it's the best possible thing. Everything is somewhat smarter. You get better care because your doctors have access to better tools. There's more, you know, like there's all this stuff about there being less software engineers. And I think we just build more software and we just build a ton more software. And like, you know, I see, you know, we're not slowing down hiring of software engineers. We're just building more things. And I think that for the consumers, that just means better to look more software, all those good things.

[Host 1]

Almost like everybody has their own team for everything, right? You have an agent which helps with your doctor. You have an agent that helps you learn stuff. You have an agent that helps you organize your life.

[Tuhim Srivastava]

It's concierge. It's concierge.

[Host 1]

Yeah. Concierge is everything for everyone.

[Tuhim Srivastava]

Yeah. And I think like, what that means, that's amazing. I think that's great. And I think education, same thing. You have concierge education, you get personalized access to everything. I think then you go one step back in how it affects developers, I think, you know, and companies. I think if you don't embrace this, I think it's the extension moment for a bunch of folks which is like, you know, everything needs. And I don't think that means that, you know, Claude, design needs figma. I think that's the thing. I think what's more interesting is just like, you know, all these workflow and software companies need to figure out what is the intelligent or intelligent inserted versions that drive the amount all that user value for those end consumers that we talked about.

[Host 1]

Yeah, very exciting. Thank you so much for joining us today.

[Tuhim Srivastava]

Yeah, thanks guys.

[Host 2]

Find us on Twitter nopriorspod. Subscribe to our YouTube channel. If you wanna see our faces, follow the show on Apple Podcasts, Spotify or wherever you listen. That way you get a new episode every week and sign up for emails or find transcripts for every [e-mail].