How Agentic AI Will Save Millions: Rapt AI CTO Anil Ravindranath Reveals the Secret

Summary

Episode Summary: How Agentic AI Will Save Millions: Rapt AI CTO Anil Ravindranath Reveals the Secret

Podcast: Liftoff with Keith Newman
Host: Keith Newman, Former Journalist and Startup + GTM Executive
Guest: Anil Ravindranath, CTO of RAPT AI
Release Date: July 17, 2025

In this insightful episode of Liftoff with Keith Newman, former journalist and Silicon Valley entrepreneur Keith Newman delves deep into the transformative world of AI infrastructure with Anil Ravindranath, the Chief Technology Officer at RAPT AI. Anil shares his expertise on how agentic AI is revolutionizing GPU management, leading to substantial cost savings and enhanced scalability for businesses operating in the AI landscape.

1. Introduction to RAPT AI and Agentic GPU Management

The conversation kicks off with Anil providing an overview of the current state of AI infrastructure. He emphasizes the complexity and unpredictability of AI models, highlighting the need for advanced management solutions.

Notable Quote:

"AI models are very dynamic and unpredictable... there's a lot of variability in each of the models." (00:37
— Anil Ravindranath)

Anil introduces the concept of agentic GPU management, explaining how multiple autonomous agents can oversee and optimize GPU infrastructure, ensuring seamless operations without constant human intervention.

2. Economic Impact and Cost Savings

A significant portion of the discussion centers on the economic advantages of implementing agentic GPU management. Anil underscores that cost efficiency is paramount in AI operations, where even minor inefficiencies can lead to substantial financial losses.

Notable Quote:

"If you have a GPU infrastructure which is 5% underutilized, then the effect of that is anywhere $120,000 living on the table." (01:56
— Anil Ravindranath)

By automating GPU management, businesses can maximize utilization, reducing wasted resources and enhancing profitability. Anil points out that without such automation, companies risk leaving significant amounts of revenue untapped.

3. Challenges in the AIOps Pipeline

Keith probes into the existing gaps within the AI Operations (AIOps) pipeline. Anil identifies that while AI models have rapidly advanced in their capabilities, the supporting infrastructure has lagged behind.

Notable Quote:

"There's a huge gap wherein a lot of advancement has happened in the models, but not at the infrastructure." (03:13
— Anil Ravindranath)

This disparity poses challenges in scalability and efficient deployment of AI models, necessitating innovations in infrastructure management to keep pace with model advancements.

4. Future Breakthroughs and Innovations in AI Infrastructure

Looking ahead, Anil shares his vision for the next wave of innovations in AI infrastructure. He predicts a shift from model-centric to infrastructure-centric advancements, particularly focusing on chip technology and data center management.

Notable Quote:

"The real innovation will happen at the chip... how can I manage all this infrastructure, manage the data centers, the liquid cooling." (04:07
— Anil Ravindranath)

Anil distinguishes between evolutionary and revolutionary changes, suggesting that while some improvements will be incremental, breakthroughs in chip design could lead to transformative changes in how AI infrastructure is managed.

5. RAPT AI's Role and Future Outlook

Concluding the episode, Anil highlights RAPT AI's pivotal role in addressing these infrastructure challenges. The company focuses on automating GPU management to help clients scale efficiently and avoid hitting operational plateaus.

Notable Quote:

"We help customers... get more tokens, more compute... we come and then automate and then make all that happen." (04:33
— Anil Ravindranath)

Anil expresses optimism about the journey ahead, emphasizing the continuous advancements and the critical support RAPT AI provides to its clients in navigating the evolving AI landscape.

Conclusion

This episode offers a comprehensive look into the essential role of agentic AI in transforming GPU management and AI infrastructure. Anil Ravindranath's insights elucidate the economic imperatives and technological innovations driving the next phase of AI advancement. For founders, innovators, and influencers in the tech space, this discussion underscores the importance of integrating automated, agent-driven solutions to harness the full potential of AI technologies.

---

For more episodes exploring the journeys of industry leaders and innovators, visit Liftoff with Keith Newman.

Transcript

[A]

Hey, Neil, welcome.

[B]

Hi.

[A]

Keith, the CTO from RAPT AI.

[B]

Nice, nice talking to you.

[A]

How is everything going at rapt?

[B]

It's pretty good. Yeah, it's pretty amazing. Amazing crowd here. Pretty intimate and a lot of peer understanding of each of these technologies.

[A]

See where we are in this crazy circus jungle. Revolutionary time.

[B]

Yes, absolutely. And then AI infrastructure is right in the middle of everything. Yes.

[A]

Can you tell me about agentic GPU management? How is that such a game changer?

[B]

Yeah, sure, yeah. So the main hypothesis or the reason why it's going to be a game changer is because AI models are very dynamic and unpredictable and then it's complex and variable. There's a lot of variability in each of the models. What you run now you need something wherein you can go and then autonomously work everything together. Right. So if you use agents, multiple agents in there which can go and manage the infrastructure for you. Right. You can do it a manual way wherein I can come and intervene and then do it. The other way is automation or autonomous way of doing things. And that's where agentic managing of GPUs become. So, so if you have multiple agents to do work for you and that's where your infrastructure is more robust and it's completely automated.

[A]

So cost savings and speed.

[B]

Absolutely, yeah. So economics is the biggest thing. The cost is the biggest hurdle. Right. Everybody's talking about tokens per second and tokens per watt, what they're generating. So if you leave without agentic or without agentic way of doing things or, or automation way of doing things, you're going to leave a lot of money on the table. Right.

[A]

So what kind of money are we talking about and does it hurt scalability or help?

[B]

Oh yeah, it's kind of a simple example is that let's say if I'm generating a $2 million revenue a month and then if you have. A simple example is if you have a GPU infrastructure which is 5% underutilized, then the effect of that is anywhere $120,000 living on the table, 5% of utilization of underutilization of a GPU. So now there's a huge need for somebody to come in and then look at holistically as to can I go and automate all this wherein a human intervention is not required. And that's where the rap AI comes into the picture.

[A]

Where do you see the biggest gaps in terms of the aiops pipeline?

[B]

Yeah. So the two things like is, especially now models are becoming more and more reasoning models. It means that you need more compute to generate more models. Sorry. To generate more tokens. But where do you have compute? Right. So you cannot source it forever. Right. So you need something which can. Either the models have to change or in some cases the models are changing, but at least at the same time, the infrastructure. There's a huge gap wherein a lot of advancement has happened in the models, but not at the infrastructure. Yeah.

[A]

Anil, you're in the middle of all this excitement. Where do you see the next breakthrough in terms of discovery and innovation?

[B]

Yeah. So the two things, right. For last 10 years, software has been eating the wall. Basically, software is the one which is SaaS, right? Yeah, SaaS. Now, I would say the first two, three years of the AI journey, it has been the models. Now it will be more of an infrastructure or the GPUs, because the AI models have ramped up pretty quick. But there is nothing at the infrastructure level to even work with those AI models. Right. So that's where the.

[A]

You're saying the real innovation will happen at the chip.

[B]

Exactly, the chip. How can I go and manage all this? All the infrastructure, how can I manage the data centers, the liquid cooling, all these things? Right.

[A]

Are those steps revolutionary or evolutionary?

[B]

Yeah, so in some cases it's evolutionary, some of them are revolutionary. Right. If you come up with a brand new chip, then it'll become revolutionary. But in some cases, if you're taking and then evolving the infrastructure or the data centers, it becomes evolutionary. Yes. That's exciting.

[A]

Well, you're doing some great work. What are you most excited about? Looking down the road, next few months.

[B]

Yeah. I think as I said, the more compute, more tokens and then we are right in the middle of that. So we help customers or data centers or cloud providers, whoever is running it, hey, you want to get more tokens, more compute? We are there because at some point of time you're going to hit a plateau wherein you cannot have. So we come and then automate and then make all that happen. Yeah. It's a good journey for last one year now. Yeah.

[A]

Well, congratulations on what you've achieved already.

[B]

Yes.

[A]

And good luck going down the road. It's going to be a fun ride.

[B]

Yeah, thanks, Keith. Yeah, thanks.