Niraj Bhatt on Innovation, AI, and Modern Technology Leadership

Summary

---

CIO Leadership Live — Niraj Bhatt on Innovation, AI, and Modern Technology Leadership

Date: July 20, 2026
Host: Lucas Marion (A), Senior Writer, Computer World
Guest: Niraj Bhatt (B), Fractional CTO & Advisor

---

Episode Overview

In this episode recorded live at the CIO100 symposium in Scottsdale, AZ, Lucas Marion interviews Niraj Bhatt, who brings experience advising both startups and Fortune 500s. The conversation focuses on navigating the rapidly evolving AI landscape, challenges with generative AI adoption in both startups and enterprises, managing hidden AI costs, talent gaps, and actionable advice for CIOs leading AI-driven transformation.

---

Key Discussion Points & Insights

The Impact of AI on Startups and Enterprises

• Rapid Acceleration for Startups:
  Bhatt describes how AI enables startups to dramatically reduce their time-to-market, with projects that formerly took years now achievable in months.

  • “Something which would probably take them three years, like a couple of years back is probably now taking them three months.” [00:58]

• Disruptive Pace Cuts Both Ways:
  He cautions that frontier AI efforts by major players (OpenAI, Anthropic, Google, etc.) can instantly disrupt smaller offerings, forcing rapid pivots for startups.

  • Anthropic’s specialized Claude model for financial services, and OpenAI’s new sales/marketing features, are recent examples of major hurdles for niche startups. [01:44]

• Enterprises Move Past POCs to Production:
  In large enterprises, the excitement after initial proofs of concept is giving way to the challenge of scaling AI to production, dealing with costs, compliance, and architectural integration.

  • “Now the struggle is getting them to production. That's where you run into like cost, you run into latency, you run into legal compliance, privacy issues...” [02:21]

---

Tackling Hidden AI Costs

• Cost Surprises:
  Bhatt notes that while POCs seem inexpensive, costs balloon at scale, especially with model usage—every input/output token adds up.
  • “Every token, every input token, output token, your model that you're selecting... the cost is significant.” [03:25]
  • The need for observability and SDLC rigor in GenAI projects is critical to avoiding costly surprises.

---

The Evolution: LLMs, RAG, and Agentic AI

• Explaining the Basics and the Hype:
  Bhatt stresses that the core technology is about token prediction—adding enterprise context to LLMs led to RAG (Retrieval-Augmented Generation), and now “agentic AI” takes things further by enabling action.

  • “The heart of it is all about tokens, right. It's predicting that next token is what this is all about...” [04:40]
  • “Agentic AI, it's more action oriented... but understanding in this context I think is the key.” [06:56]

• Cutting Through Hype:
  He warns against assuming every use case requires agents—understanding the true evolution and limits of the technology is vital.

  • “It's very easy for somebody to know a person with hammer. Everything looks like a nail.” [06:55]

---

Talent Strategy in the Age of AI

• Democratizing AI Across the Organization:
  Bhatt emphasizes that talent development should span all roles—from interns to executives, technical and business sides.
  • “Your talent strategy is everyone, all inclusive, starting from an intern, business side, tech side, CEO, everybody.” [08:04]
  • Success is about empowering staff with tools beyond mere training—especially supporting non-technical roles to leverage AI practically.
  • “Going beyond copilots... that's essentially where the innovation can really kick in.” [09:09]

---

GenAI Use Cases in Software Development Lifecycle (SDLC)

• Startup Use Cases:
  “Agile AI” in the startup world is real, significantly speeding up delivery.

  • “Three years to three months is something real.” [09:33]

• Key Enterprise Use Cases:

  • Test Automation: Crucial for resource-constrained startups, improves efficiency and speed.
  • Tech Debt Reduction: Using AI for upgrading dependencies and code migration allows teams to reclaim time lost to legacy “grunt work.”
    • “If AI can do that lifting, then that's giving significant amount of time back because tech debt has been a struggle for all of us.” [10:28]

---

Leadership Advice for CIOs

• Collaboration as the #1 Trait:
  Bhatt relays advice from a recent panel: bringing everyone together is essential for AI leadership.

  • “You need to really bring everybody together and… make sure that everybody's really on board.” [11:24]

• Manage Expectations:
  With immense pressure to “have an AI strategy,” Bhatt urges CIOs to educate senior leadership, define the art of the possible, and avoid chasing hype due to FOMO.

  • “What's maybe equally important or… more important is managing expectations.” [11:56]
  • “You are not reacting to the pressure... but you are really driving, becoming the change agent for good for the company.” [12:39]

---

Notable Quotes & Memorable Moments

• On AI Acceleration:
  “Three years to three months is something real…” [09:33] — Niraj Bhatt

• On Hidden Costs:
  “Every token, every input token, output token, your model that you're selecting... look at all of that and look at the scale…” [03:25] — Niraj Bhatt

• On Explaining AI:
  “If you cannot explain it to your 80-year-old grandmother or mother… because the execs, the business execs, they are really looking for that sort of advisor...” [04:33] — Niraj Bhatt

• On Talent:
  “Your talent strategy is everyone, all inclusive, starting from an intern, business side, tech side, CEO, everybody.” [08:04] — Niraj Bhatt

---

Timestamps for Key Segments

• [00:42] — Bhatt explains the AI landscape for startups and enterprises
• [01:44] — Disruptive impact of major AI launches on startups
• [02:21] — Moving from POCs to production, enterprise hurdles
• [03:14] — Discussion on hidden costs and observability
• [04:19] — Evolution from LLMs to RAG to Agentic AI
• [07:41] — Talent challenges and the need for inclusive strategy
• [09:21] — GenAI use cases in SDLC: test automation & tech debt
• [11:07] — Leadership advice for CIOs: collaboration & expectations

---

Conclusion

This substantial conversation with Niraj Bhatt delivers practical, candid insights into the complexities of leading through today’s AI revolution—from technical and operational hurdles to human and organizational challenges. CIOs and technology leaders will find actionable strategies for avoiding hype, building resilient AI strategies, and fostering talent and collaboration across their organizations.

Transcript

[A]

Welcome to CIO Leadership Live. I'm Lucas Marion, I'm a senior writer for Computer World magazine. I'm here at the CIO100 symposium and awards show here in Scottsdale, Arizona and I have with me today Neeraj Bhatt. He's a fractional CTO and advisor to startups and Fortune 500 companies. And you just came out of a private equity exit and you won a CR100 award for the last three years. Years. That's impressive.

[B]

Thank you.

[A]

Good to have you with me. Thank you.

[B]

Thanks for having me today.

[A]

Absolutely. So let's start at a high level. How are you helping your clients navigate this AI space?

[B]

Yeah, it's kind of interesting times, kind of living a dream right now. And I'm covering like a spectrum working with startups like founders, non technical founders and technical founders, and I'm also advising Fortune 500 companies. And what I'm seeing the startup in the startup space, they are loving the velocity and the momentum and the speed because they always wanted that. And AI is kind of providing them that, the products, they're able to bring their products to markets very quickly. Something which would probably take them three years, like a couple of years back is probably now taking them three months. So there's a lot of excitement there. And although on the flip side, the same velocity is also hurting them. All this frontier companies, AI companies, OpenAI, Anthropic, like I was working with a startup on the financial services and Anthropic recently released their Claude for financial services.

[A]

Right.

[B]

But just disrupted the whole offering which the startup was working on. Look at OpenAI, they have similar offerings in sales and marketing which they are coming up with. You look at Google, they recently came up with the interactive video model. So a lot of the startups working in the marketing space are also kind of getting. So I think a lot of what I'm focused on working with founders, helping them pivot in the gen AI space, ensuring that their systems and their products are built and structured in the right manner. And on the enterprise space, what I'm seeing like the wave of the POC proof of concepts kind of has gotten over and people have seen the value, there's some excitement, but now the struggle is getting them to production. That's where you run into like cost, you run into latency, you run into legal compliance, privacy issues, the customer concerns that if we get ticket to production, how this is going to look like, how are we going to scale and all of that. And I think what has happened there is those engineering teams, product Teams, they have to be ably supported by the enterprise architecture teams, by the R and D teams. So a lot of my focus and the work I'm doing is helping those enterprise architecture teams, R and D teams kind of come up to speed and build that kind of internal platform product for the production workloads. So I think exciting times on both the sides.

[A]

Do you help also unveil some of those hidden costs, inference costs and input output?

[B]

Yeah, it's a lot. People like the POCs when they get out of that and when you start tallying those they add up pretty significantly. Every token, every input token, output token, your model that you're selecting. Look at all of that and look at the scale because talking to my customers a lot of them are getting throttled, even the likes of Open Air because the volumes are growing and significant and the cost is significant. So just planning that all out observability monitoring. Right. So at the end of the day it's the entire SDLC needs to be put together for Gen solutions too. So there is a lot there. You're right.

[A]

Yeah. I'm still trying to wrap my mind around what a token is. It's a part of a word, it's not even the full word. So I don't know.

[B]

Yeah.

[A]

Anyways, I'm glad I don't have to foot the bill on that. How do you see Genai, the evolution from elements to rag to agentic AI? Do you think the hype of all this is real?

[B]

Great question and I'll just like pick on that token thing what you just mentioned at the end. So essentially if you look at the, and I think it's very important like we used to always say that if you cannot explain it to your 8 year old. Yeah. You don't really understand it. Well and I've kind of changed that that if you cannot explain it to your 80 year old grandmother or mother or whoever because the 80 year olds are getting it. It's the, it's the 80 year olds which we need to like because the execs, the business execs, they are really looking for that sort of advisor, trusted advisor who can really distill it for them. Right. So if you look at the LLMs to rag to agentic AI, now in the essence the heart of it is all about tokens, right. It's predicting that next token is what this is all about and I think understanding that is kind of the key. Right. So when the LLMs when they came out they were good at predicting, predicting that next token on the data which they were trained now when the enterprises looked at it, they, they wanted to make those LLMs work for their data. And the question kind of became that how do we provide our data? Right. How do we provide that context so that the LLMs, once they have the context, it's all about building the right context for the LLM. Agent AI is no different. So that's where the rag evolution kind of came in, that I've got my data and how can I. Because every LLM has got a limitation in terms of how much context it can carry.

[A]

It's just working off of its memory,

[B]

100 million. So there are like ranges of different LLM and Google has got the highest range in terms of the context window, what they support, the size of the context window. And the same thing with Agentic AI. The Agentic AI, it's more action oriented. Like the LLMs are able to act, but if you look at it, LLMs are not acting. LLMs are relying on the metadata which you are providing for the tools.

[A]

If, and then.

[B]

Yeah, yeah, so. And then they tell, then they, then again they're doing the prediction of that token. Prediction that if I'm looking for this, then I should be like using this tool and then it's upon the infrastructure underlying which LLM relies on to invoke the agent. So in a way I think I'm of the opinion that no, like if you bring a person and try to explain the microservices to that person, you're going to struggle. Right. But if you explain the evolution. So I think that's very important to understand the evolution and that's where you can really cut through the hype.

[A]

Right, right.

[B]

Because it's very easy for somebody to know a person with hammer. Everything looks like a nail. So if you take it, look at it from that perspective, it's very difficult and it's easy to kind of like get lost in the hype. Everybody's doing agents and I should be doing agents and everything can be done through agents.

[A]

Right.

[B]

But understanding in this context I think is the key.

[A]

Yeah, absolutely. I just, I want to interview the guy who decided what a token was worth, that's all. So how are you navigating the challenges around talent? I've written a lot about this. I write about the future of work a lot. There are talent gaps out there and we don't even know what they're going to be in six months because the technology is accelerating so quickly. How do you deal with that?

[B]

Great question. Yeah, easier said than done. So what I'm seeing, I think the part of it is like the IT side of things that no, there is so much of cognitive load on the members and how do we like really empower them to build solutions? And it's the right mix of products and platforms. But with AI, I think it's really about democratizing AI for the entire organization. So your talent strategy is everyone, all inclusive, starting from an intern, business side, tech side, CEO, everybody. Like your customer success, your revenue officers, you have to kind of have a talent strategy because in the end it is not going to be in a position to deliver for everyone in the organization.

[A]

Right?

[B]

And AI is making everybody or has a potential to make everyone in the organization more productive. So, so you have to really plan that. You have to facilitate that broad innovation across the organization. And that's where the talent strategy, working with your hr, the people officer becomes very important and providing those tools. So one part of it is training. But then if I'm a receptionist receiving calls and if I'm building an agent, how do I build the agent? I'm not going to be relying on wipe coding or things of that nature. But like what are the tools? Like where do I run, where do I host this? So I think thinking through that entire ecosystem, going beyond copilots, right, I think that's essentially is where the innovation can really kick in and that that broader talent strategy is what something I'm working with my, with my customers on.

[A]

Coming to using Genai for sdlc. What are some of the cases you're excited about?

[B]

Yeah, there is a lot of, as I was mentioning at the very beginning, the startup world, the whole Agile AI they're calling it as there's a lot of excitement around that. Like the three years to three months is something real. But if I were to like look at both the startups and the, and the enterprises in the startups world, the test automation, I think that's going to be a big use case because they're always constrained on resources and the operating margins and how can we deliver more for less. So the test automation using AI and I think that's becoming, there are companies out there doing that still. It's early but I think operating that at scale can really accelerate the whole sdlc. And the second thing which if you talk to any CIO and everybody I'm talking to is the tech debt that if AI, right? And this is like could be something as simple as upgrading. I am on running on a. NET version, a Java version, a PHP version, Python, whatever, and I Want to just upgrade the library from version A to version B. And if AI can do that lifting, then that's giving significant amount of time back because tech debt has been a struggle for all of us. So I think the tech debt and the test automation, those are the areas

[A]

I see as gives AI the grunt work. Yeah, yeah, I hear it.

[B]

You're right.

[A]

What's your advice to fellow CIOs for being successful with their AI initiatives? Kind of a broad question.

[B]

Yeah, it was interesting. I was attending the conference and there was a panel discussion yesterday happening in the evening and very, very good panel. And there was a question, I think similar. A question asked that what is the number one trait CIO needs to be successful in the world of AI? And the answer, what the panel gave was collaboration. That you need to really bring everybody together and you need to move forward and make sure that everybody's really on board. And maybe to my mind, simplifying that is more like systems thinking when you operate, just bringing everybody together and ensuring they're meeting the outcomes. But I think in the world of AI today, what's maybe equally important or maybe slightly more important is managing expectations. Because if you are a CIO today, there is a tremendous amount of pressure on you to deliver and have an AI strategy which is rock solid. Right. So what I'm doing with my customers, getting the board, getting the CEO, getting the CFO all into a room. Right. And really helping them understand like what I was kind of talking about the evolution and what's the art of possible. You don't want to be a CIO who thinks that. No, I have a hammer and everything is a nail. So just really working that and ensuring that the strategy which you are building, you have the buy in from the senior leaders and you are really headed in the right direction. You are not reacting to the pressure.

[A]

Fomo fear of missing out.

[B]

Exactly. With the top leadership, but you are really driving, becoming the change agent for good for the company. So that's something which I think would benefit a lot of CIOs today. Because the pressure is through the roof.

[A]

Yeah, unfortunately. Because I think it's rushing a lot of these projects that don't need to be. Neeraj, thank you so much for taking the time to talk with me today. I certainly hope you enjoy the rest of the conference.

[B]

Thank you. Thank you so much. Thanks for having me. It's great talking to you. Thank you, Sam.