The Digital Executive Podcast – Episode 1153

Scaling Agentic AI with Chirag Agrawal

Host: Brian (Coruzant Technologies)
Guest: Chirag Agrawal
Release Date: November 16, 2025

Episode Overview

This episode explores the technical and operational challenges of deploying AI agents at scale, featuring insights from Chirag Agrawal, a senior engineer with extensive experience in large-scale AI platforms and multi-agent orchestration. The conversation delves into best practices for bridging the gap between AI research and resilient production systems, managing developer freedom, optimizing for operational metrics like latency and cost, and building foundational layers for ethical and interoperable agentic AI.

Key Discussion Points & Insights

1. Bridging the Gap: From Models to Production Systems

• Model as Dependency, Not the Product:
  Chirag advocates for treating AI models as dependencies instead of the central product. The focus should be on the overall system that surrounds the model.
  • Notably, building custom agents from scratch leads to teams re-inventing undifferentiated components such as retrieval, orchestration, caching, and evaluation, which is inefficient.
• Recommendation for Teams:
  • Build agents from scratch as a prototype to learn, but transition to established frameworks for production.
  • Evaluation and guardrails need to be integrated from the start, not as afterthoughts.
• Quote:

  "Product teams should think about not the model, but the system around it... use a framework going forward to ship the agent in production systems."
  – Chirag Agrawal [02:05]

2. Developer Freedom vs. Architectural Discipline

• No Need for Trade-off:
  Chirag stresses that well-designed developer tooling gives developers freedom to experiment within disciplined architectural boundaries, improving velocity without compromising reliability.
• Effective Tooling:
  • Provides abstractions for developers (like binding model outputs to APIs) while handling schema validation, error management, context compression, and safety guardrails at the platform level.
  • This leverages organization-wide improvements—one platform change uplifts all dependent products.
• Quote:

  "Developer tooling provides good abstractions... The goal of the architectural discipline is to provide a safe playground where developers can move fast but without breaking the larger system."
  – Chirag Agrawal [05:01]

3. Monitoring and Optimizing Operational Metrics

• Key Metrics:
  • Latency: Track everything from prompt construction, model response time (first token latency), to first word rendered to user.
  • Token Usage: Monitor input/output token counts, cache usage, and dynamic prompt segments to optimize both cost and speed.
  • Quality: Harder to define, often task-dependent. Key metrics include tool selection accuracy, argument filling, and truthfulness, typically tracked offline.
• Balancing Trade-offs:
  Chirag likens managing latency, cost, and quality to balancing a triangle: improving one often impacts the others, requiring continual system tuning as per user feedback.
• Quote:

  "If you try to improve latency too aggressively, you might compromise the intelligence or the quality of your product. And if you try to chase the quality too aggressively, then your token cost and latency will explode."
  – Chirag Agrawal [11:15]

4. Ethics, Bias, and Interoperability: Building Trustworthy and Connected Agents

• Foundational Ethics & Transparency:
  Ethical considerations, mitigation of bias, and transparency must be built into the core platform, not bolted on afterwards.

  • All agent requests should be auditable, observable, and traceable.
  • Evaluation hooks for real-time monitoring and reflection are essential to prevent or quickly mitigate unsafe behavior.

• Interoperability via Emerging Standards:

  • Agents must communicate across system boundaries using open and typed protocols, inspired by standards like HTTP.
  • Chirag highlights excitement about emerging standards such as MCP (Model Context Protocol) and a2a, which facilitate agent discovery, authentication, and collaboration.

• The Next Frontier:
  Chirag envisions “an Internet of agents”—multi-agent systems that share capabilities yet remain governed independently, akin to the progression of early mobile apps to rich, interconnected ecosystems.

• Memorable Quote:

  "Bias and transparency, these are things that should be built at the foundational layer... All the requests that flow through these agentic platforms, they should be auditable, observable and traceable."
  – Chirag Agrawal [12:55]

  "Looking ahead, I think the next frontier in production AI enterprise is an Internet of agent or multi-agent systems where agents built by different teams can share capabilities but still operate under their own governance."
  – Chirag Agrawal [14:17]

Notable Quotes & Timestamps

• On frameworks vs. custom builds:
  “Instead what they should do is probably like build the agent once for a prototype... but then use a framework going forward to ship the agent in production systems.”
  – Chirag Agrawal [02:25]

• On developer freedom & system discipline:
  “A developer platform provides general solutions to essential problems. It doesn't really curb developers’ freedom at all.”
  – Chirag Agrawal [06:18]

• On prompt engineering for efficiency:
  “Cached input tokens can be really valuable to monitor... you will end up utilizing a lot of cached input tokens which will reduce your cost and latency.”
  – Chirag Agrawal [09:29]

• Analogy to Mobile App Evolution:
  "This current scenario reminds me of the early days of Android and iOS... I think same thing is going to happen with these agents. I think they are sort of in an nascent stage right now, but they're going to improve dramatically over next few years."
  – Chirag Agrawal [14:00]

Important Segment Timestamps

• AI infrastructure beyond models: [01:51]–[04:02]
• Developer empowerment and tooling: [04:52]–[06:33]
• Operational metrics and trade-offs: [07:19]–[11:57]
• Ethics, bias, transparency, and the future: [12:45]–[14:46]

Conclusion

Chirag Agrawal provides a pragmatic and nuanced view of scaling agentic AI—emphasizing the importance of robust system architecture, disciplined experimentation, and foundational ethics. He advocates for leveraging shared frameworks, monitoring key operational metrics, and building for interoperability and transparency at every step. The vision for AI’s future? A world of interconnected agents—each robust, trustworthy, and able to collaborate across domains.