Practical AI Podcast: Tool Calling and Agents

Date: February 14, 2025
Hosts: Daniel Whitenack (CEO, Prediction Guard) & Chris Benson (Principal AI Research Engineer, Lockheed Martin)

Episode Overview

This episode of Practical AI dives deep into the practical applications and industry trends surrounding "tool calling" and "agents" in AI, with a core focus on how large language models (LLMs) interact with other systems, real-world business models, and the growing open-source landscape. Amid lighthearted banter about recent AI news, Daniel and Chris clarify confusing terminology, explain technical distinctions, and explore how tool calling and agentic approaches are reshaping workflows in AI adoption.

Key Discussion Points & Insights

1. AI Industry News & OpenAI’s Trajectory (00:44 - 04:09)

• Discussion of Elon Musk’s attempted OpenAI buyout: Daniel and Chris joke about billionaire drama, noting it's entertaining but not crucial to practitioners.
• OpenAI's Super Bowl ad & spending: Chris discusses OpenAI’s heavy investments and questions the future direction and business models for leading AI companies.

Quote:
"There's not a protagonist here from my standpoint...It's another big giant AI company, you know, like the others." — Daniel Whitenack (04:09)

2. The Shrinking Gap Between Proprietary and Open Source (06:00 - 09:57)

• OpenAI’s release of "Deep Research" and immediate open-source responses (e.g., Hugging Face replicating the tool within 24 hours) highlight rapid open innovation.
• The speed at which open source catches up undermines the ability to establish lasting moats at the general application level.
• Implications for business models: Verticalization (serving niche domains or proprietary data) appears increasingly necessary for sustained value.

Quote:
"We've seen this interval between the commercial players and open source shrink to almost nothing." — Daniel Whitenack (08:42)

3. Bundling & Enterprise AI Offerings (09:57 - 14:36)

• "Bundle effect": Companies like Microsoft embed AI deeply in existing enterprise suites (e.g., Copilot in Teams/Office), generating value through integration rather than technical superiority.
• Integration challenges remain for highly sensitive domains (healthcare, finance), opening space for specialty and infrastructure players.

Quote:
"There is a very strong bundle effect...it doesn't mean that it's necessarily the best solution, but it is a solution depending on what you're looking for." — Chris Benson (09:57)

4. AGI, Business Models, & Downstream Effects (14:36 - 16:33)

• Sam Altman claims GPT-5 will surpass his intelligence; Chris and Daniel reflect—with humor—on the AGI race.
• Broader societal questions: Do AI advancements enhance or diminish human agency, trust, and community?

Quote:
"Are we building systems that enhance human agency rather than replace it?" — Chris Benson (15:26)

5. Clarifying Tool Calling vs. Agents (19:14 - 33:36)

a. Persistent Confusion in Terminology

• Many confuse or conflate LLMs with external action-taking (e.g., "How do I make the LLM create a new deal for me in HubSpot?").
• Core misunderstanding: LLMs only generate text/predictions; actual actions on external systems require separate processes.

Memorable Moment:
Daniel reacts viscerally to such misstatements:
"That's a fingernails on the chalkboard kind of moment for me..." (22:16)

b. Technical Deep Dive

• LLMs: Generate text, not actions.
• External Systems (e.g., HubSpot): Interacted with via APIs and regular software functions.
• Tool Calling/Function Calling: LLM outputs are used as arguments for standard software functions, which then interact with systems via APIs.
• Agentic Approaches: Go further—an "agent" orchestrates multiple tool calls, possibly in sequence, reasoning through intermediate steps/tasks to accomplish complex objectives.

Quote:
"In my mind, what separates out the agentic side of things is where you have some sort of orchestration performed by the LLM...There's a series of steps that might call different tools or systems." — Chris Benson (29:44)

• Training for Function Calling: Specific models (e.g., Hermes Llama) are fine-tuned on datasets containing function-calling patterns, making them better suited for such tasks.

6. Practical Use Cases & Agentic Workflows (34:12 - 39:27)

• Accessing Legacy or Inconvenient Systems: Agents (like Hugging Face’s "Small Agents") can automate browser interactions, extracting data from poorly designed web UIs where no usable API exists.
• Text-to-SQL and Secure Data Access: Agents iterate on SQL query tasks to achieve objectives while handling errors, optimization, or complex logic, especially in regulated industries.

Quote:
"Actually using an agent as a kind of extra user that you can control programmatically to interact with the application is really, really an intriguing kind of prospect to tie in that knowledge and extract things from the app." — Chris Benson (34:37)

7. Human Workflow, Jobs, & the Future of Agents (39:27 - 42:06)

• Upside: Expanded human agency—agents handle tedious or unapproachable tasks, amplifying productivity and enabling "superpowers" for workers.
• Nuance: Potential for both job displacement and new opportunity creation.
• Anticipation of mixed impacts: Some jobs may be lost, while others are enriched or made newly possible by more accessible interfaces and automation.

Quote:
"I think there's a lot of those things where that is expanded human agency...it's amplifying the effect of that worker and helping them feel like they have superpowers." — Chris Benson (39:27)

8. Trends & Learning Resources (42:06 - End)

• Deloitte's "State of Gen in The Enterprise" Report: Tracks adoption, barriers, regulatory concerns, and ROI for generative AI in the enterprise (42:06).
• Hugging Face Agents Course: Launched to teach theory and practice of agents using libraries like Small Agents, LangChain, and LlamaIndex (33:36).
• Recommendation: Replay or catch up on the course if missed live (33:44).

Notable Quotes & Moments

• “There's not a protagonist here from my standpoint... another big giant AI company, you know, like the others.” — Daniel Whitenack (04:09)
• “We've seen this interval between the commercial players and open source shrink to almost nothing.” — Daniel Whitenack (08:42)
• “There is a very strong bundle effect...it doesn't mean that it's necessarily the best solution, but it is a solution depending on what you're looking for.” — Chris Benson (09:57)
• "That's a fingernails on the chalkboard kind of moment for me..." — Daniel Whitenack (22:16)
• “In my mind, what separates out the agentic side of things is where you have some sort of orchestration performed by the LLM...” — Chris Benson (29:44)
• "Actually using an agent as a kind of extra user that you can control programmatically...is really, really an intriguing kind of prospect." — Chris Benson (34:37)
• "I think there's a lot of those things where that is expanded human agency...it's amplifying the effect of that worker and helping them feel like they have superpowers." — Chris Benson (39:27)

Key Timestamps

• 00:44 — Episode introduction and recent news highlights
• 06:00 — OpenAI "Deep Research" and open-source replication dynamics
• 09:57 — Business model shifts: bundling, verticalization, and integration
• 14:36 — AGI race, human agency, and societal effects discussion
• 19:14 — Defining tool calling vs. agents and untangling common misconceptions
• 29:44 — Technical distinction: tool calling vs. agentic orchestration
• 34:12 — Real-world use cases for agents in commercial/enterprise settings
• 39:27 — Workforce implications: expanded agency, job shifts, and opportunities
• 42:06 — Recommended resources: Deloitte report and Hugging Face course

Tone and Style

• Engaged, humorous, occasionally irreverent (“spats between billionaires”, “fingernails on the chalkboard”), instructional but accessible for practitioners and newcomers alike.

Resources Mentioned

• Hugging Face Agents Course: Live and on-replay on YouTube
• Deloitte’s State of Gen in the Enterprise Report: For industry trends and enterprise adoption
• Small Agents (Hugging Face library): For experimenting with agentic workflows

This episode is a practical, insightful exploration of how LLMs, tool calling, and agents are deployed in real business contexts—with concrete examples, no-nonsense clarifications, and a forward-thinking look at industry and human impacts.