Practical AI Podcast – "While Loops with Tool Calls"

Date: October 30, 2025
Hosts: Daniel Whitenack (Prediction Guard), Chris Benson (Lockheed Martin)
Guest: Jared Zonderike (Co-founder & CEO, PromptLayer)

Overview

This episode dives into the practical evolution of building Large Language Model (LLM) applications, with a particular focus on the transition from traditional prompt chaining and DAG-based (Directed Acyclic Graph) workflows to more autonomous, agentic systems—most notably, “while loops with tool calls.” Guest Jared Zonderike returns to unpack how better models, richer context handling, and the emergence of tool calling have changed both the landscape and best practices for making AI accessible and productive for all users, from hardcore engineers to industry domain experts.

Key Discussion Points & Insights

1. Rapid Evolution of Prompting and Context Engineering

Reflections on Changes Since March 2024:
- The shift from “prompt engineering” to “context engineering”—terms are evolving, but the core remains: input + model = output.
- Reasoning models (like OpenAI’s newer models) have increasingly automated the “chain of thought” process, reducing prompt complexity and manual tuning.
  - “Some of the weirdness of how you persuade the model and yell at the model has changed, has gone away and it's gotten a little bit more straightforward… but I think prompting is still at the core of everything.” (D, 04:35)
Context is King:
- Context windows are now much larger, leading to opportunities—but also risks such as model distraction. The challenge is fitting just enough information to “steer” the model without overwhelming it.
- The term “context engineering” encompasses not just prompt text, but what data, how much, and in what form is provided to the model.

2. From Prompt Chaining to While Loops & Tool Calls

Classic Workflows:
- Early LLM apps used structured DAGs to strictly control outputs and minimize hallucination (e.g., in customer-facing chatbots).
Modern Approach – Tool Calls:
- “Tool calling” means the model can invoke external functions—like “issue refund,” “search the web,” etc.—within the chat context.
  - “A lot of the models have been built around tool calls and have gotten really good at interacting with them, interpreting their response, sending another message to them...” (D, 18:40)
- Instead of rigid, brittle chains, new agents operate in a “while loop”:
  - Model loops: Try tool → get result → reevaluate → possibly try again, until done.
  - Simpler to build and debug, more flexible, but also introduces new testing and reliability challenges.

3. The Builder’s & User’s New Responsibilities

For Users:
- AI agents can now autonomously explore a domain—trying, failing, and retrying, much like an intern given a general task and a toolkit (D, 21:10).
For Builders:
- Agents make it faster to prototype working solutions, but harder to guarantee correctness.
- “The core learning we’ve had at least since the past year and a half is that you don’t need to have 100% coverage when evaluating these things. If anything, if you’re trying to make a perfect test for your agents, you’re probably never going to ship.” (D, 26:45)
- Testing and iteration now rely on surface-level heuristics (“agent smell tests”) as well as step-by-step and full-conversation simulations.

4. Testing and Versioning Agentic LLM Workflows

With recursive “while loop” agents, failures can emerge deep in the process.
- Unit testing becomes both more important and more challenging.
- Tools and metrics: Tool call counts, retries, agent run time—used to catch regressions and sanity check new versions.
Version management is even more crucial due to complexity and the involvement of non-technical stakeholders.

5. Bridging Technical and Domain Expertise

Success in AI projects increasingly depends on fusing engineering rigour with domain insight.
Jared: “I think the reason we focus a lot on getting these domain experts involved in the process is because we believe actually from a business perspective that's how you win as an AI company.” (D, 29:50)
“The skill of prompting… is really a skill of tinkering. And not all coders are tinkerers, but not all writers are tinkerers either. So there’s some new type of algorithmic thinking that overlaps very highly.” (D, 32:15)

Notable Quotes & Memorable Moments

On the shift from prompt engineering to context engineering:
- “The new word people love to say is context engineering… to me they’re the same. But people like this word because it’s not just the prompt… it’s how much are you putting in the text? Are you putting too much? Is the model getting distracted? Are you using rag?” (D, 04:55)
On evolving advice due to fast-moving AI:
- “Some of the things we had put out, internal videos like that I put out a year and a half ago, are completely obsolete… anything that any of us say today is obsolete tomorrow with the way this thing accelerates.” (C, 07:58)
On “LLM idioms” and mental models:
- “How do you understand the language of LLMs? … Asking the model to return JSON or a structured format will work really well… but if you want to write a love note, it's probably not going to work as well because the model is now in I’m coding gear.” (D, 13:23)
- [Joking reference] “You may have just identified why my last anniversary with my wife just went off the rails. The JSON output for the love note—I thought I had it, but, you know, clearly not.” (C, 15:01)
On the role of non-technical stakeholders:
- “If you’re building legal AI, you win by having a lawyer involved…The differentiator [is] the non-engineering taste that’s been put into the AI product.” (D, 30:59)
On agent development pace and practicality:
- “If it takes less than two hours to do using Claude code or Codex or something, just do it. Don’t ask anyone, don’t prioritize it. And it’s helped a lot, honestly…our customers have literally told us like ‘wow, you guys are shipping so much faster now.’” (D, 39:12)
On the future of accessible AI engineering:
- “I’m very excited about unlocking AI engineering for people who didn’t study computer science… people have expertise and now they’re going to be able to build AI products around it and do AI engineering around it.” (D, 41:44)

Timestamps for Major Segments

[03:44] – Changes in prompting, the rise of context engineering
[06:09] – Distinction between prompt and context engineering, reasons for new terminology
[08:35] – AI changes so rapidly, advice quickly becomes obsolete
[11:37] – Do you need deep technical knowledge, or just a mental model for LLM output?
[13:09] – “LLM idioms” and prompts: knowing what “language” the model responds to best
[15:48] – The evolution from DAGs to while loops and tool calls explained
[16:29] – Tool calls in detail and examples (United Airlines chatbot, Coding agents)
[20:14] – Impact of agentic approaches on user and builder responsibilities
[23:06] – Testing & versioning workflows in agentic systems
[27:19] – Why so many AI pilots fail: bridges between AI engineering and domain knowledge
[33:48] – Crawl, walk, run: iterative approaches for AI projects
[37:46] – Claude code/coding agents – practical use cases and internal policy
[41:30] – What’s next for the field: excitement about accessible, multi-tool AI engineering

Tone & Style

The episode stays lively and friendly, blending technical insight with light-hearted banter (e.g., the “cake” logo and Halloween quips). Jared brings a pragmatic, tinker-focused philosophy, often comparing AI engineering to negotiation or even psychology. The conversation is optimistic about the democratization of AI while candid about the difficulties of keeping up with innovation and maintaining rigor.

Summary Takeaways

AI engineering is shifting from rigid workflows to agentic systems leveraging flexible tool calls, enabled by advances in LLMs.
Testing and versioning remain critical, though “perfect” coverage is impractical—emphasis is on smart heuristics and collaboration.
Domain expertise is now as crucial as coding ability; the workflows and platforms that best bridge these worlds are likeliest to succeed.
The future is bright and democratized: anyone with expertise can—via accessible tools—build practical, valuable AI applications.

Further resources:

For a deeper technical dive or to try agentic workflows directly, see the episode’s show notes for recommended reading and tools.

Practical AI Podcast – "While Loops with Tool Calls"

Date: October 30, 2025
Hosts: Daniel Whitenack (Prediction Guard), Chris Benson (Lockheed Martin)
Guest: Jared Zonderike (Co-founder & CEO, PromptLayer)

Overview

Key Discussion Points & Insights

1. Rapid Evolution of Prompting and Context Engineering

Reflections on Changes Since March 2024:
- The shift from “prompt engineering” to “context engineering”—terms are evolving, but the core remains: input + model = output.
- Reasoning models (like OpenAI’s newer models) have increasingly automated the “chain of thought” process, reducing prompt complexity and manual tuning.
  - “Some of the weirdness of how you persuade the model and yell at the model has changed, has gone away and it's gotten a little bit more straightforward… but I think prompting is still at the core of everything.” (D, 04:35)
Context is King:
- Context windows are now much larger, leading to opportunities—but also risks such as model distraction. The challenge is fitting just enough information to “steer” the model without overwhelming it.
- The term “context engineering” encompasses not just prompt text, but what data, how much, and in what form is provided to the model.

2. From Prompt Chaining to While Loops & Tool Calls

Classic Workflows:
- Early LLM apps used structured DAGs to strictly control outputs and minimize hallucination (e.g., in customer-facing chatbots).
Modern Approach – Tool Calls:
- “Tool calling” means the model can invoke external functions—like “issue refund,” “search the web,” etc.—within the chat context.
  - “A lot of the models have been built around tool calls and have gotten really good at interacting with them, interpreting their response, sending another message to them...” (D, 18:40)
- Instead of rigid, brittle chains, new agents operate in a “while loop”:
  - Model loops: Try tool → get result → reevaluate → possibly try again, until done.
  - Simpler to build and debug, more flexible, but also introduces new testing and reliability challenges.

3. The Builder’s & User’s New Responsibilities

For Users:
- AI agents can now autonomously explore a domain—trying, failing, and retrying, much like an intern given a general task and a toolkit (D, 21:10).
For Builders:
- Agents make it faster to prototype working solutions, but harder to guarantee correctness.
- “The core learning we’ve had at least since the past year and a half is that you don’t need to have 100% coverage when evaluating these things. If anything, if you’re trying to make a perfect test for your agents, you’re probably never going to ship.” (D, 26:45)
- Testing and iteration now rely on surface-level heuristics (“agent smell tests”) as well as step-by-step and full-conversation simulations.

4. Testing and Versioning Agentic LLM Workflows

With recursive “while loop” agents, failures can emerge deep in the process.
- Unit testing becomes both more important and more challenging.
- Tools and metrics: Tool call counts, retries, agent run time—used to catch regressions and sanity check new versions.
Version management is even more crucial due to complexity and the involvement of non-technical stakeholders.

5. Bridging Technical and Domain Expertise

Success in AI projects increasingly depends on fusing engineering rigour with domain insight.
Jared: “I think the reason we focus a lot on getting these domain experts involved in the process is because we believe actually from a business perspective that's how you win as an AI company.” (D, 29:50)
“The skill of prompting… is really a skill of tinkering. And not all coders are tinkerers, but not all writers are tinkerers either. So there’s some new type of algorithmic thinking that overlaps very highly.” (D, 32:15)

Notable Quotes & Memorable Moments

On the shift from prompt engineering to context engineering:
- “The new word people love to say is context engineering… to me they’re the same. But people like this word because it’s not just the prompt… it’s how much are you putting in the text? Are you putting too much? Is the model getting distracted? Are you using rag?” (D, 04:55)
On evolving advice due to fast-moving AI:
- “Some of the things we had put out, internal videos like that I put out a year and a half ago, are completely obsolete… anything that any of us say today is obsolete tomorrow with the way this thing accelerates.” (C, 07:58)
On “LLM idioms” and mental models:
- “How do you understand the language of LLMs? … Asking the model to return JSON or a structured format will work really well… but if you want to write a love note, it's probably not going to work as well because the model is now in I’m coding gear.” (D, 13:23)
- [Joking reference] “You may have just identified why my last anniversary with my wife just went off the rails. The JSON output for the love note—I thought I had it, but, you know, clearly not.” (C, 15:01)
On the role of non-technical stakeholders:
- “If you’re building legal AI, you win by having a lawyer involved…The differentiator [is] the non-engineering taste that’s been put into the AI product.” (D, 30:59)
On agent development pace and practicality:
- “If it takes less than two hours to do using Claude code or Codex or something, just do it. Don’t ask anyone, don’t prioritize it. And it’s helped a lot, honestly…our customers have literally told us like ‘wow, you guys are shipping so much faster now.’” (D, 39:12)
On the future of accessible AI engineering:
- “I’m very excited about unlocking AI engineering for people who didn’t study computer science… people have expertise and now they’re going to be able to build AI products around it and do AI engineering around it.” (D, 41:44)

Timestamps for Major Segments

[03:44] – Changes in prompting, the rise of context engineering
[06:09] – Distinction between prompt and context engineering, reasons for new terminology
[08:35] – AI changes so rapidly, advice quickly becomes obsolete
[11:37] – Do you need deep technical knowledge, or just a mental model for LLM output?
[13:09] – “LLM idioms” and prompts: knowing what “language” the model responds to best
[15:48] – The evolution from DAGs to while loops and tool calls explained
[16:29] – Tool calls in detail and examples (United Airlines chatbot, Coding agents)
[20:14] – Impact of agentic approaches on user and builder responsibilities
[23:06] – Testing & versioning workflows in agentic systems
[27:19] – Why so many AI pilots fail: bridges between AI engineering and domain knowledge
[33:48] – Crawl, walk, run: iterative approaches for AI projects
[37:46] – Claude code/coding agents – practical use cases and internal policy
[41:30] – What’s next for the field: excitement about accessible, multi-tool AI engineering

Tone & Style

Summary Takeaways

AI engineering is shifting from rigid workflows to agentic systems leveraging flexible tool calls, enabled by advances in LLMs.
Testing and versioning remain critical, though “perfect” coverage is impractical—emphasis is on smart heuristics and collaboration.
Domain expertise is now as crucial as coding ability; the workflows and platforms that best bridge these worlds are likeliest to succeed.
The future is bright and democratized: anyone with expertise can—via accessible tools—build practical, valuable AI applications.

Further resources:

For a deeper technical dive or to try agentic workflows directly, see the episode’s show notes for recommended reading and tools.

wavePod

While loops with tool calls

Powered by Wave AI

Summary

Practical AI Podcast – "While Loops with Tool Calls"

Overview

Key Discussion Points & Insights

1. Rapid Evolution of Prompting and Context Engineering

2. From Prompt Chaining to While Loops & Tool Calls

3. The Builder’s & User’s New Responsibilities

4. Testing and Versioning Agentic LLM Workflows

5. Bridging Technical and Domain Expertise

Notable Quotes & Memorable Moments

Timestamps for Major Segments

Tone & Style

Summary Takeaways

Summary

Practical AI Podcast – "While Loops with Tool Calls"

Overview

Key Discussion Points & Insights

1. Rapid Evolution of Prompting and Context Engineering

2. From Prompt Chaining to While Loops & Tool Calls

3. The Builder’s & User’s New Responsibilities

4. Testing and Versioning Agentic LLM Workflows

5. Bridging Technical and Domain Expertise

Notable Quotes & Memorable Moments

Timestamps for Major Segments

Tone & Style

Summary Takeaways