Episode #167. Agentic AI in Action: Reaching Patients and Improving Outcomes, with Kali Ihde, Director of Northwestern Medicine's Ventures and Innovation

Summary

---

Risk Never Sleeps Podcast – Episode #167

Agentic AI in Action: Reaching Patients and Improving Outcomes
Guest: Kali Ihde, Director of Northwestern Medicine's Ventures and Innovation
Host: Ed Gaudet
Date: December 17, 2025

---

Episode Overview

In this episode, Ed Gaudet interviews Kali Ihde live from the AIMed25 conference in San Diego. The conversation dives deep into how Northwestern Medicine is leveraging agentic AI to elevate patient outreach, improve outcomes, and ensure responsible innovation within healthcare. Kali shares candid insights on AI literacy, practical deployment of AI agents to reach underserved populations, risk-taking in healthcare technology, and the challenges of separating hype from meaningful impact.

---

Key Discussion Points & Insights

Experiences from the AIMed25 Conference

• Emphasis on Clinician Leadership: Kali praises the event’s physician- and clinician-led discussions, noting the refreshing focus on real-world questions and safety guardrails (01:01).
• "Cutting Through the Hype": Kali values the candid peer conversations focused on practical implementation over AI buzz (01:15).

Evaluating and Implementing New AI Technologies

• Vetting Solutions: Kali explains the importance of moving past sales pitches and creating a "safe space" for small-scale pilots to assess true value and workflow integration (01:41).
  • Quote (Kali, 01:41):
    "It really reinforces our structure of giving ourselves a safe space to try something on a small scale, to vet it and really understand what is real."
• Involvement Across Stakeholders: Embedding both clinician and administrative expertise ensures tools are meaningful and not just headline-grabbing (01:50).

Growing AI Literacy and Acceptance

• Shift from Skepticism to Curiosity: There’s growing understanding and readiness among clinicians to engage with AI tools (02:34).
  • Quote (Kali, 03:20):
    "You're starting to get more comfortable with it, understand limitations and considerations... they're much better able to ask questions, to be open to it, to not have... a knee jerk reaction of 'no, this is way too complicated.'"

What is "Agentic AI"?

• Defining Agentic AI: Kali clarifies that, while definitions vary, agentic AI at Northwestern Medicine describes systems that operate autonomously beyond "human in the loop" models (03:34).
• Pragmatic Pilots: Focusing on lower-risk, high-opportunity projects, such as outreach for annual wellness and colorectal screenings (03:50).

Transformative Use Case: AI-Powered Patient Outreach

• Automating Wellness Calls:
  • AI agents now make outbound calls to schedule appointments for preventive care, doing the work of 22 hours in just 30 minutes (04:10).
  • Approximately 10% of contacted patients schedule on the spot—a significant improvement over previous efforts (04:18).
• Unanticipated Positive Outcomes:
  • AI agent conversations have surfaced suicidal ideation and elder abuse, leading to timely intervention by connecting patients to resources (04:32).
  • Quote (Kali, 04:32):
    "We've found suicidal ideation and been able to transfer them to social workers... risks of elder abuse... So it's also a really great way to just give our patients a touch point and identify areas where maybe things are going unseen."

Cross-Department Collaboration

• Marketing's Role:
  • Involvement is crucial but so far limited to small pilot groups to thoroughly vet the tech before broader campaigns (05:50).

---

Personal Reflections & Career Advice

What Kali Would Tell Her 20-Year-Old Self

• Education:
  • Consider clinical education to better understand patient care and the biology of the human body (06:26).
  • Quote (Kali, 06:26):
    "From a career perspective... I'd probably get a degree in a clinical area of study."
• Personal Health:
  • Take care of your body—advice that resonates more with experience (07:10).

Meaningful Risks in Her Career

• Early Tech Choices:
  • Taking a job at Epic, when it was a small company, proved to be Kali’s most impactful career risk (07:27).
  • Quote (Kali, 08:03):
    "Took a chance on that position, and it has set my career and in many ways my life up in ways I couldn't have even fathomed."
• Influence of Early Healthcare Tech Training:
  • Exposure to real physician workflows at a young age built expertise and passion for healthcare innovation (08:40).

Fun Fact: Northwestern Medicine’s Scope

• Beyond Chicago:
  • Northwestern Medicine has 10 hospitals, 200 clinics in Chicago, and partnerships in Naples, Florida and London (09:55).
  • Quote (Kali, 10:20):
    "A lot of people don't know where all we are hoping to make a difference and change in the world."

---

Practical Advice: Evaluating AI Innovations

Putting Structure Around Evaluation

• Go Beyond Governance:
  • Use structured, consistent evaluation: identify the true problem, assess technology fit, define and measure success from the start (10:59).
  • Quote (Kali, 10:59):
    "Every single solution: what problem is it solving?... Define and measure what success looks like."
• Avoid ‘Pilot Purgatory’:
  • Set outcomes in advance to know whether a pilot succeeded; avoid ambiguous goals (11:50).

The ROI Question

• Does ROI Need to Be Seven Figures?
  • Kali emphasizes right-sizing expectations to the investment and measuring ROI in quantifiable terms, especially with AI (12:18, 12:39).
  • Quote (Kali, 12:39):
    "The promise of AI is that it can do things faster, better, more precisely. That should be measurable."

---

Notable Quotes & Memorable Moments

• On Real-World Innovation:

  • "We're leveraging AI agents to make those phone calls on our behalf. And they're doing 22 hours of work in 30 minutes... We've found suicidal ideation... risks of elder abuse... So it's also a really great way to just give our patients a touch point and identify areas where maybe things are going unseen." – Kali Ihde, (04:10–04:32)

• On Cutting Through AI Hype:

  • "We're looking to talk to anybody who’s trying to dig through the hype, get to the real world application and make a meaningful difference in the halls of our hospitals, into the rooms of our clinics, into the lives of our communities and our workforce every day." – Kali Ihde, (13:20)

---

Important Timestamps

• 01:00 – Conference impressions and clinician-led innovation
• 03:34 – Agentic AI defined, real use cases
• 04:10–04:45 – Patient outreach, wellness, and unanticipated human impact
• 06:26 – Career advice for young professionals
• 07:27 – The risk and rewards of joining Epic early
• 09:55 – Northwestern Medicine’s broad reach
• 10:59 – Building structure into tech/AI vetting
• 12:39 – Quantifying AI ROI

---

Closing Thoughts

Kali invites listeners to connect via LinkedIn or visit innovation.nm.org to learn more. Her parting message champions working with others committed to moving beyond the hype and making meaningful, measurable impacts in healthcare (13:01).

---

Summary Prepared by Podcast Summarizer, preserving the original insights and tone.

Transcript

[Ed Gaudet]

Foreign.

Welcome to Risk Never Sleeps, where we meet and get to know the people delivering patient care and protecting patient safety. I'm your host, Ed Gaudet.

Hello everybody and welcome back to another episode of the Aimed Insight series recorded live here in San Diego from the aimed25 conference. Today I have the privilege of hosting Kali ID. She is the director of Northwestern Medicine's Ventures and Innovation. Callie, so great to see you again.

[Kali Id]

Yes, great to see you too. Almost a year later.

[Ed Gaudet]

I know. How you been since Vive?

[Kali Id]

Good. Busy. Yeah, it's exciting. I didn't even know the term Magentic AI when we met in February and now I probably say it 10 times a day.

[Ed Gaudet]

It's insane. It's moving so fast. So what brings you to Aimed at?

[Kali Id]

This was a conference. This is my first time visiting Same.

[Ed Gaudet]

It's my first as well.

[Kali Id]

Same I've been curious about and just understanding what's the angle here, who's here, what are we talking about? And I love that it was very much physician led and clinician led and have just been really impressed with the caliber of thought leadership here. The caliber of folks who are both asking good questions but trying new things, but doing it with all the right guardrails and really thinking through how do we do these things safely. But it's also been nice to be among peers who are cutting through the hype a bit because there's a lot of it, a lot of what I feel my life has been lately.

[Ed Gaudet]

It's like everybody in the organization wants something the toy like how do you vet through that? Have you learned anything here that you're going to do differently as a result?

[Kali Id]

I think it's been reinforced here that get past the sales pitch. You have to. And we already know that that's a lot of what we do. Our leadership will bring new solutions or ideas that they hear about. We'll meet folks and try to surface what's. But it really reinforces our structure of giving ourselves a space, safe space to try something on a small scale, to vet it and really understand what is real. What haven't we considered? How do we embed it in workflows? How do we ensure that we're getting clinician expertise, that we're getting administrative expertise to look at it and really understand how can AI, how can these tools really be meaningful today and not just a headline grabbing statement that gets us excited about the future?

[Ed Gaudet]

Totally. And what are you seeing today that is making the most difference in application?

[Kali Id]

I think we're starting to see a shift in AI literacy, which is helpful. I think people are starting to, if they haven't already, then they're getting to the point where they're ready to play with some of the generally available AI solutions out there. ChatGPT, Gemini, all of them. You're starting to get more comfortable with it, understand limitations and considerations. And that's really great because then when we reach out to cardiologists or oncologists or anyone to get input on a solution, they're much better able to ask questions, to be open to it, to not have, I think what we saw just as recently as a year ago, sort of a knee jerk reaction of no, this is way too complicated of a space. It must be a human. You can't put a robot or any AI in there.

[Ed Gaudet]

There's more acceptance. Yeah.

[Kali Id]

And intrigue too, I think.

[Ed Gaudet]

And intrigue. And like you said, more literacy, more understanding. We're starting to. And agentic is newer. So help us understand what you guys are exploring on the agentic side.

[Kali Id]

Yeah. And you know that that term agentic, just like every term that's been coming out with all of our tech, is it's both broad and narrow. You can go into five different sites on the Internet and they're going to just define it differently. And I think the way that we see it is it's really getting more to autonomous. Everybody has been talking about human in the loop. You get to agen and we're really letting it run. And so we're being mindful of finding the right areas for that to be lower risk opportunity based. Where are we not meeting patient demand or needs that something like this could help us identify more opportunities. So the big one that we've been really happy about over the last few weeks has been we're using agentic AI to reach out to patients, to get them in for annual wellness visits, for colorectal cancer screening. And really it's areas where we know our patient population isn't always proactively coming to us for these things. We've in the past done campaigns where we bring a whole group of people in to make phone calls or to send text messages, but we're still not getting the turnaround and the scheduling and getting our patients in. And then that leads to downstream issues from a health perspective in our communities. And so now we're leveraging AI agents to make those phone calls on our behalf. And they're doing 22 hours of work in 30 minutes. They're getting amazing. You know, 10% of these patients are getting scheduled right there on the spot. And what was really impressive is this is a touch point with our patients that we wouldn't have had otherwise or with our communities that we wouldn't have had otherwise. So we've found suicidal ideation and be able to transfer them to social workers and assistants. We've found risks of elder abuse that wouldn't have been sourced otherwise. And we were able to connect them with services. And so it's also a really great way to just give our patients a touch point and identify areas where maybe things are going unseen.

[Ed Gaudet]

Wow, I love that. And yeah, this opportunity is huge to really help people get to do their wellness stuff that they need to do because it's. It's hard. And you know what's interesting now? Like, so I run a full service marketing agency. It's interesting to see this channel in particular. Right. Like is the chief marketing officer part of that at all? Like the. Because it's part of the campaign management. Right?

[Kali Id]

Yeah, Fair question. So we do include our marketing folks in a lot of the discussions we have. We've been trialing to vet the tech first before we do big promotional decisions out there. We targeted small groups and so we haven't gone with full on. How do we get this campaigns to. Yes, everyone, it's been a. Let's vet the tech.

[Ed Gaudet]

Understood.

[Kali Id]

It's impressive so far and that we would go to scaling.

[Ed Gaudet]

Love it. That's so exciting to hear that. All right, so listen, if you had the chance to go back to when you were 20 years old, what advice would you give yourself?

[Kali Id]

Can I say two things?

[Ed Gaudet]

Yeah, absolutely.

[Kali Id]

So I think career wise I'd probably get a degree in a clinical area of study. So I always thought I couldn't be a doctor because the idea of cutting into somebody or blood or things like that was. But I now know now that I have such a better understanding of the industry that there are a lot of ways to get that clinical education and understand truly the biology of the human body a lot better. And so I'd probably go that route from a career perspective because I think it would have just.

[Ed Gaudet]

It helped. So would you become a doctor?

[Kali Id]

I don't know if I'd become a doctor. Maybe a PA or a degree that kind of helps you get that understanding and get closer to patient care. And then personally it would be. And it was things that everybody tells you all the time, but take care of the body that you have and it's the only one you have.

[Ed Gaudet]

It's the only one you have.

[Kali Id]

And so now over 40, I am recognizing that some of the things I did in my twenties probably weren't the best choices. And you're trying to constantly overcome them.

[Ed Gaudet]

Yeah. No, Great advice. What is the riskiest thing you've ever done?

[Kali Id]

I would say I was in college, thought I was going to go into human resources. And I met this company based out of Madison, Wisconsin who played in the tech in healthcare space. My mother was a coordinator for a physician for a number of years. My dad worked on the business sales side and is combined the business and the tech of healthcare. That company was EPIC when it started. They were very small and had known they had one building on that campus.

[Ed Gaudet]

Wow.

[Kali Id]

And now they are this huge thing. And it was a risk I had never thought I'd live in Wisconsin. Never thought I'd do that kind of work.

[Ed Gaudet]

Where are you from originally?

[Kali Id]

Central Illinois.

[Ed Gaudet]

Okay. Here from Illinois? Yeah.

[Kali Id]

From the Midwest?

[Ed Gaudet]

Yeah. Yeah.

[Kali Id]

But took a chance on that position and it has set my career and in many ways my life up in ways ahead. Couldn't have even fathomed.

[Ed Gaudet]

That is awesome. And so how much time did you spend at EPIC?

[Kali Id]

6 years.

[Ed Gaudet]

So you were there like at the beginning and you saw it build. By the time you left, how many buildings did they have?

[Kali Id]

They had two full campuses with the third being built. The big epicenter training campus had been built. And so it was much bigger than it was when I started.

[Ed Gaudet]

So that's pretty cool. And so out of that six year experience, what would you say you got most out of that? Like how has it made you different?

[Kali Id]

They train and educate you as a young person on healthcare, on presentations, on interacting with individuals. I mean I was 22 years old. Meeting with physicians and helping to figure out workflows within electronic harassment.

[Ed Gaudet]

That's pretty cool.

[Kali Id]

I was getting real meaningful impact and I think that's what got me hooked on how do we help our physicians take care of their communities and their patients and families with technology.

[Ed Gaudet]

I love that. Yeah. And kudos to Epic, right. For doing that kind of training.

[Kali Id]

Oh, it was great.

[Ed Gaudet]

You gotta be able to develop your talent. Like because as an organizational leader you cannot put bets on talent. You gotta be able. Cause they'll leave. You gotta be able to do a combination hire, but train them so well.

[Kali Id]

And give them the opportunity to show and grow. Because that was it too. We were young, but in these meetings with individuals who are decision makers at health systems, we were partnered with the brightest individuals developing technology. And it was great environment to thrive.

[Ed Gaudet]

That's really great. What's one thing that Most people don't know about Northwestern medicine, just how big we are.

[Kali Id]

So I think a lot of people see Northwestern and they know the academic campus in downtown Chicago, but we actually have 10 other hospitals, 200 physician offices and sites throughout the greater Chicago area. We now have partnerships down in Florida and in London. And so just the scope and impact of. Oh, I didn't know about Florida and London communities. Yes, we have sites down in Naples.

[Ed Gaudet]

Oh, interesting.

[Kali Id]

And then a partnership with a London clinic as well. And so I think a lot of people don't know where all we are hoping to make a difference and change in the world.

[Ed Gaudet]

Super cool. Well, being a Chicago guy, like I definitely knew about the campuses up north and out west, but Florida, London, that's exciting.

[Kali Id]

It is.

[Ed Gaudet]

And being at the head of innovation, I'm sure you're getting an opportunity to vet all sorts of things. And here at aimed I'm sure learning some things to help you distinguish signal from noise. What advice would you give to folks that couldn't make it to the conference this time around to how can they best vet through the noise?

[Kali Id]

I think it's really, and this is a theme that we've heard throughout, it's just putting structure around how you're going to evaluate and assess these opportunities. And not just don't just say you have a governance group and everything goes through governance and therefore you have a committee and therefore you're doing it. But every single solution, what problem is it solving? Is it doing it in a novel way? Is it doing it with the right type of technology? AI isn't always the answer. And so does this really need to be solved with generative or with agentic or with insert AI here and then define and measure what success looks like? I think a lot of people say that pilots don't work, you get stuck in pilots. But more often it's simply because you didn't define up front what you were expecting success to look like. You didn't set the metrics and the measures and the factors that you were going to look at to define it. So then as it got turned on, you were pulling that together and it was tough to figure out what you needed to see. So what we've really put some structure around is defining upfront and then measuring it and then giving a very set period of time. Either you were able to show the improvement in those outcomes and those measures or you can't. And that's okay. And we have learnings as a result.

[Ed Gaudet]

I hear the the ROI needs to be at least seven Figures. How true is that?

[Kali Id]

I don't know if we've. I think it depends on how much you put into it. Okay, so if it's a $25,000 a year solution, I probably don't need to see that. But if it's a million dollar solution, fair enough.

[Ed Gaudet]

Yeah. But you know, with the bandwidth challenges that we have, like, you got to be able to show ROI as the bottom line, right?

[Kali Id]

Absolutely. And quantifiable. Like a lot of people say, oh, there's some soft roi. The promise of AI is that it can do things faster, better, more precisely. That should be measurable, that should be able to measure that.

[Ed Gaudet]

Yes. Well said. Well said, Callie. Leave us with a closing thought. And where can people reach out to you to learn more about the work you're doing? The work that Northwestern Medicine is doing?

[Kali Id]

Yeah, reach out to me via LinkedIn. We have innovation.nm.org now, too. You're going to see some rebranding coming out in the next few months as well, I think. Always love to hear about it. I would say closing thought is we love to talk to anybody and work with anybody who's trying to dig through the hype, get to the real world application and make a meaningful difference in the halls of our hospitals, into the rooms of our clinics, into the lives of our communities and our workforce every day. So that's what we're looking for.

[Ed Gaudet]

I love that. There you have it, folks. Kali Id leading innovation at Northwestern Medicine. Such a pleasure to see you again, Callie. Always good to connect with you and hear the latest.

[Kali Id]

You as well. Thanks for having me.

[Ed Gaudet]

Thanks for listening to Risk Never Sleeps. For the show, notes, resources and more information and how to transform the protection of patient safety, Visit [e-mail] that's C-E N S I N E T dot com. I'm your host, Ed Gaudet. And until next time, stay vigilant because Risk never sleeps.

[Kali Id]

Sam.