S1, E46 - Peter Embi: The Doctor Who Diagnosed Himself

Summary

Practical AI in Healthcare – Episode Summary

Episode: S1, E46 – Peter Embi: The Doctor Who Diagnosed Himself
Date: July 19, 2026
Hosts: Dr. Steven Labkoff and Dr. Leon Rozenblit
Guest: Dr. Peter Embi
Main Theme: Turning AI promise into practice in healthcare: the urgent need for algorithmic vigilance, learning health systems, and scaling AI safely, grounded in Dr. Embi’s personal diagnostic odyssey.

---

Episode Overview

This episode features a candid interview with Dr. Peter Embi, President of the American College of Medical Informatics, reflecting on his personal experience being both a physician and a patient with a rare disease. Via this narrative, the discussion explores the real-world challenges, responsibilities, and opportunities in deploying AI at scale within the healthcare ecosystem—from diagnosis and patient care to algorithmic monitoring and system-level oversight. The urgency of robust monitoring, responsible AI governance, and the potential for AI to not only augment but transform clinical practice is a consistent through-line.

---

Key Discussion Points & Insights

1. Dr. Embi’s Origin Story: Physician, Informatician, Patient

• Early Inspiration:
  • Grew up in Miami, son of Cuban immigrants; discovered medical informatics after using early computers like the Commodore 64 and Apple Newton MessagePad.
  • “I wasn’t necessarily a computer guy until I got to med school.” (04:09, Embi)
  • Saw a gap in medicine for “fixing broken things,” leading to a dual focus on medicine and improving health information systems.
• Academic Milestones:
  • Pioneered clinical research informatics, developed point-of-care alerts for clinical trial recruitment during time at Cleveland Clinic, and eventually focused primarily on informatics rather than clinical rheumatology.
  • “I thought I was going to be a rheumatologist who dabbles in informatics, but I’m actually an informatician who dabbles in rheumatology.” (07:27, Embi)

2. Diagnosis Odysseys & Human Limits

• Personal Case Study:
  • Dr. Embi experienced years of vague symptoms before himself identifying a rare adrenal tumor (pheochromocytoma)—despite interactions with multiple doctors and near-fatal episodes.
  • “I was thankfully paying attention that day in med school, and I thought, ‘Oh my God, I think I have a pheochromocytoma.’” (13:39, Embi)
  • Points out how both patients and doctors are susceptible to denial or tunnel vision, even in life-threatening scenarios.
• Broader Implications:
  • Highlights the frequency of missed or delayed rare disease diagnoses; underlines the potential of AI to change this paradigm.
  • “There are technology solutions that would have actually helped accelerate my diagnosis and accelerate the diagnosis of other people.” (16:21, Embi)

3. AI’s Real-World Promise and Challenge

• Learning Health Systems:
  • The electronic health record (EHR) ecosystem still lacks the mechanisms to “close the loop” on diagnosis and learning.
  • “We want to create a learning health system...not only [with] the tools and capabilities, [but in] privacy-preserving ways.” (19:01, Embi)
• Algorithmic Vigilance:
  • Dr. Embi introduces the concept of “algorithmic vigilance”—continuous monitoring of AI algorithms in practice, akin to pharmacovigilance in pharma.
  • “We don’t really have this capability to actually monitor and understand whether or not these AI solutions are having the intended effects.” (22:11, Embi)

Notable Tool: VAMOS

• VAMOS: Vigilant AI Monitoring and Operations System—Vanderbilt’s platform for centralized monitoring of hundreds of deployed algorithms.
• “I analogize it to sort of an air traffic control tower…now we have hundreds of algorithms running here at Vanderbilt...but we don’t really have good visibility into how they’re actually working.” (23:24, Embi)

4. Risks of Flying Blind

• Consequences Without Oversight:
  • Algorithms can drift, produce harm, or be ineffective if not monitored, especially post-deployment or when ported to new settings.
  • “We deploy with all the best intentions...and unbeknownst to us, it drifts, or it just doesn’t have the intended effect. People are being harmed.” (25:25, Embi)
• Need for Networks:
  • Monitoring AI on a network level (across multiple institutions) enables better evidence generation and benchmarking, not just local optimization.
  • “A network...could produce that...I’d have more confidence in adopting it here.” (28:29, Embi)

5. Moving Towards National Standards and Collaboration

• Open Source and Collaborative Efforts:
  • VAMOS Collaborative: An open-source initiative seeking to democratize algorithmic oversight, including the majority of non-academic medical centers and integrating standards with help from HL7 and others.
  • Also, the “Trustworthy and Responsible AI Network (TRAIN)”—over 50 health systems sharing best practices in oversight and governance.
• “FAA for AI” Analogy:
  • Hospitals need their own “air traffic control,” but there’s also a need for a centralized, possibly federal layer akin to the FAA for overall oversight.
  • “At the Nashville airport, we need to watch all the planes...then you also need something like the FAA, right?” (35:43, Embi)

6. Philosophical Challenges: “Is there Anything New Here?”

• AI is More Than Automation:
  • Debate over whether current “AI” is fundamentally new or simply better automation; distinction comes from generativity, agency, and unpredictability of modern models.
  • “It’s really hard to kind of understand how [AI systems] work entirely...they’re a lot more so-called black box.” (40:32, Embi)
• Intuitive Gaps:
  • Modern AI “confuses our old intuitions about technology”—it behaves unlike deterministic devices or human agents.
  • The need to develop new metaphors and regulatory approaches.

7. Safety is Acceleration: Brakes and Speed

• Metaphor of Progress:
  • “Why do we have brakes on cars? Not just to slow down, but so we can go faster.” (45:22, Rozenblit referencing security expert)
  • Rigorous oversight and monitoring are enablers, not obstacles, to responsible AI acceleration.

8. Patient Use, Literacy, and Agency

• Should Patients Use AI Directly?
  • Patients are already leveraging tools; the challenge is to make AI use both safe and beneficial, not to restrict it.
  • “The answer is not okay, they shouldn’t use it. The answer is, how do we...train people...to make these as effective as they can be.” (50:19, Embi)
• Education/Literacy:
  • Calls for collaboration through professional societies and organizations to develop patient-facing educational resources for safe and effective AI tool use.

---

Notable Quotes & Memorable Moments

• Origin Story:

  • “I thought, I need a break. ...I bought a legit computer. At the time, this thing called the World Wide Web had just come online...I reverse engineered my first web page.”
    (04:09, Embi)

• Self-diagnosis and Denial:

  • “Like a good doctor, I explained most of these things away in my head...we’re really good at denial. Just humans generally, and doctors especially.”
    (11:01, Embi)

• The Dangers of AI Without Vigilance:

  • “We don’t really have this capability to actually monitor and understand whether or not these AI solutions are having the intended effects...I called it algorithmic vigilance.”
    (22:11, Embi)

• Why Monitoring Matters:

  • “We can harm people or populations if we don’t watch these things.”
    (26:13, Embi)

• Safety as an Accelerator:

  • “Why do we have brakes on cars? ...so we can go faster.”
    (45:32, Rozenblit, quoting a security expert)

• What’s Genuinely New About AI?:

  • “It’s hard to get under the hood and even, you know, keep that so-called human in the loop...That’s how that would be...my answer to that first devil’s advocate.”
    (40:37, Embi)

• Patient Use and Responsibility:

  • “People seek out information in all sorts of different ways...I think we want to take advantage of these tools...but the task we have before us is how do we not only make these systems better...but also better illuminate their degrees of error.”
    (49:05, Embi)

• Optimism with Eyes Open:

  • “If there’s not a significant positive gain to be had, then no amount of risk is worthwhile...we have to take advantage of these tools to improve care and efficiency and the health system as a whole. We just can't forget there are potential perils as we do so.”
    (53:28, Embi)

---

Timestamps for Key Segments

| Segment | Topic | Timestamp | |---------|-------|-----------| | Peter Embi’s Background & Path to Informatics | 03:46–07:35 | | Dr. Embi’s Diagnostic Odyssey | 10:54–16:43 | | AI’s Role in Changing Diagnostic Timelines | 17:52–21:02 | | What Is Algorithmic Vigilance? | 21:02–25:12 | | Why Monitoring Networks Matter | 26:33–29:19 | | Open Source & Collaboration: VAMOS, TRAIN | 31:41–34:04 | | Who Should Oversee AI in Healthcare? | 34:04–36:47 | | Devil’s Advocate: Is AI Really Special? | 38:14–41:10 | | Patient Agency and AI | 46:01–52:26 | | What Does “Doing it Right” Look Like | 52:54–56:00 | | Episode Summary and Close | 56:00–57:26 |

---

Takeaways & Actions

• The Future is Bright, But Not By Accident:
  • Algorithmic vigilance isn’t a brake—it's an accelerator. Making AI safe makes scaling it possible.
• The Human Patient Perspective is Instructive:
  • The linkage between physician, informatician, and patient experiences (as lived by Dr. Embi) grounds the AI conversation in human reality.
• Networks and Open Collaboration are Essential:
  • Isolated solutions aren’t enough; national, standards-based approaches are urgently needed.
• AI Literacy for Patients (and Providers) is Next:
  • Both technical improvement and education are vital; responsible societies have a crucial role to play.

---

For further resources:

• Vamos Collaborative (Open Source platform)
• National Academy of Medicine Code of Conduct for AI
• HL7 and other standards organizations

---

Transcript

[A]

Welcome to Practical AI in Healthcare, the podcast that cuts through the noise to spotlight real world solutions delivering real world value. From patient care to clinical research, from life sciences to patient engagement, we focus on what truly matters in healthcare today. No hype, no theory, just practical insights where AI is making a true impact. Dr. Steven Lapkoff and Dr. Leanne Rosenblitt are your hosts as we explore what's real and moving the needle in this exciting new domain. Welcome aboard and let's get to it. Last September, the DCI Network hosted its inaugural Signal through the Noise Healthcare AI Conference in Boston. It brought together healthcare delivery leaders, technology innovators, government officials and engaged patients. We explored some of the most important real world use cases in healthcare AI. This year, from September 23rd to the 25th, we'll reconvene in Boston for the next chapter. Signal through the Noise. From Proof to Scaling Enterprise AI Across Healthcare Systems and Biopharma, the question this year is simple but urgent. Once we found the signal in the noise, how do we turn promising AI pilots into sustainable, trustworthy implementations across the healthcare ecosystem? If you're a fan of the podcast, we'd love to have you join the conversation live in Boston. Podcast listeners can receive 30% discounts on their registration with the discount code PracticalAI30. Come join us this September for Signal through the Noise from Proof to Practice. We hope to see you there. Hello and welcome to this week's edition of Practical AI in Healthcare. My name is Dr. Stephen Lapkoff and I'm here as I am every week with my colleague, Dr. Leon Rosenblit. How's it going, Leon?

[B]

Oh, hang it in there, Steve, back from the DCI Patient Conference, which was really exciting, but it did beat, beat me up a little bit and cost

[C]

me some health points.

[B]

But I'm recovering well.

[A]

The conference was great. I, I do want to make a call out to Yuri Quintana and to the rest of the crew up there in Boston. We had a wonderful three days and you'll. If anyone's interested, you can probably find the recordings of the of the conference up on the DCI website. That would be dcinetwork.org events and they'll be posted probably within the next couple of weeks. But we're here today not to talk about the dci. We're going to be talking to a good friend of ours. His name is Peter Emby. Peter is an informatician and he happens to have been the nation's first crio and he's currently the President of the American College of Medical Informatics. He hails out of ohsu, out of Oregon for his background. And currently he. He is at Vanderbilt and is the head of the program at Vanderbilt. We're going to have a conversation with Peter today about his perspectives on AI and where it's going and some personal stories about his life. So, Peter, welcome to the podcast.

[C]

Great to be here. Steve Leon, so good to see you guys. Yeah, it's wonderful to be with you.

[B]

And Steve is like, we're kind of underselling Peter's credentials and impacted informatics. Right. It's hard to oversell them. But like, he invented a subfield like clinical research informatics. He and Phil Page and basically just invented this field from thin air. But it worked out pretty good. So thank you guys for doing that. I was able to, I was happy to have something. Some slides that I could reference during my career as I was building systems that. No, there's an academic discipline behind it.

[C]

That's very, that's very nice. Yeah, you know, we all build on the work of others, but I appreciate that.

[A]

So, you know, Peter, the first thing we typically ask our guests is about how they got the superhero cape and what's their background story. So if you don't mind, you want to dig in a little bit and go back a few.

[C]

Yeah, yeah. Happy to. Well, so I am originally from Miami, Florida. I'm the son of Cuban immigrants born and raised in Miami, and went off to the University of Florida to do my undergrad and then the University of South Florida for med school. Like a lot of kids back then, I had a Commodore 64, but I wasn't necessarily a computer guy until I got to med school. And between my first and second year of med school, I had been going straight through college med school, and I thought, I need a break. A lot of my friends went off to work in labs and do pcr. And I decided I'm going to take a break. And so I bought a legit computer. At the time, this thing called the World Wide Web had just come online and I thought, oh, well, this is interesting. And so with my, you know, 144 blazing, you know, 144 Bob Mott on my, I, I got online and I, and I reverse engineered my first web page and taught myself a little bit about that. And then there was this surgeon in my medical school who had this elective called Medical Informatics. And I thought, well, that's interesting. Let me take that elective. And that was my first introduction to the field. I actually also at that time, these handheld computers were coming around and I had what was called the Apple Newton messagepad, and I happened to know, Steve, that you had one because you were one of the only other people at that time who was actually like using this thing and publicly talking about it. I was sort of following in your footsteps a little bit there because I started lecturing to my classmates about it. And then I started lecturing at national meetings about using these handheld computers. And I went off to. When I was getting ready to go to residency, I thought, well, I want to study more about this informatics thing. I didn't think I'd be doing it full time, but I thought, let me go somewhere where they actually have a program in this. There weren't a lot of them in the country and one of them was in Oregon. And so. And they happen to also have a great internal medicine residency. So I went there for residency and I remember, you know, walking into Bill Hirsch's office and as an intern and said, hey, Dr. Hirsch, I think I want to study informatics. He said, that sounds great. Why don't you go do your residency and then we'll talk. But no, he was incredibly generous and started working a little bit with him. And so on the heels of my internal medicine residency, and even as I was sort of doing that work and moonlighting, making websites for physicians offices and doing a little bit of work with electronic medical records back then I got my. I did my fellowship and at that time the terminal degree was a Master's of science. And so I got my master's degree in Oregon, but I still didn't think you could do this full time. And so I went off then to do my other passion, which was rheumatology and immunology, and I went to the Cleveland Clinic. And while I was doing my rheumatology fellowship, they were deploying epic. They were one of the first places beyond Kaiser to actually deploy that platform. And I went and knocked on the CIO's door and said, look, I'm a rheumatology fellow here, but happen to be a card carrying informatician now, so if you need any help, I'd love to help. And he took me up on it. And I ended up starting a project related to how we could customize the electronic medical record to actually help with participant recruitment. And I created point of Care alerts, what became known as the clinical trial alert that I invented there. And then that was the beginning of the work in clinical research informatics. So, you know, from there got my first faculty position and the rest of history. I just sort of, you know, kept doing informatics. And despite my earlier thought, it quickly became my full time job. And I thought I was going to be a rheumatologist who dabbles in informatics, but I'm actually an informatician who dabbles in rheumatology.

[A]

Well, you know, it's funny you outlined, you know, our careers in many respects ran in parallel tracks. I started life as an internist at Pitt and did my residency out there and I did exactly the same thing you did, but I think I was about 10 years before you, maybe 10 years, something like that.

[C]

Yeah, pretty close.

[A]

And the whole point of being able to make a career out of this was not obvious back in the 90s. Just never thought that would be something you could, you know, get a paycheck for. It was, you know, it was too much fun. It was like, how could you have this as your job when it's this much fun, right?

[C]

They let us play with computers and try to fix broken things about the healthcare system, you know, in between doing our day jobs, which is taking care of patients.

[B]

So. But at the same time, right, you guys both have worked at this very interesting intersection where AI, where IT and healthcare gets real, right? It's monitoring diagnosis, the learning healthcare system. So I mean, that's a really interesting converging theme, right? That problem attracts the right kind of nerd.

[C]

Well, yeah, well, and probably the same as both of you. I know, because I know you well enough to suspect this is true. You know, I, I am an engineer at heart. I never really sort of thought about doing engineering. But I, I say that because I, I think back to, you know, when I was younger and one of the things that's always driven me is when I see something that's not working or not working well or could be made more effective or efficient, I want to fix it. And one of the things I learned pretty early on about this whole informatics field is, you know, when you get to be on the inside of data and technology in healthcare, at the end of the day, what we do in medicine is manage information and that's really, this is the field that does that. And so that really allowed me to scratch that itch, which was like, I really want to try and improve the system. And so it allowed me, much earlier than I think would have otherwise been the case to actually do systems work in addition to individual care.

[B]

No, I think you hit on something interesting psychologically.

[A]

Right.

[B]

I always think of myself as a builder, not formally an engineer, but when I see a problem I try to design a system that solves the problem sometimes very badly, but that's definitely one of the very first impulses. And I know Steve is the same. Right. We see stuff like, there's got to be a way to make it work better. But that speaks to why we had to have to bring you on the podcast now. We have a novel set of problems that have to do with, with novel technologies that are coming out. And I think there's, there's just such an important time to have people who are builders and fixtures at heart who also understand how healthcare and research work.

[A]

So, you know, spot on, Leon. You know, Peter, in the prep call that we did with you beforehand, you've been part of this, a story that really was a, perhaps a catalytic story in your life and your back life about, about your own journey through the healthcare ecosystem and some left turns it took that almost actually ended your life unexpectedly. If you wouldn't mind, would you unpack a little bit? Because I think it will impact the discussion as we go forward in the, in our call today.

[C]

Yeah, happy to.

[B]

So, yeah.

[C]

So in addition to, you know, being a, a physician and as I said, a rheumatologist, immunologist who, you know, treats people with rare diseases, I happen to have also gotten a rare disease. And so in my case, I had one of the rarer diseases. The one, one of the, one of the sort of prototypical rare diseases that, that informs the phrase we use in medicine, which is if you hear hoofbeats, think horses, not zebras. We often call rare patients with rare diseases zebras because, of course. So I'm in Nashville right now, right. Same in Boston. Same as anywhere. I'm walking down the street and I hear hoof beats behind me. It's pretty silly to turn around and expect to see a zebra that's probably going to be a horse. Of course zebras exist, but, you know, they're just not common. And, and so we miss a lot of diseases because they are rare. And I, I had one of those. So what happened to me was back in 2017, I had been having symptoms, in retrospect, for probably 15 or 20 years. And the symptoms were things like headaches and I would, I would sweat a lot and I would have the occasional tremor. I would, I would get anxious at times and have resting tachycardia, wake up with, you know, night sweats, like a good doctor. I explained most of these things away in my head. It didn't necessarily go and sort of say, hey, I'm having problems because number one, it was gradual. But number two, you know, we're really good at denial. Just humans generally and doctors especially. And when I would go in for my occasional checks, my blood pressure was a little bit elevated, but not terrible. Well, in 2017, that changed and changed drastically. So I ended up having very, very severe spells where I would get this all of a sudden, this sudden onset, worst headache of my life, a so called thunderclap headache. And beyond the headache, I was panicky, I was pale, I felt like I was dying and I was in excruciating pain because it was extreme, extreme headache. And turns out I was having hypertensive crisis, but I didn't know that. So the second time, the first time it happened like a good doctor, I didn't do anything about it. The second time it happened, I happened to be in Boston and I was going to be giving a talk at a panel the next day and I got this thing and I thought, okay, I got to get this thing checked out. Thankfully, I was there with my girlfriend at the time and she said, look, we're going to emergency room. I said, you're right. So we actually got in a, got on a lift and went to, let's just say, man's greatest hospital. And so we, I went to the emergency room and I said, I'm having the worst headache of my life because I know how to get attention. And they whisked me to the back and they did what you're supposed to do when somebody comes in with a thunderclap headache. They ruled me out for a bleed in my head. And at that point I had, you know, systolic of 220 and severe tachycardia. They gave me lots of clonidine for my blood pressure and they did some other tests on me, but they didn't think about what I ultimately had. And they ultimately discharged me the next morning with atypical migraine and hypertension. And I was very sick, went home, had a few more episodes of this and went to go see a neurologist for my atypical migraine. And she said, well, I think it's this other thing. I think it's occipital neuralgia, because I was having this sort of in the back of my head, occipital manifestation of this headache. So I don't think it's migraine. Gave me some injections in the back of my head that was supposed to help immediately. It didn't. Went back to the Regenstrif Institute where I was the president CEO at the Time. Sat down for grand rounds, had another episode. And while I was telling my assistant not to call 911 because I looked like I was dying, I thought, okay, Peter, what's going on? You know, you're an internist. Work the problem, Right. At this point, I had seen a few doctors. Lots of testing had been done. So I thought, okay, maybe I'm having panic attacks. That'd be great. I'll go to see a psychiatrist. You know, we'll figure out what's going on. But then I thought, no, you know, this is very physiologic. And in my head, as I was suffering with this sort of sixth episode, I thought, okay, headache, diaphoresis, hypertension, and that's the classic triad. And I was thankfully paying attention that day in med school. And I thought, oh, my God, I think I have a pheochromocytoma. The pheochromocytoma, for those who don't know, is a very rare adrenal gland tumor. Your adrenal gland does lots of things, but one of the things it produces is adrenaline. And this is an adrenaline tube. So essentially, I had vast amounts of adrenaline all the time. And then I would have these spikes. And it was on the right. It was behind my liver. So what happened was, what I would lay down my liver, would press on it, and it would essentially squeeze the juice from my tumor directly into my bloodstream. And I'd get these massive increases. Suddenly and essentially hypertensive crisis. 50% of people with these conditions die before it's ever diagnosed because it's very hard to diagnose.

[B]

Yeah.

[C]

Anyway, called my neurologist and said, look, I know this is going to sound crazy. I just had another episode. I think I have a fio. She said, probably not, but let's check your blood work. And it came back, and sure enough, that's what I had. And thankfully, in my case, even though it was quite large, I was able to work it up, get it removed, and I'm great. That was over eight years ago now, and I'm healthier than I've ever been. And I'm very grateful, very fortunate to be alive, I think, to your point. It's also that plus being a rheumatologist and taking care of people with rare diseases that are often under diagnosed or misdiagnosed for some time, it's all been very motivating in terms of saying, look, how do we use our capabilities? Because there are technology solutions that would have actually helped accelerate my diagnosis and accelerate the diagnosis of Other people. And by the way, not just for rare diseases, but for more common things as well. So, so. But yeah, that's my story. And it definitely has been in many ways, personally and professionally, very, you know, very informative to my future.

[A]

You know, you're the second doc we've had on this show in the last couple of weeks or maybe months or so who had a very similar story. In other words, when they started having symptoms of a problem, they didn't immediately put their doctor hat on, they put on their being a patient hat. And they didn't use all the skills right at their. At their fingertips to work the. Work the problem. Barry Chaikin was an internist we had on the show a few weeks ago, and he basically said the same exact thing. He was, was diagnosed with prostate cancer and then he stopped working as a doctor and started working as a patient and didn't, didn't get a second opinion initially until somebody shook him and said, hey, if I was sick, what would you tell me to do? And he woke up all of a sudden, he did what he needs to do. You know, I gotta believe that this is such an informative thing to have lived through. And I'm sure it's colored a lot of the things you've done. I mean, diagnosis of rare disease is one of the things that AI is actually surprisingly good at. You know, back in the day, when I was a fellow, I was working on something called qmr, the Quick Medical Reference. And I know up in Boston, Octo Barnett was working on something called Dexplain.

[C]

That's right.

[A]

And these tools were early versions of decision support, but they were brutally difficult to work. It took 45 minutes to put a case together. Fast forward now to where we are 25, 30 years later, and you can talk to an AI system, give it the symptoms, have it interpret through natural language processing, and do what we were trying to do back in the day with these tools that the giants built like Octo and Randy Miller. And here we are.

[B]

So, yeah, let me, let me frame what Steve is saying is into a question for you, Peter. So having that night at the AR would make a mission personal for you, Right? So how does it change what you build now, what you're interested in building and what's, you know, specifically other opportunities you see for AI and similar technology to shorten the diagnostic journeys in rheumatology or elsewhere or rare?

[C]

Yeah, absolutely. Absolutely. No question. Yeah, you're right. I mean, I think for me, at the highest level, it's only, you know, Further reinforced my motivation to really try and improve health and healthcare with data and information and technology. I can tell you, shortly after my diagnosis and my cure, you know, I reached out to a number of people and said, look, we've got systems like, I mean, for instance, the exact same electronic medical record system built by EPIC was working in the ER where my diagnosis was initially missed and in the hospital where I ultimately was cured. And right now we have the capability of exchanging information between those systems because we've been working for decades on interoperability and other kinds of capabilities like that. And that's built in, but only for individual patient care. So if I went back to the actual place where I was in the er, they could pull my records down, but if I didn't, which I haven't gone back as a patient, they don't really have permission to pull that down. And so that's an issue. Right. So one of the ways that's motivated my thinking is if we really want to create a learning health system and we want to be able to accelerate diagnostic odysseys and we want to be able to really figure out what people have, then not only do we need to build the tools and capabilities, take advantage of all the data we have, of course do that in appropriate privacy preserving ways, but, but do that in a way that's going to allow us to build out these AI capabilities so that the next patients who come to the ER or wherever can have the diagnosis made much more quickly. And we've been working on that here at Vanderbilt, we've been working that in other places to figure out how do we create those predictive algorithms, how do we figure that out? In addition to that, we have to think of it, I think globally. And that's also informed a lot of the work I've been doing with groups nationally, with learning health system consortia with National Academy of Medicine and others to say, how do we make sure that we can actually fully take advantage of all of these AI capabilities and other kinds of informatics capabilities to improve care by leveraging all these incredible assets we have. And so, yeah, it's really been very, very motivating. And the personal part of it does lend an additional motivation.

[B]

So I'm hearing several threads I want to pull on. But before we do, we got through your superhero cape. So we kind of know the origin story and several radioactive spiders that converge to create Peter Envy. Let's name the super villain of the story. You've coined the term algorithmic vigilance, but let's start less abstract Start concrete. There's a model and it lives in a hospital and nobody's watching it.

[A]

Right.

[B]

What goes wrong? Take us from that all the way down to Vamos. Right. You know, that's one of the threads I want to put up. But let's start with a concrete problem.

[C]

Yeah, yeah, absolutely. Well, you're right. Right. So for quite some time, even before the generative AI capabilities, we've been working on developing predictive algorithms and capabilities to be able to do prediction, to be able to classify and accelerate a lot of what we do in healthcare around diagnosis and treatment and discovery. And yet, to your point, very often we do good work upfront to train these models. We train them on the data we've got. The data we got inherit whatever biases we have in our society, who has access to care, who actually can make their way into the database. And then there's biases we all have that we bring to the building and testing of these tools. Because oftentimes the data we have are not perfect. They're representative or their surrogate markers of what we ultimately care about. But we develop these tools, we test them, we put them into practice, and typically, once they're actually in practice, we don't really have good learning health system. You know, technology enabled learning health system mechanisms to really monitor and understand. Is it continuing to have the effect that we expect unless we happen to do a study or something catastrophic happens? And so we started to learn that lesson quite a bit. And I was on a panel actually in 2019, and somebody said, okay, so we're starting to see all this AI make its way into practice. What's your biggest worry? And I said, well, my biggest worry is we don't really have this capability to actually monitor and understand whether or not these AI solutions are having the intended effects. And we need something akin to pharmacovigilance. Right after we, after we approve a drug and it goes out into the market, we've got to have better ways of actually monitoring the, the, you know, yeah, maybe we did our phase three study with a thousand or even ten thousand patients, but now it is being used on millions of patients. We often see things we didn't expect. Right. What about AI? We need something like that. And I called it algorithm of. And what I've been doing here at Vanderbilt is we actually built a platform because I'm also responsible for the AI center and, and have been sitting on their AI governance committee. And one of the things we've been doing is building a capability so that we could actually have an air traffic control tower that tells us whether or not this is working and mapped to not only the actual mechanistic ML ops kind of analyses of the algorithms, but also to the outcomes of interest. So if it's a sepsis algorithm, supposed to help us predict sepsis, what's it actually doing to sepsis rates and to mortality? If it's a readmission prediction algorithm, what's it actually doing to readmission rates? And how do we map to that? So we've actually got that in hand. And that's where you alluded to it. We call the platform Vamos, the Vigilant AI Monitoring and Operations System. That's what we call it. I'm Cuban, so I love a good Spanish algorithm. My friend Lori Novak actually came up with that, so I got to give her credit for it. But it was great. And so that. That platform, it also happens to mean, you know, go, you know, accelerate forward. Right. So what is it? It's really that it's. It's an. I analogize it to sort of an air traffic control tower, where now, you know, we have hundreds of algorithms running here at Vanderbilt and, you know, increasingly at many institutions, but we don't really have good visibility into how they're actually working. And so we need capabilities to do that. So building the platform, but also building out standards, doing what we do in informatics. Right, how. What are the standards? How should we be monitoring this? What are the benchmarks? How do we know if it's working within bounds or not? And then importantly, how do we take action if we see something going awry so that we can do something about it, so that we could be much more proactive and less reactive. I think that's the part that's really been driving me lately.

[B]

So, Steve, it sounds like amazing progress. I want to make it a little bit concrete for our audience. Give us the worst case scenario version of what flying blind, meaning without algorithmic vigilance, without Vamos, looks like?

[C]

Well, I mean, worst case scenario is we deploy with all the best intentions, a good algorithm that seems to work, and then unbeknownst to us, it drifts, or it just doesn't have the intended effect. And now people are being harmed. And there's actually evidence of that. There's evidence that if you. There have been studies to show that if you don't actually watch these algorithms and make sure that they're doing what you expect, either they get to where they're causing harm because they were initially working, but then they drift in terms of their Performance or maybe it works really, really well here at Vanderbilt because we tested it really well and we validated it here. But then throw it across the street into our, you know, we give it to our colleagues at the hospital across the street or across the country and we assume it's going to apply the same way, it's going to work the same way and it doesn't. And the worst case scenario is people can be harmed. Short of people being harmed, at the very least it could be very inefficient. But really the worst case is we could, we can harm people or populations if we don't watch these things.

[B]

Yeah. Unfortunately the worst case in healthcare is rarely fun.

[C]

Right. It's, it's right, just this case is pretty darn bad.

[B]

Yeah, it's bad. So, so in you just alluded to having comparison hospitals. Why does this need a network or why does it work better in a network than an individual hospital? And then do you, how do you think about it scaling even more broadly?

[C]

Yeah, great question. So there are certain things that, that, that we can do well within one health system. Right. Which is sort of monitor what's happening at our institution and that's our responsibility to do that and we should be doing that well. But one of the benefits of a network that we can't really do individually is that, I mean, depending on where you are in the country, there's different populations, there's different, the patient populations differ, the acuity of care differs, the way we practice differs. I mean we've even seen cases where there's a different performance from one floor to another within the hospital, much less from one hospital to another. Right. Because of different workflows and other things that can affect this. It's not just about data and technology. Sometimes it's obviously a socio technical system as we like to say, and informatics. And so by creating a network of institutions that are using similar kinds of standards based approaches to monitor their own AI and by sharing as appropriate the sort of emerging data that is appropriate to share across institutions, we can actually start to develop evidence. I mean one of the real problems we have right now in AI, and you guys know this well, is that we don't really have a lot of great evidence for its effectiveness. Right. And with, with some rare cases that, that prove the rule. And so being able to have a network where we can use the data exhaust from the ongoing monitoring at each of our individual sites to sort of aggregate that it'll allow us to create an evidence network where we can actually have a Learning health system that spans more than one healthcare organization. And frankly, even at a very practical level, like as someone who's been responsible for procuring AI solutions, oftentimes the vendors themselves, they don't really have good evidence. They wish they had better evidence. Right. But they have what they have. Well, a network like this could produce that. And then I benefit because if something was used at MGH or, or at Duke or elsewhere in the country, and I knew that it was functioning well, I'd have more confidence in adopting it here. And finally, I would say the network also can help us in terms of teaching us best practices for how to actually do this better across the board. So there's lots of reasons why I think a network effect is really essential to creating a learning health system. And it's even more important right now with AI because it's different than a lot of other technologies.

[A]

Yeah, and, and Peter, you're spot on. I think the, you know, one of the things about algorithmic drift, I think individuals can actually see a very tiny version of that if they work long enough with chat, GPT or with claude. If you work the same problem long enough, all of a sudden, halfway through your, your work, it tends to get confused. And if you, and we all know that if you ask the same question of it at 9am versus 4pm or when the moon is out, or when it's a Tuesday versus a Thursday, the answers that you get are either slightly different or potentially dramatically different. And I think this issue of algorithmic drift is one of the things we've been talking about at the DCI now for quite a while about how do we put together programs for monitoring this and closing the loop. And by the way, this isn't just a problem inside hospitals. There's also problems inside of research institutions. At my last, my last pharma company where I worked in, I came in as the head of analytics, and when I asked folks, you know, as I coming as the VP of analytics, I asked my team, so what was the last time you looked at the predictions you're making with your analytics and how did they work?

[C]

Right?

[A]

And I was dumbfounded because the answer to the question was, we don't do that. I'm thinking to myself, how do you not do that? How do you not look back and say their predictions worked or they didn't work? How do you know if you're going to improve? And you know, that's a, that's another microcosm of the problem you just described. And it's amazing to me that we live and we work and we run on the treadmill and nobody stops to look back and say what happened? And this is the concept of our learning health system comes into play. You know, Dean Sidig and I have written an editorial a few years ago. It's called who's Watching the Watchers? And that's the title of an old Star Trek episode. But the point of it is, is we don't have good mechanisms aside from the MedWatch scheme, the MedWatch program to surface adverse events or other information that could be very useful whether it's in the network or whether it's national to surface these issues. And I think, you know, seeing what you're doing in your institution and around the Vanderbilt area and in Nashville, that's the kind of thing that needs to be blown off. Blown blown up as a much bigger program. How do you see that as being something that you could progress? What do you, what are the pathways you could see to making that to, to bring it national?

[C]

Well, one of the things I've done is so it sort of took that base technology and licensed it out to, to actually take it to an open source network. So we actually have launched, just launched an open source network that we call the Vamos Collaborative. We might be renaming that but the idea of an algorithm of vigilance based collaborative, whether it's based on the Vamos technology or not, the idea of getting a group of forward leaning institutions together initially with the goal of having it be ubiquitous and more widely available to everyone. Because we don't want to build these things only for the Vanderbilt s and Dukes in Harvards of the world. We want to build them for the majority of people, get their health care at non academic medical centers. And we know that's where we need AI more than ever. And so we want to make sure these capabilities are there. So part of what we've done is we've created a consortium that involves some willing institutions to start with and organizations like HL7 to help us with the standards piece and others to actually develop out what are the capabilities we need, what are the standards we should be using, what are the benchmarks. And that collaborative is now getting off the ground. So I think that's one way to do it, but we really want to take that larger. So one of the other things I've been working on the last couple of years is creating an organization that we call Train the trustworthy and responsible AI Network to be a complement to other organizations like Chai and others, which is really very practical. It's over 50 health systems now that are set up to share best practices. And among those best practices is how do you govern, how do you provide oversight and what are the approaches we need to be using to do this. So there's a combination of things that we need to do around developing and deploying technologies, developing the standards under those, and then just best practices that need to emerge. And I think all of the above is the way I'm trying to approach it, but it really is going to be a team approach. I don't think this is the sort of thing that is going to be solved if each of us just does it individually at our institutions. I think we have to grapple with and figure out what are the sorts of things that especially learning lessons from the EHR experiment over the last 30 years. Right. We want to not repeat a lot of those mistakes. So what are the sorts of things that really shouldn't be proprietary that are sort of common around standards and the like so that we can actually benefit from the network effect? And then certainly there are going to be things that are proprietary and that's great. There's nothing wrong with that, but I think we need a combination.

[A]

Who do you see as being the, the overseer of this? Is this something that the, that the hospitals would do themselves or does this need a federal agency, whether it's FDA or onc? ONC is not really a regulator, but it could be potentially. Where does X us out? How do we actually land that in a way where we can make all this as public as possible?

[C]

It's a great question. I do think that, as you know, there's a range of sort of regulatory schemes for different kinds of AI. So when we talk about software as a medical device, there is an aspect of FDA that of course regulates certain AI solutions. And to some extent this network, much like with pharmacovigilance. Right. Is going to be of interest and benefit to the likes of the fda. But of course not all AI is regulated by the fda. And so a lot of it is at the level of the individual institutions. There's emerging work that isn't necessarily about per se law or regulation, but it might be about certification. So the likes of Jaco starting to think about this also. But I'll tell you that I think, you know, there's certainly from a health system perspective, there's an aspect of this that we need and that's why we're building it and deploying it, because it helps us understand how the investments we're making to Take better care of our patients. With our technologies and all the, frankly, capital that we're spending, adopting these AI solutions, are we actually getting a return on our investment? Is it actually helping us with patient quality and safety and all those kinds of things that we're responsible for? And so we need it for that. And so at the, at one level, the institution. And that's how I've gone about building this approach, because I believe that, you know, the incentives are there for individual institutions to monitor their own tools, much like an airport needs an air traffic control tower. Even if the individual airlines might know where their, you know, Delta and Southwest and Continental or whomever's around still, you know, they might know where their planes are. But here at the Nashville airport, we need to watch all the planes. Okay. And so we got to do that at the level of an institution. But then you also need something like the faa, right. You need something that also looks at the nation. And I think that's going to evolve over time. And not everything needs to be shared. Not everything we need internally is necessarily going to be shared externally. But there's a significant fraction of that that we're sort of working through now that I think will be shared externally so that the greater good can also benefit from it. And then some fraction of that will go back to fda, and I think we'll see evolution of other agencies. But the first step right now is building the capabilities and demonstrating what's really needed and what really works. And then I think that goes a long way to informing the downstream regulations.

[B]

Yeah, it's super interesting. And I think between the discussion, between the two of you, as somebody who's mostly been outside of pharma, really reminds me that AI adverse events are currently effectively invisible because we don't have a reporting equivalent of MedWatch. And I also intrigued by this idea that the pharma safety culture, it may be the right mental model with whatever regulatory frameworks are developed. Right. We still have to think about safety as being paramount because we just, we

[C]

don't, you know, we don't want to hurt people.

[B]

Right. It's one of the. I've heard of this concept in medicine.

[C]

Yes, we first.

[A]

Yeah, that's.

[C]

That's usually our prime direction.

[A]

That's right.

[B]

I think we can all kind of agree on that one. So let me. I'm going to do the fun part for me now and play devil's advocate. And I know you and I are going to have fun with this. Right. So the devil's advocate position that I sometimes take in this podcast kind of goes like this. There's no such thing as AI, it's a marketing concept. And AI, what we are talking about AI, we talk about regulating AI, we talk about governing AI, we're really talking about it. It's just automation. Right? Tell me what's genuinely different about what we're calling AI versus a set of differential equations. I hand you as a predictive model and then we'll talk about what's the same. But that's my first devil's advocate challenge.

[C]

Well, I mean, great question. Probably won't surprise you to know that I could easily be the devil's advocate here too. I mean, there are a lot of things about what we currently are grappling with with AI that actually look a hell of a lot like the things we grapple with with any technology deployment and information technology solution. But the things that are different, I mean, there are some things that are legitimately different. So. And again, we talk about AI as if it's all one thing. But as you guys know, but just for all your listeners, we talk about AI. Of course, the AI everybody's really, really familiar with now because of the incredible success of generative AI tools like ChatGPT and others, is this generative sort of AI capability. And that is very different, right, because you're interacting with it in a way that of course we all are very familiar with now. It's much more easy, you can talk to, sort of passes the Turing test pretty darn easily. And you end up with almost as Steve was alluding to earlier, whether you call it a feature or a bug, you know, it's constantly giving you different answers. It's being creative, you know, it's doing different things almost by design. Actually, not almost by design. By design it is in fact generative. And then you've got the, and then you've got the predictive models and the analytical models and the classifiers and the other kinds of approaches, which a lot of it borrows from statistics and machine learning and deep neural network type work that's happened, of course, over the last several decades. And what makes it different when we deploy it depends a lot on how it's deployed. Right. Some of it is we use AI to develop algorithms and the like, but then we turn them into kind of dumb algorithms and then we deploy those. And that's really sitting behind an alert that doesn't look a whole lot different than a decisions, a rule based decision support alert or something like that that can often look quite similar. And then you sort of do the offline updating and then you translate that back. But a lot of this increasingly is becoming agentic.

[B]

Right.

[C]

And so one of the things that we're seeing now in you know, very real time is not only the generative tools and you know, the prompting of those and the outputs that they provide and the interactions with them through ambient listening and that sort of thing, but increasingly these concatenations of different AI tools together to create agents and to have agents working as teams and increasingly starting to offload work in a way that one of the things that makes it quite different to it is that it's really hard to kind of understand how they work entirely. They're a lot more so called black box and it's hard to get under the hood and, and, and, and even, you know, keep that so called human in the loop. Even though we think, you know, that's important, especially in very critical situations, it's hard to do as compared to sort of dumb it, if you will. So that's one of, that's how that would be sort of my answer to that first. Devil's Out.

[B]

Good question. Yeah, yeah, well it, yeah it's a, it's a really good answer and I know you could play both sides and

[C]

in fact, I mean we.

[B]

You're probably one of the first, first best people to ask this question because I know your algorithmic vigilance work has been seminal to the field and it certainly sensitize you to need to tease us apart. I think you know your answer. I think very correctly pointed out that you need fairly detailed analysis about what the specific technology is doing and what it is doing that requires various kinds of monitoring. And I think that when you encounter genuinely different properties, those are the ones that we need to watch carefully because they're not same old deployment challenges. I've sort of had this half baked idea floating in my head about why people get so confused about it. I think that modern AI confuses our old intuitions about technology. We sort of understand how to think about agents with intentions, right? They have psychological states, they have beliefs, desires, values. We can analyze them in those, in those frameworks and we know how to think about deterministic systems. Devices are reliable or unreliable, they have a reliability record, they have a design and you could start think, you could think about them that way when you think about safety and risk. But when you have things that mix those two together, when you have something that seems to have psychological states, we certainly do a good impression of having psychological states and at the same time has These reliability issues that sometimes it just breaks. Right. Like, you know, Claude MD is not. Is not responsive. Right. The. You've got. It actually requires a novel new set of intuitions and we haven't developed yet. So I think we just need to start keep working on novel metaphors that help people grasp this a slightly better level. Because what we're see, we're sort of in this Heisenberg uncertainty world where we flip from one to the other and it's awfully confusing most of the time. But. Yeah, yeah. So go ahead if you want to comment on that.

[C]

Yeah, I'll just quickly rip off of that to say, I think you're exactly right. I mean, I think the other reality of what we're facing now that is different than certainly everything I've seen in my career is that this is. I've. I've often joked this is the first time in my career where I find myself actually telling, you know, hospital leadership to slow down on technology adoption. Yeah, like, that has never happened before. I mean, like, come on, guys, we got better tech. Please, let's use it. And now it's like, hang on, I get that this is exciting, but for the love of God, we need to do a little testing here and make sure that we are. And that's frankly another reason why we need the capability of doing algorithm vigilance and monitoring. Because at a certain point, I'm kind of saying that intentionally, which is we do need to be deliberate, but we also need these solutions. I mean, at the end of the day, healthcare is flawed in many ways, and we all know that it's broken in many ways. You know, there's lack of access to care for a lot of people. And again, as we talked about at the top, you know, there's diagnostic odysseys. People are on. It takes a long time to get through the system. It's very costly. We don't have enough practitioners to go around. There's lots of reasons why AI should be helping us and we need to take full advantage of it. But if we're going to do it safely and effectively, then we need to not only move deliberately, but also quickly, as quickly as is safe. We also need to then monitor it, which will allow us to move more quickly. Right. So that's the other reason I've been very interested in really pushing on what we need to do around this because I think it, you know, a lot of people might think, oh, gosh, you know, that's just more either an academic exercise of like, they just want to do more Studies which I sometimes get given. Given who I am, but also, oh, gosh, you know, you're just going to slow things down. Right. Because we just need to get these solutions out there. Because we have a need. No, no, no, absolutely. But it allows us to move faster if we've actually know we have a safety net that we're working with here because then we can take more risks and we know that the risks are lower because we have a safety net. Right. That'll catch us if we mess up.

[B]

You know, I heard this metaphor from a security guy and I'm going to just steal it. To sort of ask his teaching question is why to have brakes on cars? And most people say, so we can slow down. He said, no, so we can go faster. If you didn't have a.

[C]

There you go.

[B]

If you didn't have a break, you would probably not want to be going more than, you know, 12 miles an hour with a break. You can speed up. But. So I, I think that's a really wise. Right. Safety net analogy is going to the same place.

[C]

No, I like, I like the brake metaphor.

[B]

Yeah, yeah, yeah. I like the little, you know, the little jiu jitsu that he.

[C]

That he pulls.

[B]

Pulls with that one. So very, you know, really interesting. I'm going to hand it off to Steve. I, you know, we have got a lot of questions for you, and we're going to try to cover them in a amount of time. Yeah.

[A]

So, Peter, I wanted to raise a question that we've recently posed to some other leaders in the field. And the response I got back, I found a little. I found it interesting. I won't mention who it was at the moment. Not the bias you. But where do you come out on the issue of patients starting to use these tools on their own without a safety net? You know, patients can do very great things. They know themselves better than anybody else. And we've had patients on the podcast like Hugh Campos and Dave debranckart, who have leveraged these tools in ways that are frankly remarkable. And we have other folks in the world who try to leverage these tools and come up with answers that are frankly, 180 degrees off the mark. And this informatics pundit, who we both know very well, came out, the question was, do we care about letting this out and, you know, patients using it without. Without a safety net? In effect, the answer that he gave was, well, what's the alternative? You know, there aren't enough docs out there. There's not enough primary care. There's not enough folks to cover it. You said it yourself too earlier. And you know, if a patient can sort of self triage themselves with a, with a large language model, maybe it'll get them to a specialist faster. I mean, you live that particular dream. I mean, absolutely would have helped you.

[C]

Oh, I think it probably would have, actually. You know, but I think this is a great question and I think about it a lot. The reality is, you know, before AI, we had patients. I mean, I can remember patients coming into my office with, you know, reams of paper they printed out from their Google search. Right. So this is sort of an evolution. Evolution of that for a long time patients have. Because of the challenges of. I think a lot of people would love to just have their doctor at their beck and call and be able to ask lots of questions. At least I think so, as a doctor. But the reality is we're not always available. It's actually very challenging. And yet people don't necessarily want to go. Whether it's for themselves or for their loved ones or for others. The fact that at their fingertips they can have a solution and a resource that could actually help answer questions is incredibly attractive, incredibly appealing, and for very good reasons. And I think the challenge and the thing that gives us all pause appropriately, since we know how these things work and frankly, how they don't work perfectly, is, oh my goodness, is harm gonna befall these people? Right? I mean, that's really the worry, right? The worry is not, oh, take away this tool that could help people. The worry is we don't want people to be lulled into a false sense of confidence and sort of be harmed by these things, which is also understandable. And it does go back to the idea of not doing harm. Having said that, people seek out information in all sorts of different ways, and I think it's perfectly legitimate and expected and appropriate for people to use these information resources the way that they think it's going to benefit them. I think that the task we have before us is to figure out how do we not only make these systems better so that they are more accurate and effective, but also that they can better illuminate their degrees of error, whether or not they're certain. They're pretty lousy at that right now. They're not really great at saying, if you ask it, are you right or not, you can't always trust that it's giving you the right answer at the moment. And so I think there's a lot of things we can do technically, and there's also a lot of things that I think need to happen educationally. But I don't think that the right answer is to say don't use it because that's first of all, it's not realistic. And it's also frankly counterproductive. I mean, I think we want to take advantage of these tools. But that gets back to this whole, you know, you guys probably know I worked with the National Academy of Medicine on this committee to, where we developed a code of conduct around AI and at the core of that are some principles. One of them is really about being human centered. Right. And making sure that everything we're doing is really thinking about, okay, how do we do this in a way that is going to benefit humans the most? And there's a whole lot of things that sort of cascade from that. But I, I do think that the answer is not okay, they shouldn't use it. I think the answer is how do we either build them and, or train people and, or do other things sort of socio, socio organizationally to make these as effective as they can be for people and then, and then hopefully learn how to do that better over time so that we can actually improve care. But we, you know, we're going to have people using these tools and I think it could end up being a good thing. We just have to do it properly.

[A]

Yeah. So I'm going to be able to click. Let me just follow. Go ahead and hand it back to you, Leon. I want to just double click on one thing you said about, about education because we just came off conference at Harvard the last three days and one of the topics that we covered there was with the former director at ahrq, Chris Demick, and that's about AI literacy. And we did a workshop yesterday around how do we actually increase literacy. And one of the things that came out was not a solution, it was a question. Should organizations like AMIA or the ACP or professional medical societies, should they take on the responsibility of helping bring patient level training to give people primers on how to use these tools better, more effectively so they stay safe? Is that something that we might want to think about?

[C]

Well, that's a great question. To be honest, I haven't thought a lot about whether those particular organizations should, should be the ones taking it on. I certainly think, you know, very close to amia and certainly there's a big group of people at AMIA who are very concerned with people, organizational issues and consumer health informatics and the like. And I think, you know, people in the informatics community have worked on this for a very long Time. There's also groups like the Society for Medical, Medical Decision Making, smdm, very, very good about shared decision making and the like. So I think there is a lot. I think that, you know, Chris is certainly very knowledgeable about this, having been at ahrq. There's a lot of knowledge we've gained before, sort of the AI revolution on how to actually work to better impart information to patients and allow them, you know, better ways to understand it. I think we should take advantage of that and not be reinventing the wheel where we can.

[A]

Fair enough.

[C]

I also would say that there's a lot of work to be done because this is different, as Leon said earlier, in certain ways. Right. And so, you know, there's. There's biomedical knowledge right through. Through, you know, our libraries and in other ways that also have a role to play here, too. So I think there's a lot of ways in which we need to be tackling this. But I think the short answer is all of the above, frankly. I think we all have a role to play in getting this to be done the right way.

[A]

Yeah.

[B]

So, Peter, you told us that one of the things you harped on most around the National Academy table wasn't the perils. It was that we need this make the case why healthcare can't afford to keep doing what it's doing. And what does doing it right look like in your world?

[C]

Yeah. So just to clarify that, I mean, when we got together as a group, I think all of us kind of understood that there is a need that, of course there are potential perils here. And it's very easy when we all get together to talk about this. Frankly, it's a lot easier and more natural for us to think about all the things that can go wrong. And so one of the things that I did around that table and others did was say, let's make sure we don't completely miss the opportunity to also say what could go right? What are the kinds of things that really could be helped by AI? Because after all, if there's not a positive, a significant positive gain to be had, then no amount of risk is worthwhile. Right. So what is that positivity? The positivity is a lot of things we've been talking about. But I'll just quickly enumerate some of them again. Like, you know, right now we know there's a lot of prop. First of all, the costs are unsustainable. Right. But the percentage GDP has been going up and up and up. I mean, throughout my career, I keep saying well, it can't go any higher, keeps going higher. And so we really can't afford that. Right. We know that, that a lot of that is because of things that can be improved, we think, through these sort of augmented capabilities, through AI. And so I think that's one reason, right. Just to. Because what happens when you make care more affordable? Well, then more people can have access to it. Right. And that's the other part. We don't have enough access to care. And so that's really important that we actually use these tools to improve the level of access people have, whether it's directly through, you know, agentic interactions or as a. A complement and adjunct to what you get, you know, through their regular engagements with the health system. And then, and then there's, as we were talking about before, ways in which, you know, people. People course through the healthcare system. I talked about my situation, but I mean, as a rheumatologist, it wasn't unusual for me to see patients in my clinic who, you know, might have been. Had lupus in retrospect for two or three years before they actually came into my office. And they, you know, were bouncing around with different, you know, physicians. And finally somebody said, oh, maybe this is lupus. We should send you to a rheumatologist. And. And they finally got a diagnosis. Well, we. We now are showing through studies we've been doing that we can shorten that from years to months or even weeks. And, you know, that not only helps those patients, but it also helps the system because now won't take six months to get an appointment with a rheumatologist because, in fact, there'll be fewer people showing up that don't need to be there when they actually should have gone to see infectious disease or oncology or somebody else. And now it'll open up slots for the people who really need to be there. So I think in numerous ways, we know, and I haven't even touched on all of them, there's certainly efficiency gains and other things, but I think in numerous ways we have to take advantage of these tools in order to improve care and efficiency and the health system as a whole. We just can't forget that there are potential perils as we do so.

[B]

So, Peter, it was such a terrific conversation. I wish we could keep going, but Steve, our official timekeeper, tells me, I've got to land this plane, man. So let me just try to summarize because we've learned so much from talking to you and what I feel I'm walking away with is that Peter's life, in a way makes an argument we have a near fatal diagnostic odyssey that a system designed to learn would have caught. Which is exactly why he's built a way to keep watching models that we deploy instead of just flying blind. And why the right question posed by Peter's experience is not AI or not AI, but what's actually different and what do we owe each other to do it right? I love your optimism, Peter, and something I share. But in your case, I think it's earned. It's not naive, it's, you can see the cost of not doing this measured in Z in Ms. Zebras and undetected drift. And you've put in the work learning, teaching us how to measure it. So I just want to thank you so much for joining our conversation and I'm excited to share it with our audience and I want to direct them to the amazing work that you've done. They could find at Vamos, which is being open source and the National Academy of Medicine code of contact with that, you know, thank you, you know, Steve, and thank you so much, Peter, for joining us.

[C]

Thank you both very much. This has been a pleasure.

[B]

It was a delight. And we will see all of you next week on another exciting episode of Practical AI in Healthcare.

[A]

Thank you for joining us this week on Practical AI in Healthcare. If you're ready to go beyond buzzwords and hype and explore how AI is truly transforming healthcare, stay tuned for more conversations that get us to what works. Until next time, stay pract.