Loading summary
Transcript64 lines
- [00:01]
A
You're listening to is business broken? A podcast from the mehrotra institute for business, markets and society at boston university questrom school of business. I'm kurt nickish. From Hollywood to Healthcare in our last episode, we explored how AI is recasting the creative industry in Tinseltown. Today we're turning to artificial intelligence at your doctor's office. AI is already helping physicians with things like taking down records and interpreting medical images. And considering that healthcare is the fastest growing industry in the United States, providers are rushing in. OpenAI rolled out ChatGPT Health. Microsoft announced Copilot Health, a tool designed to help people make sense of everything from wearable data to their medical records. And demand from consumers is growing. According to the research nonprofit KFF, about 1 in 6 U.S. adults say they're using AI chatbots to get medical information or advice. So this week we're going into the hospital to ask the question at the center of this series. Is AI a complement to human expertise, or could it eventually replace parts of it? And as the technology slides into the exam room, is it actually fixing what's broken in healthcare, or is it giving corporations and health systems new ways to scale and control how care gets delivered? Joining us today are physicians and researchers who are thinking deeply about this transformation. First, Eric Topol is a practicing cardiologist at Scripps Clinic and the author of Deep How Artificial Intelligence Can make healthcare human again. Dr. Topol, thanks for joining us.
- [01:50]
B
Yeah, Great to be with you.
- [01:52]
A
We're also joined by Sunny Jha, an anesthesiologist and pain physician. He co authored a recent article in Think Global titled protecting physicians from AI imposters. Dr. Jha, thank you.
- [02:05]
C
Thank you.
- [02:06]
A
Also with us is Michael Hanson, director and principal researcher in Health Futures at Microsoft. Thanks, Michael.
- [02:13]
D
Yeah, happy to be here.
- [02:15]
A
And finally, Kerry Morwech, professor of marketing at BU Questrom. Welcome back.
- [02:20]
E
Thanks for having me, Kurt.
- [02:22]
A
All right, Dr. Topol, in your book Deep Medicine, you wrote about how it has the artificial intelligence has the potential to transform so many things doctors do. Is AI outperforming human clinicians in some areas?
- [02:36]
B
I think the AI interpretation of medical images has far outseated human eyes, as best exemplified by the largest randomized trial in the field with mammography, over 100,000 women with remarkably increased breast cancer detection and also prevention and subsequent follow up. But there are many other examples like that. But the thing that is, I think, remarkable is that the AI interpretation of medical images can detect things that we humans could never be able to cue into this opportunistic ability. So that is really striking. And superhuman eyes, if you will, have really come to the forefront. And we're not taking advantage of it yet, but hopefully someday we will.
- [03:24]
A
You're a practicing cardiologist. Alongside your research, as someone seeing patients, where does AI actually show up in your day?
- [03:33]
B
Well, a number of ways. For one, now we have the conversations in the clinic made into the notes and all the downstream functions to reduce the amount of data clerk time and also give the patient good information and audio links, also to improve accuracy, whether it's the interpretation of scans or lab tests or other things. So we're seeing some effects that are really starting to take hold.
- [03:59]
A
Is there anything that AI is doing today that would genuinely surprise most patients, or do you think that much of that is still to come?
- [04:07]
B
Well, the big surprise to me was empathy. In that book, I had a chapter, deep empathy, and I said, oh, it will never come from AI. But as I didn't forecast properly, AI with large language models is a remarkable channeling the algorithms. The AI has no idea what empathy is, but it transmits it. And in 13 studies that have looked at this, evaluated by physicians, 12 of the 13 studies showed empathy was better with the AI than by the physician. So this is really interesting, and one of the questions it raises is, well, in the future, clinicians be getting coaching by AI to be more empathic and better communicators.
- [04:52]
A
I'm just curious, is this helping doctors who don't have good bedside manner in the first place, or the fact that AI just has not seen so many patients and actually has empathy for each situation because it just doesn't get tired?
- [05:06]
B
Yeah, there's several reasons for this, but probably the biggest one is that most physicians have a lot of empathy, but they're squeezed so much for limited time. They don't get to express it. And when the AI reviews their note and say, you know, why didn't you ask Mrs. Jones about this? Or why did you interrupt her after nine seconds? And on and on, then you start to cue into what could be better. But it really does come back to, we got to have the gift of time or we can't rebuild the patient doctor relationship, which has gone through so much erosion over the years. There's been seven studies now that have been published where the AI did better than the doctor with AI in various tasks. That wasn't anticipated, but it may reflect that we're still in the early stages, and a lot of physicians haven't gotten grounded in AI or have automation bias. And the Fact that these studies were not real world medicine. They were in a way contrived with scenarios and patient actors and that sort of thing. So we'll have to see. But everybody was banking on that the combination of human and artificial intelligence would be the best of all.
- [06:15]
A
Michael, at Microsoft, you study computational biomedical imaging modalities such as MRI or CT scanning. I'm just curious, how is your research advancing healthcare?
- [06:27]
D
I think some of the things that Eric touched on that AI is having a tremendous impact on the interpretation of images. Right? We see, yeah, the ability to detect things that humans can't see reason over lots and lots of images and reduce the workflow in that regard. But then the other thing that we're focused on in our work is how we can use AI to actually make better images in the first place. So take poor signal to noise conditions where the images are very noisy, clean them up and restore them and also produce images from instruments that we didn't think that we would be able to produce images with. So those are not. That's not AI as a language model, as a chatbot, but more as a signal processing tool and having a huge impact in that space.
- [07:15]
A
That's interesting. Is this for developing nations where they do not have the same level and resolution of equipment, or is this developed nations applications too? I'm just curious where this comes up.
- [07:28]
D
Yeah, it could be both really. So in our part of the world here, where getting an MRI is still quite uncomfortable, takes a long time, you have to lie still for a long time. But we can actually shorten those MRI scans by just acquiring less data. The images would then be noisy and not good enough. But AI can help us clean that up. So that's an application applied here. But we're also looking into much, much lower cost instruments that could be deployed in places where they have no access. So 70% of the world has no access to MRI, for instance.
- [08:06]
A
That's interesting how it can bring more healthcare to more places. How good are some of these AI applications right now? Or are you still sort of in the developing these technologies and worrying about product market fit later?
- [08:21]
D
Again, tacking on to what Eric said, there are some of these things that applications that have to do with interpreting images that are really outstanding right now. And we're seeing lots of good results with that. I think what we're still waiting to see a lot of is broad adoption of it in a clinical workflow. And so we're working close with partners around the world to see how we can get those things implemented. But there's a lot of progress in that space on this sort of image restoration and signal processing stuff that I talked about that is already being productized by all the major device manufacturers. Denoising of images. There are multiple commercial companies in that space, and that's really taken off right now.
- [09:06]
A
Michael, you use, obviously, a lot of data to train these AI systems. How do you ensure that they're based on good data sets, diverse data sets, fair data sets? Yeah.
- [09:17]
D
I gotta be honest and say that it's a very difficult technical problem that we have to ensure that the data sets that we train on properly represent the types of disease and the types of lesions and so on that we want to see. Unfortunately, it's not as simple as just adding more data because many of the data sets that have been accumulated over the years are also in some ways inherently biased. Right. Not everybody gets an mri. And so it could be tricky to guarantee or ensure that these data sets are completely fair and diverse. So what it actually requires is that once you train some models and use them, that you properly evaluate them as they're put into use, and that you assume that there are going to be biases that we need to uncover, monitor, and improve on as we go.
- [10:09]
A
Clearly, you think a lot about reliability in this. This data, this diagnosis. Do you think about patient trust, too, in what you're doing?
- [10:17]
D
I hear people talking about, like, a patient's going to trust AI or things that are being produced with AI. I think patients trust the care that they receive. If they have an interaction with their physician where they feel like they're receiving good care, and if we monitor that they're receiving good care, then I think they would appreciate that the physician that they are interacting with is present, that they, as Eric mentioned, have more time to interact with the patient, as opposed to doing data entry or trying to retrieve the information that they needed in order to diagnose the patient, or in the case where they're getting an mri, I think they would appreciate if the scan is shorter. But I don't think that patients necessarily trust a model. I think they trust the care that they receive.
- [11:07]
A
Carrie, I see you waving your hand there. Yeah.
- [11:09]
E
Just following up on what Michael has to say. So we've done some studies looking at patient receptivity to AI, and generally speaking, patients are willing to use AI. You could think about it as, like, AI resistance is not like a decision rule that people say, I'll never use this kind of technology. But it tends to be attacks. And the time it's attacks is when there's not a human supervising the AI or making a decision based on AI input. So patients are perfectly fine with a physician relying on AI input to make a decision. We don't see any differences between the receptivity to that kind of healthcare and to a fully human sort of workflow. Where we do see a reluctance to use medical AI is when the AI system makes a decision without human oversight.
- [12:03]
A
I want to stick with patients and physicians and trust here and get back into that room in the clinic with Dr. Jha. You know, we just heard from Carrie how patients may be open to it depending on it. But we also hear that nowadays many people turn to AI like ChatGPT before they even come into the office. They ask health related questions, they upload reports to help understand it, they get medical advice. Have you encountered a situation where your patients come to you for that, ask your opinion on their AI diagnosis?
- [12:37]
C
I think what you're seeing now is patients are using AI tools to learn more about their conditions. You are seeing folks that are trying to use AI to diagnose their conditions. And so that can kind of be a slippery slope. AI models are improving in that regard. They are able to diagnose better, able to provide more data, but at the same time they are biased. And that's what Michael mentioned that we need to worry about the bias that takes place with the data that's being provided.
- [13:04]
E
I could share a personal story about that if that'd be interesting. I had some blood tests done and I hadn't heard back from my physician yet and threw in like just what the inside outside ranges were. And ChatGPT gave me a very alarming diagnosis and messaged my doctor and she said that that was ridiculous. And the ranges were like just off by like a small.
- [13:31]
A
Oh my gosh.
- [13:32]
E
Yeah, I should calm down and like come and see her in person.
- [13:36]
C
Yeah, you could have the red exclamation point next to a lab result which, you know, if you're looking at a complete blood count and you know, normal hemoglobin's 10 and your hemoglobin is 9.9, is it clinically significant for it to have a large explanation point? Is it, is ChatGPT or is the AI model going to say you might be having a GI bleed or you might have cancer or you might have leukemia and certainly those are on the differential diagnosis. But as an astute clinician, you look at that data point and you look at it in the grand scheme of things is Carrie, is that something that's going to be high on the differential list than somebody who's in their 20s. No. So the context of the data is important and I think AI kind of struggles with that if it doesn't look at things in the context of a larger situation.
- [14:26]
A
I get that. You know, it doesn't know your history, it doesn't know what medications you're on, it doesn't know what your doctor knows.
- [14:33]
E
It's interesting to think about what the alternative is. Right. So we're all assuming that the alternative is that patients are going to talk to their doctor about these issues. If the comparison standard is a physician like the AI is definitely going to have a challenging time keeping up right now. But if the comparison standard is like a Reddit forum, you may find that people's general care is improved in those kinds of contexts.
- [14:57]
B
Just to add on to Sunny, if I could, a couple of recent examples. So one, ChatGPT Health and Clod Health came along to try to be more specific to help people with health related issues. Interestingly, when people ask them about their health status, they put undue emphasis on on their smartwatch or wearable VO2 Max, which is notoriously unreliable and told them that their health was horrible. Well, of course it was trained on the wrong data and so that's embedded unfortunately by some influencers in our world. And the other one that just got published at Mount Sinai, they went to ChatGPT Health, this is GPT5, the latest model and they put in hundreds of scenarios of should the person go to the emergency room or should they stay home. And over 50% of the ones that should have gone to the emergency room, they were told to stay home. Like diabetic ketoacidosis and serious stuff. These are indicators of what can go. The hallucinations is one thing, but the wrong answers that are just because of training issues or because these models are just not equipped and we should be cognizant of that.
- [16:17]
A
Yeah, the stakes are high here, Sunny. How does that feel as a physician? I mean we always have the ability to ask for a second opinion. Right.
- [16:25]
C
For some patients that use these tools, it can be utilized as a second opinion type tool. Certainly that can cause, you know, with colleagues causing friction in the doctor patient relationship and that it may appear as if the patient knows more about the condition or is kind of usurp the physician authority on treating them. So there is a slippery slope there that can lead to the deterioration of the patient physician relationship.
- [16:51]
A
Yeah, you gotta have that conversation. It's sort of a, it's a navigation, I guess. Are there Other risks that come along with people seeking medical advice from AI chatbots, digital doctors. Like, what do you fear kind of beyond just having to, you know, talk this through with everybody when you maybe don't have enough time.
- [17:10]
C
Right. So historically, a physician's authority came from physically being present in the room. And now we can essentially create digital doctors, doubles of physicians based on recordings such as this, social media clips, even headshots. So that becomes an issue where if you're using AI as a resource and you see a video of me or Dr. Topol, and it could be a deep fake, you don't know who it came from, who programmed the algorithm, who owns the algorithm, but it could be an astute group of individuals who can simply repurpose these tools for nefarious purposes. So I think we've kind of run into issues, and that's what I mentioned in my article, is that this whole idea of who owns your identity and when it comes down to AI, and that's why it's important to recognize that we do have identities. Dr. Topol's highly reputable physician. And if somebody were to create a deepfake saying that the COVID vaccine causes severe heart issues that can spread dramatically for somebody with his social media outrage and his reputation.
- [18:16]
A
I'm curious, Dr. Jha, does AI change what it means to be a doctor now and maybe some of the status that goes along with that?
- [18:25]
C
Certainly, absolutely. We're facing trust and transparency issues with regard to who can call themselves a physician, who can call self a doctor. So essentially the question you're asking about is, can we trust AI? And it's a difficult question to answer because we don't know where the response is coming from. We don't understand the algorithms too well. We don't know if the data is a hallucination, whether the data is biased that it's using to make that determination. So it becomes very, very dangerous in that you can quickly find yourself going down some very slippery, slippery slopes if you don't understand the response, how to parse that data, that response to the data that you actually know about. So in some ways, if you are a layperson, as somebody who doesn't have a medical background, trying to interpret lab results in the greater context of your health conditions, it can be incredibly, incredibly complicated. One thing we're not really talking about as well is the liability. Who is responsible for this big data, bad data? If individuals were to take the information and interpret it wrongly and use it wrongly, who's responsible for it? And you see that with Influencer type individuals, you see that, who are spouting medical advice that have essentially no liability. Like if I was a physician, Dr. Hoeple was a physician, we say stuff, we're held accountable to it, but these folks can spout advice on natural therapies for cancer remedies and all that and get away with it. Whereas for us it'd be malpractice and you'd be suspending your medical license and having years of legal litigation. There's a lot of really complicated questions here. And I tell young medical students, college kids who are worried about AI in healthcare, and I told them it's much easier to do to sue sunnyja MD than it is to go go after Microsoft Health or Google Health or something like that.
- [20:17]
A
That's interesting. Dr. Topol, does it change what it means to be a doctor? AI Now I'm not sure if it
- [20:25]
B
does yet, but I think it will. You know, it'll be embedded, but there's lots of kinks to work out before we get there. And medicine is a slow moving domain and it's unlike many of the other sectors. This one's going to be a very hard slug to get it right and to adopt it.
- [20:44]
C
I think one thing to recognize is tell individuals we're in the business of priceless assets. That's what we work with human life. And when you get to 99% reliability, 99%, that sounds pretty good. But in healthcare, 99% is not good enough. And I think for healthcare AI tools to get to get the last 1%, it's gonna be exceptionally, exceptionally difficult. It'll be a challenge for everybody, not just patients, but also the tech community and physicians.
- [21:13]
E
Yeah.
- [21:14]
B
And I think to add to Sunny's important perspective there, we're not at 99% today with humans. Okay, we have some 20 million diagnostic errors a year at 800,000 people estimated to have serious disability or death. 800,000Americans per year. And those are fairly conservative estimates from Johns Hopkins. So you have to say we're expecting higher level from AI than we already have in medicine.
- [21:46]
A
Michael, I saw you nodding too when Sonny was talking about getting that last 1%. It is interesting that sort of the level that you're shooting for here.
- [21:57]
D
Healthcare is an interesting space. We should be careful. I think having a higher bar than humans is what we should have and we shouldn't overstate what AI can do right now. There's lots of sort of very brave statements about what role it will play and we shouldn't overstate those. But there are Just things that it's tremendously good at. It's incredibly good at aggregating lots and lots of information, surfacing it, making sense of it, and assisting physicians. And I think if we just maintain that there has to be a physician in the loop, as Sonny says, taking responsibility for this stuff and applying their experience and kind of catching those cases where something gets a red flag when really it's nothing. And also the other way around, when something that should have a major red flag didn't get one. It needs to be a human in the loop for doing that. And if we do that, then hopefully the humans that are involved in it, they will perform better. Even though, Eric, I know you quoted that that's what we would expect, that the humans and the AI perform better together. And we'll have to see how the, how the numbers come out.
- [23:09]
A
Yeah, well, there is that oath, right, to do no harm. And you have a. You potentially have a tool where you can make big strides to help people. So it's also the. A really interesting opportunity. Dr. Topol, you know, what do you envision the future of the Dr. AI relationship to be and what excites you about it?
- [23:32]
B
When I first got out of med school in 1980ish, it was really a great relationship. There was a lot of trust and presence, and there was time to spend with patients before it became such a huge business.
- [23:45]
A
Yeah. And how much time?
- [23:47]
B
You wouldn't have any appointments for nine minutes or seven minutes or a new patient for 12 minutes. I mean, that's ridiculous. 30 minutes was the minimum time. Usually you'd have an hour. You know, the squeeze wasn't on. Right. And also, of course, the distraction with keyboards and screens wasn't even conceived. So all of that led to profound erosion. I just hope eventually we'll get it back. But there's one big problem here, and that's the fact that it is a big business and largely run by administrators, and they don't really understand the human bond and how important that is. They just understand one thing, which is revenue. So I'm worried about that. It's going to take the medical professionals to stand up to these bean counters and say, I need this gift of time that AI can provide. And if we don't do that, stand up for patients, they'll override this and they'll use AI to make things worse. That could happen. That's the default mode if we don't get our act together. This is one time we need to be activists or AI can be really used against us and patient. And this is a time where that we can really work hard deliberately to restore trust and a relationship that's precious.
- [25:08]
A
Dr. Jhai, I see you nodding.
- [25:10]
C
Patient access to physician led care is diminishing. It's becoming harder and harder every day as the healthcare system continues to vertically integrate and consolidate. Like Dr. Topol mentioned, I think that's an issue and it will continue to worsen. I don't see it getting better anytime soon. And therefore I think AI can be used as a tool essentially as a co pilot for physicians to extend their ability to handle these administrative tasks that are burdensome to physicians, such as writing notes or putting in orders. I think as long as we stay true to that and maintain the transparency as well, let patients know when AI generated content is being used, when AI tools are being made, in what capacity, I think that'll help and help strengthen that physician patient bond that we sort of desperately enjoy and appreciate and allow improvement in the access issues.
- [26:07]
A
Carrie, when you hear all this, what do you envision for the future of the Dr. AI relationship?
- [26:12]
E
I think that we're going to be seeing rapid adoption of these kinds of systems in the healthcare industry because of the demand is so high. But as we get closer and closer to these systems totally replacing humans, that's when we're going to hear similar kinds of issues arising that we do in these other industries. So radiology is an example where like there's been incredible advances in like our ability to detect disease and like to process things quickly. It'll be interesting to see what happens if we get to a place where hospitals cut down on the number of radiologists that they need to hire. And that may be sort of when we start to see that kind of pushback from professionals in the industry. Whether or not healthcare systems have a similar kind of pushback. I would be less surprised for them to just be openly adopting these systems if they perceive there to be sort of cost advantage. And the insurance question about like who gets sued, you know, as these systems become come online, there may be sort of serious issues too around whether or not health systems were willing to sign off on like having a manufacturer be responsible or the health system being responsible, responsible for any kind of errors, whereas previously the physician had assumed responsibility and had pretty high malpractice rates. So it'd be interesting to see sort of how physicians view that from like an economic perspective. If their insurance rates go down precipitously because they're using these systems, because liability is shifted onto the manufacturer of the AI, that might increase acceptable, like the acceptance in the offices. As we get closer and closer to total replacement. I think that's going to be a place where we start to hear a lot more pushback.
- [27:57]
A
I want to thank all of you for this conversation. It's been really, really informative and interesting and it's such a rich stomping ground.
- [28:06]
D
Thank you.
- [28:06]
E
Thanks, Kurt.
- [28:07]
C
Thank you.
- [28:08]
D
Thank you.
- [28:15]
A
That was my conversation with Eric Topol, Sonny Jaw, Michael Hanson and Kerry Morweg. And the conversation is in many ways just beginning because over the last three episodes, we've looked at AI through the eyes of the people on the front lines. We've traveled from the driver's seat of a robo taxi to the writers rooms of Hollywood and finally into the quiet of the hospital exam room. In each world, we heard the same fear that the technology is powerful and could improve things, but maybe, well, change things in unintended ways. The issue is not simply whether AI can improve outcomes, but how firms under competitive pressure will use it. In some industries, the main effect will be to cut costs and jobs. In others, AI will raise quality, expand capabilities, and help create entirely new goods and services. Since competition works differently across markets, its effects will vary. On our roads, it was about safety and who's doing the work behind the wheel or behind the screen. It was never a question about whether self driving technology would ever be good enough. It was more about comfort level, do we want that? And are we willing to share the road? In Hollywood, we saw an identity threat, the fear that if a machine can mimic a soul, the arc artist disappears. And alongside the anxiety that AI will automate creative jobs, there's another growing apprehension that the technology and tech platforms will elevate other media, eroding the stature of film and television as a cultural art form. And here in healthcare, we heard this hope from physicians that as AI technology moves us into the future, it can turn back time and give doctors more time to talk to and listen to their patients. But as we wrap up this series, we also have to look at the economic context. New technologies like AI aren't just technical challenges. As we heard, they're business model challenges. Innovation is supposed to drive costs down and create new things. But in the US healthcare system, we know that doesn't always happen. We often reward innovations that can be easily monetized over those that actually create the most value for a patient's life. In fact, better and cheaper is often viewed with suspicion, as if lowering costs must inherently mean lower quality care. And since the person making the decision isn't the one paying the bill, efficiency becomes secondary. So the key question for AI in medicine is not just whether the technology can diagnose a tumor and cut costs, but whether the market will channel it toward better care rather than squeezing out every last cent. So will AI give doctors the time machine they're looking for? Will AI actually disrupt the incumbents who profit from the current complexity? Or will the technology simply be absorbed into the existing machine used to maximize billing and scale up marginal value treatments rather than making more people healthier more affordably? From the writers rooms in Hollywood to the driver's seat to the hospital wings, the takeaway is the same. The technology is coming fast. The question, like it or not, is how our businesses and society will put it to work. Will we automate the status quo? Will we improve quality and affordability for people? Or will we upend markets and make new winners and losers? That's a choice that we and our system are in the process of making. Thanks for listening to this series on the AI Transition. Stay tuned for more episodes. We'd sure appreciate it if you would rate and follow Is Business Broken Wherever you get your podcasts. It helps more people find these conversations. Thanks for listening. Listening to Is Business Broken? I'm Kurt Nickish.