← Practical AI in Healthcare
Loading summary
Transcript56 lines
- [00:04]
A
Welcome to Practical AI in Healthcare, the podcast that cuts through the noise to spotlight real world solutions delivering real world value. From patient care to clinical research, from life sciences to patient engagement, we focus on what's truly moving the needle in healthcare. No hype, no theory, just practical insights where AI is making a true impact. Welcome aboard and let's get to it. As many of our listeners know, Leon and I work very closely with the DCI Network Division of Clinical Informatics at Beth Israel Deakins Medical center in Boston. This June, the network is hosting Patient powered Digital Health 2026. The conference will bring together patients, innovators, industry leaders, healthcare providers and policymakers to shape the next generation of real world patient centered solutions. The meeting will run from June 22nd to the 24th in Boston at Harvard Medical School. We've arranged for our listeners to get a discount on registration to the meeting. If you register between now and May 15th and use promo code PracticalAI June no spaces, you'll receive 30% off your registration fee. You can learn more at dcinetwork.org patients2026. In addition, we're always looking for sponsors. If you or your company are interested in becoming a sponsor, please reach out to admincinetwork.org see you in Boston. Hello and welcome to this week's edition of Practical AI in Healthcare. My name is Dr. Stephen Labkoff and I'm here with my co host, Dr. Leon Rosenblit. How's it going, Leon?
- [01:41]
B
Great, Steve. Really excited to talk to our amazing guest today.
- [01:45]
A
So, speaking of our guest today, this particular guest is somebody I have known quite literally since the very beginning of my informatics career. I don't know if he remembers it or not, but when I was a medical student at Temple University, I took a medical student elective at the National Institutes of Health at Lister Hill center. And that was a program that was run by a guy named Larry Kingsland and a woman named May Che. And while I was there, they brought in all the luminaries in the entire country and at the time, that wasn't all that many to give lectures to us in informatics over the course of our 12 week experience. And that was the beginning of my getting to know Ted and getting to know informatics. And it really sparked the beginning of my career in this space. We'll talk about Ted's background. Ted's got an incredibly illustrious background, and that's the whole point of today's podcast. Ted started out in the 1970s in Informatics at Stanford, and he set out to figure out what to do with this whole new thing with computers and medicine. And we'll get into that in a minute. But Ted is currently the he is the emeritus chairman or the chair emeritus at Columbia University's Department of Biomedical Informatics. He's an adjunct professor there as well. He's also held positions at Stanford University and many other universities along the way in his very long career. And he is also basically the beginning of the field. He's one of the people who got this field off the ground. And we are just pleased all get up, Ted, to have you at the table today. Welcome.
- [03:28]
C
Well, thank you, Steve and Leon. This is fun and I'm happy to have a chance to talk with you about this.
- [03:36]
A
So, Ted, you've been in the field for more than five decades at this point, and you have seen things from the very beginning. And the point of today's discussion is going to be let's reflect on those beginnings because I think it's incredibly important to understand the history of things so that you don't repeat them. And we've seen problems that have started cropping up in places like electronic medical records and data and things, and the problems that you saw very early in your career, and they persist. And I think, you know, at times when you start seeing a program that's rehashing things that have looked at, you know, decades ago and you scratch your head and you think, geez, maybe if people looked back and understood some of those things, we could advance a little faster. So, Ted, why don't you give us your, your background and tell us how you got your superhero cape.
- [04:23]
C
Sure. Well, I was, you know, I was born in Canada. My father was a physician. He got educated to do hospital management. And we ended up getting moved to the United States when he got a job offer down here. So from the age of six, I lived in the United States and I'm pretty much an American now. I grew up in New England but went to college at Harvard, pre med. Planned to be a physician for sure. Never saw a computer until I got to college. They didn't have them at home, they didn't have them in high school, but they did have them at Harvard when I got there and I had to choose a major. Pre med was not an option in those days. I don't think it is today either. And as a result, I got into the more well, initially physics. And then as I began to discover computers, which I'd never had my hands on before, I transferred into the only department that lets you do a lot of computing in Those days there was no computer science department. It was called applied mathematics. Computers started out, you know, the big old IBM machine in a, in a machine room. When I got there in the mid-60s, 66 or so, and you had to submit decks of cards when you wrote a program and that kind of thing, and then you get a printout a few hours later. And yet, soon as I started to take more computer courses, I got introduced to that hands on feedback that goes with actually writing a program and running it and seeing the results right in front of me. And I found myself just fascinated by computer science and the computing courses I was taking. And I had this huge problem which was, gee, I thought I wanted to go to medical school, but I really like this computing stuff. And I remember meeting with an advisor, a guy named Bill Bosser who was in my department and oversaw a lot of the courses that I took. And I said, I'm trying to decide maybe I don't want to go to med school after all. I really think that computer science is fascinating. On the other hand, I really have always wanted to be a physician. And he said, well, you know, it's not like you have to pick one or the other. Maybe why don't you go over to Mass General Hospital, There's a lot of computer science there. Maybe you can get some hands on experience and see what they do with computers in medicine and, and you know, who knows, you may decide you want to do both. So that was a really key conversation. I don't, I never knew there was even the opportunity to do that. Went to Mass General. As you probably have already guessed, the lab was run by October Barnett, one of the true historic figures in our field. A physician, no formal computing background, but got into computers very quickly at, at Mass General. And he gave me a chance to prove I could program and I did that and gave me a one week assignment. It was to learn the language we called mumps. And then a week later I had to come back and on the spot write a program that would do something for him. And it was a pretty good test, but I managed to pass that. That was all done on a teletype. Okay. Teletype. Wow. Yeah, all. There were no, no CRTs yet, even cathode ray tube displays that we began to use as our connection to computers. And he assigned me to work with another fellow who was there who I think you know very well, who was working on his PhD. He was a physician doing a PhD in computer science. And that was Bob Greenis. Yeah. So I became Bob Greenis research assistant worked with him for two years as he was finishing up his PhD. So my junior and senior years in college and by the time I was applying to med school, I was only interested in going to med schools that would let me do both, which was a pretty strange interest to have in 1969 when I was applying to medical schools. But I, I found a few that, that were interested. I was. And then even like, like Washington University in St. Louis had a wonderful computing lab in the medical school that was comparable to what Octa was doing at Mass General. And I almost went to, to Wash U, but I got into Stanford and Stanford wouldn't promise me anything. They said if you want to do computing, you got to come here as a med student. And we'll see. Depending a bit on how you're doing as a med student probably. So I ended up going to Stanford, which was a great med school and it had great computing stuff going on there at the time, even though they'd still had a one year old computer science department, brand new. And so the kind of the rest is history. I learned about MSDP programs. I wanted to do an MD, PhD if I could. I went and talked to computer science about as a med student about wanting to do a joint degree if I could, at least a master's. They thought it was a weird thing for a med student to want to do. They'd never done anything like that before and they kind of pooh poohed the idea. But I found a project that interested me in the medical school that had to do with drug interactions, screening in the hospital and trying to find out people were taking drugs that interacted by having a database and a warning interface that helped doctors understand that it was a good little project that I worked on. And in that way I got to know the head of clinical pharmacology, Stanley Cohen, who's a geneticist as well as a clinical pharmacologist and a physician. And Stan found out about my interest in doing a joint degree. And I got introduced to amazing people that I would never probably have found on my own. One of them was Joshua Lederberg, who was the head of genetics at Stanford at the time and a hacker. He wrote programs a lot as a chair of a big department. Ed Feigenbaum, an AI guru at the time who had just gotten there in the last few years from Berkeley and Carnegie Mellon before that, and most important perhaps was Bruce Buchanan who was a new research scientist working with Ed Feigenbaum, who had a real interest in clinical medicine and to Make a long story short, I'm probably going on too long already. Let me just say that it turned out that there was a way at Stanford to do a PhD in a interdisciplinary field. If you could get four faculty to sit on your PhD committee and testify to the university that you couldn't do what you wanted to do in any single department and that they would oversee both your course selections and your doctoral work, the actual research work, and make sure that you did a Stanford quality PhD. You know, that's one of the things they were wanting to be sure about. And if you got approved for that, you could do a degree in, you could name it, you could make up the name of your PhD, you could
- [12:39]
B
name it Bob or anything you wanted, like John.
- [12:43]
C
Well, I knew that it would be hanging over my head the rest of my life, so I think he was telling something a little more appropriate.
- [12:51]
B
But good boy, good choice, man. I would have coveted that. Definitely made the right choice at that point.
- [12:58]
C
Well, there was no informatics yet there was. The word wasn't used.
- [13:03]
B
So what did you, what did you call it?
- [13:04]
C
What was the. There was a guy, Scott Bloys up at UCSF who had a division in the medical school at UC San Francisco called Medical Information Sciences. And I, I like that because it emphasized the science of what I'd be doing rather than computers per se. It's not about the, the machine, it's about what you need to learn and do intellectually that contributes to a body of knowledge. And I, so by that time I'd already figured out that's what I wanted my PhD to be.
- [13:38]
B
You know, what a great choice. And I think you're, you know the stories. This is for our audience. Those of you who didn't recognize all the names, you know, the folks Ted is describing, all luminaries in the field of informatics, and he's sort of giving us deep roots history of the field's emergence from fairly early on. One thing that struck me when you and I've talked in the past is you, when you described getting to Stanford, you realized that AI researchers, which meant something diff. A little bit different at the time, but they were, they existed, they knew about psychology and they knew about philosophy and they knew about cognitive science. These are disciplines you hadn't connected to computing before. How did that encounter change your perspective? What lines of inquiry and thinking did that open for you about AI specifically?
- [14:25]
C
Well, first I, I really, I, I didn't mention it. But the reason that, and Find a Bomb and, and, and Joshua Lederberg worked, worked together was that they, they were working on a big AI project that was funded. It was called the Dendrol project. It was a biomedical science AI project. And what they were doing was encoding the knowledge of organic chemists. In fact, their third major collaborator was a guy named Carl Jurassi, who's also a novelist, but at the time he was really known as a major professor in the chemistry department who had come to Stanford and had spun off a company that did organic chemistry and specifically birth control pills that he had pioneered. But he was a wonderful organic chemist who knew about mass spectroscopy and how to try to determine the structure of organic compounds from their mass spectrum. And he knew it. Up here, it was really hard for people to learn how to do it. Took lots of examples. So they were trying to encode the knowledge of organic chemists to write a program, Dendro, that could be used to interpret mass spectra that they got from the machines that would do mass factor for unknown compounds and determine the organic structure of those compounds. So there was already a body of activity trying to capture scientific knowledge in a symbolic form. They were writing rules that the chemists seemed to use. And so I became grafted onto that kind of activity. But of course, my interest was much more on the clinical side. As a budding medical student, I was doing an MD, PhD at this time.
- [16:25]
B
So let's, let's focus on that a little bit. Ted, just I want us to come back to AI in clinical practice. Although, I mean this, the using it for scientific discovery is really fascinating. In your early work, you surveyed physicians about what they'd need from computer based decisions, right? If they were to trust them, what do they tell you?
- [16:47]
C
So I did do that very formally at one point. It was pretty intuitive. Early on, I was working with physicians who told me what they would want want later. We did formal studies in the early 1980s to actually look at what physicians would demand of a program before they would be willing to actually use it to get advice about anything clinical. And the paper was published in 1981 in a journal called Computers and Biomedical Research. But the gist of what we learned from that was they confirmed our intuitions, which was we don't want to use something that we don't know how it works, and if it can't tell us the basis for the recommendations it's giving us in a clinical area, we're not going to be interested in using it. We want to know why. That's the way you build trust in a program is by providing that kind of Transparency about the inner workings. And that of course became part of the design criteria that we were using in the systems that we built using AI and symbolic representation of knowledge to try to make sure that not only did it make good decisions and we could knew how to validate that, but we also wanted to verify that it provided the kind of transparency that would make it acceptable to clinicians.
- [18:17]
B
So Ted, in one thing I've heard you describe in an interesting way was that throughout the 70s and 80s, the tools got better at answering through all the expert systems work that you guys have done, but worse at explaining. Now, LLMs give us fluent explanations that sound very plausible. They'll tell you exactly where they think it a patient is lupus, for example. But you've raised a concern about how the way they operate today. Can you reflect on that concern?
- [18:47]
C
Well, the explanations that you get from an LLM are produced post hoc. But I want to point out you talked about the cognitive science interests that we developed at the time. Frankly, an expert consultant who you, who you ask to give you advice, he'll give you an explanation, but you, you're not really sure that's how he actually did it. There's all that wealth of experience and, and there's almost like an automated way in which people who are really familiar with a field come up with a, with a decision and then when you say, well, why then? Then they go through the process of figuring that out and. But the cognitive scientists showed that the process that you get described in a setting like that is quite different than the process that you observe. If you make people do speak aloud,
- [19:49]
B
process a cognitive walkthrough protocol, they speak
- [19:53]
C
a lot about what they're doing while they do it. There's a lot of good literature in the cognitive science world that shows that the real way in which people do it is a little different from what they get you get back from them post hoc. So maybe we shouldn't be too worried if what we get from LLMs is not exactly the way the LLMs solve the problem. On the other hand, we don't have a good way to really assess the kind of explanations that we get back in part because you can ask as you no doubt done, you can ask an LLM a question twice, five minutes apart and you'll get us a different kind of answer back. Kind of depends a little bit about what the question is you first asked and in the order of questions and the conversational aspects they're trying to model and the way it works. And of course, at the LLMs at least the early ones, were largely data processing programs using machine learning and deep learning methods. And I've been interested to see the introduction of new methodologies in LLMs recently. The retrieval augmented generation overlays where the LLM actually uses semantically encoded knowledge of a domain in addition with all the document kind of data sets that they're using to generate their responses. So as I look at that, I'm thinking a lot about is there finally a way in which some of that more semantically structured stuff that we were doing, encoding knowledge and rules, not just doing data analysis, but rather trying to develop lines of reasoning with the programs? Is there a way that we are going to be able to introduce more of that kind of overlay into the LLMs of the future?
- [22:05]
B
Yeah, Ted, really interesting points. I mean, I love that you're bringing in my, one of my passions, right? Cognitive science and the lack of privileged introspective access for humans is really an important touch point. I think the first paper, most convincing paper on that goes Back to like 1972, Nisbet and Wilson, right, who showed that people, they cannot tell you reliably what's going on in their heads, right? But they can make something up pretty easily. So probably the standard is not the ability to produce an accurate explanation, because that's a higher standard we hold for humans, but the ability to produce testable and transparent explanations, which I think is a really, is a very different thing. And your work is major sensitive to that. I want to bring us back to kind of industry applications of AI. One really interesting feature of your career is the tools that you developed for medicine. You actually were able to bring them and use them in industry for diagnosing. I think it was car engine problems. There was some really cool applications. What was the transition from medicine to other domains like, and why is one, are some of them more easier than others?
- [23:17]
C
Well, we did, we haven't talked about what I did. For the PhD. There's this micen program. Maybe people who are watching will have heard of it, but it was an attempt to develop a consultant that you could use. If you had a patient with a serious infection and you thought it was bacterial or probably bacterial, and you needed advice on how to treat the patient based upon their clinical findings. Typically before there are any culture results, you know, once you knew the bug with sensitivities, it would be, it would be obvious. But you didn't have that luxury. Often you had to wait for the lab to report things back. Sometimes it'd be 48 hours so we, we had to write rules that we found that our experts were using when we worked with them. They were collaborators on the project of how they would think through a case in order to decide whether it was likely to be strep or, you know, which, which antibiotics would be the right ones to use. Sometimes two or three, until, you know for sure. Now I, I'm telling you that kind of detail because there's clearly a lot of uncertainty in here. Right. It's probabilistic stuff going on. Well, it's probably this fog, it might be this, let's say, let's look at his skin. They're those steps along the way that led them to feel comfortable with a particular approach. That probabilistic aspect had to be captured in the rules. And so we actually had to develop a kind of mathematical weighting scheme. We called them certainty factors. They still are being used today in applications outside of medicine. Interesting. But it was based upon the philosophical literature about how people actually weigh certainty intuitively rather than in the classic probabilistic or Bayesian ways, that kind of stuff. Now remember, we had rules that had these weights in them and then we had a processing engine that knew how to process rules. That's what could be generalized outside of medicine, is that engine.
- [25:35]
A
Yeah.
- [25:35]
B
And I think that, you know, your, your intuition aligns well with, with where I see the advances in industry are going. I think neurosymbolic processing that combines sort of symbolic processing, use of explicit rules with neural network based approaches is solving a lot of very interesting problems. You know, but one, you know, observation that I think all of us share is that it may be easier to take decision making systems, expert systems from medicine to other domains and the other way around. And we keep seeing companies who are very, very smart at doing certain kinds of things, like IBM with Watson is the classic example. Coming into medicine and just falling flat on our face. These are not stupid people and these are not stupid companies. Why do you think smart companies keep making that same mistake over.
- [26:26]
C
Well, I, I'm not sure everyone quite appreciates except for physicians. They tend to have physician advisors, frankly, and they think that they're getting the gist of what goes on in medicine. But if you haven't lived it, if you haven't worked out on the wards, I don't think you have quite that intuitive understanding of what makes medicine so challenging. And you know, when we started, we started, a whole bunch of us started a company in around 1980 to try to commercialize some of these ideas. And we had we had systems that they could write rules and those rules would then be used. You could, as I said, you could diagnose an automotive engine to figure out why it wasn't working. But we knew exactly how an automotive engine worked. We built it. Okay? And because of that, it worked. You didn't need that probabilistic stuff as much anymore because you knew precisely what was going on and what a test meant and so forth. So I think that the nature of medicine, It's so filled with complexity and ever evolving knowledge, keeping up to date and learning new drugs and understanding where they, and all that stuff has to be somehow encapsulated and understood. I've seen LLMs give out of date advice, for example, because they don't own that something that's, that's happening now. And you expect your, you expect the physician you're seeing to, to do their homework and be up to date on things like that. So I, I, I mean, I, that's what makes medicine a wonderful area to work in. With this kind of interest in, in computational tools, is it, it's tractable. We're doing so much better than we used to. Okay, yeah.
- [28:38]
A
So let me pick it up a bit. And you know, a few weeks ago we had Bob Wachter on the podcast and I know you know Bob.
- [28:47]
C
Yes, I do.
- [28:49]
A
And he made a comment and I was really interested when he made it to sort of get your take on the comment, which was basically at the beginning of a field of informatics. He argued that the folks who were at the beginning went after the very hardest problem, which was diagnosis. Figuring out the crux of why somebody has the symptoms and the signs and everything that they have. And I know a little bit about that. I worked on QMR back in the day, which was building rules. I was one of those medical student peons who were sitting there. Actually, I was a resident. I was a student at that point, but I was still writing all those weighted research protocols to sort of outline various diseases for that program. But what do you think about that? Was he, is he right? That that was the hardest thing? And you know, what made everybody go to that particular thing back in the day?
- [29:44]
C
Well, not everybody did go to that.
- [29:46]
A
Okay, fair enough.
- [29:47]
C
I think I, I mean, I would say that the biggest computing effort going on in the 70s and 80s and 90s was building electronic health records. That was turning into an industry. And of course it didn't really take off as an industry until the turn of the century probably, but, but we had a lot of homebrew systems. You know, we did in lots of different hospitals built by the people in those departments. So medical computing became a thing. It was not the oddball idea by the, by the end of, of the 80s and 90s as it was when I was trying to figure out how to get educated in this area in medical school. So. And I worked on medical records. That's what I did at Mass General. That was one of your introductions to the field because it was so obviously important to get data more shareable and into a more accessible form. And we all have seen pictures of the big medical record rooms filled with charts and you could find them. The one you needed had been checked out by the orthopedists and they had it in the.
- [31:02]
A
Oh, I remember those days, all two of them.
- [31:04]
C
It was ridiculous. We had to have a better way of handling data. So I all I say that just to say that it wasn't just decision making that interested people who were working on medical computing, but there was a certain subset that was very interested. We knew that this was an issue. The, the sharing of expertise. No one can be an expert in everything. I became so aware of that as I did my rotations as a resident. You know, you finally felt you in the previous month you sort of began to understand a field and then the next month you'd go and there'd be some totally new area that you had never really thought too much about before. And that's what you had to learn for the next month and you began to realize the volume of information and knowledge that you were trying to deal with. So I was one of the ones that was attracted partly because of where I was and this AI precedent that had been set at Stanford already and I became part of that group. We got a big computing resource at Stanford that supported AI and medicine for 20 years, supported by NIH. And so that was my environment. But I would say to Bob Wachter that, you know, yes, decision science arguably is some of the hardest stuff. It's not just a computing issue. It's got a lot of psychology and understanding of the domain of medicine. Much more maybe that than building EHRs did. But they were very complicated socio political topics as well.
- [32:44]
A
No, they still are to this day.
- [32:46]
C
I mean and to this day are so. And they still demand a lot of attention. They've become pretty much all commercial now. But, but those folks are still doing lots of things to develop and improve them. So I think Bob would agree that an awful lot of computing has been done in areas other than decision science. But the decision science is. That's a really hard problem that many of us were very interested in work
- [33:13]
A
on and, and many still are. I mean, there's still all kinds of work going at it. I mean, what do you think? One of the first that Beth Israel last year, the year before, one of the very first things they did was they took and ran a competition between the best internist and ChatGPT to do New England Journal CPCs. And they literally had them do it side by side. And at that particular point I think it was still 3.5, maybe it'd been to 4, but it was a tie. And that was a year ago or maybe 18 months ago. And now it's not a tie anymore. Now the systems have advanced in such a way that they see things. And I think part of the issue is the following and I don't know if you agree, but a good internist can understand and comprehend a bolus of information, but it doesn't really extend deeply into any one field in a concurrent way. Whereas in other words, he can know or she can know a lot of things at one level they might dive deep in one or two places. But the LMMs, when they're trained properly and they have all the back training, they can go deep in immunology and in cardiology and they can go deep and broad. And they have the advantage of being able to do both of those two things simultaneously, which no human really can do. I mean, I used to round at Pitt with people like Randy Miller and Frank Kroboth and these guys were like walking computers like they understood and knew this stuff so well and so cold that you'd look at them with awe that they could keep all that stuff in their heads. But you know, normal mortals like us, or like me, maybe not like us, but like me, couldn't keep those all that together. But today LLMs have that advantage and I think that advantage is only going to grow. So it also drives to the question of you covered a little while ago around trust and around how physicians today need to learn how to trust. I actually think it's a little different. They need to learn how to think differently and take advantage of the tools to help them to think themselves at a different level and different scale.
- [35:24]
C
I totally agree with that point. And let me say obviously the reason that we keep working on this is we think this is an important thing to do and getting LLMs to do them even better is an important thing to do as well. What strikes me is there's a kind of human Expertise. There are things that we agree that we can't do that the LLMs can do. So now it's logical to ask the question, well, what is it the LLMs can. LLMs can't do, that we can do. And a lot of it has to do with, for example, when you. When you got a consult, the consult almost always came and actually saw the patient, right? Yep, Absolutely. To get that human interaction with a real person that would allow them to get insights not just about what the disease was, but what kind of treatment would this patient tolerate, or those kinds of questions that you asked that have to do with everything from empathy to some aspect of your past experience that you see is highly relevant here, even though it may not be right out of a textbook. So, I mean, the logical inference conclusion from that is we need to find the right symbiotic relationship between experienced physicians who recognize the power of these tools, but who also bring in that human aspect that makes the combination better than either would be alone. And there's a whole society that's been formed in Europe, a professional society called the International Society of Medical Artificial Intelligence, which allows AI on what's happening, but wants to make sure that the physician is always in the loop and all that kind of human. Human aspects of the relationship with patients and the understanding of a given patient, you know, that ability to look at the patient, say, I know that we think we know what's going on here, but there's something else going on, you
- [37:38]
B
know, so, Ted, it's a great. I can't miss this opportunity for the segue into what else might we be missing section. Because, like, what are we missing with the patient, but what are we missing with AI? I think that you've added this as a separ section. We were planning our discussion, and I think you wisely said, yeah, but let's also talk about what we might be missing. I mean, one of your positions is that machine learning researchers may have moved a little bit too fast and just zoomed past structured knowledge representation.
- [38:08]
C
Right.
- [38:09]
B
Which is what was expert assistant, what specifically was lost, and how do we get it back? And obviously, this relates to our earlier discussion of explainability and transparency, but it's a little bit different. I'm asking about the representational layer.
- [38:24]
C
Well, that's why I mentioned the rag. The rag? This Retrieval augmented generation that's overlaid now. And some tools. I was talking. You folks may know Eric Horvitz, who's an IBM chief scientist. He's one of my former PhD students from Stanford. I see him fairly often. And we were talking about this issue recently and that's become such an important aspect for heightening the performance of this as well. So the notion of adding knowledge representation to an LLM in addition to their large, more data centric neural net deep learning approaches is already underway. I'm going to be fascinated to watch to see what kind of enhancements that that provides to performance, not only in decision making, but perhaps some of these. Look, there are aspects of the human interaction that I don't see ever being written in rules or in data sets or something like that that are going to be that distinctly human role. But there are subtleties in our ability to encode what we think and what we feel that may be able to be overlaid on a more deep learning type elements. And I always tell our students, you know, you are in the middle of a process. This is not the end. If you think anything that is hot off the press is going to be still great in 5, 10, 15 years, you're kidding yourself. If you live through what I've lived through and a lot of others like me have lived through, you see that this process is just constantly evolving and improving and maybe missing the boat here and there and forgetting about good old methods from the past that really shouldn't be denigrated, but rather understand what their limitations were computationally back then and how the technology has evolved in a ways that maybe some of those ideas really could blossom today.
- [40:31]
B
So I love the idea that you, that they, you know, bringing in ontology, semantic relationships, domain models that have been worked out over decades is something that can make LLM based or neurosymbolic approaches just much more powerful. I think we're seeing that in practice as we mentioned. But I want to, you know, pull on a thread that you just came up, you know, that optimism that you of, that you share with students and it came out in our earlier discussions where I think you've pushed back when I in, you know, trying to be amusing, describe various approaches as failing. Right. I mean, you know, I go, oh, that didn't work.
- [41:08]
C
Right?
- [41:08]
B
So, and I think, you know, Ted, you, you know, your, the wisdom of your experience was like, well, it's not, it didn't work. It's just, it was incremental.
- [41:17]
C
Right.
- [41:17]
B
We got to a certain point, could you talk a little bit about that? Like what's the distinction in your mind about failures? And we are not done yet, right. We're just on the way.
- [41:30]
C
Well, look, I've heard people say, well, expert systems failed and we had to do something different. And my reaction is, well, yeah, there was this AI winter that occurred in about 1988 through 1995. And why did it then take off? And it took off because of huge external changes that were going on. And I don't mean external to medicine or computing, I mean changes in the world of computing. The field has grown in so many ways. So the technology evolves in ways that allow what was once viewed as something that had failed as suddenly having the potential for new life. So look at what I mean. There's so many things on the technology side that have changed radically since the days we were working on our rule based systems. Memory that's shrunk down to the micro levels that allowed us suddenly to have cheap memory and lots of it. Okay, we didn't have that. It's GPUs, Moore's Law, the processing that's allowed us to have incredible powerful computers. Today. It's the Internet, which basically was in its infancy and only available to a few people and now has allowed us to pool data and create huge data sets of a sort that we never imagined. And machine learning is highly dependent upon those things, not just on good software, it needs all those elements as well. So you go down the list and you realize, well, we have a whole lot of new computational capabilities now. Some of our new methods have grown up because that we never could use them before. Machine learning was a joke until we suddenly had the computational power and data sets to actually do machine learning of the sort that we now are taking for granted with data science and the like. So I think, you know, something that didn't work once in a day when we had really different tools doesn't mean that it wasn't worthwhile. And maybe, and that's why I'm arguing, I think we need to be looking for ways to take more explicit knowledge representation and leverage it to enhance the machine learning approaches of today that we see and the LLMs and the like.
- [44:01]
A
So, Ted, we're going to be running up on time in a couple of minutes here. I'm going to ask one final question and we're going to try to wrap. And it's this one. It's, you know, you've been in this field for over five decades. You've seen all kinds of things, breakthroughs, you've seen opportunities, you've seen missed opportunities. What still needs to be done in clinical decision support or in informatics, a couple of things that you see that might be the next borderline of novelty.
- [44:34]
C
It's kind of like please predict the future. Yeah, that's exactly right.
- [44:40]
B
You, you've got Sting with Howdy on it, man. Yeah, it's, it's, it's free. The bed's free. No money.
- [44:45]
A
Yes.
- [44:46]
C
Well, let, let me answer this in a, well, maybe a slightly tangential, but I think still pertinent way. I have found throughout my career that mainstream medicine has always seen as what, what I do as somewhat aberrant. Okay, I've taken care of patients my career, but been a real doc, real doctor and ran clinics and the like. But that other thing I did always was something that seemed kind of non medical and to some people, even though it was totally motivated by the medical world. And as I have mentioned to you, I work with, with medical students now at Columbia who are taking a rotation in informatics. And the reason they're taking it is because there's this AI stuff going on and I don't know, I better learn about it because it's suddenly going to be important to me. But they come in with a kind of a sense that this is a non medical topic. I view what I have done always as medicine. It's just a different aspect of medicine, but it's medically motivated, it's aimed at improving medicine, it is as intellectually challenging as anything else in medicine and it's got the same value system behind it fundamentally. So I'd like to see the change that begins to occur is that this is every bit as much a part of medicine as, you know, urology and orthopedics and these other fields. This is an obvious part of what it means in the 21st century to be in the world of medicine. And I'd like our medical students to see that not as some aberrant thing you do as an elective, but as part of the way in which you both learn and approach patients in the future. And respect, I'd like to see people respect the folks that are trying to make these tools available to them as part of what medicine needs to be in this very computer oriented world that we now live in and will always live in.
- [47:07]
A
So, Ted, we are running out of time. I want to wrap up here and before I say anything else, I want to say thank you. I think the ability to have a conversation with someone of your stature, with your history and have this conversation is really, really special. And I just really want to say thank you so much for spending an hour with us in going through some of this material. I wish we had more time to go through it with that. I'm going to hand it back to Leon and he's going to close us out.
- [47:37]
C
Okay, I will.
- [47:38]
B
And I also just want to share my thanks and appreciation. Ted, always a delight to talk to you and thank you for sharing your wisdom and deep perspective with with us in our audience. And I look forward to talking to all of you again next week on Practical AI and Healthcare.
- [48:03]
A
Thank you for joining us this week on Practical AI in Healthcare. If you're ready to go beyond buzzwords and hype and explore how AI is truly transforming healthcare, stay tuned for more conversations that get us to what works. Until next time, stay practical.