S1, E30 - Amy Price: Patient Advocacy, Participatory Medicine, and AI Governance

Summary

---

Practical AI in Healthcare

Episode S1, E30 – Amy Price: Patient Advocacy, Participatory Medicine, and AI Governance
Date: March 29, 2026
Hosts: Steven Labkoff, MD (A), Leon Rozenblit, JD, PhD (B)
Guest: Amy Price, DPhil, Editor in Chief of the Journal of Participatory Medicine, Senior Researcher at Dartmouth

---

Overview

This episode features Amy Price, a brain injury survivor turned Oxford PhD and leading advocate for embedding patient perspectives into the design and governance of healthcare AI. Through Amy’s personal journey and expertise, the discussion highlights the critical role of participatory medicine, practical co-production, and the nuanced interplay between AI technology and real-world patient and clinician needs. The conversation is candid, moving, and focused on how collaboration and trust can drive truly practical and human-centered advancements in healthcare AI.

---

Amy Price's Origin Story & Patient Advocacy

Amy’s background:

• Former world relief worker, suffered a life-altering brain injury in a car accident ([01:49]).
• Faced severe cognitive and physical challenges; told she might need lifelong institutionalization ([03:14]).
• Refused to accept a passive role in her recovery—insisted on being actively involved in her care ([03:41]).

Amy: “It's better being in denial than living a life I hate. So I'm going to continue that way. Just leave me alone.” ([01:49])

Strategies for engaging clinicians:

• Entered each appointment with a set of three well-prepared questions ([04:16]).
• Gathered information proactively through research before each clinical encounter.
• Humanized each interaction through eye contact and gentle touch, transforming herself “from just someone sitting on the other side of a desk” to a partner in care ([04:16]).

---

Journey to Oxford & Early Lessons in Teamwork

Cognitive and functional hurdles:

• Memory limited to three digits, severe speech and hearing impairment, loss of drawing skills—yet embarked on post-injury university studies ([06:12]).

Leveraging technology as a cognitive prosthesis:

• Embraced technology to compensate for deficits and found it to be a “lifesaver” ([07:15]).

Community and mutual support:

• Built learning communities at the Open University with peers who believed in collective success—“we made a pact…we would all go and become doctors of something” ([07:49]).
• Unexpectedly landed at Oxford after her colleagues sent applications on her behalf and was accepted ([07:49]).

Amy: “It was, like, the only school I ever felt that I fully belonged to. And I was the only person that didn't have experience in healthcare or medicine. So I depended on Dr. Google a lot.” ([10:17])

---

Participatory Medicine in Practice

Core concept:

• Medicine should resemble a symphony: diverse expertise working in genuine concert, not isolated silos ([12:29]).
• True collaboration means “practice, play together, and develop relationships”—essential for impactful outcomes in both care and AI ([12:29]).

Amy: “You can get people that are geniuses, but when they play alone, what they come up with isn't fit for the masses… It’s only by playing together that we get it done well.” ([12:29])

Analogy expanded by Leon:

• The difference between cranks and contributors is “willingness to participate in a conversation” ([14:33]).

---

Concrete Example of Co-production

Twitter Data Project:

• Amy’s first co-production: teens collaborating to analyze social media data ([15:57]).
• Teens challenged jargon, reframed the classification system, and co-authored a meaningful paper.
• Trust, curiosity, and mutual contribution “cracked the code” ([15:57]).

Amy: “So much cracking of the code comes when people feel there's trust, not artificial trust, but a trust that's built in conversation… That’s success in co-production.” ([17:57])

---

Group Intelligence, Human in the Loop, and Model Hallucinations

How do we make groups smarter than their individuals? ([19:46])

• Relies as much on human trust and communication as on information technology.

Human in the loop versus knowledgeable human in the loop:

• Critical distinction: not just a human, but one who cares and is empowered with knowledge and tools ([20:53]).

Amy: “Anyone can become a knowledgeable human given the right tools… not to correct it, but to make it better.” ([20:57])

Hallucinations in AI:

• Amy dislikes calling model errors ‘hallucinations’—prefers to frame them as plausible inferences or assumptions given incomplete data ([22:22]).

---

Amy’s First Encounters with AI

Initial use case:

• Deployed AI tools in a Pfizer grant project to train PIs in co-production; learned collectively, including from their errors ([24:11]).

Amy: “We learned from the models and we learned from each other and it was just a wonderful experience. That actual instance actually went to Congress as an example.” ([25:17])

Amazed by:

• AI's reach into history and stored human experience ([25:29]).
• Its helpfulness for someone recovering from cognitive injury.
• The rapid adaptability to cultures but also frustrating presence of human biases and research errors ([27:14]).

---

Everyday AI for Patients & Providers

Amy’s current use as a patient:

• With experience, AI became “spectacular” for health research ([29:05]).
• Brought AI-generated findings to her cardiologist and collaborated, yielding validation and new insights ([29:05]).
• Used AI for nuanced issues (bone growth) and to clarify clinical notes ([30:17]).
• AI reduced barriers for non-native English speakers due to potent translation capabilities ([32:39]).

Amy: “Sometimes you could feed that in…when a clinician says this, what might they mean?… Either way it’s a win win.” ([31:29])

---

AI Literacy and Bridging the Sophistication Gap

Problem:

• Stark contrast between tech-savvy patients (“I use it for everything”) and those frustrated by bad outputs ([32:39]).

Informal study:

• Ran a mini-study—some patients got “junk information”; best practice was hands-on coaching: breaking tasks down, focusing on simple, single prompts, iteratively building skill ([34:05]).

Effective tactics:

• Sharpen skills through guided learning.
• Constructive peer communities sharing experiences.
• Exposure to how AI models work demystifies the process ([36:21]).

---

AI in Scientific Publishing

• AI is a tool that must be directed by someone with domain expertise (“a very bright but junior associate”) ([37:30]).
• Surface results may look fine, but flaws become apparent under scrutiny.
• Users should recognize AI’s limits, check sources, and “never just assume because it came from AI, it’s going to be correct” ([37:30]).
• Growth in AI sophistication amplifies the need for a “knowledgeable, caring human with the right domain expertise” ([40:32]).

---

Dangers & Limits of AI in Healthcare

Key concerns:

1. Mental health:

   • AI can inadvertently reinforce harmful patterns in personality disorders, or encourage misguided or even violent actions ([41:49]).

2. Bias & equity:

   • AI often perpetuates medical inequalities (e.g., skin cancer detection in people of color), demonstrating the need for critical human oversight ([41:49]).

3. The irreplaceable value of human empathy:

   • Amy describes a pivotal moment when a doctor’s tears—and caring—saved her life during severe illness ([41:49]).

Amy: “Doctors and clinicians are never going to be replaced by AI because what they do best is actually being a doctor… Those warm tears touched my body and I thought, someone cares… and that made the difference for me.” ([41:49])

---

Future Directions—Systems, Training, and Governance

On building better AI in healthcare:

• If Amy could change one thing: “I would have everyone at the table from the beginning in developing the tool and in using the tool,” emphasizing workflow, end-user needs, context, and critical thinking ([48:31]).
• Don’t build ‘guidelines’ or automation in a void; co-create with those impacted ([48:31]).
• Keep AI as a servant—not a taskmaster; ditch systems that are inefficient or lack person-centered value ([48:31]).

On the system itself:

Leon: “You’re not inheriting a broken system, you’re inheriting an unfinished one.” ([52:16])

Amy: “A broken system means you throw it out. ... An unfinished system means that there are errors all along the way…it’s a part of how we get better.” ([52:16])

• Build incrementally—apply “minimum viable product” principles, build on what works, iterate, and always prioritize research methods and critical evaluation ([54:06]/[54:11]).

---

Call to Action: Participatory Medicine Journal

• The journal is “collaborative” and focused on making contributions better, not on rejection or shaming ([55:20]).
• Offers support for unfunded contributors and training for early-career investigators ([55:20]).

Amy: “We are here to work with you to help make your paper better. And we welcome your work. We welcome working together. ... If you’re under a PI and they’ve used all the research money, consider us.” ([55:20])

---

Legacy and Leaving Doors Open

A moving conclusion on mentorship, legacy, and the power of open doors:

Amy: “Every door I can. My lifetime is not going to be forever. And the legacy is the open doors that we leave behind…being a mentor isn’t solving all the problems. Being a mentor is opening that door to that dark key that we might be afraid to go through and giving people the tools…they own the future. And the future they build is going to be different…But it’s going to be a beautiful future if it’s built on those kinds of tools where we put humanity first, we do it with intelligence and we recognized and special gifts and intelligence in every human being. And stay open, stay fun, be curious and yeah, join the party. We're celebrating.” ([56:32])

---

Key Segment Timestamps

• [01:49] – Amy’s accident, initial advocacy and patient engagement
• [07:49] – Community learning and Oxford journey
• [12:29] – Symphony analogy for participatory medicine
• [15:57] – Co-production example: Twitter research project
• [20:53] – What makes a “knowledgeable human in the loop”
• [24:11] – First AI experience in a real-world project
• [29:05] – Health use cases: bringing AI outputs to providers
• [34:05] – Informal study: why some patients get better AI results
• [37:30] – Editors, AI writing, and the need for expertise
• [41:49] – Bias, mental health, and the limits of AI empathy
• [48:31] – Biggest needed change: everyone at the table for AI design
• [52:16] – “Unfinished vs. broken” system mindset
• [55:20] – Call for contributors to Journal of Participatory Medicine
• [56:32] – Mentorship and leaving doors open

---

Notable Quotes

• “[Everyone] at the table from the beginning in developing the tool and in using the tool. Because no matter how good you make something for someone else, you don’t know them and you don’t know the people group, you don’t know the workflow.” — Amy Price ([48:31])
• “A broken system means you throw it out. … An unfinished system means there are errors all along the way … it’s a part of how we get better.” — Amy Price ([52:16])
• “Being a mentor is … opening that door to that dark key that we might be afraid to go through and giving people the tools … they own the future.” — Amy Price ([56:32])

---

Takeaways

• Healthcare AI will only succeed if it becomes truly collaborative—engaging the right experts, patients, and stakeholders at every step.
• A human in the loop isn’t enough; you need knowledgeable, caring humans with contextual expertise.
• The systems we inherit aren’t broken—they are unfinished and ready for continual improvement.
• Building practical AI means honoring legacy, applying rigorous methods, and creating communities of learning and mentorship.
• The future of healthcare AI is participatory, person-centered, and fundamentally unfinished—ready for the next generation to complete the work.

---

To contribute to the Journal of Participatory Medicine or learn more, visit their website.

Transcript

[A]

Welcome to Practical AI in Healthcare, the podcast that cuts through the noise to spotlight real world solutions delivering real world value. From patient care to clinical research, from life sciences to patient engagement, we focus on what's truly moving the needle in healthcare. No hype, no theory, just practical insights where AI is making a true impact. Welcome aboard and let's get to it.

[B]

So most AI in healthcare is built by engineers, evaluated by researchers, and deployed on patients in that specific order. Today's guest thinks that order is exactly backwards. Amy Price is a brain injury survivor who bootstracked away from a hospital bed to a doctor of philosophy at Oxford. And she spent 15 years building the case that patients belong inside the design process, not outside it. She's now editor in chief of the Journal of Participatory Medicine and a senior researcher at Dartmouth. Amy, welcome.

[C]

Thank you. I'm honored to be here.

[B]

So delighted to have you. And you and I have known each other for years and written stuff together and are good friends and you're just a lovely and delightful person. So I'm so looking forward to having this conversation with you. So I want to start by taking us back to your origin story, right? How you acquired your superhero superpowers. And like a lot of superhero stories, it starts with a terrible accident. So. But take us a little bit before that. You know, before the research journals, who were you?

[C]

Well, that's a difficult question because I was so many things in this lifetime, but I. I ended up. We, my husband and I did world relief work for years. And then on our 25th wedding anniversary, we were going to take a trip around the world, just us, not helping anyone, just having a great time, enjoying our anniversary, like a sabbatical. And we were at a stop sign. We were hit by a yellow Corvette who had no, no intention of stopping. And we were both seriously injured. I had over $4 million in medical bills, and my husband also was seriously injured. And they said that I would probably need to be institutionalized because the damage was quite extreme. I had significant upbringing, brain injury. I had difficulty moving. And then I thought, I'm going to get back to where I am. People thought I was crazy and I was in denial. And I said, well, it's better being in denial than living a life I hate. So I'm going to continue that way. Just leave me alone. And that's how I started.

[B]

So. But it sounds like the injuries were no joke. I mean, that sounds. I mean, the amount of money doesn't quite convey it. Right, but you're talking about broken neck, serious Brain injury, this, you know, and being told you were an institution, you need to be institutionalized, couldn't have been something you wanted to hear. What did you do instead? I mean instead of being taken it and lying down, how did you manage to get your way back to standing up?

[C]

Well, you know what, this comes, this also comes back to the point where patients need to be involved in their own care if they want to be. So I took charge. I made sure that we had doctors that would listen even to a brain injured person. And I, how did, how did you do that?

[B]

I mean I, I, doctors don't listen to me and I only act brain injured half the time. So what, you know, what, what did you, what was it that enabled you to get doctors to actually listen to you?

[C]

I didn't complain. I went in with questions and I went in with well timed questions. I thought I can get three questions in an appointment and I'm going to get them all answered. And he wouldn't let. The appointment continued until I got my questions answered. And most of the most questions I could get answered in other ways. I could go on the Internet, I could ask, I could ask doctor friends, I could look up in libraries and so I knew a lot going in by choice. And I basically would say, is this your best answer? What else can you think of that you've never done before? What could we do together? And then they were engaged in the journey and it's always important to make eye contact and to touch just briefly, doesn't have to be rude, but so that you become a person and not just someone that's sitting on the other side of a desk.

[B]

In a way you were like discovering the principles of participatory medicine as you were living through your recovery. Right. You had to invent a way to actually deeply engage with clinicians and get treated not as a abstract patient, but as Amy, a person who needs help. That's really, really interesting. So, so how did your, you go from the accident through recovery and rehab to Oxford?

[C]

Well that's, that's an interesting story because I am after I finished the regular health rehab that can only get to get you to about low normal. And I didn't think I was quite ready to just go get a job at McDonald's or something. And so I went back to, to, I went back to university, started all over again, even though I only had a three digit memory at the time. And I said if I can you

[B]

give us a, can you give the audience a sense of the kind of cognitive impairment and the level, the digit span is reduced. Normally it's four to seven. You were down to three. That's bad. What else were you struggling with?

[C]

Speaking? I, I couldn't pronounce the syllables or get the timing of the syllables right. Hearing, I couldn't hear properly, so that makes it hard to have a conversation. And I couldn't remember. I couldn't remember. I didn't have a working memory. I couldn't use mnemonics, like, you know, these little chants that people use to remember complex formulas. And I couldn't draw. So. And I, I ended up taking neuroscience. So, you know, there's a lot of drawing there. But I, I depended very, very heavily on technology, which was such a boon, like actually now, because I thought I can learn technology, technology can help me. And sure enough, that was a lifesaver.

[B]

And I love that, that you're like an early model for cognitive prosthesis. Right. As we, as we develop. You know, my teenager called me a cognitive cyborg recently because I'm, you know, of how embedded I am in my like, well, AI ecosystem. But just as a lot of people had to get used to wearing glasses and using crutches and et cetera, you had, in a way, an early start with the cognitive prosthetic. Prosthetics and relying on them just to function. It's, that is super, super interesting. So, yeah, sorry to interrupt, but yeah, Then, then what happened? How did you move, move through that?

[C]

Well, I found where I, I started was the Open University. That's kind of a last chance, regular university for, for people who didn't make it through the British system when they were very young. So they, they come in and they're, a lot of them, they, they don't have a lot of autonomy or self esteem because they didn't do well in school. Maybe the first time or something went wrong. And so, but these people were helping me, they were helping me survive, get through my classes. They were my buddies. They told me, you know, the most important thing about school is you have to have, if you go to summer school, you have to get buddies. They told me, like, how to behave, what you do, what you don't do. And we studied together and so online, we did everything was online and we studied together. And I learned how smart they were. And they didn't know, they didn't know they were brilliant. And so we made a pact between us that we would all go and become doctors of something. And they said, like, you know, like, you're crazy. And I said, yeah, but it would make me happy if you could do this. And they all. They all signed up. They all started. And then the week before, my doctoral committee was to meet at the Open University. It disbanded because of austerity measures. So I thought, great, I don't have to do this. But my friends wouldn't give up. They sent my application to universities all over the uk and people started calling me for interviews, assuming it was me that sent them, and I said, okay, guys, choose one university. They chose Oxford. And I got in. I had my. I had my interview. I believe it was 5 o' clock in the morning. I could have. If you get that far, they'll actually invite you to come on campus. But I thought, you know, there's not a chance. I'll. I'll get in. I'm just going to do mine online. So I. I did, and five o' clock in the morning. And I remember at one point I was answering all the statistical questions and things like that, and then they said, okay, that's enough. And I'm like, wait a minute. I said, I've done really well at this, and you can't just dump me. And they burst out laughing. And they said, what. What makes you think you're gonna. We're gonna dump you. And I said, well, I said, you said, that's enough. And they said, no, we're gonna take you. Yeah, that was. That was the. That was the beginning of the next chapter of my history. I loved it at Oxford. It was, like the only school I ever felt that I fully belonged to. And I was the only person that didn't have experience in healthcare or medicine. So I depended on Dr. Google a lot. Not as my doctor, but as. What on earth are they saying here? And what I found out was, it's okay, because there were areas they didn't have strengths, and I did. So we helped each other and was a beautiful kind of team science, team, mentoring, learning journey.

[B]

Yeah. I mean, what I love about your story is not just the perseverance, Right. And the talent, but that it was actually your cooperative spirit and ability to make strong, collaborative connections that made a difference. And I think a lot of us forget how important it is to just be, like, a decent human. Right. And how that can, you know, that both help. That helps other people, obviously, but it helps you. Right. And so. And I just love that part. So thank you for sharing that. So you went from being a patient who's fighting for her own rehab to becoming a researcher studying how patients should be included in research. I mean, that's Quite a bridge, right? And the theme of bridging keeps coming up in conversations that you and I have. So you've now been a bmj, a research editor for over a decade, and you're now editor in chief at the Journal of Participatory Medicine. Can you share with us your perspective? Like, what does participatory medicine actually mean in practice?

[C]

So participatory medicine, I, I think the best way that I can describe it is it's like a symphony. You can get all kinds of people with all kinds of expertise in different musical instruments, but until they practice, play together and develop relationships, what they play is not worth selling. And I see the same in for AI and in healthcare. You can get people that are geniuses, are near genius level or so gifted, but they play alone. And when they play alone, what they come up with isn't fit for the masses and the challenges that, the pressures that are on to compete, they're sent from their boss or paymaster and to come back with a certain thing done. So oftentimes they think that means that they can't pause and hear people when really that's the only way they are actually going to get it done. Well, and then when you play together, at first, you have to figure it out when you play with anyone, if I play with you, I, I have to figure out, like, how do I talk to Leon? Right. Or where's the, where's. Where's the interest? Or what's upsetting or, oh, his face just tightened up. And in all work, it's the same way. We need to be alert to each other and sensitive so that we can work as a team and also forgiving and have fun together. So you're not there for just like one event that's not. You're there for the long term to build things over time together and to make a difference. If you make a mistake in the beginning, mistakes happen. It's not a great tragedy. It's just part of life. And you pick it up, you take it apart, you put it back together, and you work at it again. And then never forget to celebrate.

[B]

Yeah, I mean, the emphasis on teamwork is just so important. I like the sympathy analogy. I often think of it as a conversation, and I kind of think about the difference between people who are scientific cranks. Right. And people who are scientific contributors as how willing are you to actually participate in a conversation? Because if you have, you know, everybody in the physics department gets letters from some guy who's invented a new theory of the universe. Right? And that letter may contain some interesting seeds, but it's not part of the conversation in the field and therefore it's not contributing anyway. You actually have to get in there and be willing to listen to people who've said, who are, who've said things before and are saying things now and then reflect back, not just talk your own language in a vacuum. And I gotta, But I kind of like the sympathy metaphor because I think it suggests the immediate harmony and the fact that you've got to do the kind of very human adjustment to understand what are people actually doing, what's the rhythm, why are they doing things a certain way? So I think that's really powerful. But let me, let me kind of zero in on co production and see if we can get a concrete example. Can you think of a example of co production that actually worked and what did it look like?

[C]

My very first effort at co production, I had four teens students and I was a student also, and I, we were analyzing data from Twitter or AX it I guess AX it is now, but it was Twitter then. And I was explaining to them the question quadrants of the social sciences and what to do. And they looked at me and I said, we're going to try to stay away from junk language. And they said, what is that? And it stopped me in my tracks. And I thought, no, we have to start from the beginning. And I said, okay, so this is complex for you and I understand because I spent years learning it and you haven't, so tell me what makes you curious? What do you want to know? And we started a conversation. And then from the conversation I then asked them, so how do you think we should classify this? And they came up with, well, why don't we just use weaknesses, strengths, threats, those kinds of things? Said sure, why not? And then from there we broke it into, we like, we broke it into, into classifications. And they ended up four teenagers bringing to this paper that I was writing as a student with them as co authors now. And the things that were forefront in clinical trial science at the time, the suggestions they were making were just breaking and they were people with no knowledge from the field. But by working together, by looking at the comments, by looking at what we had in front of us, we cracked code. And I think so much cracking of the code comes when people feel there's trust, not artificial trust, but a trust that's built in conversation and people feel inspired and curious and they contribute and that's success in co production. And then you can use it in anything. I've used It, I use it in statistics, I use it for qualitative research. I use it for, to teach people how to interview and I use it in, to build models and anything at all you can use co production because as soon as people are interested in your science that you're sharing, you win. Dr. Steven Chu, who was a, a Nobel Laureate in physics, we had him working with us when we were trying to make a better personal protective equipment during the pandemic. And I have to confess, Leon, I was a physics dunce. And then he would just share with us and it made so much sense. And I thought, I love physics. And what inspired him was if we brought ideas and he would, he would take the ideas, he would come back with them and he would teach us. And the virologists also taught us. We didn't know anything about that either because.

[B]

So Amy, I just want to, like, I think you're, what you're saying is really powerful, but I want to tie it back to AI and you know, but first let me frame this. I say, I mean, I think what you're touching on is something that's been of great interest to me and is currently of importance, which is how do we make groups of people smarter than the individuals that compose them? Right. What is it about some patterns of interactions that allow us to work better together? And I've always been interested in how to use information technology to do that. But you're pointing out that some of it is not information technology. It's the, it's a, it's a practice that is a very human practice of establishing trust, establish communication. But if we pull it back and think about the information technology aspect of it, we often worry about how we use it to make groups smarter and how to create processes, make processes more automated. One concept I think you've brought up before is the difference between a human in the loop and a knowledgeable human in the loop. What's the distinction in your mind and why does it make a big difference

[C]

not be a knowledgeable human, A human that cares.

[B]

So it means a knowledgeable human that cares.

[C]

Yes, that's it. A knowledgeable people that care. And anyone can become a knowledgeable human given the like, given the right tools. And so to have a knowledgeable human in the loop that has information from all these sides or has connections to, into all the different aspects and can ask questions and come back and really properly check the information, not to correct it, but to make it better. Not to say, you might. When I, when I sit there, I hear people and they talk about models hallucinating. I just about lose my mind. It just turns me off because it's a model, it's coming to the best possible solution. That's not a hallucination. That is the model arriving at a conclusion and we do the same thing in our human brains. And if someone, I come up with an assumption that's just slightly off because I don't have all the information and you, Leon, were to say to me, oh, like Amy, that's a hallucination. And they'll start getting all judgy about it. I'm going to shut down, right? I'm going to think like, yeah, I'm not talking to Leon anymore. Like, no, right.

[B]

And luckily, luckily, the modern LLMs don't get offended as easily as you and I would. So we can, we, we can accuse them of hallucinating. They don't mind. But, but I get, I get your point. You know, it sounds judgmental and a misunderstanding of what the models do. I will say it's a, it's a reasonable description of some kinds of outcomes that we used to get more often. Like, I'll give you an example. Like, I, I work with cloud code a lot and I have sort of separate memory models that I store in Gemini and for some Claude code keeps deciding that my email must be leon.rosenblademail.com because my name is Leon Rosenbl and I keep writing into memory. I'm like, that's not my email. Like it should be my email, but it's not my email. It's, you know, and it's now happened like three or four times and I keep putting in guardrails and it keeps making up because the inference is too plausible. Right. And like the training now is good enough. Where, if you say, where did that come from? The model itself will say, oh, I'm sorry, I hallucinated that. Right. And it's. So it's an effective way, I think, to encapsulate certain kinds of behavior. Where I probably agree with you is that getting all judgy about it doesn't really help. Like, you just need to think about safeguards and say, well, since we know that models will behave and will have these failure modes that are relatively predictable, right. You just need to put some safeguards in place and sometimes it's very irritating when they don't work the way they're supposed to or you're not smart enough to figure out how to build them. Right, but sort of back to your point of using AI, right, You've now Been using AI as a patient and as a researcher and as an editor. Take us back to when did you first start using AI personally and what do you remember about it?

[C]

I remember it's improved a lot and I remember I first did it, I had a grant with Pfizer where I had 14 teams of PIs that we were training in research and co production. So, so they had seed grants to receive the training and AI had just come out, so we felt, and that they were asking about it. So I thought well, why don't we use that as well? And I remember how much we learned from each other and how funny it was because we would find out some of the things that we said and the mistakes that we made and how we grew together and how we, we like we all learned, we learned from the models and we learned from each other and it was a, like, it was just a wonderful experience. That actual instance actually went to Congress as an example. So it was pretty exciting.

[B]

So cool. So what, what surprised you back then and, and maybe now about what AI could do?

[C]

I was awed by the fact that it could go through all through history and bring up information and that it had a conglomeration of human experiences logged inside it through different parts of research, conversation, whatever had been put into it. And that it was really great at working out solutions to human problems. And it thought what an amazing tool that would be for someone like, like me in the beginning with a brain injury who was just puzzled as to why are people responding like this. And I also appreciated the way that it adapted to different cultures and, and I was also a bit shocked that at the same time it would be full of all the same biases and research errors and mistakes that we, that we as humans made because it seemed so superior in some levels, especially its speed to compute things and it's its ability to organize. And yet with the biases it's just as crippled as we are.

[B]

Yeah, that's a really astute observation. And one of the things I think trips all of us up about this new brand of information technology is what I forgot who I need to give credit to. Maybe it was Asan Malik called the Jagged Frontier. And its performance is very uneven across domains. Whereas in a human sort of areas of performance tend to correlate, right. Somebody who is really good at sort of at math, is usually pretty good at logic, is usually pretty good at history. Not perfectly right, but we can sort of predict how good they're going to be. With non human intelligences those predictions just fall apart. Right. That boundary of what they're good at and what they are terrible at is shifting constantly, but it's very, very uneven. Right. There's not that positive manifold that we get with human psychology that's very easily for us to grasp. And I think that's one of the things that leads to just frustration and poor intuitions. I think that you almost need daily hands on experience to start developing intuitions about where the models are going. Right. And to start anticipating where it's, what's going to be good in another month, what's going to be good in two months. I know that, you know, just from, from my own experience over the last, you know, couple of years is that it wasn't until I made time to get in there and like just work with them consistently, hours and hours every day that I started developing those intuitions. So I'm curious about your experience in using those models. You know, you've used them as a patient as well. Right. And not just as a researcher. What was your experience in using them for your own health needs?

[C]

Well, by the time I used them, I was really good at working with the models. And because I had, I put in the time, I spent the hours, I read how other people got them to work. And so when I used it for health information, it was, it was spectacular. And the things that, the things that I learned from the model I brought back, for example, to my cardiologist and he was like, this is, this is fantastic. This is, this is perfect. I said, because I said, you know, this is just from an AI model. Um, is there anything we need to adjust or fix or do you want me to throw all this out and what do you think? And he said, you know, this is, this is fantastic. And what I found is, is it gets to things that nobody has time to tell you, like for example, bone growth. Because I, I needed to bone grow, grow bone for some dental procedures. And then I had some other testing and some said, well, maybe your bone growth is not so good. But then I found out, well, there's false positives because of, you know, all the injuries that I had. Yeah. And they put that into AI. Well, what can I do about this? The AI was really helpful about not getting involved with some of the common bone builders, but to do things a different way if I wanted to develop bone growth for bone grafts and things like that. And, and again I, I took that to my providers and I said, well, this is what I found out and said it very briefly and, and, and they agreed and they Said, well that's fascinating. And you know, and the results were there. So it's also really helpful, I think, in terms of language or in terms of things you don't understand because sometimes you'll, you'll see something on a report and it's, it's not traditional. It's the, it's a doctor specific kind of shorthand about what a. If some findings say, and sometimes you could feed that in. So when, when a clinician says this, what might they mean? In, in that way you get a lot greater understanding without having to ask questions of everybody else or to challenge anyone. You can just bring it back into the, into the conversation. You know, when you said this about this person particular finding, did you. Were you meaning this? And then they can say, yes, that's exactly what I meant. How did you know? Or they can. Or they say, yeah, that's dead wrong. And then I'm like, oh well, like, yeah, I'd love to hear what you actually meant. And then so, so either way, either way it's a win win. And also find it's really good for people where English is a second language and because it's a great translator and, and so they can, that's, that's by

[B]

the way, a commercial use case we discovered recently in a podcast where, you know, there's LLM powered human review discharge instructions. I mean, those save lives, man. Like, absolutely no joke. Right? So. So Amy, the interesting thing about your, you know, experience is that, that your sophistication and approaching AI with some level of experience puts you in a very different category than a lot of patients. I mean, so one thing that strikes me when I talk to patients and patient advocates is this complete divergence where some people are like, it's amazing. Like, I use it for everything. The other people like, it sucks. It gives me terrible advice and doesn't know, you know, keeps getting me to. And I, you know, the both are true, right? Like you have to say like, and the difference is sophistication and prior exposure and having the right mental models, right? So like folks like you and ep, Dave and Hugo Campos are like just out there taking advantage of these cognitive prostheses to, you know, push you ahead and make you really, really capable in participating in our own care where other folks are not just falling behind, but feeling like deeply annoyed, right? Because somebody's telling you in this tool and it like it gives them junk. What's, what do you think is a way to, I mean some people, folks talk about is like AI literacy. I'M not sure that's the right framing. But what do you think? Like, what's a way to close that gap a little bit?

[C]

I found that out because I ran an informal study with patients on a specific condition and it completely threw me that some of them got like such junk information because they reported it all, like they pasted it all back to me. So we all, like, we'd all know together. So, and then what worked the best was to work with them, to sharpen their skills in specific areas, to, to teach them how, how to, how to ask questions. And not to get too complex because I've seen people put in these like three page prompts. Well, you know, unless you really, really know what you're doing, you're just going to confuse the AI and you're going to get back what you didn't plan to. So you keep it as simple as possible and as real, one task at a time and start to build that. And also people get better because people see the AI can improve on one task, then they also get creative and use that same kind of thinking to change it in other ways. And they'll say, well, I got to do this and then the AR did this and then all of a sudden they're excited and they're engaged and then they start to get really good information and they'll come back with questions and they'll say, you know, I tried to get it to do math. Did you have any success? And I said like, well, initially, no, not at all. It was horrible at doing math for me, but I'm also a horrible mathematician, so that's probably why. And I said, so I started this way and, and then you, you trade information back and forth. And so within communities of, of people, learning together is a wonderful way and also to developing. If you know the back end of the tools and you're brought into the back end and you see them develop, it's much easier to understand what their capacity might be. They're not a black box anymore. So that's also helpful.

[B]

So I think those are really worth, you know, elevating. I mean the idea that you need learning communities that share information, at least about capabilities. You want some background mechanistic understanding of what's happening under the hood to give you, to guide you intuitions about what to do and what's possible. And then you need hands on experience. Right. Like you need to get in there and just start doing it and seeing what's working and what's not working. But I still think it's a really difficult problem to get most patients up to the level where they can take advantage of the best tools that are available. But I'm really excited that folks like you can help us make progress in that area. Turning for a minute to your role as an editor. Right, so we had Tiffany here who talked about her role as her journal editor and her worries about AI and use of AI capabilities. What are your thoughts right now about use of AI in writing, in editing and in publishing?

[C]

So first of all, Tiffany's awesome.

[B]

Oh yeah, we all agree about that.

[C]

So glad they had her on. And what I see is people are so bad at AI, like many academics that it. AI is a tool. AI works for us. It doesn't do the job for, for you. So you can give AI a specific task, but the work has to be yours. And if it's not yours, it's going to show up. You know, for example, I, I could get, I can get AI to build me an app, complex app. I can get AI to do that, but because I don't have the background to test that and to work will look good at the very surface. But as soon as you get under the hood and try to use it, there's going to be glitches. Whereas if I have some experience in that area, then I see those glitches and I say to the AI, like, yeah, let's move away from this aspect and let's look at something else. And sometimes you just close the chat and start over, just like a fresh start, you know, and that's, that's also helpful. But authors and editors and anyone in that field, it's all over. But look at the evidence also that's out there. Read, read the evidence, read the research and see how it applies to what you're doing. And don't just get your information from what somebody said on, on social media or, or some other column. And also ask the AI, you know, ask the AI, like, I noticed you have difficulty interpreting this or, or doing this. Can you explain how I'm making this difficult for you or can you explain what in your training makes it challenging for you to come up with, out with something like this without bias. And it will share with you. And then, you know, you start to know where the weaknesses are, where the strengths are, where you can use it. But it would very, it's very much like having kind of a, a really bright but junior associate working with you. They don't have the, they don't have any experience in being a human or in actually doing research. And so they can be great, they can organize things for you, they can, they can find knowledge you didn't have. And then it's always important to check it. So if there's, there's evidence, if there's research that has to be checked by other sources, not just assume because it came from AI, it's going to be correct. Because sometimes it is.

[B]

And the importance of an, of that knowledgeable, caring human with the right domain, expertise becomes more and more important as the tools become powerful. Right. Just like a guy with a shovel is not as likely to do as much damage as a guy with a steam shovel. Right. It's, you know, you want the per. A person who's operating a steam shovel to understand where not to dig. Right. And where you want, you really want the hole and the kind of hole that you want. So I don't know if that metaphor. Oh, thanks. Yeah, not sure it holds that much water, but right. There is a sense of empowering people with too little knowledge to do too much and like you just wind up, you know, tearing out the fiber optic cable that took somebody else a long time to lay down. So you want to be a little careful. I think you and I are both fairly optimistic about individuals and society's ability to adopt to these new tools. Right. I mean, I think we all recognize, we both recognize it's going to take work and there are some rough spots. Let's think for a minute where those rough spots are. Like what, what have you seen going badly wrong in any of those areas of your life with the way people are using AI? What concerns you the most about the way it's going to.

[C]

In mental health there's challenges. So for example, if you have, let's commonly call the cluster B or maybe narcissistic borderline type personalities, the AI can cater to that and tends to cater to it, which makes people like that think they have even more of the answers than they do and makes them even less sociable. So it puts them, it puts them behind and it can also encourage, for example, violence and harm in some of those areas. And because to the person they're discussing it and the, the AI is responding with an answer as if it's a discussion. But to a person that's unbalanced, they may go out and not consider that a discussion. They may consider that like a, you know, a directive and act on, and harm themselves or, or others. So I, I think that's, that's a critical area. And then the, there's other very critical areas in terms of bias so for example, if I'm a person of color and I have a melanoma, for example, the presentation of that melanoma is completely different in a person of color than it is in me. So if I'm the person of color and I maybe put my image into the machine or I ask specific questions, questions about treatment, I'm not going to get the same kind of quality or value that I would from a clinician. And I may not get treatment early enough because the machine says, oh, that seems to be fine, but see your doctor, or I may be steered in the wrong direction. So this is where I come back to the knowledgeable human in the loop. And also the need for the difference between somebody saying what you want to hear and genuine empathy and sensitivity. It's totally different. And doctors and clinicians are never going to be replaced by AI because what they do best is actually being a doctor. When I had MRSA and my husband had died, things had gone horribly at work. It was, it was just a terrible time. I thought, I'm in the hospital, good, this is my opportunity to check out. I didn't even try. They said, want to leave you to icu. I said, no, I'm not moving. And so left me there. But a doctor came in the middle of the night, probably two o' clock in the morning, and he said, please don't die. And there were tears and they were rolling down his eyes. And those warm tears touched my body and I thought, someone cares. A person that doesn't, doesn't even have a relationship with me cares whether I live or die. And that made the difference, that made the difference for me. And so many times just an illustration or a word given by a knowledgeable human being or an idea. It's like almost like comes like a supernatural idea comes up from the inside of them and they'll say, oh, why don't you try this? Or we could do this. And it makes a difference in the quality of care. And it's guided by their years and years of pattern recognition that they've developed as a practitioner. We are never going to replace that with AI.

[B]

So you're, I think a lot of your concerns are focusing on trying to the advantages of human relationships and having real human caring and the efforts where having AI try to replace it may actually be counterproductive. That's really interesting and kind of leads us into our future directions right where things are going. I always worry about statements about like, AI will never be able to do that. But you know, I think, let me grant that there's some value in human relationships that is unique to the fact that it's between humans and, and it's by, you know, in part by design. And I think it's an interesting thing to consider. But let's talk, let's anchor this a little bit in your current work. Right? So you're working, you're doing research at Dartmouth. You're currently training 20 teams of investigators, right. What are you training them to do differently?

[C]

Oh, that project's over and it's done.

[B]

Mission accomplished. We're gonna, we're gonna get a banner and you know, full lights. You can go.

[C]

Yeah. So what I'm currently, I'm really focusing on mentoring and teaching people the importance of research methods because that's what I focus on, because they're an important bridge. I started out in neuroscience, which I love, I love neuroscience. That's my first love. I went into evidence based healthcare because they, there weren't the necessary methods to hold together and to make it known for, you know, the, a general, a general audience, for example. And methods are, methods are so important. So I've been focusing on that. And then I take from your chapter where you were saying, we don't know what AI can do in the future, what I found is it's getting a lot better at neuroscience. So I'm climbing back in that boat, you know, and it's going to be

[B]

your neuroscience collaborator, right? He's going to go from. So if you could change one thing about how AI systems are developed for healthcare, what would you change?

[C]

I would have everyone at the table from the beginning in developing the tool and in using the tool. Because no matter how good you make someone something for someone else, you don't know them and you don't know the people group, you don't know the workflow. And we've made so many mistakes in, in human life, in, in clinical care, for example, with guidelines. That way we give guidelines that someone that's fit and we give them an old lady guideline, if they're me or you know, an old guy guideline, and they're fit, those guidelines are not really effective. And we see, I mean, that's just such a small example. We see like the discharge notes, we see the help with exams, we need to keep training people to think critically because then no matter what comes along, if you think critically, and I don't mean by being critical, I mean by examining all aspects of the question and applying the methods and improving and not just accepting things at face value, I think that this will make AI and us as a people working together stronger. People are concerned they're going to be all out of jobs. We heard that with Internet. We heard that with when things started what electric typewriters, you know, so many years ago. We heard that when computers came on board and they became small enough that you could carry them and you know they're going to replace humans. No, humans evolve. Humans evolve though. When the old jobs didn't work, you evolved to do, to do new tasks and different tasks. And I think we need to, we need to be cautious that the AI doesn't take more of our time and stop becoming our servant. So if it takes too much time and it's not useful, then we need to drop it and we need to stop spending money on things that are literally not efficient, not person centered and are not cost effective. Like prior authorization, for example. Now we have say AI doing it. Like how useless is that?

[B]

Well, we can have AI on both sides, right? You can have AI declining and AI fighting for prioritization. It'll just be an arms race. So it'll be, it'll be fun for somebody either way.

[C]

But that's where you need, that's where you need like clinical judgment in the loop. Right.

[B]

So Amy, I just, I'm going to try to bring us to a landing, but you know, before we do, there's, you know, I think if fundamental messages really optimistic about researchers and clinicians. Right. That we by, we could teach them to co create with patients early on. And I think that's a really important message for our audience to hear. One thing you see said before that you know, in our conversations, it really jumped out at me is that, you know, as a message to kind of younger clinicians, is that you're not inheriting a broken system, you're inheriting an unfinished one. What's the difference? What, what's the difference in your mind?

[C]

A broken system means you throw it out.

[B]

Yeah.

[C]

If you ate an unfinished system means that there are errors all along the way. It's a part of the process and it's a part of how we get better. So if you throw that out or are dismissive of it, you throw away the history of everything that will bring you greatness. And you also forget what brought you to where you are in terms of, in terms of knowledge and, and in terms of experience. And so same, same. I've. The greatest truths I've, I've learned are too many hundreds of years old, maybe thousands of years old. And, and they still hold, they, they still hold true today. And

[B]

so that's, so I, I love that insight and, and the, the metaphor, actually, the, in the difference between sort of a broken and unfinished system because we, we have frameworks, we have Cochrane reviews and we have review cyst, we have reporting guidelines, and there are things we can start applying to evolving information technologies. Better. Right. And I think that the vision I have, you know, when I've heard you say that was like, I imagine somebody coming in, you know, watching a civilization trying to build a canal system, right, for an irrigation system, going, this is completely broken. Those fields are never getting, not irrigated. Let's ditch the whole thing and start from scratch. And you're like, no, man, this is going to take another, like three generations to get right. Let's keep, you know, keep digging. And I, that distinction, I think, is super interesting. It just reminds me of an old principle of, you know, the only way to develop a sophisticated, a sophisticated system that works is to start with a simpler system that works.

[C]

Absolutely. Your, your minimal minimum viable product.

[B]

Yeah, Minimal viable.

[C]

Right. Viable being, keyword. And if I could say something, the most important thing that we could do in terms of research and health knowledge is bring all the research methods that we have learned into AI and have AI operate by them. But to do that, we can't just have technicians building AI and presenting it. What we need is, we need that to come in from the beginning so that it's built according to standards, so that the errors are clearly understood, the errors and the biases that we have today, and not building on top of error, but making it work as a minimal viable product and then building on top of streams.

[B]

So, Amy, you're the editor in chief of the journal Participatory Medicine and are deeply involved in the work of that society. If somebody is listening to this, wants to contribute, a patient, a clinician or researcher, what do you want them to know?

[C]

I want them to know that we're a collaborative journal and that we are not here to reject your paper or criticize it or shame you. We are here to work with you to help make your paper better. And we welcome your work. We welcome working together. That will make us a better journal. It will make me a better editor. It will make our reviewers better. And if you're unfunded, we have programs for that. If you need a training, we also can help. So if you're, you have a, A, you're under a PI, for example, and they've used all the research money and there's nothing for your research, consider us.

[B]

So, Amy, it sounds like you guys are applying co production principles in practice, right? To your journal. Like it's co production all the way down. I love it. So. So last question. You started out as a patient who was so injured that she couldn't open a door for herself. What doors are you leaving open for others today?

[C]

Every door I can. My lifetime is not going to be forever. And the legacy is the open doors, the open doors that we leave behind and help other people come through them. And, you know, being a mentor isn't solving all the problems. Being a mentor is opening that door to that dark key that we might be afraid to go through and giving people the tools and then once they have the tools to help them, but help them as little as possible so it becomes their tool. They own it and they own the future. And the future they build is going to be different, maybe even than the future that I imagined. But it's going to be a beautiful future if it's built on those kinds of tools where we put humanity first, we do it with intelligence and we recognized and special gifts and, and intelligence in every human being. And stay open, stay fun, be curious and yeah, join the party. We're celebrating. I love it.

[B]

I love ending on a party note, man. Let's do it. So, Amy, it was just a delight to have you and thank you so much for sharing your deeply personal and really inspiring stories. This was Amy Price, editor in chief of the Journal of Participatory Medicine, a researcher at Dartmouth Health and proof that a patient perspective isn't the nice to have. It's the missing piece. So thanks for being with us, Amy. And I want to thank our audience for joining us and hope to see you all again next week for another exciting episode of Practical AI in Healthcare.

[A]

Thank you for joining us this week on Practical AI in Healthcare. If you're ready to go beyond buzzwords and hype and explore how AI is truly transforming healthcare, stay tuned for more conversations that get us to what works. Until next time, stay pract.