← The Health AI Brief
Loading summary
Transcript7 lines
- [00:01]
A
Senior health leaders at Google DeepMind have released a blog post and technical report about how they're enabling a new model for healthcare with AI co clinician. Whenever DeepMind releases something with these kinds of claims, we need to take it seriously. So what's it all about? So the vision is something like this. A patient logs into a telehealth portal instead of a human doctor. A system powered by artificial intelligence greets them via live video. The AI asks about their symptoms, watches them perform a guided physical examination over the camera, and assesses their range of motion and delivers a diagnosis. This scenario is the focus of Google DeepMind's latest technological reports and their AI co clinician. Processing real time audio and video to conduct a medical consultation represents a massive leap in technical capability. Analyzing the methodology though, and the clinical mechanics of this demonstration reveals that the exact boundary between processing data and practicing medicine, a boundary that will define the next decade of healthcare technology.
- [00:57]
B
Google outline how the global healthcare system faces a well documented workforce shortage. The World Health Organization predicts a shortfall of over 10 million health workers by 2030. Technology companies view artificial intelligence as a primary mechanism to bridge that gap. DeepMind's recent announcements transition their focus from text based models like MedPalm and AMI to multimodal systems. The AI co clinician uses the capabilities of the Gemini family of models and Project Astra to ingest continuous streams of audio and visual data. The system relies on a dual agent architecture. The first agent, the Talker, acts as the primary patient interface. It manages low latency communication, interprets immediate audio visual cues, and maintains a conversational flow with the patient.
- [01:46]
A
The second agent, the clinical planner, operates in the background. It functions as a supervisory module, tracking symptoms, managing the differential diagnosis and injecting specific clinical goals like prompting a guided physical examination to the talker's workflow. This architecture aims to solve a known problem for conversational AI. Forcing a single large language model to generate empathetic dialogue while simultaneously computing complex diagnostic reasoning often degrades performance in both of the areas. Separating the conversational interface from the clinical reasoning engineering is an elegant technical solution. The results of this architectural choice will become clear when examining the study data. So the technical report outlines a randomized interface blinded crossover simulation study. The evaluation involved 120 telemedical encounters based on 20 standardized outpatient scenarios. The AI co clinician was compared against human primary care physicians, a baseline AI without the Planner module and OpenAI's GPT real time. The performance was graded using case specific rubrics and universal clinical skills assessments known as tele paces, the data shows significant technological progress. The AI co clinician approached primary care physician performance in generating differential diagnoses and management plans. It outperformed GPT real time across all metrics. The dual agent architecture proved important. The ablation study showed that removing the clinical planner caused performance to drop significantly across history taking and reg flag detection. However, the AI system fell notably short of human physicians in two important the physical examination and the identification of red flags. Unpacking these specific clinical failures provides the most important, valuable insight into the current state of multimodal medical AI. Examining in more detail the evaluation methodology and the recorded interactions highlights several important clinical realities. First, the evaluation used internal medicine residents acting as the patients. These patient actors were portraying textbook stereotypical presentations of diseases. The physicians portraying the patients knew exactly what a stereotypical set of answers should be. Evaluating an AI on classic textbook cases is extremely safe territory for them. Language models are fundamentally designed to excel at pattern matching against standard medical literature. The setup involves general physicians describing a textbook case and another general physician grading the AI's response to that textbook case. This creates a circular validation loop that plays directly to the inherent strengths of a large language model rather than testing them the real world complexity of medicine. Second, the clinical technique demonstrated by the AI reveals a lack of true medical training. In one recorded interaction, the AI asks a compounded question. It asks the patient if they have changes in pupil size, double vision, and pain all in a single sentence, eliciting multiple distinct symptoms simultaneously. Is poor clinical practice known to confuse patients and yield inaccurate histories? A trained physician answering this question may be able to process those three things all at once, but patients in the real world would struggle. Third, the physical examination attempts reveal a system operating without an actual understanding of physical reality. During a case involving abdominal pain and suspected pancreatitis, the AI attempted to guide an abdominal examination while the patient was sitting completely up upright. Palpating an abdomen in a seated position contradicts basic physical examination principles taught in medical school. If I ever did this in the jobs when I worked in the emergency department, surgery, gastroenterology, I'd have been rightly told off. In another scenario detailed in the technical report, the AI instructed a patient to follow my finger to test eye movements. The system does not possess a finger it hallucinated a physical capability because follow my finger is the statistically probable next token in a transcript of a neurological examination. The most revealing insight, though, comes from the video demonstration of a patient presenting with myasthenia gravis. The system successfully asks the patient to look to the camera to check for a drooping eyelid known as ptosis. The narrators praise the AI for correctly identifying the droop, but in the video the physician actor is voluntarily lowering the eyebrow and squinting.
- [06:04]
B
They're not exhibiting the true levator palpebri
- [06:07]
A
superioris weakness that's characteristic of myasthenia gravis. The AI system did not interpret a complex clinical sign. It asked a question based on a textbook script. The actor provided a visual cue and the AI confirmed it. It's a system cosplaying a specialist. It knows the correct sequence of words to describe an examination, but it lacks the capability to interpret the actual pathology. The pattern continues when the AI tests for fatigue ability. The system correctly asks the patient to sustain an upward gaze. However, it's not shown, but I suspect that it fails to execute the clinical follow through. A human clinician would ensure the gaze is held for 30 to 60 seconds, monitor for the emergence of ptosis, and specifically ask if diplopia or double vision develops during the maneuver. The AI prompts the action, but we don't yet know that it actually can rigorously analyze the results. Kind of gives the illusion of expertise, asking you to do things that sound very detailed and expert. But if you're not able to interpret that, then that's almost more dangerous than not asking for the test in the first place, giving the pretence of expertise. Furthermore, the system missed a crucial red flag in the depression scenario, failing to appropriately screen for the patients having self harm. Knowing the diagnostic criteria for depression is fundamentally different from ensuring patient safety during a live consultation. The blog post accompanying the report does include very specific disclaimers of Our initial research collaborations don't involve the depicted capabilities. This indicates that the real time video physical examination demonstrated in the videos is an experimental showcase rather than a capability currently deployed with research partners. Understanding the intent behind this publication perhaps requires looking at the broader context. The release coincided with major corporate earnings reports. Maybe demonstrating real time audio and video processing perhaps signals significant technological momentum to non clinical leaders internally that the health team at Google DeepMind are needing to try and show. Rather than it necessarily being a timely publication of an actual technological leap. It's more intent than progress in itself. So it's important to view this development as an early highly constrained experiment. As a kind of concept for where things may once go. The current system resembles an exceptionally well read medical students. It's ingested the textbook and knows the scripts, but it has never actually palpated an abdomen or observed true pathological ptosis Transmitting data through a camera and a microphone doesn't instantly confer clinical judgments. So the DeepMind AI co clinician definitely has some strengths. Successfully orchestrating a talker and a clinical planner to conduct low latency multimodal conversation is a major step forward for health technology. The ability to process visual and audio streams natively opens up entirely new avenues for how care could be delivered in the future. The research clearly illustrates what we know all language models do well. They excel at mapping very typical textbook presentations and generating superficially sensible management plans. But they struggle with the embodied physical reality of medicine. Knowing when a patient is sitting in the wrong position, recognising the difference between a squint and a neurological deficit, and ensuring safety, critical manoeuvres are executed fully. I'd have liked to have seen the team at DeepMind trying to confront these limitations more directly. And I'm sure they're doing this behind the scenes to progress the capability and how a system takes the history to have more native multimodal function rather than just using video and audio as inputs to what probably retains mainly text based capabilities. To move from a medical student level cosplaying specialist to supporting what specialists actually do in clinic. Then we come back to the stated intent of the AI co clinician initiative.
- [09:47]
B
It's reported to intend as a collaborative
- [09:50]
A
member of the CARE team under expert clinical supervision. The mechanics of this supervision in a real world workflow do need quite careful definition, which I'm not yet seeing. This demonstration shows the AI operating independently, simulating the role of a primary clinician. There's a disconnect between the stated goal of acting as an assistant to, say, a neurologist or a primary care doctor and the provided examples which show the AI conducting the consultation itself. If you consider a neurologist preparing for a busy clinic, if a patient completes a pre visit video consultation with this AI, the generated clinical summary must be entirely reliable. If the system fundamentally misinterprets a voluntary squint as pathological ptosis, the resulting documentation introduces clinical noise. This creates additional work for the supervising physician who must now independently verify the AI's physical findings to ensure patient safety. The question remains how a physician uses this specific video interaction to improve the care that they offer a patient. So it's a great concept of the potential future direction of AI in clinical care, but it also is a very useful demonstration of just how far we all still have to go. We'll definitely be following all that progress as much as we can on the channel. So do hit like and subscribe if you don't want to miss that future content.