Loading summary
Transcript96 lines
- [00:03]
Dena Temple West
From Recorded Future News and prx, this is Click here. When Kevin Menes was a kid, he used to watch this show called Emergency. It was super popular in the 1970s.
- [00:23]
Kevin Menes
Rampart this squad.
- [00:24]
Dena Temple West
51.
- [00:25]
Kevin Menes
Go ahead, 51.
- [00:27]
Dena Temple West
It followed a team of paramedics based out of Station 51 in Los Angeles and their work alongside doctors at a hospital there. It was high stakes, full of flashing lights and split second decisions. But for Kevin, one part of the show always stood out, how calm the emergency room doctors seemed to be.
- [00:47]
Kevin Menes
They'd always bring a victim in and there was a doctor who would go out to the ambulance bay and kind of meet the patients. And I thought, wow, that's kind of cool. That's what I wanted to do if I ever became an ER doctor.
- [00:58]
Dena Temple West
Kevin Menis did grow up to be an ER doctor. He trained in one of the toughest hospitals in the country in Detroit, and eventually became the attending in charge at Sunrise Hospital in Las Vegas. And like those doctors he saw in Emergency, he developed a kind of calm amid the chaos that came into the ER every day.
- [01:18]
Kevin Menes
You wouldn't make it through training if every heart attack that came in you, you freaked out and went into the corner, curled up and cried, right? In general, what scares most people doesn't scare us. And so, you know, cases that in training were initially difficult eventually became very easy and almost routine. And so I started looking at other things that I needed to prepare for.
- [01:47]
Dena Temple West
Things like a mass casualty event or something like what happened one night in the ER and an armed man burst in.
- [01:55]
Kevin Menes
And I remember as a first year ER resident, realizing I was not prepared to mentally handle something like this, in that I didn't know what to do from then. It made me sort of contemplate and think of a plan that I would do if something like that occurred.
- [02:18]
Dena Temple West
Of course, there's a difference between abstract contemplation and being thrown into the middle of something like. Kevin was on the night of October 1, 2017. Las Vegas is on lockdown after a gunman opened fire at a popular outdoor country music festival. That night, Kevin was doing the rounds in the ER in Las Vegas when a nurse motioned him over and told him she'd just gotten an urgent call.
- [02:43]
Kevin Menes
Over the radio that, you know, there was a mass shooting and that victims were coming our direction.
- [02:50]
Dena Temple West
As he waited for the ambulances to arrive, Kevin became aware of the full weight of what was coming his way. Ambulance sirens are the soundtrack of an er, and he says they have a familiar cadence to them. Their sirens blare and pause at each stoplight, then blare and pause.
- [03:09]
Kevin Menes
But that night when I was out there in the ambulance bay listening, it was different because you could hear the siren coming towards us, but it wasn't. The volume wasn't holding steady, which told me that they weren't waiting at the stoplights, they were running them. And that was a little bit unnerving. But what really started to make me realize what we were about to deal with was I'd start to hear this, you know, the high and the low sound of the siren at the same time.
- [03:44]
Dena Temple West
Because there were so many of them.
- [03:46]
Kevin Menes
Yeah, because there was so many of them.
- [03:48]
Dena Temple West
Wow. Had you ever responded to or witnessed something at the Las Vegas scale before?
- [03:58]
Kevin Menes
No one has ever witnessed anything at that scale. This is the scariest thing you can ever handle or deal with. You can talk about it, you can prepare for it, but you really only get one chance at it. Even though you think you could potentially handle it, there's no practicing this.
- [04:20]
Dena Temple West
And what happened that night, the way Kevin and his team responded ended up sparking an idea. An idea that would travel over 2,000 miles to Washington D.C. to a little known researcher at a lab inside the Department of Defense.
- [04:35]
Matt Turek
I'm Matt Turek. I'm the deputy Office director for the Information Innovation office at darpa.
- [04:41]
Dena Temple West
Darpa? The Defense Advanced Research Projects Agency. Essentially the R and D arm of the Defense Department, responsible for some of the most ambitious and futuristic military research in the world.
- [04:53]
Matt Turek
DARPA was founded with the mission of creating strategic surprise for national security.
- [05:00]
Dena Temple West
What brought these two together could change not just the way emergency rooms around the country operate, but could change how the military deals with wounded on the battlefield as well.
- [05:09]
Unknown
Foreign.
- [05:17]
Dena Temple West
I'm Dena Temple west and this is Click here. A podcast about all things cyber and intelligence. We tell true stories about the people making and breaking our digital world. In mass casualty situations, people like Kevin Menes make life and death decisions under intense mental and emotional pressure. Could doctors and medics make even better decisions with a little help from artificial intelligence?
- [05:43]
Matt Turek
Imagine yourself in Dr. Menes position where you're making life altering decisions and a succession of them. It just sort of grounds how impactful and challenging that is for the decision maker who bears the responsibility for all those decisions.
- [06:03]
Dena Temple West
Stay with us. Click Here is brought to you by Progressive Insurance. Do you ever find yourself playing the budgeting game? Well, with the name your price tool from Progressive, you can find options that fit your budget and potentially lower your bills. Try it@progressive.com Progressive Casualty Insurance Company and affiliates Price and Coverage match limited by state law. Not available in all states from recorded future news. I'm Dena Temple west and this is Click here. As soon as Dr. Kevin Menes got word of the mass shooting in Las Vegas, his training kicked in.
- [07:02]
Kevin Menes
I didn't really have much emotion. I think I was on autopilot at that point.
- [07:07]
Dena Temple West
He called down to the secretaries, open up every operating room, get every available scrub tech, every nurse, every anesthesiologist, every surgeon, and get them down to the ER now. Then he and his team rolled every empty bed, every wheelchair out to the ambulance bay and they waited.
- [07:28]
Kevin Menes
And then when I got that first look into the car, that's when I knew the scale that we were about to deal with was something that, you know, was nightmarish and just bigger than anything you could even imagine.
- [07:46]
Dena Temple West
The victims started streaming in one after another. Hundreds of people would come through his ER that night. And a situation like this is as much about treating patients as it is about logistics, because decisions about where to put staff and stage equipment are a matter of life and death. In this kind of emergency, it's a race against time. Because when you have the resources for one trauma bay, but two patients show up, you have to make a decision because you have a line for life saving treatment.
- [08:16]
Kevin Menes
You know, you turn that to three, you turn it to four, and you now you have patients who are waiting in the line, but they're dying while waiting.
- [08:25]
Dena Temple West
You have to decide fast who gets treated, who waits, who might not make it at all. Fortunately for Kevin, he's not the first person to have to make these kinds of choices. There's actually a system for this, a system that dates back centuries.
- [08:39]
Kevin Menes
This is a system back from Napoleon's time period. And that's the mass casualty triage system taught, you know, all over the world.
- [08:48]
Dena Temple West
It's a system of flags, essentially, that indicate how serious a victim's injuries are.
- [08:53]
Kevin Menes
Red is you're going to die unless we do something about this. Yellow. We have a bit more time, but you do need medical care. And then green is this person will not die from their wound.
- [09:06]
Dena Temple West
And one last flag. Black.
- [09:09]
Kevin Menes
This patient is dead already. So it's black, red, yellow, yellow, and green.
- [09:16]
Dena Temple West
For hours. As the victims poured in, Kevin tagged patient after patient.
- [09:21]
Kevin Menes
Red, yellow, and green. Yellow, green, red, red, red.
- [09:25]
Dena Temple West
More victims, more judgment calls.
- [09:27]
Kevin Menes
Yellow, red, red, green.
- [09:29]
Dena Temple West
Until finally, at 5am, his shift was over and he finally went home and fell into bed.
- [09:37]
Kevin Menes
When I woke up the next day, I had never been that sore in my entire life because I had ran I didn't run. I sprinted from patient to patient for the, you know, seven, eight hours that we were working.
- [09:52]
Dena Temple West
He starts replaying everything in his head.
- [09:54]
Kevin Menes
How could I have done this better? Where was I slow? And how could I have sped this up? What did I not recognize? What did I do right?
- [10:03]
Dena Temple West
There were a few things he'd changed, but overall, he felt really good about what they had done the night before. And others seemed to agree. In fact, a doctor he trained under nudged him and said, you should really write an article about this.
- [10:16]
Kevin Menes
I'm not one who seeks notoriety. I don't want any of that. But she guilted me in saying that if I didn't share this information, you know, somebody could learn from you.
- [10:30]
Dena Temple West
The journal article came out about a month later, and he was surprised to find out that it wasn't just ER doctors who could learn from him.
- [10:38]
Kevin Menes
When DARPA reached out to me, I was completely out of the blue. And I never would have thought that a researcher working on AI would think that this is something that could have.
- [10:54]
Dena Temple West
Helped DARPA remember it's part of the Department of Defense, which, of course, also deals with mass casualty events. So as Matt Turek explained the kinds of problems he was wrestling with at darpa, the connection started to make sen. Here's Matt.
- [11:11]
Matt Turek
So things like battlefield triage, for instance, even exquisitely trained trauma surgeons presented with the same information might take different courses of action.
- [11:23]
Dena Temple West
Matt Turek didn't start out his career thinking about this kind of thing. His primary interest as a kid was just straight up traditional tech.
- [11:32]
Matt Turek
I was for sure one of those nerdy kids in school and did interesting things like take television sets apart to try and turn them into other things.
- [11:43]
Dena Temple West
That's good. Did you ever successfully turn it into something else?
- [11:47]
Matt Turek
A pile of parts?
- [11:49]
Dena Temple West
But as Matt grew up and started working in tech, he always kept his ear to the ground for what was next, which meant he was thinking about artificial intelligence long before the rest of us.
- [12:01]
Matt Turek
I started my career in the mid-90s, and that was really the time when I remember hearing terms like artificial intelligence.
- [12:10]
Dena Temple West
And by the time he made his way to DARPA in 2018, he wasn't just thinking about the technical aspects of what AI could do, but also the philosophical quandaries that come with it.
- [12:22]
Matt Turek
I really wanted to start a computational ethics program. So, not like, what is the ethics of an AI system, but how do we build AI systems that can capably make these ethical decisions?
- [12:37]
Dena Temple West
Decisions like, should I give this victim a red flag or a yellow flag? Those aren't straightforward technical questions.
- [12:44]
Matt Turek
You know, there are domains where a lot of the decisions don't necessarily have right answers.
- [12:53]
Dena Temple West
He wondered, could AI make those kinds of fuzzy decisions easier? Would removing the human component and turning to a machine not weighed down by fatigue and emotion make better decisions in life and death situations?
- [13:07]
Matt Turek
Having AI systems that can reason about things and do that in a way that's consistent with the humans that are using the system seemed to me like a really important capability to have, particularly as we are making more and more capable AI systems and then potentially giving them more decision authority.
- [13:31]
Dena Temple West
That's when we come back. Stay with us.
- [13:46]
Unknown
You come to the New Yorker Radio Hour for conversations that go deeper with people you really want to hear from, whether it's Bruce Springsteen or Quentin Westlove or Olivia Rodrigo, Liz Cheney or the godfather of artificial intelligence, Jeffrey Hinton, or some of my extraordinarily well informed colleagues at the New Yorker. So join us every week on the New Yorker Radio Hour wherever you listen.
- [14:09]
Dena Temple West
To podcasts from recorded future news. This is Click here. I'm Dena Templerest. By the time Matt Turek reached out to Kevin Menace, he'd been put in charge of a new DARPA program to develop AI that could one day independently make decisions, the kind of decisions that doctors make. The idea to teach AI how to make life and death decisions under pressure, like during a mass casualty event or battlefield triage. And DARPA is hoping the ITM program will allow them to encode human attributes and characteristics into the algorithms themselves. In a sense, they want to make AI more human. And Matt was particularly interested in triage because at its core, it combines the kinds of logistics and decision making that AI is already really good at. But it also involves something that so far has eluded machine learning, complex human judgment. And in situations like Las Vegas or an active battlefield, when you have multiple casualties right there in front of you, an old ethical dilemma crops up, one that explains why you don't want to just hand off decisions to a computer in situations like this. Philosophers call it the trolley problem. You're on a trolley going down the track, there's a fork, and one path leads to five people tied to the rails. The other path, there's just one person tied to the rails. And you have to decide, kill one or kill five. What would you do? Why don't you just tell me the right answer? Well, that's what's so great about the trolley problem, is that there is no right answer. This is why everyone hates moral philosophy professors. What if those five were wounded so badly they wouldn't make it anyway. What if the one person is a murderer? How do you teach an AI system to juggle that kind of decision to make those kinds of trade offs? There isn't a one size fits all answer. Some situations require humans to break protocol. Which it turns out, is exactly what caught Matt Turek's eye when he started reading Kevin Menes article about the Las Vegas shooting. There were moments when Kevin broke protocol. Like as the night wore on, a nurse came out screaming for Kevin to get inside to help because the doctors were falling behind.
- [16:42]
Matt Turek
So he literally turned to a nurse who'd been working alongside him in the parking lot and said, you see what I've been doing? Make sure people go to the right places. And that also contravenes typical triage policy, which is that the most senior physician, the most experienced one, should be the one making the trauma decisions.
- [17:03]
Dena Temple West
How does AI learn to do that? And then there was the matter of the black flag, the one put on patients who have already died. Kevin Menes broke protocol with regard to that too, and I asked him about it. I read somewhere that you didn't black tag anyone because you wanted sort of a second opinion that someone was definitely dead, is that right?
- [17:21]
Kevin Menes
That's correct.
- [17:26]
Dena Temple West
Ambulances and cars from the music festival were lining up and he had to look at each victim coming in and decide what color to assign to them. So when in doubt, he didn't want to assume a patient was dead.
- [17:39]
Kevin Menes
You know, safety net was to send him back and then have another doc look at him and make sure that the patient didn't have pulse.
- [17:48]
Dena Temple West
But looking back on it, he says he probably should have made a different call.
- [17:52]
Kevin Menes
That first card, I should have blacked tag all five right in the beginning. And there were a couple more patients that had come in and I should have black tagged them.
- [18:06]
Dena Temple West
He said as much in his article and explained why he decided to go against policy and convention. That night, he was face to face with a trolley problem. On the one hand, his not black flagging people made sense. You want to be safe. On the other hand, every choice is a trade off when resources are limited. So sending already dead patients into the hospital bay means doctors can't focus on the people they can actually save. And it's this, the human struggle, the second guessing, the improvisation that AI can't do yet. So implementing AI in triage is still a long way off. For now, Turek's team is doing extensive testing to see how it might work. The transport has arrived and they have room for three Patients, who do you want to send? They built a VR simulation with real war zone scenarios and they're bringing in emergency doctors and medics to have them play out these situations similar to ones they've encountered in real life. And they put on a pair of VR goggles and become immersed in the scenario.
- [19:11]
Matt Turek
There was one environment that was reminiscent of some of the places that the US military has been deployed in the past, in the Middle east or other places. And then the way the scenarios unfold, they are carefully designed to present some of these forced choices to the decision makers. And certainly there were folks on the program team who've been in operational settings where it's like, oh, that took me back.
- [19:39]
Dena Temple West
In these scenarios, a kind of trolley problem typically unfolds.
- [19:42]
Matt Turek
You've got two victims in front of you, they have similar injury patterns. You only have resources to treat one of them. Who do you treat? You now find out, well, one of them was the perpetrator, the other was the victim. Does that change your decision?
- [20:00]
Dena Temple West
As the expert navigates the virtual scene, they're asked to explain what they're seeing and how they would react so AI can learn from them.
- [20:10]
Matt Turek
There are some surveys that we use to understand the individuals and also potentially to go back afterwards and understand what some of the rationales were, what comes.
- [20:22]
Dena Temple West
Out on the other end, whether it's an app on an iPad at the hospital, wearables, something like a robot that's still on the drawing board. But if they can sort the details out. Matt's hope is that the systems could, in a sense, extend the medical team's decision making abilities to amplify what they.
- [20:40]
Matt Turek
Can do today if programs like ITM are successful. You could imagine if an AI system was doing triage in Dr. Menez's place, that that AI system could know the status of all the operating rooms, of all the supplies of all the medicine, of all the blood, for instance, that are available. That's not going to fit in one person's head, but it could fit in the span of awareness of a AI system. It might have situational awareness about the entirety of. And so an AI system might find solutions for how to treat an inrush of patients that a human would agree with, but might not be able to figure out themselves.
- [21:32]
Dena Temple West
But of course, even if they can get the technology to work, it's a whole other thing to get doctors to buy into it. Maybe the hardest thing in medicine.
- [21:42]
Kevin Menes
We tend to be very suspicious of anything new. I think the medical legal responsibility that we as physicians feel terrifies physicians enough to be cautious about taking on new technology or new techniques and procedures.
- [21:59]
Dena Temple West
Would you trust it?
- [22:01]
Kevin Menes
Me? Yeah. If it is what I envision, yes. I think that the technology can keep up and eventually do something to this, you know, that could mimic the decisions that we had done that night. But again, in the current form of the way that ChatGPT just, you know, sifts the Internet for information and then compiles it in a paragraph, I think those type of algorithms are, they're, they're step forwards from what we had before. But in order to make a difficult decision on the fly, it's a completely different, more complex. It's much more complex than that.
- [22:49]
Dena Temple West
And there it is. A possible future of medicine with a technology that promises precision but requires a great deal of trust. For now, physicians like Kevin are cautious and understandably so, because an algorithmic misstep isn't just a bug, it's a life. Would you trust it? This is qlik here.
- [23:23]
Unknown
If you're looking for a daily guide to cybersecurity news and policy, sign up for the Cyber Daily from Recorded Future News. It serves up today's most interesting and important cyber stories from our sister publication the Record, and then aggregates all of the big cyber stories you might have missed from news outlets around the world. Just go to the Record Media and click on Cyber Daily to get all you need to know about the world of cybersecurity right in your inbox.
- [23:52]
Dena Temple West
Today is Tuesday, February 18th, and here are some of the top cyber and intelligence stories of the past week.
- [24:03]
Kevin Menes
I feel like the luckiest man on earth right now.
- [24:06]
Dena Temple West
I want you to know that I am not a hero in this at all.
- [24:10]
Kevin Menes
And President Trump is a hero.
- [24:15]
Dena Temple West
That's American history teacher Mark Fogel at the White House. He was released from a Russian prison last week in a prisoner swap exchanged for Alexander Vinnick, a Russian national who had been in custody in California on charges related to BTCE, the cryptocurrency exchange he operated. U.S. officials said Vinick was a money launderer for cybercriminals and helped some of the world's most prolific ransomware gangs wash some $9 billion in ill gotten gains. He was arrested in 2017 in Greece at the request of the US on cryptocurrency fraud charges and he was later extradited to the United States where he pleaded guilty last year to conspiracy to commit money laundering. Fogle, the wrongfully detained American teacher was picked up at an airport in Russia for having a small amount of cannabis in his bag. It had been prescribed by a doctor for his bad back. TikTok is back on the App Store this is not a drill Last week, Attorney General Pam Bondi provided a letter that paved the way for Google and Apple to put the Chinese video app back in their app stores. Both companies took the app off their online store doors after a ban on TikTok took effect in the US on January 19. The Bondi letter essentially told Apple and Google that they would face no legal liability if they put the app back on their sites. It's the latest twist in a very complicated TikTok story. Congress passed a law last year that mandated that TikTok must be sold to a non Chinese owner or face a ban in the United States. United States lawmakers said it needed to be done for national security reasons. Since then, President Trump has signed an executive order that delays enforcement of the law. Right now, ByteDance, the app's mainland Chinese owner, has until early April to find a buyer. Treasury Secretary Scott Besant, who leads an interagency panel that reviews deals that have national security implications, is thought to be working with Trump himself to vet potential buyers. And last week, new news on Russia's notorious Sandworm hacking unit. They're the state hackers who may be best known for their attacks on Ukraine and their release of the destructive NotPetya malware. According to a new report from Microsoft, a hacking group within Sandworm is now taking aim at western targets, including Australia, Canada, the U.K. and the U.S. the group, which Microsoft calls Bad Pilot, specializes in initial access operations. They basically break into networks and then hand over that access to Sandworm hackers. And finally, well, first of all, some.
- [27:05]
Matt Turek
Of the things that I say will be incorrect and should be corrected.
- [27:09]
Dena Temple West
That's Elon Musk in his unusual half hour long Q and A with a press in the Oval Office, and he said that his Doge team started publishing information about his activities on a simple website last week. Only problem is, it turns out the Doge site is pretty easy to break into. Two web developers working independently found that it's possible to push Updates to the doge.gov domain. According to 404Media, the website uses a database that can be edited by anyone online. The two experts who broke in left a couple of messages on the site to prove their point. This is a joke of a.gov site. One read and the other says these experts left their database open. The messages stayed on the website for at least 12 hours and remained visible for some time on Friday. Maybe that's one of the things that Elon Musk will be corrected. Today's episode was produced by Sean Powers, Megan Dietry, Erica Gaeda, Zach Hirsch, and me, Dina Templewreston. It was edited by Karen Duffin, Fact Checked by Darren Ancrum, and contains original music by Ben Livingston with some other music from Blue Dot sessions. Our staff writer is Lucas Riley and our illustrator is Megan Goff. Click Here is a production of Recorded Future News and prx. Tune in on Friday for Mic Drop, which features our favorite interview of the week. We'll see you then.
- [29:09]
Unknown
Looking for more of the cybersecurity and intelligence coverage you get on Click Here, Then check out our sister publication the Record from Recorded Future News. You'll get breaking cyber news from reporters in New York, Washington, London, and Kyiv, among others, and you'll see for yourself why it attracts hundreds of thousands of page views every month. Just go to the Record Media.