Loading summary
Transcript28 lines
- [00:05]
A
Hey, everyone, and welcome along to Seriously Risky Biz. This is our podcast all about cyber security policy and intelligence. My name's Amberly Jack and in just a moment I'll bring in Tom Uren, our policy and intelligence editor, and we're going to have a chat all about the Seriously Risky Business newsletter that he has written up and published on our website this week. You can of course read that and subscribe over at Risky. But first I'd like to thank our sponsor for this week, which is Socket. And on our Risky Bulletin feed this week, Casey Ellis has done a sponsored interview with Socket founder for us to book a dj. So you can search for that on the Risky Bulletin feed. Tom, great to see you.
- [00:45]
B
G', day, Amberly. How are you?
- [00:46]
A
Yeah, really good, thanks. Great newsletter this morning and I loved your first piece. Want to dive straight into that one? We've got a bit of a difference of opinion, I guess you could say, between Anthropic and Pete Hegseth, and words have been said, ultimatums have been given. And as you've written in the piece this morning, Tom, this all kind of begs the question of who should actually be making the rules when it comes to military use of AI. So why don't we sort of start at the beginning there and tell me what's happened this week that kind of made you start thinking along those lines?
- [01:21]
B
Yeah. So the news this week, there have been several articles that have come out about the Argy bargy between the Secretary of Defence, Pete Hegseth, and Anthropic. So the backstory is that Anthropic's Claude models have been the first ones that have been allowed or authorised or approved for defence classified uses. And apparently Claude was used in some role in the raid on Maduro. Hegseth has in the past, well, as recently as a month ago, really wanted to accelerate the AI adoption by the Department of War and has, you know, railed against, basically, you know, DEI aware AI, and we're not going to have AI that doesn't want to fight war. And he has basically issued an ultimatum to Anthropic saying, we want you to drop all the restrictions you have and it'll be up to us how we deploy, how we use your technology. Now, Dario Amade, who is Anthropic's CEO, has said, look, removing restrictions is fine, except we won't allow our models to be used for the mass surveillance of Americans and we won't allow our models to be used for lethal autonomous weapons. I totally get Hegseth's position Like what you do in a Defense Department is sometimes you blow things up and kill people. That's not the way you want normal AI models to be trained. So you've got to get those safeguards out of those systems. At the same time, what Amada is saying is like, like that actually seems pretty reasonable. Right.
- [03:19]
A
I was going to say it's a, it's a really tricky one because when you, when you look at it and you're like, well, yeah, the Department of War, clearly they want to do some war at some point. And then, and that's the point and Anthropic's point you can kind of see as well. So it seems like a really unwinnable argument almost.
- [03:38]
B
I think it comes down to what you think CLAUDE is. So the Pentagon, their view, Claude is a tool and we'll use it how we like. Yeah, Anthropic. When you look at their material, they view CLAUDE as like an entity, an actual sort of maybe thinking being. And it's not an, it's not a really necessarily well behaved entity unless you give it the right training. So like just the last month CLAUDE came out with this, Anthropic came out with this Claude constitution, which is a very long document about how CLAUDE should behave, how it's expected to behave, what it's trying to do. And the idea is to get Claude to do what a person would expect it to actually do. Like to be genuinely helpful rather than helpful in a very narrow way. That's kind of counterproductive. And that Constitution's not just a set of words on their website. It's actually used in Claude's training. So it's incorporated into its behavior. So their view is that, well, you can't just have an AI that is totally valueless because it will just make random decisions. So I think that my view is that what the Department of Defense or war really wants is not Claude with claude's standard constitution, but Claude with like a warrior ethos. And in, in a serious way that is like what are the right trade offs to make in a warlike situation? Or when you're the Department of Defense, like how do you value, say American military lives versus enemy combatants versus, you know, civilians versus collateral damage? These are decisions that you're not going to, or trade offs that you're not going to train mainline CLAUDE to make because that's not what it should be doing. But I think that's a very valid thing for military AI to be considering.
- [05:42]
A
Do you have any sort of idea of what this warrior ethos is going to look like over time?
- [05:48]
B
I've got no idea. And I think no one, maybe people have some ideas.
- [05:53]
A
I think to use a commonly AI used phrase, it's a rapidly evolving landscape.
- [06:01]
B
That's right. Yeah. And so I think that there is a strong argument that neither anthropic nor the Department of Defense should be laying out what the actual rules of or the limits of AI use are, but that Congress should be doing that. Like that's the role of Congress is to like make these sorts of decisions. And I think there's lots of gray areas where like for example, the use of autonomous weapons that are lethal, you can quite quickly come up with scenarios where the speed of an attack means that you need like an automated response like hypersonic weapons. Coming at a Navy vessel, for example, are you just going to go, no, we must have a human to press the button when within seconds your ship will be destroyed. I think there are difficult trade offs where there's no clearly right answer. So I think there's role for Congress, but I think there will be limited cases where you're able to say this is clearly right and this is clearly wrong. Now I think the mass surveillance of Americans, I think that's probably something everyone could get behind as a. AI models should not do this or government AI models should not do this. Like that seems like one case where it's pretty cut and dried. But I mean even that case, I think you can argue that when does it become mass surveillance? What about particular types of criminals? It's complicated, but I think that because it is a space that changes and because you need some guiding principles that will be trained into these AIs, I think that's the place that Congress should be getting involved is what are the transparency and oversight mechanisms to make sure that not only are AI companies making models that are appropriate for military use, but that that military use is, is consistent with American values.
- [08:04]
A
Yeah.
- [08:04]
B
So I think there's a long road ahead where people experiment, I guess. And it's not just the how the models are trained, but also how they're used by, by the department. I don't think there's like any set. I think there's very few set and forget laws that can be implemented.
- [08:22]
A
Yeah. I liked how you sort of phrased it in the newsletter and said, you know, Congress, Congress needs to take a kind of a mentor role here and you know, almost to stop the kids from fighting too much and getting out of hand.
- [08:34]
B
Yeah. So I think that Congress probably does have a very firm idea about what American values are, and that those involve making reasonable trade offs between some of those different questions I talked about. And those are the sorts of things that that should be getting put into models. I think models probably right now are not making those kinds of really significant life and death decisions, but as they get more capable and you trust them to do more and more. Like every single person who joins the US Military is indoctrinated with the values of whatever core they're in, Marines, army, whatever. And so that's part of why you then give them responsibility and trust them to make decisions which are pretty consequential. And so I think it would be silly to say, yeah, we'll give this machine those responsibilities, but we won't tell it what we're actually expecting. That's part of how you train the machine to do what you want it to do, is to, you know, instill those values. I think part of it is that Hegseth just likes, doesn't like the idea of values restraining people. So I think there's in a way a sort of ideological or philosophical difficulty with Claude Constitution or even the idea of training models.
- [09:53]
A
Yeah. I want to move on, Tom, to your second piece this week, which to me immediately feels like rather unsurprising news. Volt Typhoon is Volt Typhooning in the US but this comes after the US Actually kind of declared victory on Vault Tofing. Tell me about what happened there.
- [10:15]
B
Yeah, so Volt Typhoon is the group that's been compromising U.S. critical infrastructure. It's very concerning because it appears to be pre positioning for sabotage in the event, particularly of a conflict around Taiwan. So back in July of last year, a number of officials at a conference, like an NSA official and an FBI official, said basically, we've won Mission accomplished. Vault Typhoon failed. One of the the FBI blokes said, we gave the private sector the tools to root out and find Vault Typhoon. And I didn't really write it about it at the time. I think the, just the timing in the newsletter didn't work out. But at the time I remember being like, really? And thinking back now, this would be the only time a government has ever declared, yes, total victory against a cyber threat. Because the entire history of cyber threats is that they don't go away, they just evolve. And you've got this big picture. We want to be able to take Taiwan. You know, here's a supporting plank, a very small supporting plank, and it's not even the biggest one. But unless the whole idea of potentially taking Taiwan disappears, why would we not keep Trying like, what's the, what's the. Why is it off the cards all of a sudden? And so at the time it struck me as very strange, but I didn't write about it. And yes, lo and behold, Dragos, which is an operational technology or an industrial control system CyberSecurity firm, their 20, 26 year in review, they've said yes, they're still absolutely out there. They're compromising stuff, they're embedding themselves and it's just like, oh, yeah, well, of course, this makes sense.
- [12:10]
A
What was the wording? And then you color us completely surprised or something.
- [12:14]
B
Totally unsurprised. That's a different color, Amberly, the color of unsurprise. And the problem with declaring victory, like it looks good in the short term, but the problem is that most critical infrastructure in the States is in private hands. And the way you get private sector people to do something is that you warn them about the threat and say, it's worth doing something about this threat. It's serious enough to warrant some response. And so if you say we've totally won, everyone's going to go, well, why should I bother doing anything? That threat has been dealt with, let's move on, let's make money instead of making security decisions. I think that that's the problem. And so I'll be interested to see if there's a shift in the communications. There's been nothing about Vault Typhoon, really from government officials since July last year. Since that moment.
- [13:21]
A
Yeah. And it's quite funny because you and I were sort of talking about that as we're reading through the newsletter this morning and sort of saying, you know, as you said, there's been nothing about Volt Typhoon. Salt Typhoon we hear about quite a bit.
- [13:33]
B
Yeah. So Salt Typhoon is the one that compromised the Chinese state backed group, that compromised a whole lot of US telcos and telcos worldwide. And the US government is still talking about Salt Typhoon. So just this week the FBI, an official came out and said, yeah, Salt Typhoon is still around. We encourage you to talk to us so that you know, the companies that have engaged with us have done much better responding to it. So they've kind of said it's a problem and here's an open door and that's how you get cooperation. And so I was going to say
- [14:04]
A
it seems like a fair way to get working.
- [14:07]
B
So it just seems very strange that they've got this Salt Typhoon. Yeah, we're an open door. Still a problem of Vault Typhoon. Nothing to see, no problem. I don't know, it's very weird.
- [14:17]
A
So, Tom, as you've kind of written in the newsletter as well, the strategy with Salt Typhoon seems like a pretty good way to get these organizations and these telcos to cooperate. And that's kind of exactly what should be happening with Volt Typhoon. Do you think they're going to start not declaring victory again now that this has come out?
- [14:38]
B
I think there's a bouncing act, right? You don't want to say Vault Typhoon is the worst thing ever and they've totally owned us and we're wrecked and go ahead and take Taiwan. I don't think you want to say nothing to see here. Everything is perfectly fine. It's Vault Typhoon is around. They're continuing to try. Here's some tools and techniques and we've got guides on how to deal with the threat. A campaign like that is not a one and done. We've won. Move on. It's a constant. It's a battle, I guess it's a contested area where you need constant effort to push back against what the Chinese are doing. And you don't get a constant effort when you've won. That's not the sort of psychological dynamic. So I think it needs a reframing.
- [15:32]
A
Yeah, pretty sure we all as kids learn very quickly the lesson that pretending problems don't exist doesn't mean that they go away. So, yeah, just a little acknowledgement goes probably a long way with getting that cooperation.
- [15:45]
B
I'm not sure that we did all learn that as kids. I find that there's many adults who employ that strategy still.
- [15:51]
A
Fair enough. Yes, absolutely. On that note, Tom, we will leave it there. But hey, thank you so much for joining me again. And you can of course read Tom's newsletter over at our website, Risky Biz. But Tom, thank you so much. Always a great chat and look forward to doing it again next week.
- [16:07]
B
Thanks, Emily.