← Threat Vector by Palo Alto Networks
Loading summary
Transcript47 lines
- [00:02]
A
You're listening to the Cyberwire Network, powered by N2K.
- [00:12]
B
Welcome to Threat Vector, the Palo Alto Networks podcast, where we discuss pressing cybersecurity threats and resilience and uncover insights into the latest trends. I'm your host, David Moulton, Senior Director of Thought leadership for unit 42.
- [00:25]
A
AI security isn't just security tools. It's about judgment, leadership, and responsibility. Responsibility. The choices we make now will define who is protected, who is exposed, and who holds and keeps power.
- [01:06]
B
Erica, welcome to Threat Vector. I'm really excited to have you here and have been looking forward to this conversation since we started planning it.
- [01:14]
A
Same. I am very, very excited to be here today and be able to just have a conversation and hope that your audience finds it actually very valuable.
- [01:26]
B
I'm sure they will. I know when I was looking at your background, I was impressed of your. Your time in the intelligence community and then how you shifted that service into the private sector, helping out a number of different companies. Think about AI and cybersecurity and that. That intersection where things come together, even going into national security. Could you talk to me a little bit about that journey? You know, two sides, but kind of the same mission.
- [01:54]
A
For me, my North Star is always thinking about the human first and what human centered design is. And my whole mission is working at the intersection of where people and technology collide. And when I take a look back at the work that I've done and walking through that path, for me, it's been having grown up in the FBI, in the US Intelligence community, that work started very early on where I was focused on very much so, you know, national security and criminal matters from counterterrorism, counterintelligence, transnational organized crime, and also national kidnappings, crimes against children. Like, I literally worked a gamut of different programs. And what was very unique for me is that when I came into the FBI, I was in a very small satellite field office. And there I had the opportunity to work all the things that I'm telling you about, like, any. Pick any hour of the day. I can't even say any day of the week. And I could be working very matters just because of the nature of how the office was set up and also the location of where I was. And that really set this, you know, very young, naive professional up, really to be able to, what I would say, kind of tip my toe into a bit of everything and actually understand it and do it very well. Because I understood no matter whatever I was working, the analytic tradecraft in itself is the same. Even though the threat in all of the emerging trends might be different. That piece to me was like all of it was the same. And so that's kind of what I think about taking myself back to the beginning of my career in national security.
- [03:57]
B
So it sounds like what you developed was a framework for dealing with threats or assessing risk, and then you could apply that to different domains or specific instances. Am I understanding that right?
- [04:09]
A
Yeah. No, you're 100% right.
- [04:11]
B
Well, let's shift away from that sort of environment that you grew up in and some of the national security risks of that post 911 era to another big thing that's hit, you know, with a, with a ferociousness is AI. And I'm curious how you react to and what you think stands out most about how AI is being integrated into national defense, into cybersecurity. What stands out for you right now?
- [04:41]
A
Great question. What stands out for me most is that AI is really being operationalized in national defense and cybersecurity, quite frankly, before we've even fully internalized how it changes the threat dynamics. And we're not just automating tasks, we're automating judgment under this real pressure. Right. You have the additional points that you're thinking about that you have to layer in. AI compresses time. Think about detection, think about decision making and response. They all move faster because of AI, which can be, which can be a good thing. But then on the flip side, you have to think about how your adversaries benefit from this too, especially our non state actors and legacy cyber frameworks that assume human pace escalations. Really. AI breaks that entire assumption of what is possible and what is not possible.
- [05:48]
B
Yeah. I was reminded of a conversation I had some months ago with one of our AI researchers and she was talking about this idea of infinite patients within an attack. And threat models might be out for five years, but with infinite patients you could go much further out. Right. Like it's not like a machine gets tired or bored.
- [06:10]
A
Right.
- [06:10]
B
Or it can. Yeah, it can basically sit there in its sort of quiet phase for 10 years or more. And it's making me wonder what your thoughts are on our, you know, our policies and our oversights and our mechanisms to handle things. Are we moving faster with AI than those things can keep up?
- [06:34]
A
Yes and no. Depending on the part of the world we're in. I would say oversight was built for these very static systems. Right. Not these adaptive ones that are literally changing like milliseconds. Right. Get new information, is continuing to train, is continuing to build on whatever is being fed into these large language models. But most governance is still like happens after deployment. And it's actually one of my biggest pet peeves. I remember coming into tech and thinking, wow, they're deploying all of this amazing innovation. But policy is very reactive, governance is very reactive. And yes, there is a lot of framework talks out there of responsible AI, responsible emerging technology, but the reality is there are things that are still being deployed, product features where policy is still the afterthought. And the one thing that I challenge companies, organizations and leaders is what if, what if we decide to do this business different and say, hey, our customers, our users are so important to us that what if we waited maybe a week, maybe two weeks, maybe a month, if we just did that people would actually buy into the company more because you have really kept the human at the center of what you are building. Yes, I know that there is this idea of like being fast and being the first, that's cute. But I always say innovation is great until it's not. So to really think about how to bring it all together and intelligence. We never assumed systems would behave as expected under stress, but I think we're pushing AI to say you behave yourself under the stress and doing business bigger, more efficient and faster. And AI governance needs that same mindset of that. Because I'm asking you to work faster and harder. We need to say that yes, that computer, that supercomputer can do things faster, but we must also say that it can get stressed and humans are still feeding that system that under stress it can also still make real mistakes.
- [09:10]
B
Well, let's shift gears here a little bit. Much of your work focuses on human centered design and really high stakes systems. Why is this critical to cybersecurity and AI governance?
- [09:25]
A
It really isn't about having empathy as a value, it's about having operational clarity. And analysts really need to think about and understand why a system flags something. Right? Their cognitive overload is in and within itself is a security risk. Trust in AI outputs must be earned and not assumed. And I know we had a funny conversation off mic about that. Right? Sometimes we've gotten so comfortable with what the output is that for, let's even say for 95% of the time it's right. But what about the 5% or even let's say the 1%, but the 1% of the time is so consequential that we can't even afford to have that 1% wrong. And I think that that's the piece we have to keep telling our employees. The innovators that don't get so comfortable with AI and its systems that we forget that we need to continue to audit the AI within itself. An example of failure when humans are ignored are this over reliance on our model outputs without context, interfaces that obscure uncertainty. And then lastly, you have these incentives that reward speed over verification. And I'm seeing that a lot inside of companies where we've also seen a lot of also layoffs for various reasons. Right. Regardless of whatever the reason is, whether it is you want to go leaner, you're doing business differently. The one thing we know, at the end of the day, we cannot replace institutional knowledge. And I would challenge companies as we're thinking about how we move forward is to really think about the risk versus the rewards of going leaner and really thinking about speed and what to automate and what not to automate. And, and also how many humans that we continue to have as a part of the auditing process, like AI can't audit itself. It can, but it can't. And the reason why I say that it can't is that you want to make sure that you keep humans first. And we need to still have that human piece because that is where the emotional intelligence comes into play. And I know there's a whole bunch of discussion about AI and emotions and like making it more human, but it's still not human. A human designed it. Right. And so the last thing I will say on that is that failure isn't usually the algorithm, it's the systems around it. And that includes the people in the organizations that are designing it.
- [12:19]
B
I think that one of the most pervasive design patterns I've seen in the, the frontier models and then, you know, the larger tools is this. Your idea is brilliant. That's a wonderful thought type of conversation. Even if the idea is terrible. And it's this false sense of confidence in that conversation being the right thing. And you can imagine how that would become pervasive as you're looking at, you know, the, the confidence in this is a threat that you need to look at, or it's a threat you don't need to look at, worry about.
- [12:57]
A
Yeah.
- [12:58]
B
And. Or it's the type of thing that the system is giving the human an emotional reaction to something that isn't warranted and you start to sense that trust. And I think that's a very dangerous design pattern in general. And I think it's particularly dangerous in the security environment.
- [13:22]
A
Disagree.
- [13:24]
B
I want to move on to policy and power. A lot of governments are trying to regulate A.I. right? There's this, this race to figure out how to, to regulate AI, but then there's also this other side where different teams, different state actors, whatever, are trying to deploy for their defense. And I'm curious, how do you balance a, you know, an innovation side and an innovative use of a technology while having the right levels of restraint so that, you know, you're not just wild, wild west, but you're not, you know, at a point where you're falling so far behind no matter what you're doing. And I think, I think that tension is really interesting. I'm curious what your thoughts are.
- [14:14]
A
Yeah, and I think I kind of alluded to this a bit earlier too, about building and having policy or policy governance around whatever that innovation is. And really restraint is not, I think where people maybe become frustrated or are very anti policy sometimes is that restraint is not anti innovation in national security. It's how you prevent a catastrophic failure. And we've seen these unfortunately more times than we want. And ways to frame this is policies should function like guardrails, not breaks. Goes back to my point earlier. And innovation without accountability creates strategic fragility. And the goal is durable capability, not flashy deployment.
- [15:15]
B
You've said that public trust is a national security asset. I love that phrase. By the way. How does that principle translate into cybersecurity strategy?
- [15:26]
A
Ooh, this is good. So I would say trust determines whether people believe institutions or adversaries. Distrust accelerates our misinformation. It's my conversation every day, actually. Once trust erodes, recovery is slow and it's costly like it's quick. Trust is easy to break. The rebuild is what is most costly. Trust failures create openings for influence operations and, and those are the things that I think about and for your listeners, practical steps for the CISOs and leaders as we think about this really explains the decision logic, not just the outcomes is so important for these leaders. Building the audit trails into AI systems, again, super important. And being honest about limitations and uncertainty. That transparency piece is what's going to keep people believing in your product and your company and the government for that matter. And the overconfidence is one of the fastest ways to lose trust is what I would say to any leader that's listening now.
- [16:39]
B
I think you're absolutely right. And when you're, you're talking about this idea of trust and how quickly you can lose it, you know, part of it is if you can't explain what you're doing and why you did it, it seems like you're hiding things. I was recently talking to our CEO and know he's got a policy that says, you know, nothing's going to go perfect, but if you're the type of person or organization that the next day shows up and says we're going to fix this and we're going to be there with you, then you've earned the right to be a partner. I think what you're talking about is if you show up the next day after something breaks and you have no answers, you have no audit trail, you have no transparency. Like you're suspicious, you're not, you're not somebody I want to listen to. And now I think the human mind comes in and says something's up and you fill in those stories and you become susceptible. Yeah, you become very susceptible to other things. So in real world security operations, I'm curious, how do you ensure that the ethical principles survive the pressures of mission urgency or that hot threat response that's going on?
- [17:46]
A
Great question. Ethics. Love it. Ethics don't survive because people are good. That's what people want to believe, but that's just not how it works. They survive because systems enforce them. So holding the accountability piece right, ethics must be embedded into our workflows. Accountability must also be predefined. What is that criteria? What is accountability? What happens if I do this or the system does this? Then what is the consequence of that? What am I being held to if this thing fails? As the person who is leading the thing, pressure tested before real world deployment also is part of that, that we need to always keep top of mind. And when we think about the tools and processes we want to think about, again the human, human in the loop for high impact decisions is a must. It is a non negotiable and we have to really think about that. Kill switches and escalation protocols are also necessary. Again, we're dealing with what we talked about earlier, fast technology. We have to have a way to be like, we got to kill it now. Even if you're like, oh my gosh, this is going to cost so much, we got to do the right thing and think about that part later because there are real people in front of this technology. Post incident reviews that focus on learning not blame is where we keep the ethics at center and not the finger pointing. Right. It's so easy to try to find someone where they're going to fall on the sword. When we want to just think about the lessons learned. Particularly again we talk about not if it's going to happen, the when, if we're working from that Standpoint from the beginning. We can always continue to have our active action post mortem where our people still believe that this company is doing the right thing and we followed all the steps and if we didn't, what was the mishap and why? And being able to lean into that is what people care about, I believe the most too. Sam, what's the secret to building that
- [20:33]
B
culture or those structures?
- [20:35]
A
The secret to that is one you yourself, the leader starts at the top and it's. And it literally is ingrained down. And the secret to building that is having a leader that you talked about it, you mentioned it earlier about building a culture of transparency and honesty, having real conversations with your people, no matter what, starting small and going big and keeping your people in the loop and ensuring that your leadership is also held accountable for sharing that information with the people. If there is not room to have those conversations with the entire org or company, it is important that the leaders are building that same culture that you are asking. You have to make sure that that's being implemented and, and you have to check in with your people. I would say every, you know, three to six months, especially if you're a company that is building, if that spirit of transparency is being drilled top down is what I believe.
- [21:37]
B
Erica, let's shift gears and talk about your personal advocacy work around health and equity. I know that's shaped how you think about systems. How has that perspective influenced your approach to national security?
- [21:55]
A
As I've worked through my career in various sectors and intersections of, I would say the pieces of me, I would say public health and national security, they share a real truth and that is systems fell first at the margins. You have vulnerable populations experience harm a lot earlier, a lot sooner than the average person. Those signals are the warnings, not the exceptions. And sometimes because it's happening to the marginalized, those signals aren't necessarily believed immediately, even if they're believed later. And ignoring them weakens the overall system's resilience. And as I think about the work that I've done, both at the advocacy level, from public policy, public health, and also in cyber and threatened intelligence, prevention beats reaction any day of the week. Data without context misleads. And unfortunately, I myself have been a victim of this and I won't even say a victim. I'm going to say a warrior of this because I was able to see it and be able to call it out and really change the lens of where it was coming from. Bias isn't a side issue. It is a real system vulnerability. And every Single one of us have them. And I mentioned it very early in our interview that even when I'm doing the work of analyzing, I have to check my own biases and also go back and look at the data to ensure that my biases isn't over influencing for what I see in the data versus what I feel. Because we can't make decisions, particularly in this world, based on feelings alone. We have to really try to keep objectivity first with our facts being of the utmost part of this piece.
- [24:15]
B
You know, I really like how you think about that. Years ago I was talking to a colleague at IBM and she was talking about security teams. Having consistently trained in computer science on a team and they all are trained the same, they tend to look the same and they have the, the same gaps in their understanding of the world and they're going against an adversary that didn't go to the same schools, didn't grow up in the same culture, didn't have the same, you know, norms or, or code for how they think about things and don't always look at computer systems in the same. This does that right? Like they actually look at it as like I can make this do anything I want. And by bringing in a variety of perspectives that weren't very matchy matchy, you ended up with this extremely strong team to think through all of the edge cases, to look at the margins or to look at the signal that you got from that and go like wait a minute, I've lived that before. I know that that's indicator what's coming next. We've got to look at that now rather than going, well that seems like an outlier, let's just go ahead and suppress it. And here you are saying like you learned that lesson from your advocacy work and health. How would you advise a security leader to take those lessons and apply them to their systematic bias and checking that and or even looking at that as a way of building their resil.
- [25:58]
A
We need different perspectives. We need, and you need as a leader folks that look different, you need folks that look like me. I'm just gonna point blank say it. You having a homogenous team again, you said it so well. Similar schools, similar background, you know, homogenous upbringings. Even if some of it was hard, it's still in many ways the same. And our adversaries aren't operating from that lens. Like I talked about at the beginning of our interview, how with the western mindset can trip you up and this thing of what is not possible when you are working with people again who have had a different lived experience. And so I would challenge any leader to really look at your teams, even now you're in the beginning of the year, take an assessment of what your teams look like, their experiences, their backgrounds and the makeup, and challenge yourself to say, I'm going to do maybe what feels uncomfortable in this season that we're in and still build teams that look like the world and the people that we serve. Because my business counts on it, my consumers count on it, and at the end of the day, when I bring it back to profit, my profit actually depends on it.
- [27:28]
B
Let's talk about AI and global competition. As AI becomes this geopolitical resource, what do you think is the biggest risk to global competition in this space?
- [27:43]
A
A race mentality that prioritizes the speed over safety, I think is again the biggest geopolitical risk. And I actually spoke to that kind of earlier about what are we racing to and for is something that we should continue to ask ourselves, particularly companies, government organizations. What is the ultimate goal? Fragmented norms across allies also is something we should be thinking about from a geopolitical risk standpoint. And also the weaponization of AI enable influence operations is those things that we should be thinking about and from being here in the US how the US and allies should respond to some of these things is sharing standards for AI risk. I know that before there was a whole Bill of Rights and blueprint of how we should be thinking about this right now. That's not so much here in the US I know other countries do have various frameworks and I think that that is something we desperately need because that keeps the North Star aligned and coordinated. Intelligence sharing on misuse is also important and something we can't take for granted. And then lastly, the collective response frameworks, not unilateral ones like you don't want to continue to look like the left hand is not talking to the right. And right now I do see in the world where it feels like at least looking at various online reports, even how policy is being created, that there is what the rest of the world is doing, particularly in the Western world is doing versus what we are currently doing here in the us And I think that that needs to have a better alignment if we are really focused on on how our how we are honestly positioned in the world from a geopolitical standpoint.
- [30:01]
B
So, Erica, look ahead and talk to me about what a secure, ethical and resilient AI future looks like to you.
- [30:11]
A
Yeah, so I'm optimistic. Hence is why I smile. The next generation needs more than Technical skill. We know this of course, but I think because we have all of these big systems now, there is a lack of thinking, a lack of critical thinking because we have become a lot more reliant on the systems thinking for us versus thinking about. Critical thinking doesn't have to be a big think piece. Critical thinking is just continuing to press even the technology that you're working with and continuing to ask questions that the system can't think about alone. So systems thinking one, two, policy and regulatory literacy is so important. People don't even know what's being passed these days and even what it means. And so ethical reasoning also under pressure is so important. Again, if you're building, I want to build fast. Building fast for what and for who and and why? We should continue to always ask. And then lastly just going into like the training part of this is thinking about training, thinking about schooling, keeping up is something that I think is going to be, we will have to continue to evolve on, but we must still think about what is still being siloed. Engineering, our policy and our ethics. And we have got to get away from these siloed effects of thinking that engineering is engineering, policy is policy and ethics is ethics. It all needs to be a conglomerate where it's working together. Even if you're not the person that's doing the tech, you still need to understand what tech is doing and tech needs to understand what non tech is doing and care about it. Either way, the separation is a liability in an AI driven security environment.
- [32:14]
B
As you describe that. I'm reminded of a great kinesiologist that put me back together a couple times and he described this idea of a doctor will look at your knee pain and say, here's some ice or some Advil. And a kinesiologist will look at it and go, you've got arthritis in your foot and you've got an underdeveloped glute and your knee is getting hammered. So let's go ahead and you know, work on your, your, your strength and it's all connected, right? You can't separate these things in medicine. And I think what you're saying is, you know, the world we live in, everything is connected. There is no separation between ethical thinking and engineering. There is no separation between what you're going to study and wait for somebody else to work on. You got to have some understanding of it. And I think that that broad ability to bring things together may be more possible than ever with a, you know, AI future where we can turn to, you know, a chatbot or a robot or some level to help us think. But it still comes back to, we got to do our own thinking.
- [33:23]
A
Yes. Oh, my gosh, you hit the nail on the head. And I, again, being in the public policy space overall is really how I think about these problems. And as someone who has been put back together again a few times myself, I also am always, like, arguing with the doctors that I bring it into my professional life. Like, let's stop being so segmented. Everything is a system and it all works together. Nothing is separate. And if we think about that, because I'm always, I'll ask a doctor, like, because I've, I'm sure, like, you, like, built these great relationships. I'm like, but then you go to med school and I know you've learned about the whole body. So I asked people who are doing this work in the tech space. I'm like, you went to a whole school, right? Like, yeah, you had a focus, but your brain is allowed to think beyond just that focus. And I think in workshops, I say this all the time. I'm like, you are able to think beyond what your expertise is. You get to have a thought. I mean, it works. So think about it in that regard.
- [34:28]
B
Well, Erica, thanks for this awesome conversation today. I really appreciate you giving us the time to share your insights. You've obviously deeply thought about this space and bring such an interesting mix of expertise and lived experience together. And I appreciate that you're putting it to good use.
- [34:49]
A
Thank you.
- [34:50]
B
To make sure that we arrive at this optimistic future that you just described.
- [34:55]
A
Thank you so much. Thank you so much for having me. Really appreciate it.
- [34:59]
B
Where can listeners find you on the Internet if they're interested in reading more, learning more, those sorts of things?
- [35:05]
A
Absolutely. So you can find me on Instagram at the Policy Goddess. And that's T H E Policy P O L I C Y and Goddess is G O D D E S S. And you can also find me on LinkedIn Erica S. And I will also, of course, share these links as a part of this conversation as well. And also you can go to my website, which is www.leadwithne strategy AI and so yes, that's where you can find.
- [35:40]
B
We will go ahead and make sure that those links are in our show notes if you want to follow along with Erica, I really recommend you do go check those resources out. That's it for today. If you like what you've heard, please subscribe wherever you listen and leave us a review on Apple Podcast or Spotify. Those reviews and your feedback really do help me understand what you want to hear about on the show. You can email me about the show@threatvectorlotonetworks.com I want to thank our executive producer, Michael Heller, our content and production team, teams which include Kenny Miller, Joe Betacourt and Virginia Tran. Original music and mix by Elliot Peltzman. We'll be back next week. Until then, stay secure, stay vigilant. Goodbye for now.
- [36:54]
A
Sa.