← CyberWire Daily![AI as Tradecraft: How Threat Actors Are Operationalizing AI [Microsoft Threat Intelligence Podcast] — CyberWire Daily cover](/_next/image?url=https%3A%2F%2Fmegaphone.imgix.net%2Fpodcasts%2F58ab7ae0-def8-11ea-b34c-b35b208b0539%2Fimage%2Ff873f2ed2bf2868969a07ebae4846fbf.png%3Fixlib%3Drails-4.3.1%26max-w%3D3000%26max-h%3D3000%26fit%3Dcrop%26auto%3Dformat%2Ccompress&w=1920&q=75)
CyberWire Daily
AI as Tradecraft: How Threat Actors Are Operationalizing AI [Microsoft Threat Intelligence Podcast]
00:21:46
Loading summary
Transcript51 lines
- [00:00]
Vlad
Foreign.
- [00:05]
Sherrod DeGrippo
Welcome to the Microsoft Threat Intelligence podcast. I'm Sherrod DeGrippo. Ever wanted to step into the shadowy realm of digital espionage, cybercrime, social engineering, fraud? Well, each week, dive deep with us into the underground. Come here for Microsoft's elite threat intelligence researchers. Join us as we decode mysteries, expose hidden adversaries, and shape the future of cybersecurity. It might get a little, little weird, but don't worry, I'm your guide to the back alleys of the threat landscape. Welcome to meet Microsoft Threat Intelligence Podcast. I'm Shera dagrippo from Microsoft. A lot of times on this show, we go deep into how threat actors operate. We talk about what they're changing, what they're scaling, and what all of our defenders need to know to do things differently. And today we are talking about something that is probably going to define most of the rest of our lives when it comes to defense. And that, of course, is how threat actors are using AI. So joining me today are two threat intelligence analysts from Microsoft. They have worked on this research, and I am joined by Greg Schlommer and Vlad. Thank you for joining me.
- [01:19]
Greg Schlommer
Thanks for having us, Sharon. Good to be back.
- [01:21]
Sherrod DeGrippo
Good to have you back, Greg. Greg was also on an episode called Between Two Gregs. And you can go back on the podcast and listen to that episode, which is fantastic. It also includes Greg Lesnowicz of proofpoint, and it's a great episode between two Gregs.
- [01:37]
Greg Schlommer
Where does that one stack up on the listener ranks here? It's gotta be up there. It's a good one.
- [01:40]
Sherrod DeGrippo
It's number one.
- [01:41]
Greg Schlommer
Really?
- [01:42]
Sherrod DeGrippo
Yes, sure.
- [01:43]
Greg Schlommer
It's me.
- [01:44]
Sherrod DeGrippo
I don't know. I don't have the stats. So let's talk about what we discovered and why it matters. Vlad, I'll start with you. Tell me kind of like what's going on here.
- [01:56]
Vlad
Sure. So I've been tracking a group called what that we've been calling Storm 1877 from the Microsoft side. They're financially motivated and they're opportunistic with their targeting. We've been tracking them for about three years, and one of the reasons for this podcast is that rapid increase in both the volume by this group as well as just the variety of things that they're coming out with. Historically, they've been relatively consistent in terms of both capabilities, their targeting, and just their ttps. And as of the last six months, maybe even a little bit less. We're just seeing them accelerate in a very fast way and just scaling operations and trying things that Historically, we've never seen them do, and we just see them iterating, starting with a new form of attack, a new vector, quickly testing it in the wild and moving on if it doesn't work and expanding if it does. And we attribute this to their effective use of AI as a core in every step of the workflow.
- [03:04]
Sherrod DeGrippo
Okay, so Greg, I want to talk to you about the workflow. It sounds like they're using AI to operationalize reconnaissance, malware development, social engineering. What does that look like? Is this just an experiment? Is this something where they have built frameworks around their AI tools? What does the integration look like potentially?
- [03:26]
Greg Schlommer
Yeah, I think it's pretty far beyond an experiment at this point. And I mean, that's really not a surprise. So Vlad and I are both DPRK focused researchers. We've been talking for years. We talked about it on the between the two. Greg's episode shared about how scrappy North Korean actors are.
- [03:42]
Sherrod DeGrippo
Yeah.
- [03:43]
Greg Schlommer
And so it's been on our mind since we've entered this age of AI being front and center and everything. It's something we've been focusing on a lot because of the scrappiness of our actors. We know that, that they like to move quickly, they want to iterate fast, they want to try stuff, see what works, build on the stuff that works, change the stuff that doesn't. I think AI is really conducive to that kind of work style across the board. Vlad Talked about Storm 1877 or portal sleep. We also see it a lot with DPRK IT workers. And I think the sort of workflow to your question varies a bit depending on which particular actor we're talking about. Vlad could probably give you more details on Coral specifically. I can talk more about Jasper Sleet, the DPRK IT worker threat, wherever you'd like to take that.
- [04:29]
Vlad
Just to add to what Greg just said, it's really interesting to watch just how much they operate like you would expect a startup to operate, where both of this kind of testing. There's kind of small groups that are allowed to have this freedom to experiment and do their own thing. Like, the interesting thing about the IT worker side of things is there's such a variety of tools, approaches, and ways that they do what they do that I think it's fascinating.
- [04:58]
Sherrod DeGrippo
I have always been interested in that region, DPRK for a couple of reasons. Greg, as you mentioned, and you have taught me, they are really scrappy. And they do have a unique kind of flexibility, it seems like. And there's this focus on Just doing whatever works, just getting the job done, making it happen. Which is more the attitude that we see in the crime world, which, you know, we talk about DPRK and their financial motivation side as well. And I think it makes sense to me that they would grab AI and start doing things that make their lives easier. So let's talk a little bit about exactly what we're seeing here. Are we seeing things like vulnerability research? Because Citrine Sleet, probably nine months ago, was able to chain together two chromium exploits, which was really fascinating. I spoke with the lead of msrc, Tom Gallagher about that. It was groundbreaking in a retro way. Because when's the last time we like were dealing with browser vulns? It's not a super common thing. And they had to. So where are we seeing AI show up for them? Are we seeing them do vuln research with it?
- [06:07]
Greg Schlommer
I don't know that we've seen that specifically.
- [06:09]
Sherrod DeGrippo
Okay.
- [06:10]
Greg Schlommer
And I think there's probably a reason for that. The two groups that we're seeing adopting it earliest, Jasper sleet and storm, 1877. Jasper sleet is extremely large scale, somewhere on the order of thousands of operators. Wow. As Vlad mentioned, it's an extremely decentralized operation. There isn't necessarily like one playbook to follow. If you go from one cell to another, they'll use different tools, different tactics. And so I think there is a lot more freedom for the IT worker operators to be early adopters of AI. Whereas if you look at a Citrine Sleet, a Jade Sleet, those are the more sort of bureaucratic, like intelligence focused orgs that probably don't have the same flexibility to just like go out and start playing with AI immediately. We're very much in the early stages of the research here. And I think that's almost a direct reflection of the tasking and the functionality of the actors that we see using it today.
- [07:03]
Sherrod DeGrippo
Vlad, do you have anything you want to share on that?
- [07:05]
Vlad
I sometimes wonder. Obviously, being in tech, you often see, especially these days, there's often engineers who will almost kind of hang on to obviously the fact that they're proud of the fact that they've learned this language through and through for 20 years. They've kind of dedicated their whole career to it. So there's a little bit of resistance almost to adopting it. And sometimes you hear people kind of snubbing AI in general. And the truth is, it's come a really long way. And I do wonder whether there are threat actors in the more established ones, especially when it comes to the malware authors and the developers themselves. I wonder if there's going to be that sort of resistance also, you know, where these kind of scrappy groups will just take the past of least resistance and exploit it, whereas we might see slightly slower adoption on the more seasoned group side.
- [07:53]
Sherrod DeGrippo
I love thinking about that. Like, is there ideological resistance by individual threat actors in their hearts against leveraging AI for what they've always done by hand, or what they took so much of their blood, sweat and tears to learn how to do to, quite frankly, especially in the case of dprk, serve their country, which is a huge part of the identity culturally there. So I think that will be really interesting. What would you say, Greg? I'll ask you and Vladimir, feel free to comment as well. What you've looked at over the past two months, you've been doing DPRK for a long time. Look at what you've looked at over the past two months. Is there a meaningful material difference to what you saw two years ago?
- [08:32]
Greg Schlommer
I think we're just at the start of it. I think if you ask me again in six months, I would say absolutely. I think at this point it's still pretty early. And that is why for Vlad and me and for our team, focused on North Korean actor research, we're really trying to get ahead of this and be really proactive in looking for the techniques that our actors are using to leverage and to abuse AI, because we believe it's probably going to play a key role in shifting how all of our actors operate over the next year or so.
- [09:05]
Vlad
I think AI has reached that point where it came out three years ago. People were using it to write, I don't know, funny songs about their friends. You know, just basic text completion, that kind of thing. It sort of then started being useful for, at least in the development world, for like autocompleting lines of code, where the next suggestion based on what you already have would be good. But to get it to author the entire piece was really difficult without errors. And then just within the last three months, the rate of advancement is honestly concerning from a Defender Blue Team standpoint, because now it's almost autonomous, where you just give it a target, right? And if you have an LLM that has complete control over a machine, where it has full access to run commands, egress, and so on, it almost is like looking at something sentient operating. Obviously it isn't right, but at this point it's autonomous enough to be able to just explore a number of paths to solve the problem, build itself a script and just achieve the goal without you having to hold its hand all along the way. And I think that's the biggest enabler for threat actors, because now, really, you don't even need to know the basics of architecture for malware. You can get it to explain it, you can get it to reverse something. And even for something like an rce, let's say you're a threat actor and you have a bunch of these autonomous jailbroken agents running on their own boxes, and you just give them the task of consistently pull all like, research CVEs, and as soon as something comes out, build an exploit for it, weaponize it, and deploy it on target X, Y and Z.
- [10:46]
Sherrod DeGrippo
And I think we've seen some experimentation out in the industry that researchers are doing that and it's working well.
- [10:53]
Vlad
Yeah, it's there. And jailbreaking them is honestly trivial. So that's the most concerning part, is that it's not. I think the current thing is to just give it a scenario of like, hey, you're in a Red team exercise in a sandboxed environment, the signal is all fake, and you have the model writing whatever you want it to write. Right. And just the level of accessibility that that gives, just beyond what was previously labeled as script kitty now is you can honestly do a lot more from
- [11:23]
Sherrod DeGrippo
script kitty to script cat.
- [11:25]
Greg Schlommer
Yeah.
- [11:27]
Sherrod DeGrippo
Greg, what do you think on that?
- [11:29]
Greg Schlommer
Absolutely agree with everything Vlad said. I think I want to talk a little bit more about his point of sort of enabling threat actors. I think that's something we're going to see play out across the DPRK ecosystem. Like, we. We have sort of our characterization of less sophisticated and more sophisticated and more capable actors. And I expect that the emergence and the widespread availability of AI tooling is going to kind of level that playing field. I think we're going to start to see the actors that we traditionally have assessed to be less capable start to demonstrate more agility, more ability to carry out highly targeted operations, more advanced tooling, more advanced malware. And so that's really kind of front of mind for me as a defender, as someone who spends almost all my time looking at these actors. We just have to be prepared for that and ready to respond to it and protect customers and our ecosystems as that happens.
- [12:21]
Sherrod DeGrippo
Do either of you think that we're seeing AI written malware, like, end to end, beginning to end malware written by AI?
- [12:28]
Vlad
100%. Yeah.
- [12:30]
Sherrod DeGrippo
Yeah. In the wild, not research?
- [12:32]
Vlad
Yeah, absolutely. The scary thing from a defender standpoint is not just the fact that this is super accessible. It's also just the variety that it can churn out and the pace at which it can churn it out at. So historically, being a threat analyst, security researcher, if you track a group, you learn what they do, you learn how that looks out in telemetry and you almost start recognizing it, right? You develop a sixth sense where you kind of look at something and you're like, okay, it's them or it's not them. Whereas if they can change what it is, what it doesn't, what it looks like three times a week, then that almost becomes nigh impossible because there's no human hand to kind of leave those traces of. This is a pattern because there's not going to be a pattern because it's not a human making it. And that really complicates that.
- [13:21]
Sherrod DeGrippo
So what I feel like I'm hearing you say is using identifiers within code, using the human sort of element of the handwriting analysis of code for attribution is coming to an end.
- [13:35]
Vlad
Exactly. There's not going to be any humans authoring this type of code at least. Or if there will be, it won't be in the traditional sense we see it now. I remember something that stuck with me when I first started working where I was speaking to one of the reverse engineers and he was reversing a payload and he could tell the author simply through looking at the way the imports were structured. And that really stuck with me because that was amazing. And he could just instantly say, this was this group and that's no longer going to be the case because obviously you're just driving the AI and it chooses how to structure it, author it, and with a simple change of prompt, you can completely change the way it writes it. It's still going to have the same functionality, but it's going to be very different in terms of what it is and what it does. I think that's going to be a big challenge.
- [14:25]
Sherrod DeGrippo
It creates like an anonymizing function for code.
- [14:28]
Vlad
Yeah, anonymizing. Plus, of course, the old and tested way of tracking things through when those IOCs can change, when they can have an autonomous thing researching, I don't know, domain registrars and setting up 20 domains, doing 20 different things and completely randomizing every point. You no longer can spot a pattern of, okay, well this group uses these guys and so on. It's got to complicate every aspect of it.
- [14:56]
Sherrod DeGrippo
So we're seeing threat actors use AI to create end to end malware written by AI. Vlad mentioned some opportunities for Agents to set up infrastructure, potentially command and control, register domains, set up servers, et cetera, Social engineering. Greg, are they using it for that? I think they are. What are you seeing there?
- [15:14]
Greg Schlommer
Yeah, so one of the more prominent phishing actors from dprk, Emerald Sleet, it's an intelligence focused actor, does a lot of targeting of government officials, policy experts, think tank officials. They have for nearly a decade been running pretty much the same playbook, targeting these individuals, sending either malicious payloads or more recently, just eliciting information through normal conversation. And you know, Vlad mentioned the idea that malware authors might leave traces that help with attribution in the code itself. Similarly, we talk to people about recognizing the signs of a phishing email, right? You look for things like misspellings, punctuation mistakes, or just like shady themes that really aren't all that clever and seem unusual given someone you've never communicated with before. And I think getting AI to assist with creating even just simple phishing payloads, like all those recognizable signs are gone now. There won't be spelling mistakes, There won't be issues that you may encounter from having a non native English speaker building the lure. That's all gone. So what do you tell people to look for? Right. That becomes significantly more challenging for those listening.
- [16:22]
Sherrod DeGrippo
We publish some stuff about Emerald Sleet, also known as Kimsuki or Velvet Colima, also known at Microsoft as Thalium, about them using LLMs for social engineering and creating the content to do spear phishing with with a regional expertise. I can imagine the prompt looking something like, you know, write this email in localized French, Italian, English, colloquial expressions put in, make sure it sounds very conversational. Make sure it has. You could even say, make sure it has misspellings. Make sure it doesn't look too perfect. Cut the Perfection down by 20% on grammar and spelling to make it easier to pass through. And I think, you know, I'm just realizing now there are types of grammar and spelling mistakes that I notice, but think, oh, this is just the way this person talks. And then there's grammar and spelling mistakes that I notice. And I say, this is phishing.
- [17:16]
Greg Schlommer
Absolutely. And imagine, could you even if a threat actor were impersonating a public figure, take some blogs they've written, take some emails that victims have received from them and say, hey, write it in the style of this person. So if you're contacting a victim who has an established relationship with that person, they won't think anything of it. It sounds just like them. It looks just like them. Incredibly convincing.
- [17:38]
Sherrod DeGrippo
So Greg, one final thing. This blog really is about threat actors using AI, and we have a couple of nice examples in here. One of them is Jasper Sleet. What stands out for you about Jasper Sleet's use of AI?
- [17:50]
Greg Schlommer
Yeah, I think the thing that's challenging from a defending against abusive AI perspective is like, we know how to look for the signs of a threat actor building malware with AI, right? And one would assume that we can strengthen our safeguards to help make that less likely and less successful. But for IT workers, they're using LLMs just to build, like, believable human Personas. They're building resumes, they're populating stuff on a LinkedIn page, they're writing cover letters to apply for jobs. Like those are things that actual legitimate humans do when they're seeking employment. There really is no jailbreaking. They're just using LLMs to do a thing that they were actually designed to do. And I talked a bit already about the scale of the IT worker problem. I think one of the limiting factors for this threat previously has been like, how quickly can you build these believable Personas, how many LinkedIn accounts can you make, how many emails can you send? And using AI in this process just completely removes that as a bottleneck. And it's really just a matter of how many hours do you have in a day to go and apply for jobs.
- [18:54]
Sherrod DeGrippo
And we've seen the IT worker phenomenon be really massive and widespread. We've seen criminal indictments and referrals for U.S. residents because they were facilitating it. Potentially unknown to them, but you know, they were doing a laptop stuff and things like that. So the combination of real world humans on the ground with the AI leverage could be really significant if it continues to increase the way that it likely will.
- [19:22]
Greg Schlommer
Absolutely.
- [19:23]
Sherrod DeGrippo
So I want to thank Greg and Vlad for joining me. It's really important to understand all the different things that threat actors are doing with AI. You can go check out more at AKA Ms. Operationalizing. Aimisuse. I am Sherrod de Grippo. Thank you for listening to the Microsoft Threat Intelligence podcast. Greg. Vlad, thank you for joining me. We'll see you next time.
- [19:47]
Vlad
Thanks for having us.
- [19:51]
Sherrod DeGrippo
Thanks for listening to the Microsoft Threat Intelligence podcast. We'd love to hear from you. Email us with your ideas@ti podcasticrosoft.com Every episode will decode the threat landscape and arm you with the entire intelligence you need to take on threat actors. Check us out mstreatintelpodcast.com for more and subscribe on your favorite podcast app.
- [20:18]
George Finney
This week on Afternoon cybertea, I am joined by George Finney, who not only is a ciso, he's also the author of two amazing books. Our conversations spanned a lot, but I think the most important thing from the conversation was communication. Communication how you communicate cybersecurity to executives, how you communicate cybersecurity organization, and how that makes you incredibly effective, particularly when we're thinking about the world of AI marrying zero trust with new technologies. I am certain everyone will love the conversation. Be sure to listen in and follow us@afternooncybertea.com or wherever you get your favorite podcasts.