Loading summary
Transcript28 lines
- [00:04]
A
Hello everyone, this is Tom Uran and I'm here with AJ Williams, who is the product manager for Sublime Security. G', day aj, how are you?
- [00:14]
B
Hi Tom. I'm doing good, how are you doing?
- [00:16]
A
So, AJ Sublime, you're the product manager for some AI agents. We were having a chat beforehand and I had started on the assumption that your agents were looking at all the email that was coming in and, and figuring out whether they were bad or not. But it turns out that I'm in fact incorrect and that it's much more like a person who's been tasked to look at some bad email and so that there's a filtering process, a flagging process, or users can flag them. And so that then kicks off a process to do, I guess, a human level amount of work on those emails that have been flagged. Is that the sort of framework for thinking about what these agents are doing?
- [01:08]
B
Yeah, I would say that our agents do operate a lot like a human and we've really modeled them to operate like a team of individuals that work alongside a team of actual humans. They're doing work as they need to. We're not just throwing generative AI, generative everything and raising through your emails. We're being selective about our usage of AI and our usage of that processing of your mail when we're applying these agents. So it's much more selective than just pushing every single message that you receive or send to our agents.
- [01:56]
A
Okay, and so what do those agents do when you get the email? I guess in the good old days or the bad old days, there would have been a person actually analyzing to decide, is this email bad, is it good? What do we do about that? Is it the same work?
- [02:16]
B
Yeah, ASA works in a non deterministic way. So when a message comes in, it's sent to ASA in the same way that historically would be escalated to an analyst instead of an analyst. We've got asa. ASA takes a look. It uses its knowledge base as its expertise to make a decision on what to get started with first. Maybe it notices a link, maybe it notices an attachment. Whatever it might be, it gets started there. And then from its first set of analysis, it moves through the total analysis that it wants to complete on that message. It can dispatch smaller sub agent workers to go do targeted tasks, go detonate this link, go detonate this file, and then from there come back with its verdict based off of that analysis and make a decision on what type of message this is. Is it malicious? Is it spam, is it graymail? Which means that two Messages with very similar surface characteristics could actually have completely different investigation paths, which really mimics how humans go about investigating. No two people are going to do it the same way.
- [03:36]
A
Right. So Acer Autonomous Security Analyst. And then there's another one you've got here called Autonomous Detection Agent and Detection Engineer. Engineer. What does ADA AD do?
- [03:54]
B
Yeah, so AADA takes and takes missed attacks, you know, novel attacks that are found in your environment that have been deemed to be malicious and writes novel detection coverage to cover that attack moving forward.
- [04:12]
A
So the engineering part is we need to change something in our system so that it matches that or detects it and stops it instead of remediating it afterwards. Yeah, right. Okay. And so again earlier you were saying that you take these models and there's some training process and I was kind of thinking that maybe it's like you've got a new intern that arrives on the job and that you've got to teach them the ropes and after, I don't know, however long, I presume an AI agent is a bit faster than an intern, but maybe is it less trustworthy or what's that process like? And how do customers move through training them and how trustworthy, how much trust do they invest in them in the longer term?
- [05:04]
B
Yeah, our customers aren't responsible for any training for ASA or ade. There's no lift for them to get started using or benefiting from our agents and product. We do a process of, or did, I should say a process of training ASA and ADE by giving them a knowledge bank and building up their expertise and helping them understand the landscape in which we're inserting them. So I would almost think of it as like a little bit of pre job training before they ever get started on the job. Maybe they're taking a LinkedIn course or, or doing a little bit of learning ahead of time and then they get started and they're able to utilize that information to make smart decisions from the beginning. So it's a lot more like an associate or a junior employee where you have some set of information, some set of logic that helps you to make smart decisions. And more than anything, you are showing the organ what you can do and gaining trust from your boss or your boss's boss or your peers, not necessarily learning net new capabilities.
- [06:19]
A
Right, right. And so how does that trust develop? Like I can imagine that you might have some customers who are, we are never going to trust an AI. What's the, I guess what are the checks? Or how does it say, here's a decision, I want to make or I'm recommending and then do those. And you might have some customers that are just risk averse, but you also might have some customers that are like yolo, have a high risk appetite. I don't want to do this work ever. Do customers move from I'm not certain or I don't know that I trust this AI to the trusting camp. What does all that look like?
- [07:00]
B
Yeah, so the way that we gain trust is by doing a great job. So like I mentioned, it's an associate employee, it's joining, it's got all the skills, it just wants to show off,
- [07:11]
A
you get a place.
- [07:12]
B
Yeah, yeah. So it starts off by doing analysis passively, what we call passive mode. And this is something that all of our users can leverage and enable and maintain as long as they would like as well. There's never a requirement for them to move from passive mode to autonomous mode, but they can use our agents passively where it does its analysis, it does its thinking, and then it presents that information to our users so they can understand how ASA or ADE got to this answer. What was it thinking? What did it look at? How did we get here? Which helps to build the trust for those folks who are just in the boat of not trusting the agents. There's a different category of customers who don't move from passive to autonomous mode, which is more of the like regulatory folks who for some reason within their organization cannot move to autonomous mode. And no amount of us explaining or the agent explaining itself will get them to move to autonomous mode. It's an outside or external reason why they can't. But for folks who are just risk adverse or maybe not trusting AI, still seeing it as an intern, that process of seeing ASA and ADE run passively is what gets them bought into or interested in going to autonomous mode once they see, hey, I can cut the number of manual investigations for user reports down 95% with ASA. I don't need to look at these. I can spend, you know, send two whole full time employees to go do other cool, brilliant activities instead of reviewing, you know, 90% spam user reports all day long. Like there's no, there's a lot of people who, it's just a no brainer choice to start using ASA and AADA to do that work instead.
- [09:22]
A
Right. And so at some point those agents will say, here's something we found and this is what we reckon you should do. And over time, I guess the customer is, is reading those reports going, oh yeah, I agree with that. And just saying yes, go and do that. Is that the sort of process? And then eventually they go, oh, okay, I'm sick of just rubber stamping this because I've never said, you know.
- [09:47]
B
Yeah, exactly. And we have reporting to try to help support that process as well. Like, we know there's a lot of folks who doubt the agent's capabilities to start and, and we expected that. And we want to make sure that they have all of the information that, that they need to make the transition to trusting our agents in an autonomous mode and make it easy for them to do that. So for asa, that means that once you remediate, you know, you've checked asa's work for however long you would like to, it's as simple as switching from passive to autonomous. And we've already pre filled all of the remediations that correspond with the different verdicts that it can produce.
- [10:32]
A
Right, right. And how extreme do those remediations get? Is it like, does it vary from I don't know, we'll just put it in a spam folder to we're going to delete your email or even further than that?
- [10:46]
B
Yeah, our defaults depend on the verdict. For malicious, we want to quarantine that. We want to make it inaccessible in the email environment for that end user mailbox. So we completely nuke it, we yank it from them. Um, but for something like spam, it doesn't really need to be, you know, blown off the face of the mailbox. They could move it to the spam folder. That's our default that we recommend. And we've got a wide variety of customers. Some people want to trash spam. They find it inappropriate to be in the workplace mailbox. They want to get rid of it and it's into more of a like trash folder rather than just letting it sit in spam.
- [11:25]
A
Right.
- [11:26]
B
There's optionality there. They can pick and choose what works best for their organization.
- [11:31]
A
So in the case of a malicious email, where the policy would be to delete it out of the inbox, which like, I mean, that makes sense. Right?
- [11:40]
B
It's pretty standard.
- [11:42]
A
You want to do that regardless of whether you've got a human deciding or a machine deciding, does that mean that if you're not operating in autonomous mode, you have to wait for someone to
- [11:53]
B
approve that you could leverage any of our existing detection coverage to make a decision. So we, we do have, within our core feed, I think it's over 800 rules. We've got a lot, we've got a lot of rules and policies there. And you could create automations based off of those policies to automatically triage your messages. You don't have to rely exclusively on asa. There's a variety of different ways that you could complete remediations automatically, but ASA's almost like the last line of defense. If nothing else has given you a remediation or a path forward automatically, EASA at the very least has.
- [12:40]
A
Right. So it is like the human, I guess. So I was kind of thinking again that it was like one of the first steps, but it's the. I guess in any email system there'll be a whole lot of automated remediations and then you'll get to the human. And so I guess it's acceptable that you've got some delay there, because that's what would happen if it was a person doing it in a system anyway. But what I was thinking is that I guess there's risks on both sides. There's risks of asking for human approval because that email is then sitting in somebody's inbox for a long time. And so it's not just that there's risk in trusting the agent, there's risk in not trusting the agent. So it's a balanced decision where you. You actually have to think about it, I suppose.
- [13:26]
B
Yeah. And we do have users, especially when they're in that passive mode, that want to escalate to get a human in the loop as soon as possible. And so we create opportunities for alerting. Let's say you're in passive mode. ASA finds something to be malicious. You could hook up a Slack alert, an email alert, or a webhook to then go ping your team and say, hey, ASA thought this was malicious. There's no remediation applied right now. Here's a bunch of information about this message. You should probably go take a look. Like, you should probably go check this out sooner rather than later. So it's not sitting in the mailbox for longer than it needs to be.
- [14:11]
A
Right. All right, that makes a lot of sense. AJ Williams, product manager at Sublime Security, thanks very much.
- [14:18]
B
Thank you, Tom.