A (3:05)

Yeah, and this is very much your research. But I want to put this in context because I think we've both been in discussion, you've had work with them, we've invited the University of Montreal onto our program, we've done things with them because there's a lot of research that's out there that is really just marketing bump. I'm sorry, but I spent 10 years at a publisher looking at some of this stuff and some of it was, oh wow, that, thank you, Mr. Obvious. There's also, geez, there's some things in here that don't seem to be in here because they don't favor your product. And I think that's fair. So people confuse marketing and research. And this is research. And I'm not saying that all companies do this, by the way. I'm just saying everybody has a, a bit of a bias. And by the time you run it through your marketing department, I will tell you, I'm not going to say the name of the company. I have put stuff through one of the larger tech companies before, had it shoved back. And you say we don't use these words, we don't talk about this. So there are filters on the research and I want to point that out. I don't think they're there. I think partly because of your ethics, but also the discipline of doing this as real research.

B (4:15)

And if we do have biases, because there are philosophical positions we've taken, we're doing our best to be radically candid about them. And what I'll say with this is that we are very explicit in how we position the work that we do as security awareness, behavior and culture. We do not call ourselves a human risk management platform on purpose. I think when employees at organizations find out you bought software that's to manage the human risk, it's incredibly demotivating. I think it's dehumanizing. I think it feeds into something that I'm Seeing metastasize from the previous stupid user fallacy or peb tack to something far worse that a fundamental distrust and dislike for people. That's known as misanthropy.

A (5:03)

And yeah, so you've admitted to a fatal bias of caring about people.

B (5:09)

Yeah, that's my feeling.

A (5:10)

We'll watch out for that one.

B (5:12)

We're going to die on the hill of believing in the positive potential of people.

A (5:17)

And yes, and I, and I think that. And I don't want to say it's not all sunshine as well. There are difficult things we do with people. But I think that's a fundamental. If I'm going to have a bias, I'll vote for that one. I want to put this in context here just to. For the listeners, because I think sometimes I worry about this on the program. And by the way, if you're out there, I'll give you contact information at the end. You could send me a note and say I'm right or wrong on this. But I sometimes think we get ahead of it. We think everybody's on the same page as us. So I just want to establish the groundwork. We use the term phishing. What do we mean by that?

B (5:54)

Absolutely. And by the way, dear listener, dear viewer, if you are confused, befuddled about what phishing is, you are not alone. This industry uses this term radically inconsistent for some people. They defined phishing as specifically the delivery of an email containing a link designed to capture user credentials. That is one outcome of phishing. But it's not all. And phishing can happen via email, it can happen via text message, it can happen on social media. It is, at its core, the expert use of electronic communications as a delivery mechanism to do manipulation, to get somebody to do something that is against their interest and benefits you. And it's only slightly more different than marketing in that in ideally, marketing has some value for the customer who is a willing participant in the exchange. But it is psychological, it is manipulative, and it works on a fundamental human level. It works on a biological, neurological, psychological level. And it's the human as the target. And so it can be credential capture, it can be QR codes, it can be attachments, and you're opening payloads and those type of things. Or it can be, hey, here's my updated banking information. And to do something incredibly devastating known as business email compromise, which actually generated six times the economic losses last year that ransomware did. $6 billion.

A (7:43)

Let's go back over those. So it could deliver viruses and malware. Yes, that's one piece of it, right?

B (7:50)

Yes.

A (7:51)

It is an entryway for ransomware. Most people know what ransomware is. Freezes your system, encrypts your data, or it steals your data and you pay people a ransom for that. There's also credential theft, which I think is more and more, as we do stories, this is becoming the way of that. It's not just the killer thing for phishing, but it's a killer thing in terms of security because you put up all of the barriers in the world you want. But if somebody could steal your credentials. We locked the door, but we gave you the key. It's that. I think it's. That's a pretty clear analogy.

B (8:29)

It is. And what's fascinating is that when you read reports like the Verizon data breach incident report, which is also a really great example of as pure research from a vendor perspective as you can get, it's actually if we're all cards down, it is what inspires us at Boseron with our annual report. But they have this interesting feature and report. And so when they look at the root causes of cyber incidents, reused credentials appears as a category distinct from phishing. And the reality is most credentials are still stolen from end users via phishing or they're acquired by a backend hack of an organization and dumping all their data and all that fun stuff. And that's less and less common. So phishing is still, at the end of the day, it is still the kingpin behind the scenes of many of the things that go wrong.

A (9:22)

Yeah, and that's the old give a person a fish and you can feed them for a day. Teach them to do fishing and you can feed criminals for a lifetime.

B (9:30)

Absolutely. And in the industry, God, I hate. This is something I Mea culpa. This is the industry that I work in. We've got these weird portmanteaus of QR code one and it just sounds awkward to say squishing and like, I refuse to indulge it, but it's smishing when it's sms, it's vishing when it's by phone. I'm like, oh, man, can we just call it phishing?

A (9:54)

And it's boring to most people, unfortunately. It may entertain us, but it's boring to a lot of people. I just. So that's. Basically. We talk a lot about email and you said that there's a whole world of this and I think it's because email is just cheap and easy to do. The email, SMS, that stuff, you can pump a thousand 10,000 or 100,000 of those out and you don't have to work a lot to do it.

B (10:19)

The biggest thing is that there is. There's no penalty for launching an attack. The the ability of organizations to then hunt down and shut down a phishing operator is extraordinarily limited. We've seen and we've covered stories this over the last year, in particular phishing as a service sites taken down that had tens of thousands of customers that were automating this activity. But there was no penalty. It's not like when you commit a crime and you're going to burglar a place, right. So you're breaking in the window and maybe a cop's gonna drive by and catch you or there's an alarm that's gonna go off and you're caught in the act. There's zero penalty for every attempt that you have. And you can play it in volume, almost picture like a VLT, just keep pulling or dealing with ChatGPTs. One more prompt, one more prompt and then you just get lucky. And that's the percentages game that we're talking about. And when we talk about the work that we're doing to help build human resiliency to phishing, it's not that you will ever make a human being 100% of the time, 100% immune to 100% of fishes. That's never going to happen. You're never getting a zero click rate. There's no perfect person. But what you're trying to do is get the click propensity down to a point where your technological tools can have the greatest probability of preventative or reactive success.

A (11:45)

Yeah. And so there's really three levels to this. You've got the basic technology of filters and we've all gone through that. And there are a lot of people who say we filter so well, we don't need training. And I think we've, you know that. Yeah, I'll take that as your as not in favor of that idea. And then there, there's the training level, the awareness. And you can take this and you can take the awful thing that I think people do, which is people are our greatest weakness. Or you can say there's another line of defense and that you can use, which is people. I obviously I'm biased towards one of those. And then there's the third was if it gets through, how do we react? Right. How do. What do our systems do That a fair assessment.

B (12:26)

Yeah. I would add a nuance is the ability to tap into your people when it makes it by the filter. Do they know how to tell you something bad has arrived? Are they motivated to do that and give it? And do they get meaningful feedback when they do that activity? Because that's when you know the point of Beauceron is that humans and technology together, and this is a theme that you and I constantly talk about, the right sort of way of thinking about these artificial intelligence technologies. It's not about one or the other. It never is. And you will never find a responsible security awareness company saying by Boseron, do not have email filters. None of us have said that. On the flip side, however, the technologists, the techno utopianists, the folks who believe we're just one more thing from the perfect system. Just one more thing. We've been one more thing for 30 years. They consistently denigrate the work we do on people and it's dangerous. And done.

A (13:30)

Yeah, I think that's the difference between a product pusher and somebody who's a cybersecurity professional. Everybody who's a professional who does this knows you do it in layers and you do each layer knowing something's going to get through. That's just, that's just the way of the world 100%.

B (13:46)

Like, for example, we talk about the flaws of the human, that we may fall victim to a compelling fish because it's about a disaster and we have empathy. Empathy is not a bug, it's a future. Yeah, God helps.

A (13:58)

And we've got to remember that in order for that you to do that, it got by a technology. So something failed and maybe the human failed and maybe something has to pick up. But it's not that the human is the only source of failure in there. We'll come back to that because we'll do that. Talk about the report. But now this, you know, this whole idea that we've got a whole pile of IT filtered, we have a layer that, that is the human response layer. And we know that anything gets past there can do a lot of damage, or people making mistakes can do a lot of damage at that point. So we try to train them and test them. I think that's the world that we live in. And let me correct me on this because I just want to do this, the setup. You basically, you people do training and then they do simulations and testing, fishing simulations. And then from that you get a response. You either get a page that says don't do this anymore, or you get sent to remedial training. That's the construct that's out there in.

B (14:55)

The world that's the common construct. And that's where when Boseron entered the space 2017, we realized there was a fundamentally missing piece. Because in the typical fishing game it's either lose click or draw didn't click. There was never any win to the game. So when we designed our system which was based around the understanding that it's not just about disseminating learning, it's about motivating people to want to expend the energy to practice this thing and to do this thing which is a totally different problem set. And so we came up with this concept of rewarding people when they reported simulations and oh my God, Jim, did people start reporting more and more. And in particular there's a group of people who once they get stung by a phishing simulation and they're feeling some of the negative emotions that can come from this process. They start reporting everything because they're. They go into a hyper paranoid mode and you need a feedback mechanism to give them more confidence to tell them what they're doing. So what we noticed was when you close that feed loop feedback loop of people, you actually improve your report rate by 60% and the quality of the reporting gets better over time. So it's seen as meaningful. And the report a fish button goes from a is it good for the company kind of vibe. If you've seen the movie Office Space, you'll get the reference. I just for those listening, I made the visual banner motion that happened in that very classic 2000s comedy. But when you have a button that actually gives people meaningful feedback, it becomes the staples. That was easy. It was easier to do the right thing than the wrong thing.

A (16:43)

Yeah. And I think so at the end point. The positive endpoint of this training is the. Is two things. One is I don't click. That's good. But the. An equally important one and I think this, this is ignored and you've. Something you've picked up is do you report? Do you report it accurately? Do you report it? So you flood the people who get these. You don't want to do that but do if you report them accurately. Even many systems now if you report something is phishing will then pull that email from everybody. So you can stop a lot of damage by reporting not just. It's not just what you do wrong. You can actually be a defender at that point.

B (17:22)

And this is a great pivot moment. And you still see in some IT departments these walls of shame when they've got mug shots of their clickers and look at all these people. I've got look at all these stupid humans. Oh, demotivational anti human. What we've started to see work for.

A (17:44)

Us on expense claims. Though I must confess I made the wall of shame a number of times in my career. But different story. Sorry, but not on fishing.

B (17:52)

Yeah, but I'm just saying that the gains are not outweighed by the potential losses. When you emphasize the shame activity of that there is a difference between blame and shame and meaningful personal accountability and what that means. But when you have walls of faith. So we did this. We saw this with a police department. Again, that is an interesting. You're talking about a paramilitary culture. You're talking about lots of different rituals, norms and other things. And the chief sent out a memo to the entire team thanking everyone for taking their security awareness training. Why it was important to the mission of the department to protect the integrity of the justice system and the cases they prosecute and highlighting that it was a 911 dispatcher who had the best record that year. She caught all 12. And you want to believe that she felt good about that. You want to believe that maybe we had a national telecommunications carrier create a president's club not just for sales, but for cybersecurity. And some of the deep things that we talk about, the work we do about creating a security culture beyond just delivering training and phishing simulation testing, these are the things you can use these critical moments to shape culture.

A (19:02)

Yeah. Let's talk about some of the findings of the report. And I want to get into a couple of things. One is basically there's a whole group that don't click. 74% somewhere around there and 26% do. That seems to be a pretty consistent piece of data that goes through the report. So that's the basic world we start with.

B (19:21)

So what's interesting is we often start with a much higher click population than that. In fact, you know that University of California San Diego study, when you look at the course of the many months they run their simulation, their click population is significantly larger than 26%. So a really good program get you down to a certain limit. And click population is different than click rate. And it's a new kind of lens that we've added to this. Click rate looks at the template and campaign. And if you think about yourself as a marketing hat for success, how successful was this messaging clicking population is? Okay, of the group of people, how many people potentially would have fallen victim to a real attack over the course of the year? And I think the really cool part is the ability to say 74% of people didn't. That's success. There's a lot of different reasons why they didn't click, but we should just acknowledge that for a second. The vast majority of people post training don't engage. Great. And then you get into of that 26%, how many would happened once the human. And the value of that experience speaks to something incredibly important that I want to zero in on. The experience of the phishing simulation itself. It lands in your inbox, you see it in your routine, in your reality, helps break down biases that we can all carry in. I'm not a target. I wouldn't fall victim to that optimism bias. The experience of doing this helps overcome those psychological effects. And there's a second thing that happens that's completely ignored in a lot of the research because they don't often approach this from a social sciences perspective. In the social sciences, they discovered something interesting many decades ago and it was called the Hawthorne effect. People on a factory floor noticed that were people in lab coats with checkboxes being observed, their behavior changed under observation. Nothing else had changed. Just the noticing that, okay, this is something that's being measured, productivity improved, etc. Phishing Simulations act as the Hawthorne effect inside an organization. They may not believe that a Russian would bother trying to fish them, but they know these simulations happen and they become more vigilant for all emails. To take another analogy that I've used, these are your traffic cops that sit at those great little intersections and they catch you speeding because you just got into that habit. And by the way, speed is something important in this conversation. 40% of all clicks are because people are operating on autopilot. They're speeding on the information superhighway.

A (22:19)

Yeah. Yeah, I think that's fair. Let's go back. Let's back up to that. So this 7426 mix is after training. So basically you're saying the training's having a positive impact. That alone is having a positive impact changing your world.

B (22:35)

Yeah. Let me thank you for chance to dive into this. Just the activity doing simulation has a massive impact because what happens is without training, the percent, the click rate on any given template can be as high as 35%. And if you've got 35% high click rates across a population, you're going to get to 50% plus of the population in a year. Clicking right. So this is how this plays out. The presence of simulations alone does decrease the click rate. So we saw that from our data. When you look at people who don't get training, they have a click rate of about 22.87% across all different templates. And people that have the best training outcome with the best kind of feedback, their average click rate drops down to 13.56%. So you're lowering that pressure. Lowering that pressure means you're going to have less chances that population is going to grow. Does that make sense?

A (23:30)

So it makes a difference, but this degrades over time.

B (23:34)

Yeah, please.

A (23:36)

No, I was just going to say that the training one is not one and done. And this is something I think we get into the research report is. And I think this stems back to your earlier research. It's not one and done, but the other end of that is doing it all the time can have a negative impact. So it's not just that you do the training, you get a great impact, but how often you do it also has a big impact to prevent that degradation or going back to the mean however you want, regression to the mean however you want to describe it. So can you talk about what the report told you about that?

B (24:11)

And this is the research that Michael delivered back to us. This was the incredible gift of taking a smart risk and saying to researchers, because we didn't know what was going to happen, God, here's the data when he comes back and he blows up our work. But we're committed to being empirically led. And so he comes back and he says, okay, I found something interesting. It is the probability that someone will click and the distance from the training that they took. And so this is on page 36 of our report. And so there are two things that he noted. There are two distinct behaviors, the click behavior and the report behavior. And as Michael has taught me, reporting is not the opposite of clicking. The opposite of clicking, the calorie efficient, fastest thing is not to click. Reporting takes a whole series of additional steps. So immediately after training of the percentage of people who would click on any given fish over the course of years, that 26% of people that probability that they will click immediately after training is 3.5%. That's not the click rate. That's the chance that if they're going to click this year, it would happen immediately after training. Ninety days after training, that chance rises to 15% and continues to escalate. When you get to 180 days after training at six months, it's 45% probability, almost a coin toss. At three hundred and sixty days, you hit the peak, which is 95% plus probability the click is going to happen at that point. That's really interesting curve and what it shows is that not that people forgot knowledge, that's not what the problem is. It's their willingness to apply the knowledge, to be vigilant giving all the other competing needs for their energy or put another way, their attention.

A (26:13)

And I think that's an important piece and I want to go back to this because. And keeping in mind those two behaviors clicking and reporting and I think that's because we lose that reporting. We all everybody talks about clicks and they're both important, both behaviors we want the question that you focused on in this report, it's part of your title is from what you know here we do this to why. And I think that why is incredibly important for us in understanding what to do. And I think that's the theme of your report. But I think that's. So why do people they've been trained. We assume and I'm assuming that most people have passed whatever tests to take to get a job in the place they are. We assume they're reasonably intelligent people and they can read and write. I mean that anything that's beyond that can't deal with. But we assume that as a going in position they know they shouldn't, they retain the training. You've seen the stats. Why do they click even though they know they shouldn't or they think they shouldn't?

B (27:23)

And so what's really interesting and to put this into context, Beauceron is a 45 person company. We're the only Canadian owned company left in our space. Which is an interesting time to be that my biggest competitor has 2,500 employees and there are about 150 other companies in our space. It's a highly competitive space and we're the first to ever put a qualitative survey into the field after someone clicked on a simulation in a close ending questionnaire format to ask them why did you do it? And we got 5,000 plus quality responses. We had more than 6,000 but we actually had a question that said please put strongly agree here to make sure they're actually paying attention and not just so we had to weed out about 20%. So this is really good science on this.

A (28:18)

5000 is a pretty good amount to actually come up with a conclusion.

B (28:23)

It is a really good starting point and by the way, bit of humility on our side. Why is really hard when you're talking about humans. And oftentimes the social sciences, I think Michael has pointed this out in the past are often seen as quote unquote the soft sciences. They're not the technology or mathematics or there's no laws of gravity as human behavior. It's moody and complex and interpretive and other things that actually makes it really hard science. But the search for why is where the greatest value is generated and we'll learn and improve and iterate and that's our commitment to the process. So that's why. But here's what people told us why based on the questions we asked. It's the number one reason it looked legitimate. It's about 25% makes sense. Thought it was safe. This is what I do. I get emails, I act on my emails. Perfectly logical. Another almost dead heat. Another 25%. I was expecting something similar.

A (29:28)

And this is a killer. I want to point this one out because this one is one that we work on in our little two person pistons and my wife came down to me one time and said I got this thing from a courier, I didn't order anything and I went oh okay. So we had a conversation with that month later I know I got a parcel out there but this looks weird things you're expecting you're likely to take action on and I think this is a big one in business.

B (29:56)

I think and when we talk about how AI tools are enabling criminals these are the two best opportunities for automation and scale. AI has fundamentally leveled the playing field on grammar, language, formatting, visual look. A low skilled criminal can use a phishing as a service platform. In fact it was Tammy Harper who's on frequently on our panels and everyone can tell us one of those people that I look up to and adore. She took us through one of these platforms in depth and it looked good and it was easy to use. It was criminal SaaS for phishing and it had AI throughout all of it to do exactly that. And here's the other part. AI can take these big data sets of all these data breaches. The compound radioactive effect of all of this exposure on all of us is it is now fed into a machine that knows a lot about us. David the hugh Toronto Maple Leafs fan. No kidding. I'm posted on LinkedIn. I got to see my first game of my life last year. Do I can I reasonably expect a Maple Leafs fish? Probably. It's probably not a good one because I'm calling as a specific example but so those are the two biggest areas.

A (31:12)

But this is big. It looks good. It you're expecting it the average thing in there and I'm getting a lot of them. Now you pay this bill or your cloud, your cloud storage is about to expire. I've just seen lots of them. Most of them are pretty amateurish. But I sent you one and I've been doing this for a long time. I sent you one and I could not see. I knew it was wrong. I just couldn't find out why. And that was scary.

B (31:43)

And the old advice, this is the interesting thing about working with humans and then all the work we're doing to understand psychology and understand brain science in terms of how we often have an anchoring bias. A lot of people we taught 20 years ago when we started teaching about phishing. There'll be typos, there'll be formatting issues. All of those things are gone away. And what's dangerous is that the human brain will always prioritize the first information they've learned about a subject. This is the anchoring bias side. And so they develop these really important energy efficient modes where they're like, okay, what's the anchor say? It does okay, it doesn't have these things I was taught in the past might not be a fish. And this is why we've embraced research around teaching people emotional intelligence. That's a key part of how we do the post click remedial training is that we are teaching people to tap into the power of their humanity, to be mindful, to think about the way that this makes them feel, to engage critical thinking faculties, and to understand how the manipulation could be working rather than teaching people the specific tactics. And it works. You know that, that, that click rate percentage for people that have that as a remedial training is dramatically lower, massively lower than people that get the landing page education and people that get education whatsoever. So it works. Now I do want to say that's 50% of all clicks. The question listener might be wondering, okay, well, when did David work? What about the other half? 26% of clicks are, I don't remember.

A (33:25)

They just did it.

B (33:27)

They just did it. And this is strong evidence of a feature of humans. We burn a lot of calories. Our little meat space data center ramps up big time when you're accessing the neocortex and you're using that phenomenal, unrivaled human intelligence. But it comes with a bill, and that bill is in the form of calories. While we evolved as a species, we didn't always have regular meals. So we developed ways of low energy mode thinking, battery saving mode that relied on gut instinct, remembered patterns, other things. This is the system one versus system two point. And we could, we, we've seen the evidence of it and so it's really powerful and about 14% of people on reflection well and I have enough self awareness to label it as I was rushing but effectively they were also just being efficient. So 40% of all clicks have nothing to do with the training that we have delivered to people with respect to here are the cynicals or indicators or other things because they're not even engaging that part of the brain to do that. So the training now has to be about good email habits, good healthy technology use habits. Taking breaks the greatest gift Microsoft could give us is not another Copilot feature right now that freezes my Mac. First time I've ever had a Mac memory collapse crash it was in this week when I turned on Copilot's cool all its features and literally it was like that meme where Chrome was eating the memory and it crashed in and out. The best thing Microsoft could do is a coffee icon. If I've been in Outlook for an hour that pops up like a distracted driver. It says go take a walk.

A (35:08)

That needs a that's on my Toyota. I don't know if it's on your car but if I'm driving and I'm and I'm moving a lot it the little coffee light pops up and says take a break man.

B (35:19)

Yeah, you know that's a big yeah. Very rarely in all truth like I'm I think this is the true about people who are great thinkers. You'll acknowledge you rarely have truly original ideas. You see a really cool idea in one context and you go geez, this would really make sense here. And yes. So the coffee icon in my car that inspired me to say this would be the coolest human intervention. That would be amazing because the data.

A (35:40)

Says that and you've talked about your quality. Do you have other things in the qualitative survey Because I want to jump to where what I saw in your pod.

B (35:46)

We do have some really interesting oh lastly now I want to figure this from the why people click because this.

A (35:51)

Is really just on the why on.

B (35:52)

The why people clicked 5% click and this is where I was I said this in past episodes when this data first started to emerge that I was convinced that fear was going to be the one of the top answers and it was the lowest answer. It was only 5% of the respondents and it was I was more afraid of the consequences of not clicking than clicking. But we learned something interesting about this group when we drilled into this group their click rate average was massively higher than the other groups.

A (36:19)

Put that but put that into context for the people who are just listening to this is that classically, you've given some reasons why people click. None of them were, I was afraid if I didn't do this, I'd get in trouble.

B (36:30)

Yeah, right.

A (36:31)

It's a tiny percentage.

B (36:33)

Yes.

A (36:34)

It's the one that we would have thought, oh, I have to do this because that's for my buy. I got better click on this. That's a really small percentage. But you're saying we need to unpack that a little.

B (36:44)

But they have the highest average click rate on all templates. So when you compare it to people who are curious, when you compare it to people who said, I just. I expected something similar, their click rate is high. But where they're real noticeable is that they have a dramatically lower chance of admitting they made a mistake. This is the new North Star metric that we're tracking for psychological safety called post click report rate. And it is, I screwed up. Do I tell someone? And Verizon taught us this in 2024. It was one of those moments where you sit back and you're reading a report, you go, oh, that's good, that's clever. But also, oh, my God, they're saying it's only 1 in 10 people will admit they made a mistake. This is the people we need. Absolutely need to know, get help. We made a small change in March and you can see this in our data, because when we first reviewed our data from the same time frame we had the. We replicated it almost precisely. One in ten. Okay, bang on. So key principle here with good science is, can you replicate the experiment? Yes. Then we changed how we nudged people and our rate in this report is now north of 25%. One in four. That is a massive risk reduction for organizations, a massive improvement in psychological safety. And in top performing organizations, we can now see on the curve things like 80% of people who click admit they made the mistake.

A (38:11)

Which is really amazing. Yeah. Except if they're management in you, this really. And because we'll talk about the cultural aspects of this. But that when I was going through the data. You're talking about the qualitative stuff. When I was going through your report, a couple of things jumped out to me and one of them was there's a real correlation in the report that says that one of the things that's going to really make you have a strong culture and success in this is if people feel that management is on board. You called it. I forget what you called it, but it was a great term. But for me, it was what we used to call tone from the top, that people in management care and will behave. And that's. We used to lecture people about this when management people would come and say, oh, break the rule for me. I said, everybody's going to know we broke the rule. And as a cio, I insisted that I obeyed every rule and if I wasn't, I should be called out for it because it's dangerous. So there's that whole piece. But that just struck me was management was more in fear of saying we screwed up. And they happen to be the people that would be most dangerous to fish, by the way.

B (39:24)

And what's interesting is on the whole they perform relatively well because one of the things that we did for the very first time in this report is in some of our competitors reports they had proposed that tenure within the organization, how long somebody was there, had a dramatic impact on click rates. And we went to one of our large global banking partners who takes our data and then was able to in their own environment, because we use minimum viable data in our data. So they take it from us via API into their environment and combine it with a grid chip with HR data. And they were able to look at click rates by tenure. And they replicated and were like, okay, this is worth studying more. They also were able to look at click rates by compensation level, which also had a very interesting impact in terms of turns out, more loyal your people, better they feel about you longer they're with your organization.

A (40:19)

So the longer they're there, the less likely they are to click.

B (40:22)

Yep.

A (40:23)

And if you have a better pay rate, they're less likely to click like.

B (40:26)

And these things intuitively make sense. But what's interesting is that, hey, how you treat your employees has a tangible impact on the number one way you could potentially get hacked. So when you think about things right now, like the amount of stress and anxiety we've seen in reporting about the prolonged changes at Shopify where they were dragging out their layoff cycles and they're creating all this fear, uncertainty, stress and anxiety, the dark side of hr. They were also tangibly increasing their cyber risk. And so one of the cool things is that there's this opportunity for security awareness programs, which for the most part typically sit in the IT function or under the ciso, to have a deep, meaningful conversation with the HR team and to speak their, for lack of a better word, love language around the value of people, the value of employing and empowering our employees to want to stay all that to say high, high, unwanted Attrition rates in your employees creates massive amount of risk. So we wanted to pivot over to some of the, the insights. So you specifically mentioned about management alignment. So there are 41%. That's huge. 41% higher report rates when people believe their managers, care managers and senior leaders. In fact, senior leaders matter the most. They are the cultural standard bearer. This is not new to Bosros. Lots of really good literature covers this. But it's really clear there are 47% higher click rates when people think phishing is just Aki's problem. That's interesting. The second year in a row we have seen a relationship between people's perception that security tools provided by the organization, firewalls, email filters, antivirus, et cetera, completely protect them from threats. This year they had 83% higher report rates. In our first year we studied it was 140%. So that's currently our range, still huge. The people that.

A (42:23)

Okay, let's back out because we're throwing a lot of numbers of people in there. But you're, but there's a code. You're saying if people believe that their it, if this is it, they tend to click more, don't make more mistakes.

B (42:38)

So they believe that cyber and phishing and all this stuff, that's not my job in sales. That's it's job as a specific chin set in terms of roles, responsibilities, et cetera. They have a 41% higher click rate if they believe the tools provided by the organization, that is the car, the Tesla drives itself. Nope, they don't. They click 80%, 83% more on phishing. And this is down a bit, which is good to see. Compared to last year it was 140% higher. But now we've got a range 80 to 140% higher. Ooh, that is. And here's the thing, you need to have that balance. You don't want to have people completely disregard tools and devalue tools because again, tools are important, but they need to have a healthy respect for their limitations on that side. And, and that's just important. And then the other key one, this goes back to what we're talking about, giving people feedback. 62% higher report rate when you give automated feedback. Meaningful by the way, contextual feedback. Yeah, that was totes of fish. That makes sense.

A (43:50)

Yeah. No, let's go back to this feedback thing because I think there's a couple of things in there. If people know that management cares, if people think that if it's part of the culture, if it's the way we do things here. And I'll tell you, I remember sitting as a consultant in one company, I took my foot off the floor. I think it was Dow Chemical, I really honestly can't remember. But they had a big safety culture. I took my foot off the floor because that's how I sit. I tend to when I'm thinking a lot, I'll bring my. I'll just put my foot over my leg or cross my legs. And the guy looked at me, said, don't ever do that here. He said, both your feet are on the floor at all the times. That's part of our culture. That was a senior person. Another one. I did a lot of work for INCO at one point and I was walking around without my hard hat on and one of the people in the mine came up to me and said, I don't care who you are, I don't care where you come from, you wear a hard hat here. If people feel it's part of the culture and everybody feels that no matter who you are, because I was a visiting dignitary in a couple of cases and people still come up to you say, that's not how we do things here. So that, that's an important piece and.

B (44:56)

That'S the power of culture. And it wasn't always that way in industrial sites in Canada. When you think about the west raid disaster here in Atlantic Canada. And then you find out in the inquiry about all of the gaps in safety process and procedure and safety culture and other privilege of working with a number of energy utilities who are customers and they have ones where you don't see people jaywalking across the street to go to their office from that team ever. You can spot their employees in million miles away. Because Fredericton loves its jaywalking and we're small town like for mini Ottawa, but. But not them. In fact, there's a time where the CISO at the time when he spotted people doing the right thing and this is again the power of culture as a leader. They got Starbucks gift cards. Hey, I saw you doing this. You're always wearing a haircut. And you didn't have to give that reward to everybody, you had to give it randomly to people and you tapped in some powerful things. This report, by the way, contains for the very first time a section on security culture. It actually has security culture scores that we have created on a per industry basis. And the security culture score is. We've got a whole other white paper about we're transparent. Here's how we Calculate it. Here's all the inputs, here's why we've chosen the inputs to it. So now we're starting to really draw the relationships between here's Everyone loves the really cool table of industries. Okay, who clicked? Who reported? What's the most clip or right? What's a repeat click rate? How do I measure up? And that's a fundamentally human experience. Right? I love it. But then a lot of the questions start to become, but why is telecommunications in such a rough place relative to banking? And it comes down to culture. And then the question is, how do I change that? And that's where we evolve security awareness training from A, oh, it's January. We can get this compliance check done. Everyone's going to get their training done by the 30th. Let's go. And oh, great, it's done. And two, this matters to us. This is important. And when you do it really well, we're equipping people with skills to protect them at work and at home. And it's interesting and respectful and it never talks down to them. These are the things like, you know how much this past year has just driven me around the bend of the headlines I've seen it says security awareness doesn't work. Garbage phishing simulations don't have value. The juice is not worth the squeeze. If you do them poorly, like anything, you're going to get poor outcome and you can do them the right way.

A (47:33)

The other piece I wanted that I think you came up with in the report that jumped out to me was, and we go through this, people say training doesn't work, but the response to reporting or the response to making a mistake is important. And you've. Your report seems to indicate that the standard thing of I clicked, I go to a page and it says, you're a bad boy. Or I. Or whatever it says, I don't know. But those. That stuff just doesn't work.

B (48:01)

No. And so we, when we created Boseron, we, I was at the University of New Brunswick. I used at the time, the market leader in the space. And I pulled up their analytical reports and at the time, most people were on that page for less than 10 seconds. So on page 34 of our report, we had our. Yeah, I feel like the Avengers. I've got a Hulk. I couldn't take a scientist. It's really good tiny billion guy. And he did this distribution median time on landing page curve, right? And so when you look at that, the arithmetic mean is about 15 seconds, the median is 11 seconds. So when we look at this 11 seconds. What the hell are you going to learn in 11 seconds? No, that's just ridiculous. And only a tiny fraction of people stay on that page for more than 30 seconds. So here's what's interesting and I'll throw this. As my acknowledgement to the University of California San Diego grew, we took our data set and we looked at the people where we had 0 second time on the landing page. So the JavaScript didn't even load to start doing the calculation on the page. And we said, okay, these are the people that represent. There was no training presented to them whatsoever. Never happened. And so their average Click rate was 22%. When you look at people who got the landing page alone, again, average time on the page, 11 seconds, it was 20.35. On a percentage point basis, that's only 2 percentage points. Their argument was always the juice isn't worth the squeeze. And they said this was the state of the art delivery mechanism. Okay, no, my brooch, it's not. But sure. But by the way, like on a declining basis it is 10% less. Okay. You know that 11 second average does get you a 10% drop. Sure. But then you look at what happens if you assign training inside a learning management system that they can't click away from that they get reminders for and they take it. And this is the very much standard vanilla. So you clicked on a fish and this is what you need to know. Yeah, it's not customized, it's generic. Rinse and repeat. That click rate average drops to 18%. Okay, so that is starting to get meaningful. It's less than the 22. Sure. But then when you look at real time remediation, which is a mechanism of delivery delivered via the LMS assigned to the individual that shows the person the fish that talks about mindfulness, that talks about about the cues, that uses the NIST fish skill framework. So we didn't invent this. We again inspired like the coffee icon NIST amazing group peer reviewed academic research said this is how you need to score fishing difficulty. We said what if you taught people this crazy thought that has an average click rate of 13.56%. That's a huge drop from 22. Is it zero?

A (51:05)

No.

B (51:06)

But is of cutting it almost in half by a better delivery mechanism. Yes. And here's the cool thing maybe and.

A (51:12)

So just to stop you there. Sorry, go stop you there. So the, that's what's the big difference you've got One is I show you a page. The other is I walk you through the basic Learning management system. And what's the next level of that?

B (51:26)

So there's three levels to this. So one, they didn't see anything, they just clicked on a fish. They failed. They can get any kind of a feedback loop. So that, that, that click rate average across all simulations for that for those folks is 22%. The second mode is I got a landing page. I did actually, honest to God, see a webpage pop up that had some kind of content on it and that drops down to the 20% click rate. The third you may get in combination you may get the landing page, but what also happens is you get an email in your inbox. So you click on fish. We've assigned you a training module inside the lms. You have to take it and you have to take it within X amount of time prior. And so people go and they're on it. And here's the other cool thing, and this is in our, in our work you can see the percentage declines and the time spent. Remember I said that the landing page average is 11 seconds. The remedial training, the basic supplementary course is about three minutes of time on average. So I know this is a radical concept, but it turns out when people take the time to read something, they retain it and it does have an impact. Now our fourth layer is a customized course that's dynamically generated every time you click and it shows you embedded into the course the fish that you did click on, it explains how that fish worked. Here are the top three cues you may have missed and it says this is the emotions that it was trying to elicit with you. So these are, this is actually, and by the way, for those listening, this is one of the few times I'm going to put a brag out but we have a patent pending on it because it's damn cool and sounds like.

A (53:07)

You'Re using a little bit of AI there David. Getting close. Geez, I don't know, I don't know, but that's great. So you customize it and you say here's what you did.

B (53:18)

Yes, it's dynamically generated at the time of click. And there were competitors that did something along the same broad theme with the landing page as the delivery mechanism. But we the fun, the fundamental UCSD finding is delivering content via a web browser that pops out that people weren't expecting, doesn't work. That's it, that's what they found. And the average time on that real time remediation course, that most advanced version of the remedial process that we have five minutes People engage with it far more. So that's the point that we're saying and we don't think we've even scratched the full surface. This is where we talk about continuous improvement. This is where small changes in courses, visual presentation, maybe a video, maybe other things in that delivery mechanism where we can get that rate down from 13 to 10. This is the hard work that we're going to do.

A (54:12)

Another thing that jumped out to me from the findings was small companies doing it better. And that was, I just looked at that and it seemed counterintuitive. You're a big company, you've got all these resources, you've got the money to hire the best people and all that sort of stuff. And small companies were doing better on average for in terms of their experience with training and with phishing.

B (54:34)

So what our hypothesis is since this has been the way, since we've been measuring this data, doing these reports. So number one, your turnover for employees is lower. People stay longer at small businesses. Number two, people know people, they will turn around and go, did you send me a wire transfer request? No. Great. They're more comfortable, they know you're breaking down barriers, you're doing those things. So on that side of things and their, I think cultures are stronger in many of those respects. So scale and size sometimes works against organizations. In fact, I know it works against organizations like take a tens of thousands of employee bank or telecommunications company, big part of their population are in their call center still or in their frontline service desk and they have very high turnover. These are tough jobs, not necessarily the best pay. You're just starting off in your career, you're going to transit those people in and out. Turns out those are really good people to fish.

A (55:36)

Yeah, yeah. And I think also the idea that my actions matter, I think this is one of the things that kills me about big companies and that people don't get because everybody does this thing and I hear it, it just makes me insane. Oh, we're going to be the world's biggest startup. No, a startup is where everybody knows that the actions they take have an impact on the results of the company that's a startup. And when you treat them still like they're a big glob of people, that you're never going to be a startup. That would be part of my thesis on this.

B (56:06)

Yeah. And then I think if we go back to what we were talking about earlier, in smaller organizations you see senior leaders more often. There's more opportunities for them to demonstrate how much they care about people. Things do these things. I think future directions for the science that we're doing is diving more into that smallness, proving that. And we've got more and more data now with our tenure data and other things to correlate these things back and forth. So this is the future direction. Direction that we're going in is like better understanding, improving. Why small? What can we learn from small on those side of things? And then why specific industries and what are the cultural values in those industries? And how could they cross pollinate to others?

A (56:44)

So just as we wrap this up, what surprised you the most? What was the thing that you looked at and went, wow, I didn't know that.

B (56:52)

Oh, so number one on the. I was wrong, was I? I was the guy who's like, oh, fear, man. Fear is my. Fear is my winner in this horse racing, dead last. Right. So David's not going to bet any ponies anytime soon. But the positive one that I absolutely love because it speaks to the power of the science that we base some of our work on, which is behavioral economics and nudge theory. We changed two small things, Jim, like in an email template. It was just a couple of sentences and wording and the timing in which people would get that encouragement. Hey, don't forget to go back and report this. You can get your points back on your score. And to go from an average of 1 in 10 post click report rate matching the Verizon data set to north to 25% in a single year. And it wasn't even all the ideas we have to work on this nudge factor, but we've got a tangible. I could write an academic paper just on that one tiny change. And here's the really cool thing. How many organizations that we serve with the 1400 caught real fishes where people admitted they made a mistake and gave up their credentials and shut down an incident because of it. I guarantee it's more than one. In fact, I know on our client base we've saved multiple clients from various things and that's it right there. When you can take an idea put into action, see the positive results and know that somewhere down the line this actually did help. That's the best. That's the highlight of my job. The best part of leading the pack.

A (58:26)

Yeah, yeah. I'll tell you, for me, just in terms of dealing with the reports that you got, the one thing that still sticks with me is this idea that when you start scoring people, when you shame them and they have no way back and they'll check out and it's that thing of saying but if you but and this is that tiny little things matter like saying hey, you can get back on track, you can get back, you can get your points back or whatever it is. I think those types of things without them people say dumb things like training doesn't work.

B (59:01)

They do. And they say things like people can't learn. Right. It is without irony at some point I got to get off this hobby horse and just stick with Beat none hacker lore. But again, not without irony that a highly educated, incredibly intelligent medical doctor with a specialization and passion for cybersecurity turned around and told me that it's not worth educating people. Oh my dude. Position, heal thyself. And that's all I will say about that. As Forskump would say.

A (59:33)

Yes. And I admit you admit to a bias that if you treat people well, if you expect the best from them, you'll tend to get it. Because I'm happy going to my grave with that as my bias.

B (59:46)

Because here's the truth. If you expect the worst in people, you will get the worst. And we're seeing too much of that these days. And I just.

A (59:55)

And by the way, that's a 50 year old thing. When I first started in management I was told you get what you expect.

B (60:02)

Yes. And you know what? Again I, I lots of people in the industry will challenge me to come with my receipts and I gots to my seat.

A (60:12)

I'm going to warn them, don't do that. Of all things you do to Shipley, don't tell them turn up with the receipts because you'll be sitting there for a long time walking through the receipts.

B (60:21)

And it's going to go on for a long time. Because I'm petty.

A (60:25)

Yep. And yeah, no Patty be too thorough. And the report is thorough. It's at least 50 pages, 55 pages I think with the appendixes. I'm not, I'm going from memory now.

B (60:34)

Because I yeah Hank is closer on the 40ish side on that side but in depth. We try and improve it. I guess the one asked I have for listeners. If you enjoy the report, if you have feedback, if you want to challenge us, if you want to point out an academic paper that we could be looking at or how is this different from others? And again you don't. We are not that vendor where God forbid like you download a research report and then for rest of your life like you've got like a cultist following you around trying to why haven't you bought my. That's not our gig. That's not what we do, we're Canadians, man. We, we just want to have a conversation and help you. And here's the non vendor sales pitch. My sales team hates when I do this, but it's just, this is my Miracle on 34th street moment. A lot of the stuff that we're talking about here today, you can do this across a variety of platforms. It's the choices of how you choose to use them, how you look at these things. I would argue that because we study and lead in these areas, we do them better. But you don't have to run out today. But I both surrender. We'd love if you did. But you do have to pay attention to the data and the insights you're doing and give some thought as to how it fits into your organization.

A (61:48)

Yeah. And as I said, the report is thorough, it's well written, and if you see it and you see it's got a lot of pages to it, it's a really easy read. I think that was one of the things I would say about it. And there's just, it's like one of those things of getting that the thing where you reach into the box and you pull out something new every time so there's a different surprise. Everything. When does this come out, David? How can people get it?

B (62:12)

So it is available now. It launched Thursday at 10am My marketing team is super excited to see how people engage with it. You know, the funny thing is we created this for our customers. We didn't create it just to be marketing collateral. That's why it has the voice and tone that it does is that our customers said, help us understand how do we grow next. It's actually a tool for my customer success team more than my sales team. And my CRO constantly points this out to you, but I'm like, no. It's also the really cool ways of also giving independent researchers a different data set, something they can reference back. Yes. You can still refer to us in research about some of the things that we found. And if something we've done here inspires you to do an experiment or if you're an ethical identified university scholar and we can talk to you, we are open to having those conversations. We're not exclusive. We love Michael and the University of Montreal, but Michael and the University of Montreal also want more people to do more good science. So yeah, let's go for it.

A (63:14)

And if I was your marketing VP, right now, I'd be saying, David, please tell them the URL.

B (63:22)

Yes. And in fact, I do have that message from my marketing team hilariously. And I actually just have to make sure I remember it because it is bosonsecurity.com report or if you're in Canada. And by the way, this is a sneak reveal. We have launched BoserOnSecurity CA. So BoserOnSecurity CA we are, we are waving the flag and proud of it because there's a lot to be proud of in this country.

A (63:48)

Yep. And you can catch it all as well. I'll put a link to it in the show notes so you can put it@technewsday ca or dot com. We're equal opportunity. We have US listeners. We have Canadian listeners. So Tech Newsday ca. Just go to the podcast page. You'll see a picture of David, I'm sure on there. Click on it. You can re listen to this and you can actually find the link there as well. And probably we'll be. I'm sure we'll be running an ad in our Tech News Day publication with also this. So there's lots of places you can get this. Check it out. Not just because David's my friend, but it really is thorough. It's really well, well done and worth your time.

B (64:24)

And I'd say that if you're a security professional and you've had bad experiences with other cybersecurity, marketing and other things that have happened, you want to connect with me on LinkedIn and say, David, I just want your report. Can we connect on LinkedIn and I will give you the report. You don't have to fill out the gated content and all that. We'd love if you did. But if you're like, I had my fill. I just want to read your research and think about whether you're worth continuing to chat with. Have the conversation.

A (64:50)

Great stuff, David. Thanks a lot. It's nice to see you in the interviewee seat. You'll be back on Monday morning with the news and. Which allows me to have Sunday night off, so. Which I'm eternally grateful.

B (65:04)

I listen. I get to scratch my inner journalist. But I could tell you it is also sometimes really good to be back on the other side of the camera and thinking about that. So appreciate this. Thanks for the chance to be a nerd for. For data.

A (65:18)

Yeah. And thanks for you who are listening out there. If this is your weekend, then we're pleased to have shared the time with you. I'll try and have edited this down so that we make it as go as cleanly as we can. But we obviously enjoy this chat. We hope you do too. Always willing to hear from you. Always love to hear from you. You can again. Technewsday ca Go to the contact us form. Let me know what you think. Aside from that, have rest of your weekend. If you're listening on the weekend, talk to you later. I'd like to thank Meter for their support. We're totally supported by your donations and sponsors who will only ask for a mention and no editorial control at all. All we offer them is a description of what they do and in this case, let me go ahead with that. Meter delivers a full stack networking infrastructure, wired, wireless and cellular to leading enterprises. Working with their partners, Meter designs, deploys and manages everything required to get performant, reliable and secure connectivity in a space. They design the hardware, the firmware, build the software, manage deployments and run support. It's a single integrated solution that scales from branch offices, warehouses and large campuses all the way to data centers. Book a demo@meter.com CST that's M E T E R.com CST and that way they'll know you found them through our show. I'm your host, Jim Love. Thanks for listening and have a great weekend.