B (3:52)

Oh my gosh, it's so difficult, especially as somebody who's new to security. I think about the show floor, for example at RSAC conference. You've got football field size of vendor booths, they're all over the place. You've got hundreds and hundreds of vendors, they're all cloud, native AI first. You know any buzzword you can pick. And I'm curious because you've got the technology experience as well as the security experience, how did that background as both CISO and CTO shape the way that you evaluate technology providers and vendors?

A (4:39)

I think the, I would say both advantage disadvantage of managing both sides of the floor, both the security bit and the operational bit of it, makes you understand much more about the impact that the technologies are having on your wider IT organization and the users as well and the challenges involving in implementing this kind of technology. So you're getting to a point where you as a ciso, you can't just implement the latest and the greatest in cyber technology and just tell the ops team to implement IT and put the change request and go onto your weekend. Because when you create a problem, you're going to deal with that as well. On the other side of that one, when you're looking from the IT operations bit, you can't just, you know, when something doesn't work, just open all the firewalls, rules to allow all the traffic and you'll be done with it because you're going to deal with the security issues coming out of that. So the stability that is required and the wider understanding of what thing is impacting which other kind of tool that you have in your, in your wider IT state that's becoming IT complicates your life. But I think Ultimately, it makes your decision more mature and the company is doing better for it because it's more stable and it's more clear, clean.

B (5:53)

What it is that you are doing makes total sense. And so when you look for a potential provider or vendor, what are some of the first things that you assess to determine whether they're a good fit?

A (6:07)

Well, I think the first and foremost is that are they solving an issue that I actually need solving. You will be surprised how much that's not a standard thing. I've seen quite a lot of practitioners, and in many cases they're lucky because they either have the width of resources to actually invest in new things just because they're new, so staying ahead of the curve. And they also have the support in the organization to try new stuff, even if they don't solve necessarily a specific thing. From my perspective, I always looked at, am I solving something that I need solving or am I improving something in my existing estate? So it can't be something out of context. So that's, that's the first and foremost thing. I think the, the next thing that you do is that, and that's something that I try very, very hard to do, especially when you deal with new startups, is you either look for references from people that you trust. When I say people interesting, someone from the industry, or you try to figure out who the, the founders of the company are. And I'm talking, of course, not on the mainstream or the big players, you know, like the Palo Altos and Microsoft of the world, because, you know, you more or less trust what they're doing, that they know what they're doing. I mean, hopefully so more of the newer companies, which are less mainstream, you will be surprised how much, or maybe you're not surprised how much the opinions of security practitioners can be different from all the opinions of the various analysts that you are dealing with. So you know all the reports you're getting about which company is doing well against which company. In most cases, they're not giving me a lot of useful information. But when I find out if a CISO A or CISOB are working with them and I can have a conversation with them, that gives me a real good information about how practical it is, what kind of issues they saw. And that's a good opening for me to actually go into a further discussion with that company. I had one specific case where I, I had a CISO friend calling me and running from a big enterprise, and he said, I just implemented a new solution. No one ever heard about them. They're Doing something outside and you need to talk with them. And I did, and it wasn't an outlining solution. I ended up buying them and they actually solved me an issue that I didn't know that I need solving. That was a bit of an unusual thing.

B (8:15)

It makes a lot of sense what you say, and we actually see it in the behavior of people that go to the conference, especially folks who are decision makers inside of the business. A lot of their time is networking, right? They're finding others, they're trying to calibrate with them, they're asking them about vendors or how did you solve this particular challenge? It's such a communal thing, security, I think, based on what you're saying. And I'm wondering what advice would you give to somebody who's trying to sift through all the marketing noise and find real capability? You talked about talking to others, talking to peers, or evaluating the founders. But imagine you've got a project that you have budget approved for, like a zero trust project, or we're going to secure AI and we're going to bring it in house and everybody in their cousin has those things in their description. What other resources can people go to, you think?

A (9:18)

I think try to go into the technical discussion from a very early stage, right? If you're sitting in the initial pitch and you have a salesperson that is giving you half an hour of size, why this is important and why phishing is horrible thing, and why is the big next thing, and you know, the things that you've been listening to for the last 50, about 50 bucks last 30 years, then you know they're wasting your time a little bit. And when you start asking the technical questions quite early in the pitch and you realize based on the answers, whether they actually understand what it is they're selling and they believe their solution, and whether they can go to the nitty gritty details, even, you know, just for the point of interest from the, for me and the, the person or the technical people I'm bringing with me, that's a good sign for me. If it's a salesperson or sales team that have very little technical knowledge and are trying to do the normal sales pitch, that we are the bestest, we are the greatest, and you know, you buy us, we solve everything, it's two days to implement, you tick in the box and you're done and we make you a huge amount of roi, then that's already turning me off, right? Because I've heard all these things before. So it's really sifting through the marketing noise from the actual content that you need to hear. And I think the second thing which is really important is if you can do it, go for a technical proof of concept as soon as you can. It doesn't need to be the entire company, it doesn't need to be huge. But if you just play with the solution, even in a closed demo environment to begin with, you get a very good feeling for if the solution is actually doing what it's supposed to do. Of course, there are more steps on the way and you need to see whether the solution is scaling up to deal with the entire enterprise. But that's already giving you two data points which can be very useful for a decision whether you want to go into a more deeper conversation with that group.

B (11:05)

That's great. That makes a lot of sense. And I'm wondering just to unpack some of the early signals that you might get. Let's say you do this proof of concept. You've talked to them, you've gone into the technical details early. But some of those signals that after you sign a po, after this person, this company, is an actual partner of yours, they're under contract, that they're going to be a good partner, they're going to be there in the hard times as well as the good times. I'm sure you've developed a set of signals or things that you look for. And I'll just give you a personal example. I've been the program chair for RSAC conference for 18 years now, and we get, I don't know, maybe 3,000 submissions to come in from around the world. You have the short abstract and the long abstract. There is a tell always in the long abstract. If it's submitted by a vendor and they're not just trying to share knowledge, like they're actually trying to sort of sell a product, which we don't allow through this process. The way you can tell, it's like a poker. Tell is the last sentence of the long abstract. It always ends with, and we'll show you an example of an optimal solution that's been implemented to solve this problem. It's like as soon as you see that, you know, what signals do you look for from a vendor when you're evaluating them? I guess before you sign the purchase order, that they're going to be with you for the long haul, that they've got the right security maturity, that they're aligned with how you're thinking and the problems that your company has.

A (12:55)

I think one. One thing that is quite obvious is how realistic their Plan of implementation is normally when I go, especially with large scale programs, you ask for the implementation team to actually be present in the negotiation, in the presentation or at least the person is going to run the program. And if that person is sending you, you know, they come with a reasonable gantt and they said this will take us nine months, this will take us half a year. These are the steps and this is what we need from you in order for this to succeed. Because this is not a one way street. Then I already feel better about this. Because you see that person is not trying to sell you fairy tales. They're trying to really do what's right for you as a customer. On the other side of that you have companies that say we'll do it in three months and zero risk or very little risk. What do you need from us? Nothing. You know, we need a steering committee every now and then. And you know, after you've implemented these kind of projects for many years, you know that there's nothing is going along according to the plan ever. There's always things that will fall down, there's always going to things going to break. And it's fine, it's part of, it's part of the life of implementing large scale IT project. But as long as the other party does not recognize that there are going to be issues and they're ready to deal with them, that's a big warning sign for me because they're trying to look as optimal as they can, but they don't show me they can actually deal with crisis.

B (14:17)

That makes so much sense. So just the realism from them on and does it compute with you based on your experience, security? Like does that what this person saying actually line up to what you actually expected? And maybe another question along those lines, how do you ensure that a provider aligns with your organization strategy and your risk tolerance and your operating model? Because those are, those are very different company to company. How do you know if there's alignment there? Maybe you get it out of that same process, out of the operational folks being in the room during the discussion.

A (15:00)

I think it's that one. But also it's part of the negotiation that you do when you already sign the contract. Each company is always trying to make lives easy for itself. And for me as a customer, I can't allow a third party provider to bring their own operational model to overlay that on top of mine. Because by the end of it I need to be able to keep an eye on stability, on resiliency, to report to my management how everything is going and I can't use another reporting system. So from my perspective, I need to be able for them to liaise to whatever system I have in place in terms of service calls, in terms of the definitions of what an incident is, in terms of SLAs, and that's something I have to negotiate. There's always room to move left and right a little bit, right. Because I can't force them completely to give up on what it is they need to do. But they have to align somehow to my, I don't want to say values because it's not values, but let's call it my IT core values. In order to operate in terms of alignment to strategy and culture. I think something that works very well is that you actually open up and you talk to them. As soon as all the NDAs are signed, everything is ready. You need to treat them as a partner, not as a supplier. So that means you tell them what it is you are about to do in the next six months, you tell them how they can help you with that one, you tell them whether they see any kind of risks to their service or maybe even to the wider services. And of course these kind of relationship, the bigger the scope of that vendor, then the bigger the conversation is going to be. If I have a vendor that's doing something relatively niche, of course I'm not going to open the entire IT strategy, but if I have a strategic vendor that they operate most of my estates and they're responsible for my MDR for example, and my operations and I definitely going to have a conversation with them because they might have insights that I don't have because they work on a wider market than me. Let's say I'm going to now invest in a new S4 HANA Real Estate and I want them to tell me what kind of risk they're seeing in, in the sense of security monitor. Right. They've done it before, I haven't done it before. So as an example, I want to

B (17:02)

ask you about startups because you started off by talking about how dynamic this space is. You've got well resourced attackers that are constantly changing, they're upping their game. You've got new technology that's being brought into the business. I'll hit AI only because it's the buzzword of the day. It's this big disruptive force. How do you think about net new problems like I'm thinking about securing AI itself, things like model integrity and how do you deal with prompt injection? Probably you're dealing with startups in the space because it's a relatively new problem, eventually the more mature players will be in it. But how do you balance when you're evaluating a startup that maybe has a solution that you feel like you need the stability of that company versus their ability to provide something that's a net new solution, it's like are they, are they stable enough as a business? How do you check that? What's your appetite for bringing in sort of very new companies into the business? We'd love to get your view on how do you evaluate a new entrant in the space and say are they going to be around even?

A (18:27)

Yeah, it's always a bit of a risk with startups, but I think you can't really avoid working with startups today if you want to keep more or less in line with what's happening in the market. AI is a very good example because this is run very, very fast and almost all of the significant players in the field today are startups or let's say, I want to say mature startups, but we've only been dealing this for the last two years, right? So a mature startup here would be a two year old company essentially. I think there's a few things you can look at first if you want to implement the solution. What is the operational risk of implementing that in the company? Are you going to cause more damage than good? And I work quite a lot of startups and I also seen a couple of VCs to evaluate new startups. If it's a brand new startup, you will not give it right permissions on operational systems or you will not allow to automate processes just because they put something wrong in their code and half the company is going down. So normally you start with these kinds of contracts or these evaluations with a read only visibility type of solutions, advice kind of solutions, but you're not allowing them to automate anything. And over time, when you get a sense of how good the startup is, the people, the knowledge and whether they're able to retain their people as well over time, then you're starting to give them a bit more permission, maybe in specific units, specific departments, you take time until you actually do the whole enterprise just to lower the risks. In parallel to that, you look at the founders and that's what I mentioned before, some of them today are already past several exits or they are very, very experienced in that field. And of course that makes me trust them more that they know what they're doing. And I think also you look at the, at who is actually Supporting in the background, whether you know, the, the investors, either VCs or angel investors, if they have already advisors in their advisory board that you can trust that they're keeping them a bit grounded. One thing I see quite a lot is that they might have a brilliant idea, brilliant technology, but it's not feasible in the real world to actually implement the way that they want to operate. And then you need someone to keep them grounded. This is where you look at their advisory as well. And again, like with the big solution, you just play with the technology and play a lot of it and you grow gradually with the company so both you can enjoy. Right. And you don't just take too much risk.

B (20:40)

Now let me ask you about price because at the end of the day you're looking at a solution, you like it. There's going to be a discussion and a negotiation on price. When you're evaluating cost of a solution that you're going to bring into the enterprise, how should CISOs assess that value beyond the price tag? Like how do you, how do you think about value versus actual cost and maybe how do you communicate it to other stakeholders internally?

A (21:15)

Yeah, it's always a difficult one. I think first and foremost to all the CSO out there is that the, the price you're getting from the vendor, the initial one, is probably 30% higher than what you're actually going to pay.

B (21:25)

Right.

A (21:25)

So don't be, don't be shy on this one and just negotiate as extensively as you can and bring in your procurement people. They're not your enemy, they're your allies in that sense. I think they would, I would say there are very broadly two types of price tags when you look at security solutions. One is a pur. Lower cost to the organization. You can't put any ROI against that. This is something you put to lower risks. I mean, you could say that the value is risk lowering. You can't always put a price tag, but you can kind of estimate. And I had many conversations like that with the, with the company's management or board and said, look, we are investing right now, let's say, for example, on active directory backup or entra backup or you know, tertiary backup, not because I like doing that, it's a headache, but because if we have another not Payti event, want to recover from it. Right. And that's something they can relate to. Especially with my last company where, you know, cars is literally down the road from Maersk, then you have the other type of solutions which can either bring you a real roi, say digital identity Solutions in many cases they can actually improve a lot of the onboarding and offboarding solutions and actually lower the cost you have. Or they can improve the life quality of the users, which there's a big value there, which even if we don't put the price tag there, I think we can't be discard it. It's quite important. So for example, when you invest in a very convenient two factor authentication authentication solution or a passwordless authentication solution, you're making the users your allies because they want to use it. It's much easier to use that then actually start putting, you know, that 12 character password that changes every 22 hours. You can't repeat the 25 past ones. You know everyone's going to hate you for that one. But if you make it easy for them, they will want to cooperate with you. They don't want necessary to be bad corporate citizens. So that I think also has quite a lot of value. So all in all the value is the cost of the solution, the cost of you implementing it, because that's also on top of whatever it is you're going to pay the vendor and how much value you're bringing to the organization in the sense of roi, peace of mind or better life for the users.

B (23:36)

So let me get into a rubber meets the road. So now you've convinced yourself this is a great solution. We want to bring it in, the price matches the value or is less than the value and we need it, right? You know, we've got a real risk to buy down. How do you evaluate whether a partner or vendor can integrate effectively into your tech stack? And I ask this because there's probably limited capacity to do full blown proof of concepts, right? Because then you could really figure it out. How do you sort of weed out early the folks that maybe you shouldn't even go to a proof of concept, not because you don't like the solution, but because you know it's not going to integrate with all of the backend systems that you have or the cloud providers that you've chosen. What are some of the things you do to evaluate that?

A (24:32)

I think in a sense it actually got easier last few years because it's a trend which I find very positive in the industry that most newer systems talk with almost everything else, right? They are like APIs and standards everyone supports. So it got easier than it used to be. It's still not easy, but easier. I think the few things you need to look at first of all is whether they can connect to your IT service system, the ServiceNows of the world because most enterprises will be using something like that at and this is where you're going to manage all of your IT operations. And that solution, whatever it's going to be, is going to become part of it. So that's first and foremost. The other thing is that you need to define a few core systems that whatever solution you're buying has to talk with it. No. Without any negotiation involved. So for example, if you, if it has to talk with your idp, then it has to talk with the idp. Right. You're not going to make any concession. And you know introducing Okta when you're using Entra, right? Just because one one provider doesn't want to work. I had one example that to give an examp example, this one that I had a phishing simulation provider that I really wanted to work with but they were not willing or they couldn't interface with the email reporting with the phishing reporting button I was using, which was something that came from the email security company we were using. And I told them look, as much as I want to work with you, I'm not going to introduce a new phishing reporting button because I spent a long time training the users to actually know where to press. You wouldn't believe how much of a headache it is to now to tell people to use another button because the one they used to is vanishing. There's a few other technical reasons but I will do just a bit. One thing also that I think you really need to look at that maybe more for multinationals is it which languages does the solution support? Especially if it's user facing because that one you will be surprised how much that's not not a common thing. I mean everyone supports English obviously, but when you start asking them about supporting Mandarin or Vietnamese or all sort of languages which are either non Latin or less standard. The answers are becoming very, very different here. And I had for example a discussion with quite a well known DSPM solution which I want to test. They told me we're only supporting English. That was the end of the discussion because most of my data at that point was actually not in English. So what am I going to do with that solution? And somewhat related to that and that's becoming more and more of a challenge in the last, I would say five to 10 years is the regulatory environment. Can your solution adhere to the regulations in the various jurisdiction you're working? And let's say that everyone knows most people know how to operate within the eu, most people know how to operate from the US Most people know the main stuff, but not a lot of people know how to operate in mainland China. And when you have a huge operating in mainland China, you have to know how to operate within the regulatory environment over there. And that's not easy. So there are big vendors out there, you know, not only big vendors, there are definitely vendors out there, they know and there's some vendors that explicitly want to avoid operating in that environment because it's, it's not, it's not easy. And that's another telltale that for a multinational you need to keep an eye on as well.

B (27:39)

And over the years I'm sure you've seen a lot of mistakes. And so I, I'm wondering what are the, some of the most common mistakes that you see organizations make when selecting or onboarding a provider?

A (27:54)

Probably the first one is not using it, or at least not using it. It's a great one actually not using it to capacity because you buy something and then it ends up being like a nice server sitting with blinking blue light somewhere on your rack and you're not actually using anything, right? It's just sitting there getting, getting dust on it. So if you buy something, make sure that you don't only you're not only going to use it, but also use all the features or at least most of the features that you actually need. Don't just buy licenses to make yourself look good. I think a classic example is a Microsoft E5, right? A lot of organizations buy the E5 without actually understanding how much technologies or different types of technologies are involved in there. They end up using Defender for Endpoint, they end up using the main stack of identity, but it's like dozen other things you're not even using which are great why you're not using them. So I think this is definitely that one. I think the other one is making sure that you keep the partner or the vendor interested along the life cycle of the, of the contract. I think you see that and I think you see that especially with service companies, that the interest from the vendor goes down over time. Especially when you're talking about long contracts, the support is becoming a bit sluggish, innovation is going down the drain. You don't really have governance meeting anymore and the quality of the contract is going down. So instead of you educating good relationship mutual updates and you keep to a point where you can get to the end of the contract and either decide if you continue with them or part as friends, you know, in a good way. You know, people just get sick of each other both on the vendor side and from the customer side. And that's a good way to manage relationship.

B (29:30)

Brings me to another question I wanted to ask you, which is related to this, this ongoing relationship. How do you make sure that there's good governance in place after the contract signed? Like I'm sure there's, there's a fair amount of diligence that you do before you sign the contract and the questionnaires, sometimes third parties that go in and evaluate the company or the product or the solution. But when the contract signed, the vendor, the provider could go and change certain things on the back end and it's different from what it was when you originally signed up with them. Any tips for how do you get that ongoing governance of these providers?

A (30:12)

I think that probably the most important tip would be when you sign or when you negotiate the contract, have someone who is a specialist contract management person to actually look. Oh, help you write the contract. Right. Because by the end of it we are mostly, you know, we are IT people, we're security people. We don't necessarily know all the angles of how to manage large scale contracts. If you have someone that put the right governance in place in all the SLAs and the expectations and how would the contract look like when you want to change it, or what the vendor needs to do when they want to change something, or what you need to do when you want to change something? When you have this kind of control that both parties are reasonably happy with, say reasonably, because you're never going to be 100% happy with these kind of contract, then you should be okay, right? Because then you have the regular meetings, you have the vendor that knows that they have to deliver something and you know that you also have obligations as part of this relationship. You know, and that's something I think a lot of people forget. When you sign the contract, you can't just do like this, okay, now it's your problem, right? You know, I bought a from you go and deliver. You have to be an active part of this as well. And if you're not part of it, it's not going to succeed. And if you have this kind of good bilateral relationship and you treat the other side as a partner and you keep them abreast of what's actually happening in your organization or your projects, then it should be okay. Not always, but in most cases.

B (31:30)

Let me expand on that. In most cases. Can you share a time when a provider exceeded or failed your expectations and what you learned from it?

A (31:41)

I had a very substantial MDR security operations provider which ran most of my estate. And they were amazing. And they were amazing because they really wanted to deliver what's best for the customer and not looking all the time or most of the time at the commercial angle of it. Right. You have companies, by the end of it, they want to deliver, they want to maximize on their income, which is fine, it's an approach. But by the end of it, that's not necessarily something you always need. And with these guys, I always had a very practical answer. I came to them one day and I said, you know, I want to look at our sim. It's starting to age a little bit. Do we need to look at that solution which is really like cutting edge? And they said no. And the reason we say no is because you are actually getting a maximum what you need, you don't need anything more at this stage. We are keeping an eye on the market. When we think you need to go to something which is substantially more expensive, we'll tell you that they could have said yes, you know, increase their income by about 20% easy. But they were the people that, they valued their professionalism much more than the income, again, to an extent that they had. On the flip side of that, I had another provider which they got to the point where they. I don't know what it was, maybe because it was a long contract that was running out, they didn't care anymore. And we. They actually caused a couple of significant security incidents because of lack of caring. And. And I was having an escalation call with their head of operations for Europe. And you can see the guy either doesn't understand what I was telling him or didn't care. And I was thinking you, basically, you put my entire Western European network in risk. Right? That's potentially tens of million euros. Why are you guys not rushing to fix it? And that was. I mean, they knew they were already on the way out, but it's also kind of added a note next to their name that I'm going to be very careful working with them in the future in other capacity because of that lack of caring? Yeah.

B (33:44)

I love these two examples. Right, because one of the pieces of advice that you gave at the very beginning was talk to others that have gone through the journey with this company and imagine, you know, that first one MDR provider, they're going to get great references because if they do it with you, they're going to do it with others. And it's just, it's naturally going to build their business the other place, you know, the references are going to come out bad. It just highlights the importance of what you said, that it's important to connect with others and talk to them.

A (34:17)

I think people forget how small and how tribal the security industry is, and these things go around. And I definitely saw that in the, in the last RSAC that I went to last April, that. It's San Francisco, right? You go on the main street in San Francisco, you have like 10 different people calling your name. It's like, I haven't seen you in 15 years. I haven't seen you in two years. It's like a big old reunion meeting. So it's like that. And words on good or bad will

B (34:43)

go around it really well. I couldn't agree with you more. And it underscores the importance of getting together and having that community. And one last question for you. If you had to give another CISO one piece of advice about selecting the right cybersecurity partners, what would it be? I mean, you had so many great nuggets during this discussion. What's the biggest thing? The single biggest thing?

A (35:14)

Choose a vendor or a partner that you can see yourself sitting with in on, you know, in a, in a pub five years from now and still get engaged by them. If you've done that, then, then you're gonna be okay.

B (35:29)

I love that. The pub test has to be right.

A (35:32)

The pub test. Yeah. Yeah.

B (35:34)

That's great. Tal, thank you so much for being here. Thanks for being a part of this. And listeners, thanks for tuning in. Please keep the conversation going on our RSAC membership platform by visiting onersac.commembership and be sure to check onersac.com for new content posted to your wrap. Tal, I can't tell you how much I enjoyed this and just the great wisdom that you're giving to our audience. So really, really appreciate it. Thank you.