The Future of Offensive Pentesting with Mark Goodwin

Summary

Podcast Summary: The Future of Offensive Pentesting with Mark Goodwin

Title: Software Engineering Daily
Episode: The Future of Offensive Pentesting with Mark Goodwin
Release Date: December 12, 2024
Host: Gregor Van
Guest: Mark Goodwin, Director of Operations at Bishop Fox

---

1. Introduction to Offensive Penetration Testing and Bishop Fox

The episode begins with Gregor Van welcoming Mark Goodwin to the show. Mark introduces himself, highlighting his background as a former US Air Force officer specializing in cyberspace operations. He has been with Bishop Fox for four and a half years, where he oversees operations that focus on offensive security testing.

Key Points:

• Offensive Penetration Testing (Pen Testing): Actively probing systems to identify and exploit vulnerabilities, mirroring tactics used by real-world attackers. The objective is to uncover security weaknesses and provide actionable insights to bolster defenses.
• Bishop Fox: An 18-year-old professional services firm renowned for its offensive security testing. The company emphasizes continuous training, tool development, and maintaining cutting-edge offensive capabilities.

Notable Quote:

"I consider it the future of offensive pen testing to be able to stay up with nation state actors, follow emerging threats and identify attack surface that customers may not know about." – Mark Goodwin [03:40]

---

2. Evolution of the Cosmos Platform

Gregor delves into Bishop Fox's Cosmos platform, a significant development in their service offerings. Initially, Bishop Fox was predominantly consultative, but Cosmos represents a shift towards a scalable, platform-based approach to continuous offensive security.

Key Points:

• Genesis of Cosmos: Launched around March 2020, initially named CAST (Continuous Attack Surface Testing), Cosmos aims to automate and scale offensive security efforts without compromising on quality.
• Three Core Components:
  1. Automated Attack Surface Discovery: Continuously identifying and mapping out all potential attack vectors across a customer's digital footprint.
  2. Automated Application Pen Testing: Focusing primarily on web applications, automating the detection of vulnerabilities while reserving critical findings for human analysis.
  3. Internal Pen Testing (Future Development): Planned enhancements to include internal network assessments, enabling deeper penetration testing within a client's infrastructure.

Notable Quote:

"We don't want to build Ultron, we're building Ironman, to propel us forward." – Mark Goodwin [04:58]

---

3. Automation and Scalability in Pen Testing

Mark elaborates on how Cosmos leverages automation to handle large-scale attack surfaces, enabling Bishop Fox to serve enterprise clients with extensive digital infrastructures.

Key Points:

• Efficiency Through Automation: Automates routine tasks like subdomain enumeration and initial vulnerability scanning, freeing up human operators to focus on critical and complex issues.
• Prioritization of Vulnerabilities: Uses intelligent filtering to identify and report high-impact vulnerabilities, ensuring clients receive actionable and relevant findings without being overwhelmed by noise.
• Integration with Threat Intelligence: Continuously updates the platform with emerging threats, leveraging resources like CISA KEV (Known Exploitable Vulnerabilities) and industrial research to stay ahead of adversaries.

Notable Quote:

"We're continuously looking for Attack Surface to bring in. We're scanning customers to make sure that we're having a right picture of their attack surface." – Mark Goodwin [24:38]

---

4. Handling Subdomain Takeovers

Subdomain takeover is a critical focus area for Cosmos, given its potential to expose organizations to significant risks.

Key Points:

• Definition and Impact: Subdomain takeover occurs when an attacker gains control of a subdomain due to misconfigurations, allowing malicious activities like phishing or credential harvesting.
• Automated Detection: Cosmos automates the identification of vulnerable subdomains, ensuring timely remediation before they can be exploited.
• Case Study: Reference to a recent incident involving Watchtower in Singapore, illustrating the real-world implications of subdomain vulnerabilities.

Notable Quote:

"If I was hosting something clients.example.com and then for whatever reason I misconfigured... it can now become a phishing angle or a way to harvest credentials." – Mark Goodwin [16:08]

---

5. Collaboration and Customer Engagement

Mark emphasizes the importance of collaboration between Bishop Fox and its clients to maximize the effectiveness of their pen testing services.

Key Points:

• Customer Involvement: Encourages clients to actively participate in verifying and managing their attack surfaces within the Cosmos platform.
• Transparent Communication: Maintains open lines of communication to ensure clients understand the findings and the necessary remediation steps.
• Building Trust: Moves away from the traditional "hand over a report and walk away" model to a more integrated, collaborative approach, fostering trust and long-term partnerships.

Notable Quote:

"We're here to help. We want to keep you safe. The best way to do that is open communication." – Mark Goodwin [28:35]

---

6. Managing Stress and Workload with Cosmos

The introduction of Cosmos has implications for the workforce at Bishop Fox, particularly in managing the stress and workload associated with offensive security operations.

Key Points:

• Reduction of Manual Effort: Automation handles repetitive tasks, allowing security professionals to focus on higher-level analysis and creative problem-solving.
• Prioritization Framework: By automating the identification and prioritization of vulnerabilities, Cosmos helps in managing the backlog of work, ensuring that critical issues are addressed promptly.
• Supportive Leadership: Ensures that team leaders are equipped to make informed decisions about vulnerability prioritization, alleviating individual stress.

Notable Quote:

"Because we're leveraging five plus years and lots of operators and analyst input, we're able to say we believe as a service we're looking at the most important things. So it takes off that stress off the individual and really puts it on the shoulders of the leaders to make sure that we're making those right choices." – Mark Goodwin [32:15]

---

7. Regulatory Compliance and SOC 2

Gregor inquires about the impact of regulatory frameworks like SOC 2 on Bishop Fox's operations and Cosmos platform.

Key Points:

• Compliance as a Driver: SOC 2 compliance has necessitated rigorous internal processes and best practices, ensuring that Cosmos operates within defined security standards.
• Enhanced Service Quality: The adherence to regulatory requirements has streamlined operations, making the platform more reliable and trustworthy for clients.
• Focus on Data Protection: Aligns with Bishop Fox's mission to protect customer data, reinforcing the importance of maintaining high-security standards.

Notable Quote:

"SOC 2 forces you to figure out, hey, here are the non-negotiables and how do we make sure we're doing these right so that we can continue to fly and do the right things." – Mark Goodwin [35:50]

---

8. Future Directions and Innovations

Looking ahead, Mark shares insights into the future developments planned for Cosmos, highlighting the integration of advanced technologies and continuous enhancement of service offerings.

Key Points:

• Next Phase of Cosmos: Transitioning to a more self-service model, allowing clients to interact with and analyze their data more independently.
• Expansion of Service Lines: Introducing Cosmos Chasm for continuous attack surface management and Cosmos Capt for ongoing web application pen testing.
• Incorporation of AI and LLMs: Exploring the use of artificial intelligence and large language models to further automate and enhance vulnerability detection and analysis.
• Staying Ahead of Threats: Continuously adapting to emerging threats by leveraging threat intelligence and maintaining agility in response strategies.

Notable Quote:

"Emerging threats are going to be the thing that continues to set us apart and set continuous pen testing apart." – Mark Goodwin [41:52]

---

9. Personal Insights and Advice

In the concluding segment, Gregor asks Mark for personal reflections and advice based on his extensive experience in the cybersecurity field.

Key Points:

• Embrace Continuous Learning: Encourages professionals to continually enhance their technical skills and understanding of complex problem sets.
• Importance of Leadership: Leaders should strive to understand the technical challenges faced by their teams to foster effective collaboration and support.
• Value of Hard Training: Highlights the benefits of rigorous training and the importance of partnering with highly skilled individuals to drive success.

Notable Quote:

"Embrace that hard training and know that it's worth it because from my experience, those individual contributors will lean in and partner with you." – Mark Goodwin [42:45]

---

10. Conclusion

Gregor wraps up the episode by thanking Mark for his insightful contributions, emphasizing the value of Bishop Fox's innovative approach to offensive penetration testing and the promising future of the Cosmos platform.

Key Points:

• Collaborative Future: Reiterates the importance of collaboration between security firms and their clients.
• Continued Innovation: Anticipates ongoing advancements in automation and threat intelligence to enhance pen testing capabilities.
• Commitment to Security: Affirms Bishop Fox's dedication to maintaining high-security standards and protecting clients from evolving threats.

Notable Quote:

"It's a really good call out. It's changing the mindset of what a service such as Bishop Fox can do, which is its collaborative like end to end and security." – Gregor Van [44:24]

---

Conclusion

This episode of Software Engineering Daily provides an in-depth exploration of the future of offensive penetration testing through the lens of Bishop Fox's innovative Cosmos platform. Mark Goodwin offers valuable insights into the evolution of automated pen testing, the significance of collaboration with clients, and the ongoing challenges and advancements in the cybersecurity landscape. Listeners gain a comprehensive understanding of how continuous offensive security measures are shaping the industry's approach to safeguarding digital assets against sophisticated threats.

Transcript

[Narrator]

Offensive penetration testing or offensive pen testing involves actively probing a system, network or application to identify and exploit vulnerabilities mimicking the tactics of real world attackers. The goal is to assess security weaknesses and provide actionable insights to strengthen defenses before malicious actors can exploit them. Bishop Fox is a private professional services firm focused on offensive security testing. Mark Goodwin is the Director of Operations at Bishop Fox and he was previously an officer in the US Air Force where he did cyberspace operations. Mark joins the podcast with Gregor Van to talk about Bishop Fox and the future of offensive pen testing. Gregor Vand is a security focused technologist and is the founder and CTO of MailPass. Previously, Gregor was a CTO across cybersecurity, cyber insurance and general software engineering companies. He has been based in Asia Pacific for almost a decade and can be found via his profile at Vand hk.

[Gregor Van]

Hi Mark, welcome to Software Engineering Daily.

[Mark Goodwin]

Hey, thanks for having me.

[Gregor Van]

Yeah, so great to have you here today. Mark. You are at the company Bishop Fox, which we're going to hear all about and understand about what is called the Cosmos platform and really just what Bishop Fox does in general. I think one of the first things would be really just interesting to hear about. Obviously some details I don't think you'll be able to go into, but just your background is very interesting and sort of how you then ended up at Bishop Fox. So maybe if you just sort of introduce yourself a little bit in terms of, I don't know, straight out of college or high school, what did you do?

[Mark Goodwin]

Yeah, again thanks for having me on. My name's Mark. I've been with Bishop Fox for four and a half years. I graduated college back in 2012, 2013, immediately went to the Air Force to be an active duty officer where got to do offensive cyberspace operations. So a lot of late nights, a lot of long days and learned a lot about threat posture. How do we go after hard targets in ways that keep the toolkit safe, keep the platform safe and you need to go after nation state objectives. So did that for six years, six and a half years and at the end of my time in the Air Force I started applying and Bishop Fox was starting this new service called Cosmos and it was all about continuous offensive security at scale and you know, definitely very exciting. Very. I consider it the future of offensive pen testing to be able to stay up with nation state actors, follow emerging threats and identify attack surface that customers may not know about. So like I said, I joined Bishop Fox in March of 2020, started as a senior analyst running the analyst team, which is all about discovering attack surface, did that for about a year and a half, led the operator team, which is all about taking that intelligence that we derive, that attack surface that we see prioritizing potential vulnerabilities and providing true positives and true negatives. Right. So true positives, here's a finding that we know exists, that we've validated and we've proved you're vulnerable to with a finding or we give them a true negative to say we went and identified that this host is out there. We thought it was going to be vulnerable and it wasn't. So, you know, you're good. Right? Your defensive team is doing the right things. They're applying the patches. This misconfiguration that we thought we saw didn't actually exist. So we're able to provide that safety blanket to say even if you're not getting findings, you should be getting something to say. We're looking at your attack surface and we don't see anything.

[Gregor Van]

Yeah, super interesting. So I'd love to understand. So you've given a great sort of quick overview, I guess of Cosmos. My understanding is that Bishop Fox started as more like consultative company and the platform aspect has kind of developed over a significant amount of time. I believe Bishop Fox started back in like 2013 or around then.

[Mark Goodwin]

I believe it's older than that. I believe we're an 18 year old firm. So yeah, they, they have been doing consulting for a long time that has been, that is how we've made our name. We bring on some of the best offensive mind in the country across the world and let them work on really interesting projects. We dedicate time to training and you know, developing tools that is going to make the offensive cyber community better. I have to shout out Joe Demessi with his sliver tool kit that he's built. Pretty exciting and you know, we use that almost daily. So you know. Yes, Bishop Fox started as a consulting. It's still our primary way to get after securing customers, keeping them safe and protecting their customers, data and trying to find vulnerabilities. You know, we have our consulting firm and then we have the Cosmos firm and they're both, you know, continuing to grow.

[Gregor Van]

Amazing. What was the, I guess do you know, kind of what was the moment when it was thought actually bringing, turning sort of this knowledge into a platform made sense and like what was the kind of the thought process behind that?

[Mark Goodwin]

Yeah, so when I joined In March of 2020, we had been doing, we had called it Cast, which is like continuous attack surface testing. And then we had been doing that for almost a year. And I think it started from this mind of like, you know, how do we take our insanely skilled consultants or operators and give them a suit for Ironman, Right? Like, we don't want to build Ultron where we're just automating everything, but how do we instead enable and make our operators better? And so that was kind of the driving force. So it's on our very old documentation that from the way back times we have that, you know, we're not building Ultron, we're building Ironman, you know, to propel us forward. And so part of that was looking at enterprise customers, right? Their attack surface is huge. How do you look across all of it when they just need help identifying what is the most important thing? Right. We only have so many hours in a day, in a week, in a year. How do we find those things that advanced actors are going to go over? Right. Hackers by trade, very lazy. We know that if we can take the knowledge of 10 operators or 10 consultants, consolidate that data, make it easily attainable, take off some of the scanning that like every external pen test starts with, what if we instead offload that to automation so it is on a weekend, or you know, smart scanning, as we identify new targets, it gets scanned and then it prepares that data for us. So I'm not wasting cycles waiting for a scan result to return. Instead that data is there ready for me. And then when I go to pick up the work, because we've had five other super sharp consultants or operators do a similar investigation, we get a jumpstart on what we're looking at.

[Gregor Van]

Yeah, So I think this is where it gets super interesting. So my understanding is the platform has. It kind of has like three parts to it. So you've got the automated attack surface, you've got the automated application pen testing. And then does this understand also automated internal testing and pen testing, is that correct?

[Mark Goodwin]

Right now we're not at the automated internal pen testing yet. That's definitely on the roadmap though. How do we do that smartly? How do we make sure that we drop into a right location? And again, it's for Cosmos. The entire point is scalability, Right. I need to be able to do this on five targets, 500 targets or 5,000, and way beyond that. Right. So right now the main focus is that web app pen testing, doing the research on customers. Attack Surface, we know it makes up the majority of their attack surface, right? There is going to be SSH and FTP services out There that may be vulnerable and we're going after those. But most of a company's attack surface is going to be web based. So that's what we spent the first four years really focusing in on, making sure that we can automate what we should like, hey, this is an exposure, you need to go fix this. Preparing the work for the team of, hey, we've identified that there's a version mismatch or there's a potential SQL injection here. We're not going to throw that. We want to keep that to be a human decision, right? Because at the end of the day the last thing we want to do is knock over customer infrastructure, right? Like that's one of the worst things that could happen. So using this insight, using this, you know, prioritized attacking, we're able to say like, hey, we're going to automate these things that we don't care about. You need to fix it, right? Directory listing is something that you should not have. Like that's not a good configuration because of the potential to put something sensitive there. But we don't necessarily have to go look at that, right? We can build automations that will look for directory listings, tell you about it and then look for things that were exposed and identify like was there something sensitive here that needs follow on work. So you know, continuous web app pen testing is what we're started on. Continuous external pen testing is kind of that next thing of all. Right, now that we've got web app pretty much locked in, now that we know what we're looking for, how do we look for other services, you know, SQL and other endpoints that may be a little sketch to be out there. How do we start targeting those and take care of those?

[Gregor Van]

So when it comes to the automated pen testing, and I might just sidebar for a second, I mean this is what could also be classed as effectively automated red teaming. Is that a good way to think about it or.

[Mark Goodwin]

I think it's definitely nuanced. I think there are some portions that could be considered that when I think of red teaming, again with the background that I have coming from the military, I always think of red teaming as this like, you know, very high stakes. I need to move silently, I need to, you know, I'm competing against that blue team, right? If I'm a red team, I'm competing against the blue team. I don't want to get caught. So red team will have to operate and act and make different opsec decisions, right? For our continuous external pen testing, for our continuous web app pen Testing, I hope you find me. That would be a good thing, right? Like if you have smart filtering or smart logging to say, like, hey, we see this thing, this isn't good, drop those packets, that's a win for you. So that's kind of the nuance, I would say. But the methodology is still there. Right. Like I want to be safe, I want to make sure we're not knocking over infrastructure. I want to make sure that we can get the data that we need out without causing logging to occur because it makes our job easier.

[Gregor Van]

Yeah, I think that's a good nuance. I mean some of our listeners, whilst very technical, security is probably not something they actually have had to touch a lot. So, yeah, the sort of term of red and blue teams. Yeah, Red being the offensive and blue being the defensive. And I believe there's even at this point now purple teaming, which is.

[Mark Goodwin]

Yes, there's purple teaming, where the blue teams come together like, hey, we want to see if we can identify this attack patterns. Right. The offensive landscape is huge. You're not only competing against white hat or even gray hat security researchers, you have black hat actors out there that are willing to cause harm and willing to like, ransomware is on the rise and at an all time high. Right. So purple team is this way to address that where you take red team folks and say, run your playbooks with us. Right. Like tell us when you're going to launch an attack so that we can look at our logging and figure out like, do we have these systems in place to catch that? So yeah, sorry about the nuance. Right. Like red teaming to me is a very like specific subset of offense. But I would say everything that we're doing is on that red team offensive side.

[Gregor Van]

No, that's great. I think it's really good to be able to explain these. As you say, it is nuanced. So I'm just kind of interested to understand maybe more how did Cosmos evolve? So, I mean, it's Cosmos, I think you said it's now been in, I guess production for four to maybe five years. Is that correct? Right, yeah. Could you just walk us through almost, I don't know, year by year, if that's possible. Yeah. Where did it start and what was added and why? Because I'm just super curious how you take sort of this human process and then decide what and when that's going to get automated and brought into this platform. So yeah, if you could walk us through that.

[Mark Goodwin]

Yeah, it's an ever changing process. I know since being here in 2020 what we do now is not vastly different, but it is significantly different than what we did, you know, back then. When I joined the team in 2020, we're operating out of just the goods, right? We're only going to provide findings for things that are highs or critical severities, right? Like this is the pull in your folks. Like they don't get to take a weekend, they have to fix this now. Everything else was a notification of, hey, you should probably fix this. And as we continue to work with customers, we realize like there is a wider range of severities, right? Like it goes from informational all the way up to critical for a reason. So around 2021 or early 2022, we started reporting low to critical. We still don't do the informationals just because there's so much noise associated with that. And again, Cosmos is all about the validated findings. If we're writing you something, it's something you actually need to go take a look at. So after about being here for around a year, we started reporting these low to criticals. And so that was more work coming in as we were getting more customers. And that's where we started looking at. All right, how do we identify those vulnerability types that we should automate? Like, does it really bring value? I gave this example earlier, but does it bring value for me to look at a directory listing and say like, yep, there's folders in there with files, right? Like that doesn't really always equate to something good, especially if it's something that is relatively benign, like a CSS folder, right? Like very rarely is there ever going to be anything sensitive in there. But you do need to be aware of it so you can fix it in case you have an intern or even somebody super senior that goes to drag and drop and puts it in the wrong folder, right? Like there has been findings that we've reported where like, hey, we found passwords in a docs folder, right? Like that's not good. You need to make sure you don't do that. So those become two findings. And so it really became, you know, out of survival and scalability, what do we need to automate? And so it is those like these low level, medium level of this is an exposure that you need to fix. This is a system configuration problem and we're going to save that harder follow on work for the operator cycles, right? We let our analysts make sure the target's in scope and then our operator will do the work to make sure that like, hey, here's the full business Impact of what we found in these files as we continue to grow. Around 2022, we started automating subdomain takeovers. Again. This is one of those vulnerability types that doesn't take a lot of time, but you know, a customer lifecycle. With Cosmos you get onboarded into the platform. My analyst team goes out and identifies all of your attack surface that they can find, right? They use open source intelligence, they use publicly identifiable information to figure out like, all right, this person is in your org, this person has registered this domain through whois information. So we believe this domain follows. We do this work and we would always get this huge spike of trashed DNS records, stale DNS everywhere, domains that were for sale that used to be registered and had clear ties to a customer or just again, misconfigurations where you have subdomain takeoverable subdomains. And so because we were at a point where we were getting a lot of new customers coming in, we looked at that and was like, all right, let's prioritize subdomain takeovers. And so we spent some cycles to report it out as fast as we could. Because there's this other entity out here that we're working with called Bug bounty, right. You know, there's a lot of bug bounty organizations out there that are also trying to help keep customers safe. And so, you know, we bring in a mature, validated findings. We have, you know, quality assurance checks. There's a lot of like, not a lot of bureaucracy, but there are a lot of checks to make sure that everything you get still is that Bishop Fox standard. Where with Bug Bounty there's not always that, right. Like it can be five people just trying to get first report in so they're not considered a duplicate. And so that's how we've kind of driven our automation approach. It's all right, what are the low weight things we can knock out and then what are through a customer lifecycle, what are the high impact things that take up too much time that really don't need to be there? Right. Like if I can write a script that can go out and validate this vulnerability, do I need to go look at that or can I use my creativity somewhere else? And that's what we always try to lean back on. Is, is this a creative exploit or is this just something that with enough time and repetition, we can train and build a good enough tool that can tell a customer about it so that they can go fix it?

[Gregor Van]

So yeah, I think a lot for the listener base, the subdomain Takeover, I think is super interesting and also very relevant given some news that came out recently around a company called Watchtower, who's actually based in Singapore, being able to take over the Mobi whois records, I believe. Can you elaborate on why was subdomain takeover so important for Cosmos to focus on? And what does that even mean? And what can someone do if that is brought into effect?

[Mark Goodwin]

I guess, yeah, it's not the first order, it's the second order effect. Right? If I was hosting something clients.example.com and then for whatever reason I misconfigured or the application, like the hosting provider that I was using lapsed and that clients.example.com instead gets hijacked or taken over by somebody else, it can now become a fishing angle or a way to harvest credentials from customers that are trying to log in. And so it's bad in that it hurts customer, it would hurt our customers public perception and making sure that we're keeping our customers safe. We try to approach it from that angle.

[Gregor Van]

But when we talk about subdomain takeover, I guess is there a reason why the subdomain is more vulnerable or say this subdomain, but if you have access to the DNS of a domain, then you can just create whichever subdomains you choose. So are we talking about both effectively domain takeover and subdomain takeover, or how does that look?

[Mark Goodwin]

That's a great call out. So it's both, right? Azure subdomain takeovers, you're not going to be able to take over a domain that points back to an Azure. But if they misconfigure their Azure instance, then it would be possible, given that the right things appear when you do a dig, you're able to investigate like, all right, this is vulnerable because of the configuration that they have. And then you can do work through the API gateway to take it over, right? There are other applications where you go out and it's a third party that you're hosting on, and if the third party is vulnerable or you didn't complete a setup, Shopify is a great example, right? Like if you don't complete a Shopify setup, you're able to come in there and scoop it up and now that's your subdomain. Congratulations. So there's a lot of examples out there and they kind of run the gamut of yes, they are harder because you actually need the domain. And that's why we also report on like, hey, this domain is for sale, right? Like it used to belong to you. It would make sense that either you make it very clear that it no longer belongs to you or you go out and you purchase it just defensively. There's a subset of offensive security called like brand protection. That's not something my team does, but our customers ask us about it and we try to lean in and help where we can, but because it's like more defensively focused, we don't do it. But we're able to say like, hey, you know, the common things to look for are typos, squatting, you know, off by one character, those kind of issues.

[Gregor Van]

Yeah, that's a very good way to explain that.

[Height Ad]

Introducing Height, the only autonomous project management tool. Backlog grooming, bug triage, keeping documentation up to date. Those aren't why you got into product building, right? Well, height handles all that grunt work for you. Using a first of its kind AI approach, Height proactively takes care of time consuming workflows without you lifting a finger. Height recognizes when you've agreed to trim, scope and handles, mapping the necessary edits back to your product brief. When new tickets are added to your backlog, Hyte combs through them, adding feature tags, time estimates and more. And it's not just you. Everyone on your team manages projects, tracking updates, scoping work, balancing priorities. But whether or not your product succeeds shouldn't depend on project management. With height, autonomous workflows handle that mundane upkeep so your team can focus on building great products. If you're ready to stop managing projects, it's time for height join the new era of product building where projects manage themselves. Visit Height app sedaily to get started.

[Gregor Van]

So when we think about, you know, you mentioned having enterprise customers and yeah, if you go on, you know, Bishopfox.com, you can see a whole bunch of names there. How do you approach that when it comes to like companies with such large attack surfaces? You know, whether you maybe want to bring out any examples that you can or just talk at high level about a sort of industry. I've worked on a tax surface, but I'd say probably this was a tax surface of fairly small companies. So I mean, maybe the largest I think I worked on was Singapore Airlines and it was large.

[Mark Goodwin]

Ish.

[Gregor Van]

But I wouldn't say, I think for probably the kind of cases that you work on, it was what would be defined as large actually. And we did find a couple of interesting things like tons of staging sites and that kind of thing. And to your point, it's not that critical, but there are various things where you said, look, if someone is looking for as much information as possible to try and say spoof being Singapore Airlines, they've got a ton of resources now, so you might want to just lock that one up. So yeah, talk to me about maybe some of your largest attack surfaces and what were the actually, because I think Bishop Fox talks about the needle versus the haystack. We find the needle, not haystack. What were some of the needles, I guess in enterprise?

[Mark Goodwin]

Yeah, that's a great question. Our largest attack surfaces are six to seven digit targets. Right. So that is a scheme IP or domain and then a port. Right. So HTTP on an example.com and then HTTPs example.com on 443. That would be two targets. So we look at it as a whole though, because we've seen through investigation and, and the reporting that we've done that, you know, port 80 and 443 may serve up different information. And so we want to take that unique approach to each one of them. The way that we find that needle in the haystack goes all the way back to my analyst team. So they are the ones that are tasked with identifying attack surface. Right. So we do this breadth first and then we go depth. We start with breadth of customer. A has a lot of subsidiaries. So let's go out and identify each one of those. You know, we use things like SEC filings if it's a US based company. If it's not, we get to research like, all right, how does Germany do their filings for what customer owns different companies. And so it's a lot of fun investigative work there to figure out this customer that has the Service actually has 10 other companies underneath it. And so we do this iterative approach until we get to the point where, all right, we've identified all of the main companies that they would own that have a domain. From there we start using reverse whois information to figure out like, all right, this company generally uses these email addresses or they have this as their org name or their tech name. And we're able to just search back through current information as well as historical information to figure out like, all right, did they have these domains and then they privacy protected them. We're going to bring those in and have a conversation with the customer to say like we found these. Are they still active because they're private and we don't want to include them if they're not yours, but, but if they are yours, we do want to include them. So we try to get that whole approach of we call it domain based investigations. Right. Like we're Going to go out and yes, we're going to cover your cidr blocks, we're going to research ASN filings and make sure that we're grabbing 24s or 20s and bring those into the platform. But we really want to make sure that we're identifying your domains because that's again, how most of the things talk to each other on the Internet. From there, we're going to do subdomain enumeration with lots of different tools. The cybersecurity community is not shy when it comes to creating a tool and throwing it out there. And then we get to go through and figure out like, is this useful or is there a nugget in here that's good that we can take and make better? And in some instances we'll go back and do a pull request to help the community. And in other instances, like, we've built a lot of things in house that, hey, this works really well. And we're going to do this for our service to give back to our customers, to make sure that we're finding as much as we can. We let the platform help with reachability checks, resolving checks. So we never cut off a target too soon, right? Like we give a target several chances to become validated and become something that we look at. So it's a really long winded way of saying we do a lot of prep work to build out a haystack that is less haystack and more needles. Right? Like, I know if I'm looking at something, it's definitely theirs. And then from there I leverage my T team. They're like my threat enablement team. And they create scanning engines, they create, you know, things that we're going to go look for that are threat informed. You know, we use the CSA known exploitable vulnerabilities list, right? Like that is one of our driving forces because customers care about that. Right? I may not care about this thing that a security researcher found because it's probably not going to be out there. But I know that my C suite or my senior executives are going to be asking about these things. And so again, we always want to partner with the customer, meet them where there are, you know, get those baseline things taken care of and then we can start working on, oh, and by the way, here's this, and here's this, and here's this. And we continue to get better that way. So it really goes back to good attack surface discovery and then looking for the right things so that when the operators pick it up, they're guaranteed to Have a needle.

[Gregor Van]

That makes a lot of sense. I mean, when it comes to the, I guess, continuous aspect of this and sort of continuous, I'd almost like analogize with being proactive versus reactive in the sense of you're able in theory to, as it sounds, continuously understand the Attack surface and I guess match that up with, especially with any newly declared vulnerabilities. How does that work? I mean, it must be a lot of, I'm just talking total layman terms here, but a lot of filtering going on where you're like, we looked at this yesterday, so we're not going to look at it again today. Or how does that work?

[Mark Goodwin]

Definitely that is one of the hardest things that we've had to solve for. We say that keeping Attack Surface under control requires constant vigilance. Right. Like it is a whole party team. Right. We do that by my threat enablement team. They track emerging threats. And so again, we use the CISA Kev to help guide that. But we also look at things like Microsoft Patch Tuesdays, what are the things that they're reporting on that may not be a news article yet, but it will be once other people start researching. And then you brought up Watchtower earlier. They do some great write ups. And so it is looking across the cybersecurity landscape to figure out who else is doing really cool research. I know Bishop Fox, we frequently do blog posts of like, hey, I have to shout out the capability development team. They're awesome. They're always researching those next things. And so we really try to leverage what they're finding as well as what other researchers are putting out there to figure out. Even if there's not a proof of concept and an exploit to go with it, what are the versions that we need to identify? What are the things that we need to tell customers about? And so that's where that continuous cycle comes in. You know, we're continuously looking for Attack Surface to bring in. We're scanning customers to make sure that we have a right picture of their attack surface. Because if, you know, if we're only scanning once a month, that's 30 days, where something can change. And so we need to scan more frequently than that. We need to make sure that we're staying aware of the changes in the nuance of versioning to make sure that if an emerging threat drops for version 5.2, we're not going out and reporting to customers 5.1 or 4.8. Right. Because again, that's not delivering a good service and we want to keep this high signal to Noise. If you're getting a notification about an emerging threat, it is, hey, this is important. Try to do this before your Friday kicks off. Which is unfortunate because a lot of emerging threats drop on Fridays. It's a classic joke.

[Gregor Van]

Humans are humans, humans are humans.

[Mark Goodwin]

Right. They got to wait till Friday. But that's how we get after it, right? Yes. There's a lot of filtering with AI and LLM becoming a thing. Computers are good to talking to other computers. So knowing that a CVE comes in and it has a cvss, a score that says this is a local privilege escalation, we can use an AI to help. It could be a simple regex, but we are leveraging LLMs to help make sure that. All right, why is it a no? That way if a customer asks and they give us a random CVE string from 2022, we're able to look in that and say like, hey, you know, we're not going to cover that because that's not an external unauthenticated, that's a local privilege escalation. So, like you're going to care about that once they're already on your machines. That's not something that we're going to identify from where we sit externally at scale.

[Gregor Van]

And I think there's mention of sort of like a live collaboration feature on the platform and I think a lot of what you've just been speaking to the client needs to be involved to be able to. Because I think quite a few things there. You say, we know it's yours, but perhaps. Can you clarify? Xyz? From my experience, we had a scan done by someone on past company and they found a notion instance with the same name as our company, but it wasn't ours, but they absolutely thought it was ours. And that platform did enable me to say that is not ours. I can guarantee that. Is that what the collaboration aspect is?

[Mark Goodwin]

Yeah. So our platform allows customers to do that with or without us.

[Gregor Van]

Right.

[Mark Goodwin]

Like, they're able to say, like, through the platform, they're able to see their targets and they can say like, actually those things, they don't belong to us. And we're able to not scan those things and not report on those things. But we always tell our customers, hey, we work best from an informed perspective. Right. Like, if you want to get the most out of this service, lean in. Because we're here, we're here to help. Like, we want to keep you safe. And the best way to do that is open communication. You know, we've had some customers come in and with Folded arms, like, hey, we're not going to help you. You figure it out and we'll still deliver value and we're still going to do a good job. But those customers that meet us where they are and they're able to say like, this is everything we think we own. What do you see?

[Height Ad]

Right?

[Mark Goodwin]

That is always a better conversation because we're able to say like, hey, yeah, like we found all of those things. But you've missed this section or this company business unit that you may not know you own yet, but according to the news and lawyers, you do own it.

[Gregor Van]

Why are some customers. I mean, I like that sort of analogy. Well, not analogy, but coming with folded arms, like, why do they have that attitude?

[Mark Goodwin]

Security has been around for a while, right? And I think we're at a stage right now where we have to work, like offensive security has to get back some goodwill, right? Because it is so easy to walk in, punch somebody and be like, hey, go fix it. And frequently that's what it used to be like, right? And so in people that have worked in the industry long enough, it is like, I don't want to give you anything because if you find it, I'm going to get in trouble for it. Right? But instead if it is like, hey, we're going to find it and we're going to make sure that we work with you, it goes better, right? And so part of our investigation life cycle always, it should always end with a remediation test, right? Like, you know, we're going to work with you until it's fixed. And I think that is like the one thing that we try to shout out to customers when we get to talk to them. It's like, hey, you know, customers will fail a retest and we'll give them specific reasons why. We'll jump on calls with them to help maybe not troubleshoot because, you know, like, we're not developers, but we are able to attack it with our mindset and help them work through the problem. And so most customers that come in with arms folded by the end of it, you know, they see the value, they see my team and how much they care and we get through.

[Gregor Van]

I think that's a really good call out. It's just, yeah, you know, in days gone by, pen testing kind of done by a team that maybe was never seen before. And all that happens is someone gets this report on their desk kind of saying like, tons of problems. And quite frankly, they're just like, why did we have to find these problems? Could this not just happen? Next month or next year or I'm about to leave. Yeah.

[Mark Goodwin]

Can it be the next guy's problem?

[Gregor Van]

Exactly. I think it's just as you call out, it's changing the mindset of what a service such as Bishop Fox can do, which is its collaborative like end to end and security. If it's the right security partner, it should be a collaborative exercise, end to end. And obviously confidentiality is just such a huge part of that, which is saying, look, we'll be as open as we can as a client with you, but at the same time, we have to trust that this stays within the walls of our relationship. And even as working at a security company, previously we had a third party security company come and audit us. That's just the way it works. But you have to obviously trust if one company is auditing another security company, you have to really trust that. So I think it's a really good call out. So one thing I'm interested in is within the industry anyway. It's no secret that it's a very stressful job. Anything to do with security and finding vulnerabilities or helping remediate. Would you say introducing Cosmos within Bishop Fox has that sort of enabled. I'm not saying it was a stressful environment specifically in Bishop Fox, just more in the industry. Has Cosmos enabled it to become a more manageable profession?

[Mark Goodwin]

I guess I think the short answer is yes. Right. The long answer is always it depends. When you get attack surfaces for extremely large customers, you see the amount of work that exists, right. We call it our backlog. Right. That's a pretty common name for the way to refer things. But to make sure it's always that added pressure of did we prioritize all the vulnerabilities that we see? Because at the end of the day, because we're not a scanner and we're not fully automated and we want to keep that bar to tell a customer to go fix something, we want to go look at it, There's a backlog of work. And so it's that the stress of the unknowns of like, I think we prioritize all these, right? And I know that we're working in what we think is the most important thing, but what if, you know, there's this random offshoot and there's actually a critical vulnerability that may be hidden as a low. And you know, the way that we approach that problem is like, all right, let's attack it from all sides, right? I have my operator teams going priority first. But then we also look at things that like, all right, what's our oldest potential flag? Like what's the oldest thing out there? And let's go make sure that we're staying in line and not letting things get too old. Because the worst thing that would happen is something is vulnerable. You know, there's, there are passwords out there and then they take it down on accident, but then those passwords don't get rotated right. Like that's still bad. So we want to make sure that we're attacking it from both angles. So that helps with the stress at night to know that, you know, if it's not the most important thing, we're going to get it on the back end and then we're going to be able to tell customers like, hey, you know, we've identified this a while ago, but because of priority thresholds we haven't found it. But now we did. And now we can work together to make sure that it is remediated and we're able to give those timestamps to customers like, hey, if you are really that concerned, you can spin up an ir, an incident response and make sure that nobody beat us to it. But that is one of the things that we look at is when we land remote code execution, we're looking to see did anybody else beat us here? Because that is incredibly important, because it helps build the picture for the customer of risk, but it also gives business impact. Right. If I'm throwing an exploit that puts a file on disk and I get there and there's 20 other files like, hey, you need to fix this tomorrow. So that's how we approach it. Yeah, it's still stressful, but I think it is more manageable knowing that we're trying to approach it the best way we can. On a traditional consulting engagement, you scan and then it's kind of on you to figure out what looks the most important. And because we're leveraging five plus years and lots of operators and analyst input, we're able to say we believe as a service we're looking at the most important things. So it takes off that stress off the individual and really puts it on the shoulders of the leaders to make sure that we're making those right choices.

[Gregor Van]

Yeah, it's not as if Tony Stark is not stressed just because he's inside the Iron man suit.

[Mark Goodwin]

Exactly. I like that.

[Gregor Van]

I think that's a good call out. I mean, technology will help. It will. At the same time as you've call out, by now getting to almost have these sort of X ray vision that can lead to Maybe more sort of thinking about what to prioritize, et cetera. And that can potentially just be a different. It's just a different stress point, potentially. Moving on just to kind of the final area before I just ask about some future things. Regulation and things like SoC2. Like SoC2 has become so, dare I say, popular. You've got companies like Vanta who've made it, in theory, very. I wouldn't say Easy to get SoC2, but they have definitely streamlined the process big time. How has that sort of impacted Bishop Fox generally? Has that sort of been a great driver of business or is that kind of actually been more of a problem to have to think about how to adapt things like Cosmos to be in line with regulation drivers or just how's that evolved?

[Mark Goodwin]

Yeah, that's a really good question. A driver for business. I don't know if I can speak to that. That might be more of my sales team's perspective on is this driving them to help with customers that are SOC 2 compliant? I know for us, the work that went into it to make sure that the Cosmos platform and all of the operators, analysts and engineers, that we are doing the right things, I know that that was good for us to make sure that the common saying is everybody's building a plane as they're flying it. But I would say SOC 2, it forces you to figure out, hey, here are the non negotiables and how do we make sure we're doing these right so that we can continue to fly and do the right things? I don't know if that completely answers your question, but that's kind of my perspective on it, just with my limited interaction with the work that went into it.

[Gregor Van]

Yeah, no, for sure. So I think the internal aspect is very interesting because again, perhaps it's not always clear to maybe non security industry people that just because you are a security company doesn't automatically mean that you've got SOC2 or something. You should still go look and check.

[Mark Goodwin]

That is correct. It was a lot of paperwork, a lot of interviews, a lot of making sure that we followed best practices.

[Gregor Van]

Yes, exactly. I guess I was just also interested to if with, let's just say current clients as opposed to anything to do with driving new business. But with current clients, do you get questions to do with. Well, if we don't fix this, does that make us uncompliant with SOC2 or do you get those questions or does that actually go to kind of. I guess that would go to Avanta or something?

[Mark Goodwin]

I think those questions get driven Internally by the customers infrastructure and IT teams, we're able to say this is a problem and you know, you have PII or phi exposed and so like to me that means you're already failing and you gotta go fix that. And you know, some of it is like they're just not aware that it's out there. None of our customers currently or historically have asked hey, does this vulnerability or does this finding affect our SOC 2 compliance? Instead it's just like, hey, how do we help take care of our customers data?

[Gregor Van]

Yeah, okay, makes a lot of sense. So just kind of, I guess looking ahead, what can you share about sort of how where Cosmos is going? Like what are you guys thinking about in terms of when I talk about new features, I guess the customer kind of sees features like they have minus, I think they have a UI that they interact with. But I mean a lot of, I imagine the development is kind of more like under the hood and what you actually expose to the customers doesn't maybe always look new as such. So. But what can you share about that?

[Mark Goodwin]

Yeah, this is actually a really great question because we are at this precipice of moving off of our old Cosmos platform that has gotten to us to where it is or gotten us to where we are. Right. It's successful, it's able to generate information that my team can pick up. We work the cases, we do that work and we report it out to customers. Once we kind of launch into our, into our next phase of our Cosmos evolution, there's going to be a lot more push for self service for customers so that they can go out and take the data that my team has found and dissect it and look at it in different ways in ways that make sense where they're able to determine confidence values or why this asset exists in their endpoints. Like why are you showing this to me? All of that confidence information is going to be made readily available to the customer so that they can self service and figure out like oh, you know, this subsidiary, they're making it too easy for everyone to go out and find our assets. We're also going to. It's kind of like two answers for reporting, right? So the Cosmos is kind of the portal and the service and everything that goes into it and then we have different service lines. Our chasm, which is kind of the thing that started all of this is our continuous attack surface management that's going to continue to grow and approve. You know, like I said earlier, we focus historically on like web apps and FTP and SSH and we're going to continue to grow that scheme and that list of things that we're not only showing to customers through the portal, but we're actually providing investigations. Right. Hey, you've got anonymous login for this service that you should definitely turn off. Right? We do that for FTP, we do that for other logins, but to make sure that we continue to grow that so that the Cosmos Chasm service is kind of that. Here's everything from a threat informed perspective. We recently just launched our Cosmos application, Pen testing, we call it capped and that's web app pen testing that is delivered with, you know, some upfront testing similar to a consulting engagement, but then they continue to test those assets through the lifetime of the contract, looking for things like emerging threats specific to frameworks and doing, you know, tail testing to make sure that like, hey, has anything changed from the last time we hit this application really hard. So that's our chasm, our capt, and then we have a Cosmos external pen test and that's meant to be a point in time external pen test similar to consulting, but delivered through our Cosmos platform where we take everything that my analyst team on Kasm has found and we use that as the scoping. And so the future of Cosmos is continuing to be operations led discovery, relying on the customer to work with us and partner with us and provide feedback, but to continue to grow and improve what we look for, what we report. And then there's a whole new world out there with AI and LLM of how do you discover things faster, how do you help rip through SEC filings so that you can generate that subsidiary list faster with better confidence?

[Gregor Van]

Yeah, I think we don't have time to dive into, I guess what could be done with LLMs and AI today on this one. But I think you've mentioned it a couple of times today, which is a good call out, which is effectively replacing things like Regex with an LLM lookup. Now obviously from a cost perspective that's expensive, but at the same time in many areas that's going to give much actually clearer data and be able to react kind of faster effectively than a Regex. So it must be quite an exciting time for you guys thinking about integrating those 100%.

[Mark Goodwin]

Yes, I'm very much looking forward to it. And then I'd be remiss to say as we continue to grow, I think emerging threats are going to be the thing that continues to set us apart and set continuous pen testing apart. How do you stay up to date and do what threat actors are doing not only with the old things that we know work, but also the new vulnerabilities and the new things that are dropping through either, you know, white hat security researchers or, you know, this apt got busted by a government. And now that ttp, you know, that tactic or technique is now available, how do you leverage that to keep customers safe? I think that's going to be super important as we continue to grow.

[Gregor Van]

Definitely. Well, just as we wrap up, I sometimes ask either two or one question more. Back to you yourself, Mark. So what do you know now that you would like to have told yourself when you were starting out in this industry?

[Mark Goodwin]

Ooh, man, that's tough. You know, it's tough because I started as an officer so many years ago where we were trained to lead and think a certain way. And a lot of that has done me well. But it's also good to know to be technical and to be able to work alongside some of the sharpest people I've ever met in my entire life. And so something I would tell myself going back 15 years is embrace that hard training and know that it's worth it because from my experience, those individual contributors will lean in and partner with you with a leader better if they're able to speak their language and understand the problem set that they're trying to solve. I've had a lot of leaders and managers in the past who, because they didn't understand the difficulty, would oversimplify the problem set. And I think that's something that leaders need to be aware of. And then for individual contributors, it's continue to embrace the hard thing because that's where you learn the most.

[Gregor Van]

I love that. I really like that. I can definitely side with having not gone through any, I guess, military background myself, but was in a company that was started by US military operatives. And I think it was just having that mutual understanding is what would drive things forward very well. You know, I don't go from a military background, but I understand a lot is learned there. Meanwhile, I come from a very technical background and I hope that sort of a lot was learned on that side as well. And I think the sort of the collaboration and combination was what made that work. So it was a great call out. Mark, thank you so much for coming on, giving up your time in an evening over in North Carolina today. I really appreciate it.

[Mark Goodwin]

Yes, sir. Thank you again for having me on, man. This was a blast.

[Gregor Van]

Yeah, thanks a lot. And we'll catch up soon, I hope.

[Mark Goodwin]

Yes, sir. Thank you.