Loading summary
Transcript131 lines
- [00:00]
Daniele Perrito
There is this saying in security circles that in order to survive a bear attack, you don't need to outrun the bear, but you need to outrun the person running next to you. That's the way that the business has been operating for a very long time. But with AI, you can think about the fact that there isn't just going to be one bear, there's going to be a thousand AI bears.
- [00:17]
Interviewer
That's terrible.
- [00:18]
Daniele Perrito
So we're really trying to secure the whole software from AI bears, really.
- [00:23]
Interviewer
Today I'm here with Daniele Perrito, who co founded Fair. Before that you were a founding team member at Cash App at Square and you also ran Data and Security there. And then most recently you've now become the founder of Depth first, which is an awesome AI security company. Really excited to be doing this podcast with you today.
- [00:39]
Daniele Perrito
Thank you for having me.
- [00:40]
Interviewer
I want to start by learning about Fair and sort of like what your experience was like there. But maybe if you could take us back to sort of the founding insight or what sort of led to the creation of the company.
- [00:50]
Daniele Perrito
I would say that Fair was probably a little bit of a contrarian bet. People at the time didn't think that brick and mortar retail was this place where there was going to be a lot of growth like Fair. FAIR proved. But at the time, Max Marcelo and I were talking about ideas on companies to start together with Jeff Collison as well. And Max was introducing a high end umbrella from the from New Zealand to the US market. He was seeing there was a side gig, you know, he was working at Square, but I had a little side gig and he was seeing how getting sales on Amazon was extremely hard. Getting into Nordstrom or Walmart was also extremely hard. And working with the company, hundreds of thousands or millions of retailers was just impossible because there were many, many regional sales reps and things like that. So we thought that there had to be a better way. And then from Square we knew that sort of taking risk on behalf of your customers was always a good way to create value because Square is in the risk management business in a sense and we have learned that there. So we decided to give retailers the ability to order and not have to pay for 60 days and be able to return anything that they don't like and taking the discovery risk off of balance sheet and then not even asking brands to offer that value prop, but us sort of trying to use technology to offer that value prop. So that was a big insight starting there.
- [02:10]
Interviewer
Did it go sort of the way you expected from the beginning like, how linear was it from like that concept to just like the company, you know, taking off and going the way that it ended up going?
- [02:18]
Daniele Perrito
Yeah, I would say the pre true product market fit there was a little bit meandering. I think at the time we were unsure exactly what was going to give enough value to retailers to order on fair. We were experimenting with a lot of things. We were experimenting with something called consignment, which is a little bit of a technical term in the business, but it's just the ability to put something on someone else's shelves without them having to actually buy the merchandise. And we were experimenting with consignment that was extremely capital intensive for us and very, very risky. We were experimenting with points programs and we were experimenting with other things. And I remember one night, this was maybe August of July or August of 2017, Max was at a trade show in Atlanta. Every single day he was at the trade show. And Marcel and I were in the background, like, coding changes so that he could sell them the day after. And he was just like, I think we need to go with, like, try before you buy. Which is the same thing we've been experimenting around, which is like net terms plus the ability to return. But I'm going to talk about it before you buy. And so a night I coded that behind the scenes. The day after he went in front of customers, they immediately got it. And that's when we knew we had something.
- [03:32]
Interviewer
Yeah, it's amazing how, like, those early days, the speed you can move, the way customers react when you do that is crazy. You know, like, I remember we had some experiences like this early on at Lattice where you, like, take, you know, take some customer's like, bug and you fix it within the hour. And then it just like it completely changes the relationship. I'm sure you had a lot of that.
- [03:51]
Daniele Perrito
Yeah, 100%. And I think it's really, you know, product market fit is this thing that everybody talks about, of course, But I feel like until you see it now being there to. Then it being there.
- [04:01]
Interviewer
Yes.
- [04:01]
Daniele Perrito
And that shift, that is real, because before that actual shift, you try to convince yourself, I think we have product market.
- [04:08]
Interviewer
Yeah. Because you don't want to tell yourself you're wasting all your time, right? Yeah.
- [04:10]
Daniele Perrito
That's the alternative after that is that you actually say, oh, yeah, we've put myself before. And this is actually what it is. Yeah.
- [04:17]
Interviewer
I don't know. I mean, for us, we had some. Something. Something similar where it's like a product was built. Of a sudden it went from no to yes. I think maybe there's some companies where it's more of a gradual thing that happened.
- [04:28]
Daniele Perrito
I don't know. Yeah, we can talk about this when.
- [04:29]
Interviewer
We get to depth first, but I'm curious how that experience compares to the current experience. Before we get there, I want to stick with FAIR a little bit longer. You've talked about how it was an operationally intensive business that required a lot of rigor. Can you talk about what that looked like in practice?
- [04:45]
Daniele Perrito
Look, one of our values of FAIR is seeking the truth, and that is sort of necessary. In operating a marketplace business, you are providing value to retailers by having more brands on the platform that can sell to them. Providing value to brands insofar as there are retailers on the platform that buy from them. There are all sorts of balances. You're trying to make sure that exists between supply and demand. You want to give retailers an amazing discovery. You want to manage risk so that retailers can get as much payment terms on the platform. They can order on terms as much as possible, but without risking too much on your end. And to brands, you want to onboard them as fast as possible, but also making sure that in their first week, in their first month, they get as many orders as they possibly can. So there is many, many factors in a marketplace business. And you're trying to make decisions within a system that is highly recursive, where small is chaotic, small changes can ripple out. And so intellectual rigor and data analysis is crucial. But of course, that always needs to be paired with sort of intuition and sort of a vision, because otherwise you can be a little bit too incremental. Right. So you want to balance out these two things. But in operating a marketplace business, you really need to be rigorous.
- [06:07]
Interviewer
Yeah, I often think about that. We're like a software business, you know, another. Another cut on. This software business has these high margins and that affords a lot of, you know, error underneath. When you're operating a business that has a different margin structure, let's say you have a 10% margin versus a 85% margin. There's just a lot less underneath there that you can operate within. So I imagine the sort of daily workings of the company have to be more precise and measured.
- [06:32]
Daniele Perrito
Yeah. I mean, one thing to say is that micro businesses are not, you know, depending on how you compute them, but usually you compute them over sort of gross revenue. So in general, they're more like 50% margin businesses.
- [06:42]
Interviewer
So I was even thinking of, you know, there's like a business like Amazon or something like that. You know, it's like at the extremes, like how careful do you have to.
- [06:48]
Daniele Perrito
Be day to day? There is a lot of rigor that you need to build in. You know, you have all of these machine learning models that are making predictions at all times and the business really relies on them to actually operate and flourish. The marketplace is really something that needs to flourish where demand must meet supply.
- [07:09]
Interviewer
And really, I'd imagine that means you also need like a culture of like a lot of testing rather than like maybe in like a B2B company you can do a lot of sort of just like, here's a plan. We kind of know this is roughly going to work. It takes a lot of effort and you like do the plan once versus I'd imagine in a marketplace business there's a lot more like test things like see what happens in reality and then you can grow programs over time.
- [07:28]
Daniele Perrito
I would say that operating fair has given me a healthy amount of epistemic modesty and humbleness because about like how.
- [07:36]
Interviewer
Much you can really know, how much.
- [07:37]
Daniele Perrito
You can really know. And I come from academia. Right. Where I was trained in some sense to be skeptic of my own beliefs. But there is nothing quite like trying to test your beliefs in the market.
- [07:49]
Interviewer
Yeah.
- [07:49]
Daniele Perrito
To actually know the limits of your beliefs.
- [07:51]
Interviewer
Yes.
- [07:52]
Daniele Perrito
Where you will launch an A B test and be like, I am certain that this a B test will land. And then you'll discover that there is a second third order concern that you will never have anticipated that completely sort of undermines the hypothesis that you had. And so this idea that there are limits to human knowledge and you kind of need to experiment your way into things is really just bitter into you if you're operating a marketplace business.
- [08:17]
Interviewer
Yes. I mean, in general, I really appreciate when people can express the confidence interval that they have on whatever they're saying. When someone's like, I'm really sure about this thing, 85% sure I'm right. I really like that actually when people can just sort of admit that whatever strong view that they have, unless it's a piece of arithmetic, it's like you probably don't know for sure.
- [08:37]
Daniele Perrito
Yeah. I have this little quip which is I say that the market is an incredible truth seeking machine for the type of questions that you can investigate. Now, you're not going to know whether neutrinos are a certain way or another way with the market. But for the types of questions, like does a certain type of signup flow work best for retailers or another type of signup flow work Best. Yeah, the market is really good at getting the answer to that question. Right.
- [09:01]
Interviewer
On the other side of product market fit, when you're like, okay, this thing's, you know, we've got something people want, it's working. Did you experience ease from there? Like, did employees and candidates did, like investors? Like, did people see what you saw at that point? Or was it like, this is still niche, you know, the tam's not that big. Or was it easy to get people excited?
- [09:21]
Daniele Perrito
I think there was probably another two or three years. I don't remember the exact dates, but another two or three years of just pushing just to get people to understand.
- [09:29]
Interviewer
Even though you knew it was working.
- [09:30]
Daniele Perrito
Even though we, I mean, I think we had a very good idea that it was working. We had no idea about tam. And that's why when sort of founders talk to me and they're like, how do you think about the time of this business? I think TAM is just a directional thing where for the first two or three years we were trying to estimate the size of the fair market in a thousand different ways. There were these signals that will tell us there are millions of stores across the globe, trillions of dollars of wholesale orders, and it was just hard to wrap our heads around it. It took really like two or three years for us to be like, okay, this is a gigantic market and we have a place in it. But investors didn't quite get that. Or like many investors did. Of course, the ones that bet on us, it did, but. And I was in a lot of just one on one conversations with candidates where I was just like, no, this is going to be big, trust me. But yeah, I think it took a while to really just get the word out. And I remember maybe like it was four years into starting the business where now people were starting to like repeat back to me things that I was trying to tell people like three years earlier.
- [10:33]
Interviewer
You're like, oh, where did I hear that before?
- [10:35]
Daniele Perrito
Right. And finally it was just like, finally people get it now. Yeah.
- [10:39]
Interviewer
When you look around like AI landscape right now and you think about comparing this moment in time, starting a company today versus when you started fair, does it feel qualitatively different in the psyche of founders or how people are thinking about these types of questions, or is it similar, very different.
- [10:57]
Daniele Perrito
I feel like you could rely on an assumption of some type of steady state system underneath you nine years ago, and right now the assumption is that everything is about to change every three months in ways that are hard to predict and you have to just stay alert. To all the potential changes and everybody's sort of trying to see where the puck is going and it's extremely hard. So I would say the level of energy, paranoia, the level. And I think it's also because the rewards are much bigger than, much bigger.
- [11:28]
Interviewer
Than ever and stuff growing faster than it ever.
- [11:30]
Daniele Perrito
Yeah, exactly. And so there is just a level of the stakes are higher and everything is just so intense all the time. And this was, I mean, don't get me wrong, things were extremely intense affair too. But it was kind of like we had our market where we knew that it was there and we, we just had to figure it out within that market.
- [11:48]
Interviewer
Right.
- [11:48]
Daniele Perrito
But now things are just changing all the time.
- [11:50]
Interviewer
Right. Both because it's like, you know, the market might completely change what product you could build might completely change. Competition might completely change just like way faster.
- [11:57]
Daniele Perrito
Yeah. Or we might get the singularity in a month and yeah, everything changes. That's right.
- [12:01]
Interviewer
Yeah. Can you talk about Cash App, your time at Square and sort of the beginnings there. So I, you know, you were on the sort of founding team at Cash App and you talked to me about how like there was a certain mindset that you went into that with where you're like, you know, taking a bet inside a big company. Can you talk about that?
- [12:18]
Daniele Perrito
Yeah. So this is something that I like to tell people a lot. Just establishing my frame of mind. At the time I joined Square as my first sort of corporate job. I was a researcher before, like an, in academia. I was doing a postdoc and I joined corporate America and at first went into industry, went into industry and at first my feelings were oh my God, everybody's going to be on top of it. I don't know, you know, sort of imposter syndrome. But then I think right after that I started having this belief and the belief was simply like literally was stated in my mind as individually. In a company of a few hundred people, there has to be a way for me to 2x the value of this entire business. I don't know why I had that belief, but I did. And I think that when you have a belief like that, a belief like that, it has a way of being self fulfilling. Why? Because I think you another way of saying it, if you knew that success was guaranteed, what would you do to achieve that success? If you knew there was a way, then your brain is just going to try to find a way through solution space to try to find the set of actions you can actually take. The way that manifested itself for me was Cash App was a hackweek project. It was spearheaded by Jack. At the time, we were using a trick which was sending an email, CCingcashappsquare.com and as you know, emails can be spoofed and things like that. So everybody was a little bit affair at Square that things were going to get weird with security. I was working in security at the time at Square and I was just like, hey, put me in coach, I want to work on this problem. And I was like, I want to make sure individually that this is implemented correctly. Now, a few months later, we moved away from the email trick and luckily because we built an app and it was much better. But the other thing that then happened is that our risk losses, our fraud losses from sort of stolen credit cards and things like that were a little too high. And I remember going to my boss at the time and being like, I want to work on this problem. I think I can make a big dent. And it was still through this mindset that I had at the time, which was like, what is the biggest thing where I can have an impact? I think my brain is very sort of anxious, paranoid and I try to always find ways in which things can go wrong. But that was very well suited to the problem of fraud and combating fraud. So I came up with a whole system. I implemented all these rules and these machine learning models where I had one or two people helping me at the time. And so we implemented this system and we reduced the risk losses by 80%. We brought them into a sort of range that was actually like healthy. And then in some sense that allowed Cash App to thrive and survive and go on to become the massive business that is, I think according to public data and earnings calls, I think it's a 10 plus billion revenue business.
- [15:14]
Interviewer
It's amazing.
- [15:15]
Daniele Perrito
So in some sense that did end up happening. That belief ended up materializing. Of course, it was a large team, like many, many people had.
- [15:22]
Interviewer
Yeah. But it's a good mind, It's a good mindset. Because I also think when you, when you either feel like I could work really hard and nothing's going to come of it, that's super demotivating. Or if you're like, I can work really hard, but you know, the best I can accomplish just doesn't matter that much. It's like that sort of mindset. It's hard to. It's hard to care when you think those things.
- [15:39]
Daniele Perrito
Yeah. And absolutely. And I think it's actually a mindset that is related to security in a sense. Right. Because what do hackers do hackers find a way in where nobody else sees a way in? This suspension of disbelief is similar for the hackers, similar to how a hacker would think. It's like there has to be a way to create value. There has to be like a path for me, a set of actions, a few words to whisper to the right people at the right time, a piece of code that I can write, an idea that I can have, a partnership that I can form, a customer, you know, whatever that may be that will inflect the business. And there is, you know, I guarantee you, no matter who you are in.
- [16:18]
Interviewer
Whatever company you work at, there basically always is.
- [16:20]
Daniele Perrito
There is a way for you to have just an outside impact.
- [16:23]
Interviewer
So let's talk about depth first. So you're doing it again. You got sort of the motivation to go back through the journey and you're sort of doing it with full force. What's the sort of idea behind it? What's the mission that you care about with that first?
- [16:38]
Daniele Perrito
I think I'm doing this again. For me, it's a very mission driven endeavor. Maybe a year and a half ago, I was listening to a podcast episode between Sam Harris and Max Degemark and they had this point that really resonated with me, which was without much better computer security, we do not get to play the AI safety and control game. If you think about it, AI safety and control are going to be mediated by software. And to the degree that our software is not secure, which it isn't, and we need to make it a lot more secure, then what are we even talking about? And so I was like, okay, if I can create a business that is both commercially successful, but it's aligned with admission of making the whole software more secure, then maybe I can create a flywheel there. And the flywheel is like helping secure open source software, building better AI to find vulnerabilities and fixing them in the software runs the world, creating infrastructure, open source, anything, creating goodwill with that. On the other hand, using the same technology that we build to create a product that customers want. And here we're talking about corporations like Square, Fair, Lattice, you know, companies that are trying to secure their perimeter, making sure that their customers data is secure. Yeah, and I really thought that there was a way to create a massive business with a tremendous amount of positive impact by creating this like flywheel. I would say that I think we're, we're starting to get a good way of the way there and like the pieces are really falling in place and I'm really excited about the mission, I could not be more excited.
- [18:08]
Interviewer
What's like broadly speaking, before we get into the specifics, what is the sort of landscape for security with AI? If you had to sort of try to describe the most important parts of the new territory now that there's AI, AI generated code, sort of the ability to do reasoning to look at if you're an attacker. What does this all mean for security?
- [18:31]
Daniele Perrito
There are multiple lenses through which we can answer that question. But at the macro level, like at the mission level, I'll tell you the mission level and then sort of the commercial side, at the mission level, software runs the world. There are billions and billions and trillions of lines of code and systems and configurations that make the turn the lights on and they operate the banks and all the things. Every sort of serious security professional will tell you that there is always a way in. I think AI is fundamentally changing the equation there. We can go maybe into that a little bit later, if you like. On the commercial side, I think people are figuring it out. Our take at that first is that two years from now a company like Fair will operate pretty differently than the way that it operates today. Today a company like Square or Fair will buy a certain number of SaaS security products. They scan certain subsets of their code or their infrastructure. They do so largely using old school techniques like heuristics and rule based systems. Those techniques necessarily have higher false positives, lower detection rates, and can only discover shallower problems with reasoning and AI. What we really see happening is a convergence of all of these subcategories in security. And essentially what we're building is an AI security engineer. Think about a swarm of independent agents that are going through your organization, going through the lattice infrastructure. And they're saying, hey, there is a code bug here that allows someone to log in as someone else. And nothing before could detect that that was not possible. It needed the intuition and judgment of a human. But today we're starting to approach the point where we can do that or there is a misconfiguration in your cloud that will allow someone to get in into this way. The pieces were there, the detection rates were lower, the false positives were higher, the technology only before only allowed to solve a little sliver of the problem. But with AI we really think we can put it all together and make it feel like you have an AI security engineer all the time.
- [20:35]
Interviewer
Should it end like self driving to a degree where it's like you don't need to name, okay, here's it for permissions and here's what we care about for logins, and here's what we care about for API keys and whatever else. And you're able to just say, I want this thing to just very intelligently say, like, what are all the possible vulnerabilities? And just swarm and look at it all.
- [20:53]
Daniele Perrito
I think a lot of that is true. I think there is probably the human element is still going to be something.
- [20:58]
Interviewer
Different, like call somebody, be like, oh, I dropped my password, can you give me your login?
- [21:02]
Daniele Perrito
Yeah. I think the human element, I think there is always companies that need to understand how to interact with the human side and make sure that they authenticate properly and they don't do.
- [21:10]
Interviewer
But on the software side, do you think that's basically where this side is getting.
- [21:14]
Daniele Perrito
I think there is going to be a great unification because the technology, I mean, to me it's just a mechanistic claim. Before technologies could address small slivers of problems, and now the technology is actually able to generalize a lot better.
- [21:28]
Interviewer
Do you think, in theory, at the end state, like, let's go ahead four or five years and just assume things kind of stay what we expect, which. Who knows what we expect, whatever that means. Do attackers or defenders have the edge over time?
- [21:40]
Daniele Perrito
So I think it's a dynamic system. I'll use an analogy. Perfect security is not achievable. And I think this might seem like a scary claim if you're not into security, but everybody understands this intuitively. Everybody knows that there is no such a thing as a perfect bank vault, that a bank vault is only as secure as two things. Number one is how difficult we can make it to attack it, to get in. And that's a matter of, like, cost, equipment, expertise to actually, like, drill into it or the lock or things like that. And then the second aspect is how likely it is that you're going to get caught. And, you know, what are the sort of disincentives there? So you do the equation and then, you know, sort of thieves and attackers do pretty rational math there where they say it's not worth it or it is worth it. You know, in software, it's similar. There is no curfew vault. That's just impossible from a purely theoretical perspective. And attackers are making the same judgment call. And it has to do with how hard is it to get in, and it has to do with how good your protections are and then how good enforcement is. But the reality is that online enforcement is last because it's much more difficult, it's much more anonymous. You're not going to leave fingerprints. And you might be in a state where a nation where There is no delighter 3Ds for things like that. So the equation has the same factors. Enforcement is a lot of a smaller thing now. So therefore it's really about cost. How much does it cost to get in? With abundant intelligence that cost is bound to go down. And I think what's going to happen because of that is that we're going to see a lot more frequent attacks for organizations. And on the other side of that, I think we need to get far ahead of the attackers. But I'll get to the actual question which is, so I think a company like that first is going to have to get in front of the problem, help organizations secure themselves for this coming wave. And I think we're doing that. On the other hand though, I think that the balance of attackers and defenders is not going to change drastically because defenders still have a certain advantage, which is they have full context at that. First, we spend hours and hours of compute on one of our customers code bases to fully understand how it works. The AI spends hours and hours in there understanding, oh, those are the ingress point, those are egress points. This is the inputs and the outputs. This is how everything works together. And using that knowledge, the AI helps secure the business. Attackers need to fly blind. Now there is another advantage that the attackers have, which is defenders need to find every attack, attackers need to find one. But I do think that with the technology that we're building at depth first, we can tilt the scales in favor of defenders.
- [24:30]
Interviewer
You also, I guess have some advantage as a defender because you can know everything about your own systems versus an attacker. You can't know everything about the system.
- [24:37]
Daniele Perrito
Exactly. So yeah, this is the context point that I was making. Our AI spends hours and hours just mapping out everything.
- [24:42]
Interviewer
I guess every time you push new code though, it exposes a potential for new vulnerabilities too.
- [24:47]
Daniele Perrito
Yes, exactly. And that's why we have one of our products is one that scans all of your pull requests as they're being written. And you know, I think one thing that I've noticed both square and fair is that there's always been a little bit of a difficulty between prioritizing security versus productivity. And I really think that this is a false dichotomy at this point. I think another big thing that I think AI will enable is a great reunification peace between the infosec teams and the security teams and they'll be able to achieve that security that they want without impacting. Productivity.
- [25:24]
Interviewer
Yeah, it's like a drag.
- [25:25]
Daniele Perrito
Yeah. Security engineers had to sort of sometimes say, hey folks, we need to look at this. Give us a day or two. And we need to. And that was totally irrational. It was the way to do it. But, you know, if you move at the speed of AI, you can sort of do those reviews much faster. And so I think we also want to see a little bit of reunification. I think we will be able to achieve security with productivity as well.
- [25:50]
Interviewer
Yes. It's funny also because, like, outside of engineering, I think there's like, people understand security as like password protection and random stuff. You know, like, hey, if somebody like sends you a phishing email, be careful. But like, and that is part of it. And there's probably a whole separate, you know, approach needed there. But for the software piece, it does seem like everything's about to probably look very different.
- [26:09]
Daniele Perrito
That's why I'm here too, because I want to sort of tell how to people how cool security is. I got into security originally in grad school because of how fantastical it is. You know, hackers and defenders and firewalls and bastions. It's really just like a fantasy world.
- [26:26]
Interviewer
It's really funny because, like, it was security, like, the excitingness of it ranges from like password manager and, you know, just like somebody at your company telling you, like, hey, you got to follow these protocols. Like, that's one side and then the other side is like Ocean's Eleven.
- [26:40]
Daniele Perrito
Exactly. So people, I think the first thing, if I ever say, tell someone that I'm working on security, I think the first thing that they think about is just like, oh, shit, the other day I had to reset my password. That was painful. I think that's the first thing they think about. But the reality is that what they should really be thinking about is those crazy hackers that are doing daring things to get into systems.
- [27:02]
Interviewer
High level government agencies.
- [27:04]
Daniele Perrito
High level government agencies. That actually is what security is at the limit. And it's incredibly intellectually stimulating. It's really just at the edge of technology.
- [27:13]
Interviewer
It's also possible that like, sort of like by the end of like the cloud software generation, like, it was getting a little bit boring. And now with AI, it's like back to this like, very fresh thing, right?
- [27:22]
Daniele Perrito
Because I think it goes back to that point I was making, which is security is relative to the level of attacks, right? So we had reached sort of a steady state where a company like Federal Lattice could operate and having a team of ex security engineers and business will go on and the likelihood of attacks was relatively low. So you could just do your thing and put security a little bit on the back burner. There is a saying in security circles that in order to survive a bear attack, you don't need to outrun the bear, but you need to outrun the person running next to you. I think that's the way that the business has been operating for a very long time. But with AI, you can think about the fact that there isn't just going to be one bear, there's going to be a thousand AI bears.
- [28:03]
Interviewer
That's terrible.
- [28:05]
Daniele Perrito
So we're really trying to secure the whole software from AI bears, really.
- [28:09]
Interviewer
Why does it seem like security is its own sort of ecosystem? Echo chamber world to me? I'm not lucky to invest in you, but in general, I don't do security companies. And what I found is it seems like its own world. Why is that? Why is it not similar to just other software categories?
- [28:26]
Daniele Perrito
For context, I was in charge of security affair during my tenure there and I had a team of folks that was super talented, is super talented, and as well as Cash App that was within the square ecosystem. I would say that security is such a different market just because of how hard it is for both buyers and sellers to know what they're buying and what they're selling. Like when you're selling, say, observability software or databases, someone can try your database. You make a claim, I test it, I see it, and it's done. If I run a company like fair, I don't necessarily know everything that is wrong or like Lattice or anything or any other business really. And a vendor, a security vendor comes in saying, I think these things are wrong. And it was like, are they really wrong? First you don't know because the claim may be partially incorrect, because many security issues may be false positives, because it's almost true that someone could take advantage of it. But there's this one little detail that makes it not true. Right. So first you need to investigate every single claim. Right? So that's the first part, sort of really. Both the buyer and the seller might have different opinions on what's really a false positive, what's really a true positive. And then another more pernicious part is like neither the buyer or the seller know what the true positives are. So let's say that I go into your organization and I don't find anything. Did I not find anything because I'm not good? Yeah. Or did I not find anything because you don't have anything to find? Which Neither of us knows. Right. So the information of the ecosystem is weird.
- [29:55]
Interviewer
Very hard to get.
- [29:56]
Daniele Perrito
It's very hard to get. It's worse than a market for lemons in some sense, where neither the buyer nor the seller really know what's going on. And I think with AI, actually, we can overcome that, because I think the reasoning abilities of these models can actually do that.
- [30:11]
Interviewer
You can trust that the AI will find the bug if there's a bug.
- [30:15]
Daniele Perrito
Well, and also you can make sure you can trust that it will find a higher percentage of those bugs eventually. And I think this is why we're investing in technology. And I can tell you about our investments there. The percentage of bugs that you can actually find is increasing pretty rapidly. So that part of the equation is changing. And then the second part is that the AI can actually operate almost like a human in the sense that they can verify their work and they can be like, oh, those were my assumptions. I thought that that was a real security problem because X and Y and Z. And actually one of the things our customers love about our product is that we tell them all. We list out all the assumptions that we made to conclude that something was a real security issue, and we showed the work that the model has done to actually verify each assumption. Security is a different market because of those reasons. But I do think that AI hopefully will change the equation a little bit, where it's going to be much easier for buyers and sellers to be on the same pages exactly. On the value that is being exchanged.
- [31:16]
Interviewer
Getting into sort of the tactics of depth first, what are the important pieces of technology for you all to build, to be able to accomplish this?
- [31:23]
Daniele Perrito
Yeah. So maybe let's digest a little bit. So I'm one of the founders, I'm the executive chairman. Kassim is the CEO. He comes from databricks. He was director of infrastructure there, also in charge of security. And then Andrea is the CTO, he comes from DeepMind, and he was one of the authors of Alpha Dev, which is the reinforcement learning algorithm that found a better way to sort and hash Google. And I think this is almost like the perfect team to go after this problem, because AI. So to answer your question, AI has a big infrastructure component, especially when you're doing AI for security, we're doing these technical things. I don't want to go into too much technical detail, but we're spinning up Docker containers to run code inside so that the LLM can test whether certain hypotheses are true or not. And having hired a bunch of folks from Databricks has helped us a ton in setting up that infrastructure. People have been calling it the scaffold, the harness, but it's our intelligence layer that allows us to really repurpose the technology that we built on each new problem. So vulnerability discovery is one thing we've applied it to, but we also applied it to other things. And each one thing that we apply it to becomes easier and easier because we built a really solid AI infrastructure there. The second piece is more like the deep research side, which Andrea has done before at DeepMind. Fundamentally, I believe that reinforcement learning plus large language models will allow us to sort of create a superhuman hacker for defensive purposes. We were talking about the fact that systems, we were only able to find a certain low fraction of the real problems that existed. And that was because the other problems were deeper and deeper and deeper and more and more complex. And I think with reinforcement learning, we can teach these LLMs to go deeper, to, like, find those clever ideas that will allow them to put two small vulnerabilities together and combine them into something that is actually real. We have some security researchers on the team coming from Apple and security services like idf, and the way that they work is phenomenal. Like, their brain work differently. When Mavo and the team tells us that he's discovered a molabini with our LLM and how he verified it and how he actually sometimes pieces it together, it's wonderful to see.
- [33:50]
Interviewer
Is that way of thinking learned, or is that a certain brain type that exists from the beginning? Is it something that comes out of experience, or is it something that's by nature?
- [33:59]
Daniele Perrito
I feel like just almost like everything is probably a combination of nature and nurture. Personally, I feel like my inclination of being a little bit of anxious, paranoid person that's always trying to see how things could go wrong helps is definitely like the background thread in my brain that is constantly seeing how catastrophizing and seeing how things can go wrong is definitely helping in that pursuit. And there are definitely people that are more apt at sort of stepping out of the box and seeing things from a different angle, which is you clearly need. But I will say that it's probably just almost like every other thing. It starts with probably a small talent, which then tells you, oh, I'm good at this, and then you invest more in that, and then you get better at it. But if I had to guess, the seed was actually quite small, and then it blossomed because you invested a lot of time because you were good at it.
- [34:55]
Interviewer
So this is sort of like some of the technology underlying, you know, the product. And then I guess is the idea with the product itself, should it get to a place where, you know, a customer can basically just install depth first and they just know that you're constantly exposing vulnerabilities at a way higher rate than people and you're doing it more thoroughly, faster, cheaper. Is that basically it?
- [35:14]
Daniele Perrito
Yeah, that's the goal. And I think by training our own post training our own LLM, which we're experimenting with right now, the hope is that we will have a technological edge and we can tell customers, I think it's two things. One is the technological edge of our AI stack, and then two is really thinking about the problems the right way. So, for example, we started with code, but we are now telling customers, hey, if you link your staging environment, we can test the findings against your staging environment to tell you whether something is real or not. So I think expanding into other areas that our customers care about is going to be crucial and really giving them an interface, whatever that may mean. You know, right now we have a web app, but I'm also thinking that at some point you need to be able to talk with this thing as if you were talking with a security engineer, being like, hey, can you double check this thing please for me? And then giving the AI access to the components, giving them the context. I think the context is super important. Another thing about security is that it's really context specific. If you're operating a social network, the fact that people can see a customer's profile is the way that it works. But if you're operating corporate Slack platform, a messaging platform, or corporate, you probably don't want the profiles to be public. So that's context specific. So our systems, again, as I said earlier, spends hours in a code base, sometimes going into the old commits. If you think about it, whether something was done differently earlier than it is done today may tell you, hey, we actually had an assumption a year ago about how this thing was supposed to work, but now it's not like that anymore. Why? And so it's really about building a centralized repository of context about the security posture and organization. And that's what we're building. Then adding agents, they can go in and say, let me look at your code, let me look at your infrastructure, let me look at your configurations, let me look at this, let me look at that.
- [37:17]
Interviewer
Will your system learn as a result of, you know, like, is it the kind of thing where the more customers you have, the more you will learn to be safer for the next one or is it Is each one its own new instance?
- [37:28]
Daniele Perrito
So as an enterprise company, you know one thing.
- [37:31]
Interviewer
Yeah, of course, data can't be shared.
- [37:32]
Daniele Perrito
Data cannot be shared. The customer's data never makes it into the weights. Yes, never makes it. We just don't do that. We cannot and we will not. But what I will say is that there is an outer loop, as the folks in AI like to say. There is an outer loop which is like we learn from the types of issues that are not you the people at depth first? Yes, the people at depth first learn. So what do we do when we see that like we are not quite doing as well in that type of issue? We will take some open source software, find the issues that are similar and then train on that. So that's the way in which you're sort of participating in an ecosystem that's not different from how any other product becomes better because more people use it. Because as you join a platform or SaaS offering, you're probably benefiting from the fact that other people have discovered ways to use it that have been built into the product. So in that sense there is some.
- [38:28]
Interviewer
Do you think of depth first when it's a security engineer operating on the team, is it like, is its role to help manage the human security engineers or are the human security engineers managing the AI?
- [38:41]
Daniele Perrito
I think it's going to be a collaborator. I think it's going to be like the humans will probably have the ultimate amount of final say in context. I think you still need that for now.
- [38:51]
Interviewer
Probably one day you don't.
- [38:52]
Daniele Perrito
At that point, it's not this company that changes. I think the whole society changes and.
- [38:57]
Interviewer
I don't know what happens by the time that's happening. It's like all these rules don't apply anyway.
- [39:00]
Daniele Perrito
Yeah. And then I think we need to figure we need to have an entirely different conversation as a society about what's going on. But before that, I think that the security engineers will be the ultimate judges of what's going on and making sure that everything works okay.
- [39:14]
Interviewer
I want to maybe kind of switch to back to sort of like generalities around building this company versus building fair. And we talked a little bit about sort of like the mindset of like the grounds are shifting faster, the rewards are bigger than ever. So that kind of changes some things. I also imagine just that in some ways, like the types of culture inputs that you want are a little bit different. Maybe the types of people are a little bit different. What have you found when you take that difference in the environment? How does that apply to building a company now in this era, whether it comes to recruiting or the way you manage the team or anything else like that?
- [39:51]
Daniele Perrito
There's this book called the Platform Revolution. They talk about two types of businesses, platform businesses and pipeline businesses. Platform businesses are dual sided marketplaces, social networks and things like that. Fair. Exactly like fair. Pipeline businesses are businesses that produce a service or good and sell it to their customers. And there isn't much interactions between the customers or there isn't like a lot of interactions on the other end. I would say that in a platform business, a marketplace like fair, I think you need to keep a tighter grip on the business just because everything is so interconnected that it's hard to just let people completely run with things. Because there is always going to be second and third order things that might happen in a pipeline business. I'm noticing with that first, I think there is a little bit more of letting a thousand thousand flowers bloom and seeing what works. So that's one potential, but it's a small difference. I'm not saying that it's huge.
- [40:51]
Interviewer
I mean, no, it makes sense. So it's almost like you just need greater coordination of efforts. In a marketplace business versus a pipeline business you need like a, you, you basically need systems that like let the flowers bloom.
- [41:03]
Daniele Perrito
I think so. I think so. I think that's a, I think that's a fair characterization. So that's one thing. Those, a lot of things are the same. For example, one of the things I, I tell folks that I work with is don't shy away from putting 30 data points on a spreadsheet and look at them and see what's going on. And data points here is a generic term. It may be like 30 customers, it may be like 30 issues of chargebacks on your platform.
- [41:33]
Interviewer
It may be whatever it is, 30 is like approachable. I'm like, I could do that in two days for most things.
- [41:37]
Daniele Perrito
Yeah, it could be two days, it could be two hours. Just spend some solid time. A few things are going to happen. One, you're going to build so much intuition about whatever that is. You're going to be like, oh, actually that is how that works. And that was already incredibly valuable. But then I think it forces you to overcome this almost like anti pattern that we have as tech people, which is like we want big data because that's the only way to know. Like a lot of data is the only way to know. But the reality is that with 30 data points you're going to know whether something is 60% plus or minus 10%. Or it's 10%. Plus or minus 10%. And you can know a lot from that fact alone.
- [42:17]
Interviewer
Yeah.
- [42:17]
Daniele Perrito
You know, is your conversion rate, is your chargeback rate, whatever that may be.
- [42:22]
Interviewer
It's roughly good. Or it's roughly bad.
- [42:23]
Daniele Perrito
Is roughly good. Is roughly bad. And make a decision.
- [42:26]
Interviewer
Yeah.
- [42:26]
Daniele Perrito
And then, you know, take the top three things you've learned and try to address them. So even, like, in everything, like, when it came to fair, like, get actual a bunch of search results and look at them one by one and form an opinion about when is it that you don't think they're good enough and why. And is it because they're completely irrelevant or is it because they're just. Yeah.
- [42:47]
Interviewer
Even though it's like, theoretically a little bit less accurate. I also think when you spend time in 30 anecdotes versus like 3,000, like, sort of unemotional pieces of data, it's just very different.
- [42:58]
Daniele Perrito
Yeah.
- [42:58]
Interviewer
And you learn more when you look at an anecdote.
- [43:00]
Daniele Perrito
Absolutely. And, you know, this is. I feel like I want people to form, like, a deep intuition about the data, the customers talking with customers and things. And this is one way in which it happens. Yeah. And it's a little bit of a blind spot. And by the way, I try to do this myself. I seek out times where I can just, like, put on my AirPods and put on some music and then just kind of like churn through a bunch of data in a spreadsheet, because I find that to be an important avenue in which I can actually get true context about what's going on.
- [43:31]
Interviewer
Yeah. What else do you think about? Because obviously decision making is such a central part of what, you know, you're trying to, you know, create and what you're trying to do yourself. What else do you think about at this stage for decision making?
- [43:44]
Daniele Perrito
So another thing with that first is that I'm trying to help with all of the context that have accumulated at fair, but trying to only take with me the good lessons. I feel like I could potentially overshoot my role by trying to apply the same exact learnings, the same exact patterns. I think. I mean, it is an enterprise security SaaS company and using AI, and it's just a very different type of business. And so now being a pipeline business, I think we can make sure that people can experiment. And so this is giving me the freedom to sort of step back a little bit. But I think there is also an element of trying to take the learnings that I have, but allowing experimentation. I guess.
- [44:30]
Interviewer
Yeah. And I guess probably also with that you're like as long as the guardrails are safe because obviously you need that as a security company you probably want people to try things quickly in general for sure.
- [44:41]
Daniele Perrito
And I think we have a lot of security expertise inside, so we really make sure to build things properly from the beginning in a secure way. But when I talk about experimentation, I also talk about things that let's see if the AI is able to solve this problem. You can do that. You can have someone spend two or three weeks on that.
- [45:01]
Interviewer
You know, I always thought like as long as it's a two way door decision, just like the faster we can try stuff the better.
- [45:07]
Daniele Perrito
Absolutely.
- [45:08]
Interviewer
Yeah.
- [45:09]
Daniele Perrito
I'm always about making three 90% confidence decisions every week. Yeah. Rather than one 99% confidence decision every quarter.
- [45:20]
Interviewer
Well, Daniel, this was really fun. Thanks for making the time for this and super excited what you're doing at Dev first.
- [45:24]
Daniele Perrito
Thank you so much.