Donato Capitella (41:21)
That's very interesting because when we started doing pen testing of LLM applications in 2023, we were doing a lot of stuff manually. And obviously nobody wants to do that manually. It's more similar to a data science problem than a lot of the traditional pen testing. So we started looking into tooling that we could use. And I'll be honest, the problem there is that a lot of tooling for LLM red teaming is doing exactly that. Is red teaming an LLM, an LLM application? It ain't an LLM. Like it's got nothing to do with an LLM. Like it doesn't have an inference API. Like if I have a button that I can click that summarizes an email that is not even a conversational agent. If I send an email in and there is like an entire chain of stuff that happens, like, I can't run like a general purpose tool against it. It doesn't make sense. Sense. So we started writing scripts, individual scripts that we use to kind of create data sites. And obviously for us, this thing needs to be needed to be practical. Now I, I have five days, six days to do a test for a client. And within those days, I need to be able, even in an isolated environment, to give the client an idea of what an attacker could do. So you have all of these wish list of things. So my wish list was I need to be able to run this practically in a pen test. I need to be able to generate a data set which is customized for what makes sense in that application. Like, for example, I wanted a data data sets that I could use whenever it mattered to test data exfiltration via markdown, images versus HTML injection, JavaScript injection versus harmful content topic control. A lot of our clients, for example, say, I don't want my chatbot to give out investment advice. Actually, we would be liable if that happened. But every use case is different. So I needed something that I could very quickly create these data sets and then every. And then it could be as big or as small as I needed it to be. Now sometimes we go to clients and they tell us, oh, you can send 100,000 requests a day, Fine, I'm gonna have a very large data set. Sometimes we go to clients and they say, you can only send a thousand prompts a day. So you need to be very careful because that's an application, that's not an LLM inference endpoint. So you need to be very careful that you need to create a data set that answers the questions of the client. Can people exfiltrate data? Can people make this thing give financial advice? And then you also have general stuff like, like toxic content, hate speech. Yeah, that anything covers that. But we needed practical stuff and we needed to be able to run it in completely isolated environments. Like if you don't have access to. We needed something where I didn't need to give it an OpenAI key. Okay. It is really important and you know, some of the stuff we can check with regular expressions if we've been successful. But we had to figure out a way that if I am in an isolated environment and I have a data set that I'm generating to test whether the application is going to give out financial advice, but I cannot call a judge LLM to tell me whether the Output is actually financial advice. How do I deal with that? So we had to find a solution for that. It needed to be simple, that we could have a team of pen testers use it. It needed to be extensible. So it needed to be modular so that if one of my colleagues has an application in front of them and this is something that we will see. I think one of our colleagues in the U.S. steve had a chatbot that was using websockets. Now he spent the first day crying trying to reverse engineer that protocol. And then on day two, and he can do that with Spiky, he wrote a Spiky module that's got a playwright. So the Spiky module used a headless browser to open the chatbot, send the prompt and read the response. We were the only pen testing company working on that chatbot that was actually able to programmatically test a lot of stuff. I think we had another one of our guys was working on some AWS infrastructure and the way you introduce the prompt is by dropping a file on an S3 bucket calling a lambda and then in another S3 bucket, one minute later you would have another file that was result of the pipeline that eventually called the LLM. So we needed a way where a consultant could enough a day look at whatever they had in front of them and create an easy module so that then Spiky could take stuff from the data set, send it there and read the response and then say whether the attack was successful or not. So we assume. And then we wanted to be able to extend it with guardrail bypass. So we have a lot of attacks where you take the standard data set and then you can say, okay, for each of these entries in the data set, I want you to try up to 100 variations using the best of an attack. So introducing noise versus using the anti spot lighting attack, which is another attack that we develop where you try to break spotlighting by introducing tags and strange stuff. So the LLM doesn't understand where data starts. So all of these things and it needed to be simple and Sorry, that was very long answer, but that's what we've been working on for the last year and we made the whole thing open source. We've actually had people from the community, from other companies contribute. So it's been very fun to put this together.
