Why Social Engineering Now Works on Machines

Summary

Podcast Summary: AI + a16z

Episode: Why Social Engineering Now Works on Machines

Date: December 2, 2025
Host: Joel de la Garza (a16z)
Guest: Ian Webster (Founder and CEO, PromptFoo)

---

Overview

This episode explores a new paradigm for security in an age of AI “agents”—AI systems empowered to take actions on users’ behalf in the real world. With enterprises racing to implement these agents across applications, their fundamentally different attack surface (conversational, not deterministic code) leaves them highly vulnerable to “social engineering” tactics that trick them, much as one would a human. The discussion features Ian Webster, founder of PromptFoo, which provides agent adversarial testing, and a16z’s Joel de la Garza. Together, they discuss the “lethal trifecta” of AI agent vulnerabilities, why traditional security methods fall short, and how security is evolving from code scanning to adversarial conversations at scale.

---

Key Discussion Points & Insights

1. The Shift from Deterministic to Conversational Threats

• Agents Defined:
  • Ian defines an "agent" as an LLM that is empowered to take actions (e.g., interacting with APIs or other systems) ([02:52]).
• Unique Attack Surface:
  • Traditional security methods (SQL injection prevention, access controls) don’t suffice; agents are vulnerable not to code-level exploits, but to being "persuaded" or socially engineered via conversation ([00:43], [10:02]).
  • “You can't patch persuasion, you can't firewall social engineering. The attack surface isn't code, it‘s conversation.” – Podcast Host ([00:43])

2. Enterprise Adoption and the “Year of the Agent”

• Enterprises are rapidly progressing from exploring internal chatbots to integrating AI agents with core internal systems like Salesforce.
  • “I'm on board with 2026 as the year of the agent ... that’s what we keep hearing whenever we work with folks on the corporate side.” – Ian Webster ([02:52])
• Security priorities are still lagging behind deployment as initial rollouts focused on features and not on security ([00:12], [05:46]).

3. Security Patterns: The “Lethal Trifecta”

• Coined by Simon Willison ("lethal trifecta", also called "rule of two" by Meta): An agent is considered fundamentally insecure if it has all three:
  1. Takes untrusted user input
  2. Accesses sensitive information/PII
  3. Has an outbound communication/exfiltration channel ([04:23], [07:24])
• "If you take an untrusted user input, if you have access to sensitive information or PII, and if you have some sort of outbound communication channel or exfiltration path, then your agent is fundamentally insecure." – Ian Webster ([04:23])

4. Why Traditional Security Approaches Break Down

• Agents are often attacked via non-traditional inputs (e.g., uploaded documents, images, API integrations)—risks are subtler than classic web application exploits ([09:27]).
• Social engineering tactics apply: attackers can role-play, use psychological manipulation, and leverage contextual conversation to bypass controls ([17:15], [19:35]).
• “It's just absolutely mind blowing to me that we have computers that persuasion can work on ... It's sort of like emotional fuzzing.” – Joel de la Garza ([20:21])

5. PromptFoo and Automated Adversarial Testing

• PromptFoo (started as OSS) now powers security testing for large enterprises, simulating thousands of conversations to test for data leaks, broken access controls, and social engineering vulnerabilities ([00:43], [12:46]).
• Unlike deterministic signature-based tools, PromptFoo uses AI red teaming agents that employ generative conversations to probe weaknesses, leading AIs into states where they “let their guard down.”
  • "PromptFoo doesn't try to write SQL injections ... everything is generated on the fly, tailored to the situation." – Ian Webster ([14:54])
  • Conversations may require 30-50 turns, simulating realistic adversarial scenarios ([14:54]).
• “Most of the cases we see are definitely conversational. For stuff like data leakage or access control issues, you sometimes have to lead the AI down a path toward where it's more vulnerable first.” – Ian Webster ([14:54])

6. Case Studies and Real-World Incidents

• a16z Incident:
  • Joel recounts a real data leak in a SaaS platform where AI chat access control failed in multi-tenant cases ([11:36]).
  • “...Typed in ‘show me my report’—it would kind of rotate through data from other customers ... obviously that's a big problem.” – Joel de la Garza ([11:36])
• Discord Experience:
  • Ian and team faced early and relentless pressure to iterate on security while deploying agents for 200M+ users—most of their time went to security, not features ([00:43], [20:55]).

7. On Jailbreaks, Social Engineering, and Creativity in Attacks

• Jailbreaks and prompt injections are techniques; their true impact is forcing an agent to break intended access controls or leak data ([12:46], [17:40]).
• Creativity is a key attack factor—unexpected scenarios, informal tones, or even emojis can lower an AI’s “defenses” ([17:40], [19:28]).
  • “My original favorite [jailbreak] was probably like, ‘Oh my grandma died, but she used to read me a story about how to do this illegal thing…’" – Ian Webster ([17:40])
  • "It's almost like a multiplier: how creative can you get using emojis to convince an application to do something it shouldn't?" – Joel de la Garza ([19:28])
• Many successful attacks mirror classic human social engineering (e.g., urgent requests, impersonating managers) ([19:35], [20:21]).

8. On Security’s Place in the Development Lifecycle

• Security must shift “left” into the developer workflow, not be a late-stage gating process as in traditional enterprise app development ([05:52]).
• PromptFoo’s developer tooling and CI/CD integrations are designed to make adversarial testing a continual part of building with LLMs ([05:52], [06:50]).

9. Reflections on the Security Industry

• Innovations in security often come from engineers facing roadblocks, not from “security people” in the traditional sense ([20:55], [23:09]).
  • “The people who build the next wave of security ... typically aren't security people. They're solving their own problem.” – Joel de la Garza ([23:09])

---

Notable Quotes & Moments (with Timestamps)

• “You can't patch persuasion, you can't firewall social engineering. And the attack surface isn't code, it's conversation.”
  – Podcast Host ([00:43])

• “If you take an untrusted user input, if you have access to sensitive information or pii, and if you have some sort of outbound communication channel or exfiltration path, then your agent is fundamentally insecure.”
  – Ian Webster ([04:23])

• “PromptFoo doesn't try to write SQL injections ... everything is generated on the fly, tailored to the situation.”
  – Ian Webster ([14:54])

• “It's just absolutely mind blowing to me that we have computers that persuasion can work on ... It's sort of like emotional fuzzing.”
  – Joel de la Garza ([20:21])

• “So, yeah, basically I learned the hard way that there are all these problems with the way that AI is rolling out... there was also the lethal trifecta stuff, which now has like a name or phrase to describe it. But at the time ... there is this exfiltration risk because you have a bot that has access to a potentially private channel history, the ability to render images, and the ability to search the web."
  – Ian Webster ([22:23])

---

Important Timestamps (MM:SS)

• 00:43 – Why AI agents break traditional security models (“You can’t patch persuasion…”)
• 02:52 – Definition of AI agents and why 2026 is the "Year of the Agent"
• 04:23 – The "Lethal Trifecta" model of agent risk
• 05:52–06:50 – Why security must be embedded in developer workflows
• 11:36 – Real-world a16z SaaS data leak incident
• 12:46 – PromptFoo’s automated adversarial testing
• 14:54 – From deterministic signatures to conversational attacks
• 17:40–19:28 – Creativity, jailbreaks, and social engineering in agent security
• 20:21 – “Emotional fuzzing” and persuasion attacks on machines
• 22:23 – Hard-won lessons from Discord’s agent rollout

---

Final Thoughts & Resources

• Security must now consider conversations, not just code. AI agent threats are fluid, creative, and hard to “patch.”
• For builders: It’s essential to test how agents respond in real-world adversarial scenarios—at scale and with creativity.
• PromptFoo is open source and designed for developer-driven security testing.
• "Good luck to everyone who's building with agents. I think it's going to be a pretty exciting year ahead." – Ian Webster ([24:13])

---

This summary distills the key concepts and memorable moments for listeners interested in how AI agents are creating unprecedented security challenges, and how adversarial “conversational” testing is the way forward. For more, check out PromptFoo and follow industry insights from a16z and their founders.

Transcript

[Ian Webster]

This year was all about spinning up AI and some of those initial use cases. I'm on board with 2026 as the year of the agent.

[Joel de la Garza]

It's just absolutely mind blowing to me that we have computers that persuasion can work on.

[Ian Webster]

A lot of security teams were scrambling to catch up with these initiatives and initial prototypes.

[Joel de la Garza]

Every new platform cycle, security always lives at the end because everybody's going too fast to think about security.

[Ian Webster]

If you take an untrusted user input, if you have access to sensitive information or pii, and if you have some sort of outbound communication channel or exfiltration path, then your agent is fundamentally insecure. I think it's going to be pretty exciting year ahead.

[Podcast Host]

Every company is racing to deploy AI agents. Almost none of them are ready for what happens next. Here's the problem. We spent decades learning how to secure deterministic systems. SQL injections, buffer overloads, access controls. But AI agents don't work like that. You can't patch persuasion, you can't firewall social engineering. And the attack surface isn't code, it's conversation. Ian Webster saw this firsthand at discord building an agent for 200 million users. Most of his team's time wasn't spent on features, it was spent on security, trust and safety. Because when you give an AI access to data and the ability to take actions, you created what's now called the lethal trifecta. And traditional security tools are useless. That's why he built promptvoon not as a security person, but as an engineer who hit a wall. Now Fortune 10 companies use it to simulate thousands of adversarial conversations, testing whether their agents will leak data, break access controls, or get socially engineered by a well crafted emoji. In today's conversation, Ian sits down with a 16Z's Joel de la Garza to discuss if 2026 is really the year of the agent and whether enterprises can secure them before something breaks spectacularly.

[Joel de la Garza]

Today I'm pleased to be speaking with Ian Webster, the founder and CEO of PromptFoo, AI agent testing company that focuses on security. And this whole series of conversations we've been having with founders has been really focused on what we're seeing in the market, which is that every corporate customer, every corporate cio, every corporate CTO we talk to has some sort of agentic thing that's being built. Now, it could be a customer service agent that a large airline is building, or it could be a gaming company building something to replace the NPCs. Right? There's all sorts of stuff that's happening and it's great to have you on because we know that you are an expert in agent and agentic security, having done a ton of work in this space and would love to maybe just get your really quick take on what do you think an agent is and how are people thinking about agents at the current time to maybe frame the discussion.

[Ian Webster]

Yeah, I think the way that I would start out, just very simply an agent is what you get when you have an LLM and allow it to take actions. So if you're hooking up APIs to it or anything where it can interact with the outside world and in terms of why that's important and where it's going, I mean at promfu we work with some of the largest companies in the world, Fortune 10s, Fortune 50s. And what I have been hearing very consistently is this year was all about spinning up AI and some of those initial use cases around kind of like internal chatbots and rag. But without fail, everyone on their roadmap has like, we're going to start hooking it up with Salesforce or with other internal systems and that's the plan for next year. So I definitely think or like I'm on board with 2026 is the year of the agent in the sense that's what we keep hearing whenever we work with folks on the corporate side.

[Joel de la Garza]

Well, and you've been incredibly busy and it was hard to schedule this podcast, so I assume that's an indicator that people are really engaged in this sort of stuff.

[Ian Webster]

Yeah, we are working hard for sure. And yeah, I mean we saw like a big step up at the beginning of this year as some of the like, AI initiatives and budgets kicked in. I think we're going to see another, probably even bigger step next year just in terms of like the amount of activity and as well as what needs to be tested and secured.

[Joel de la Garza]

You know, it's one of those popular cliches that unfortunately probably applies here, which is that history never repeats, it just rhymes. And so the history of enterprise applications, having yourself grown up in the enterprise as well and built a number of these things, has always been that you build the app right? And there's a process around product management and building the app and engineering, and then it's about to be promoted into production. And always at the end of that journey is some sort of security assessment of the app to make sure that it doesn't do something incredibly horrible. And your tool is kind of like the new security assessment for that app or that agent before it goes to production is that Correct. Is that kind of how it's working in the enterprise?

[Ian Webster]

Yeah. So yes. And there's a bunch of stuff that I would add. So I guess just for folks who are not already familiar with promfu, what promfu is, it started as an open source tool. It's used by hundreds of thousands of developers to run evaluations and security tests on Genai. And where it falls in the development cycle really depends a lot on how you're using it and who you are. So you're absolutely right. In many cases, unfortunately, security is an afterthought. Right. So you build this cool thing like this new kick ass agent and then you're like, crap, I cannot bring it to production or I won't get sign off for it unless I do this sort of testing. That was a lot of what we saw kind of at the first half of this year where a lot of security teams were scrambling to catch up with these AI initiatives and kind of initial prototypes.

[Joel de la Garza]

The start of every new platform cycle. Security always lives at the end because everybody's going too fast to think about security.

[Ian Webster]

Yeah, yeah. So history is rhyming again. But I think what I'm getting at is that's definitely not the ideal scenario. What we want is to give developers the tools that they need in order to actively test and get feedback and build secure systems even before they land their code. So one of the reasons why prompfu is doing really well is because it started as a developer tool and has the developer friendly CLI just very easy to integrate and that's the direction that we want to head. We've done a lot more recently in terms of how do we make it really easy to embed in CI cd, how do we do code analysis and give feedback on agent security, other relevant LLM security topics in prs and ultimately how do we bring all that intelligence to the developer in their IDE? So that's my $0.02 on just like where we land in the cycle and where AI security actually should be.

[Joel de la Garza]

Yeah, well, we're speedrunning this cycle. So it took Cloud 10 years, will probably take AI one year.

[Ian Webster]

Right? Only time will tell.

[Joel de la Garza]

Yeah, totally. I'm really curious. There's like a classic. Every technology platform has the classic list of things you need to check right before you promote it.

[Podcast Host]

Right.

[Joel de la Garza]

Like cloud was always that you've made a storage bucket, you know, world readable and old world infrastructure was silly. Things like password reuse and stuff like that. What are the classes of vulnerabilities you're seeing with these agents that are kind of like those new sort of critical issues.

[Ian Webster]

Yeah. So there, there's the stuff that you. By now we've all heard about, like prompt injections and jailbreaks and things like that. There's also a ton of concern around PII and data leakage. So whereas prompt injections and jailbreaks are more on the foundation layer, you introduce a whole lot of new risks when you start to build the system or application layer and hook it up with the knowledge base and so forth. So I won't rehash that, or I think there's another podcast on that. But I think what makes Agent Security really interesting is that it's not just AI security, it's kind of a confluence of ident identity, API security, et cetera. And the way that I've been communicating this, or kind of a good mental model for this that's been coined is called the lethal trifecta, or people call it different things. Simon Willison kind of coined this term meta, calls it the rule of two. But basically it's the idea that if you take an untrusted user input, if you have access to sensitive information or pii, and if you have some sort of outbound communication channel or exfiltration path, then your agent is fundamentally insecure. And for people building with agents, that is the thing to think about. Is my agent checking these three boxes? If so, you really need to unpack whether that's a smart idea or if there's a way to kind of slice things. So you can only do two out of the three. And just to dive deeper on that, I think it sounds obvious when you say it out loud. Yes. If you have access to sensitive information and untrusted input, then obviously we don't want to open an exfiltration channel. But the actual what we see in the wild is a lot more subtle. So untrusted data, for example, it's not just what the user types in to the front end of the agent chatbot, it's also, are you surfing the web and bringing in websites? Are you pulling in documents from a.

[Joel de la Garza]

Document, uploading a picture or whatever the case may be?

[Ian Webster]

Yeah. So there are a lot of ways that indirectly you can introduce untrusted data. And then on the kind of communication or exfiltration side, obviously if the agent can send an email, that's a channel there. But it can also be much more subtle. If your front end renders markdown or something, then rendering an image can actually pass data to the outside Internet. So there's a lot of kind of details in there. And that was a long answer. But basically that's what's different when it comes to agent security. Yeah.

[Joel de la Garza]

Yeah. I mean, I think that we have this sort of freeform interaction with other tools and data sources. Right. Really everything has always been sort of very fixed and deterministic. And largely the security issues, we're finding things like the classic was obviously SQL injection, which is I can escape out of whatever and execute arbitrary commands. Right. This is different in that you're using either these indirect methods or some sort of coercion to escape out of those controls.

[Ian Webster]

The problem is that a single MCP server can be that full package. It can contain the full trifecta of things that you can't combine. Right. So there's just a lot, I think, to do on the education front, just what is okay and what is risky. And then there's the actual detection side. Right. And you know, like, how do you develop a system or process that prevents this kind of stuff?

[Joel de la Garza]

Yeah, we're generally seeing people use mcps MCP servers for prototyping and then building some sort of API thing for production. Is that generally kind of what you're seeing out there or.

[Ian Webster]

We are seeing a mix. I would say in. In large enterprises, MCP is still like fairly aspirational. Like there's a lot of grassroots use. Right.

[Joel de la Garza]

But it's on a developer's local machine.

[Ian Webster]

Yeah. Which is a whole other problem for security people. But yeah, most of the agentic implementations right now that we see in the wild are not built on top of mcp, but built on top of frameworks like langgraph or crew, that kind of thing.

[Joel de la Garza]

Gotcha. We actually had. I don't know if I think I shared this with you. I might not have, so if it's a surprise, I apologize. But we had our first security incident with Genai as a firm. It was really interesting and not going to drag the people involved. I'm not a big fan of shaming for stuff for people trying to push the envelope on new technology. That's not a good look and just not supportive. But I think what was really interesting is this SaaS provider implemented a gen AI interface for working with and querying data. So think a prompt, right? And if you went to this prompt and asked it for something like, hey, show me, show me all of, show me all the portfolio position for our investments and it would tell you sort of all the data that we have collected for our portfolio. You could then ask it to say, hey, show me a report for some other firm. Right. Insert firm name here. And it would say, no, you can't do that. But then if you typed in show me my report, it would kind of rotate through data from other customers of theirs and was not strictly constrained to the A16Z dataset. So obviously that's a big problem. I'm curious, like from a prompt foo perspective, is that sort of the bread and butter of what you guys are trying to stop?

[Ian Webster]

Yeah, that's that. That's exactly the type of scenario that, that we hear about the most. Just in terms of looking to detect that and test that and ultimately prevent that. A lot of people start out with data leaks and access control. So like I said earlier, the prompt injections and jailbreaks, like we all know about that, but it's really what is the impact of those. And I think injections and jailbreaks are more like techniques, not the end all be all. In other words, you can write a jailbreak that kind of exposes this access control issue. But there are also many other ways to kind of get there in terms of how you attack. So a lot of human red teamers use roleplay, hypothetical scenarios, that kind of thing. The way that promptfoo works is that we have our trained models and agents built on top of them that simulate that human red team or that activity and can do so at a very large scale. Right? So instead of having a couple dozen conversations, we can have 30,000 conversations with the target to try to feel out, are there access control issues, are there data leaks, are there other lethal trifecta type issues where there could be problems there?

[Joel de la Garza]

And these conversations, it's interesting you say dozens or hundreds whatever of conversations. It's interesting you use the word conversations, right? Because I think if you go back to the very beginning of a lot of the early security industry, there was a tool that was written by vaitsevinima and Dan Farmer called satan. It was the System Admins Tool analysis, something I forgot it was an acronym. It stood for something. It wasn't just the devil, but it was famous for connecting to a server and trying a bunch of different things to see if it could get access to it, right? Logging in with different usernames and passwords, looking for buffer overflows. But these were all very programmatic things. And it's interesting now that you say we've gone from this sort of, hey, there are these very deterministic signatures to this. We have to have conversations. And so maybe just pull the thread there on sort of what that testing looks like vis a vis kind of what it was before yeah.

[Ian Webster]

So I guess what makes promfu kind of part of this, the like, AI generation of tooling here is like the attacks that promptfood generates and kind of its overall adversarial objectives are our natural language, right? So PromptFu doesn't try to write SQL injections or you don't have signatures, in other words. Yeah, we don't have signatures. Everything is generated on the fly, kind of tailored to whatever the specific situation is. So, like, when you use PromptFood, you feed our models business context. Like, what is this application used for? Who uses this application, what should they and should they not have access to that kind of thing? And yeah, in many cases these are full on conversations. And obviously it depends. Not every AI target is conversational. Sometimes it's just an API endpoint or a code hook or even like a behind the scenes, just like kind of function process that takes some data and spits out some other data. So there are many ways to, to hook in prompt fu there. But most of the cases that we see are definitely conversational. And the reason why that's important is especially for stuff like data leakage or access control issues, you sometimes have to lead the AI down a path toward where it's more vulnerable first. So the conversation always starts off pretty innocent. Hey, tell me about how's your day? Yeah, how's it going? And then 30 or 50 messages later, that's when the conversation is in a state or the AI is in a state where we can go in for the kill, so to speak. So the big difference or one of the big differences here and the reason why so many people are looking for automation, is that it's just so time consuming as a human pen tester or red teamer, to have to go in and have this whole conversation. And let's say you hit a refusal or a roadblock at like at turn number 12, you have to go all the way back from the beginning and reproduce the whole conversation. So we're talking like hundreds or thousands of hours in order to get the same result. So it's really just like the breadth of the attack surface that has driven this move toward automation.

[Joel de la Garza]

Sounds a lot like Dungeons and Dragons, I guess.

[Ian Webster]

Anything can happen, right? Yeah, yeah, yeah.

[Joel de la Garza]

It's so fascinating. You know, it's been a bit of a trend, I know I've seen on a couple podcasts lately, for people to say their favorite jailbreak would love to get your thoughts. I mean, you work with every model, every frontier lab. Like, what is generally the class of jailbreak that's Sort of the most interesting to you without obviously posting a vulnerability to the with it that's unpatched. We'd love to maybe get your take on sort of like that whole phenomenon of jailbreaks. And what's interesting to you about that?

[Ian Webster]

Yeah, I think so. So jail jailbreaks come and go. Like, my original favorite was probably like, oh, my grandma died, but she used to read me a story about how to do this illegal thing that was actually. We first saw that on Discord. I think that was baked on the very early agent work that we've been doing. I get a kick out of. So I read most of the papers in this space. I just get a kick out of the random jailbreaks that someone creative thought about that you wouldn't expect. So the other day I saw there was a researcher at VMware came up with this jailbreak that was basically talking like how millennials used to have a Millennial? Yeah, with an emoji. And I'm doing a really bad job explaining it. But it's just the things about these more informal inputs, I think kind of lower the defenses or the guard of the AI or maybe don't match exactly what it was with the reinforcement learning was as far as jailbreaks. So, yeah, it's not rocket science. I just get a kick out of all of the scenarios and tones and stuff you can come up with in jailbreak.

[Joel de la Garza]

Well, it's more. I think what's interesting about it is that, I mean, I think because as a software engineer, you know that coding and developing is actually a quite creative process. Right. It's a very creative thing. It's similar to writing a complex story or complex novel. It's just more people can read a novel than can read a code base. And so there's always been this inaccessibility, I think, to the larger population about exactly how creative this stuff can be. And so this is interesting because this is actually the combination of the two, where you have the creativity of persuasion combined with a non deterministic system. Combined with deterministic systems, Right.

[Ian Webster]

Yeah.

[Joel de la Garza]

So it's almost like a multiplier of, like, how creative can you get using emojis to convince an application to do something that it shouldn't do.

[Ian Webster]

Yeah. And it's also kind of brought social engineering more to the forefront, even for machines, which is like, very cool. But yeah, when we do these. So I guess backing up for a second, the way that Promfoo kind of conducts these conversations is we have an agent which is behaving As a red teamer and kind of like feeling around the different guardrails and scenarios. A lot of times what is most successful is just basically, basically what I would call social engineering. Hey, my manager is out today. I have an urgent request and I just really need access to this data because I have this client breathing down my neck and it's that kind of stuff that if you have that fundamental vulnerability where you've screwed up your access control or whatever that can push things over the line for the LLM.

[Joel de la Garza]

It's just absolutely mind blowing to me that we have computers that persuasion can work on. Yeah, it's sort of like emotional fuzzing. Right? Like it's just the strangest thing. It's such an interesting time.

[Ian Webster]

Yeah.

[Joel de la Garza]

So, so you're. I mean maybe let's. I mean it's. This is all frontier stuff. This is all. There is no textbook for a lot of this stuff. This is all things that have happened in the last three months or three years. Would love to maybe just hear a little bit about your background. Kind of like how did you start on this journey? I don't remember. I don't believe you were like a traditional security ops person. You were an engineer and focused on other problems.

[Ian Webster]

Yeah, that, that's absolutely right. So yeah, prior to this I was at Discord where I started the developer platform Org and then when AI got hot, switched over and focused on building out some of our early AI features and experiments and leading that team. And yeah, I mean I'm an engineer, right. I've always loved building and tinkering and AI is great for people who love that stuff. So at Discord we were building this agent. This was in 2023, so we didn't have the luxury of all these excellent agent focused reasoning models. But we were building this agent and rolling it out to 200 million users and we made a lot of mistakes and I was spending most of my time on things like security trust and safety policy and compliance and that's where my team was also spending their cycles. So it was pretty clear to me that, that like this would be the main sticking point for any more advanced AI use cases. And you know, if we were running into that stuff at Discord, then I can only imagine what the guys at, you know, like a financial services institute or you know, well, you do have more regulated.

[Joel de la Garza]

Discord probably has one of the more punishing user bases when it comes to stuff like this though.

[Ian Webster]

That's, that's true. It is, it is a bunch of 14 year olds with nothing but time.

[Joel de la Garza]

And the ability to persuade.

[Ian Webster]

Yeah, yeah, yeah. So, yeah, basically I learned the hard way that there are all these problems with. With the way that AI is rolling out and that went all the way. You know, there was like the jailbreak stuff, but there was also the lethal trifecta stuff, which now has like a name or phrase to describe it. But at the time it was, yes, we. There is this exfiltration risk because you have a bot that has access to a potentially private channel history. You have the ability to render images. So. And you have the ability to search the web. So, like, there was a bunch of stuff that we were putting in there that was like a precursor and kind of went into the first version of PromptFu, which was open source and like, ultimately is still kind of guiding some of where we want to take the product today.

[Joel de la Garza]

Yeah, that's fascinating. The other really interesting point, the other interesting point kind of in your background and your history and how you arrived here, was that it's very similar to other large platform shifts that have happened where you, at least in the security industry, it's always been the case that the people who build the next wave of new security stuff typically aren't security people. And it's always sort of like people that are adjacent that are solving problems and then kind of get drawn into the security space because it's in the way of them solving their problems. And so very much a similar account. Right. So devinima was a physicist and he wrote the first vulnerability scanner because people were messing with his research servers and so he needed a way to secure them. Right. So, like, it's really interesting that we're seeing sort of this cycle repeat. Ian, thank you so much for coming by. This has been an awesome conversation. Always fun to talk to you. You have the best stories in the industry. I know you guys are absolutely on fire. Thank you for taking the time in between fielding customer calls and customer inquiries to come and chat with us. For folks out there, any last words, any sort of like, where you can direct them if they're interested in this space and things that they could take a look at.

[Ian Webster]

Yeah. I would say for folks who are interested in kind of safety and security evaluations and testing, definitely check out prompfu. It is open source, so really easy to just take it off the shelf and start trying things out. But, you know, good luck to everyone who's building with agents. I think it's going to be pretty exciting. You're ahead.

[Joel de la Garza]

Bigger than the Internet. Thank you so much.

[Podcast Host]

Thanks for listening. If you enjoyed the episode. Let us know by leaving a [e-mail] a16z we've got more great conversations coming your way. See you next time. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com disclosures.