TruffleHog Creator: You Can’t Have AI Agents Without Secrets

Summary

Podcast Summary: AI + a16z — TruffleHog Creator: You Can’t Have AI Agents Without Secrets
Date: November 11, 2025
Guests: Dylan Ary (CEO & Co-Founder, Truffle Security), Joel de la Garza (a16z Partner)

---

Episode Overview

This episode dives deep into the escalating challenges of secrets management in the era of AI agents, highlighting why robust security for machine credentials (secrets like API keys, OAuth tokens, and database passwords) is now the critical bottleneck in software and AI agent development. Dylan Ary, co-founder and CEO of Truffle Security (creators of TruffleHog), discusses the history and future of secret-scanning tools, explores the evolving threat landscape, and gives insight into how Truffle is adapting to support both security and developer productivity as the stakes—and the pervasiveness—of secrets keep rising.

---

Key Discussion Points

The Central Role of Secrets in Modern Software and AI (00:00–04:24)

• Secrets as a Bottleneck:
  • “[Code] used to be the bottleneck…now the bottleneck is secrets. It's literally the thing that makes developers slowest to get things in production. Right now…the thing that's preventing us from moving as fast as the speed of the agent being able to write the code is the secrets.” (A, 00:00)
  • Secrets have now become a direct vector for immediate, often financial, impact when leaked.
• Human vs. Machine Secrets:
  • Humans use passwords. Machines use secrets—like API tokens, cryptographic keys, and other credentials crucial for authentication between non-human agents. These are much more likely to leak undetected.
• Cloud as a Game Changer:
  • “…the rise of secrets as a problem is pretty much correlated directly to cloud…” (C, 03:42)
  • Cloud computing has increased the proliferation and sensitivity of secrets, making leaks more impactful and common.

TruffleHog and the Evolution of Secret-Scanning (01:43–05:21)

• Origins and Impact of TruffleHog:
  • Launched as an open source side project in 2016.
  • Now run ~250,000 times a day globally with over 23,000 GitHub stars and 8,000 organizations using it.
  • Used by both red teams (attackers) and blue teams (defenders), but has no offensive capabilities itself.
• Legacy and Ongoing Value:
  • TruffleHog allows identification, validation, and analysis of leaked secrets, which is crucial for rapid response to breaches.

Current Threats & Financial Stakes (05:21–08:11)

• Monetization of Secrets:
  • Leaked credentials are now easily monetized—e.g., using stolen cloud keys to mine crypto, steal payments, or issue fraudulent refunds.
  • Financial losses now hit immediately, not years later.
• Asymmetric Impact:
  • Students and small developers suffer disproportionately when their secrets leak: “A crypto miner on a $1,000 [student] account is a huge hit…left picking up the pieces.” (A, 06:02)

AI Agents: The Next Security Frontier (08:11–13:57)

• Agents Depend on Secrets:
  • AI agents require machine credentials to perform actions (API calls, code pushes, transactions) on behalf of users.
  • Example: “When you first sign up, you go through an OAuth flow…a secret is manufactured behind the scenes that allows the agent to act on your behalf…” (A, 09:00)
• Trend of Leaks Increasing:
  • “The trend is up, not down.” (A, 10:28)
  • As AI agents act autonomously, poorly managed secrets multiply both risk and complexity.
• Shift in Bottleneck:
  • “It used to be the bottleneck was code development, and now the bottleneck is secrets.” (A, 12:34)
  • Developer speed now constrained more by secret management than actual coding.

Truffle Security’s Roadmap and Vision (13:57–16:42)

• Company Evolution:
  • Detection → Validation → Analysis → (Upcoming) Inventory (introspecting existing secrets managers for comprehensive understanding).
  • Example: “We just launched the ability to take a Google Cloud secret and see exactly what resources and what permissions…that secret is able to do. We call that GCP analyze.” (A, 14:26)
• Ultimate Goal:
  • Make secret management as fast and developer-friendly as possible, clearing the “bottleneck” without sacrificing security.

The Culture War: Security vs Developer Productivity (16:42–21:43)

• Security Tools Designed for Security, Not Usability:
  • Secret managers historically imposed security-team requirements (short-lived, rotated secrets), often at odds with developer needs:
    • “Every single secrets manager up until this point has taken 100% of the requirements from the security team…As a consequence…developers just don’t use the secrets manager…” (A, 17:33)
• Punishment Paradigm Is Failing:
  • “CISO…said I want the secrets manager to be as hard to use as possible…If the secrets manager was really hard…it would make the developer think twice about whether they needed a secret in the first place.” (A, 19:27)
  • Result: Developers “rolled their own” insecure solutions instead.
• Call to Build for Productivity:
  • “Let’s pay them to do both. So we’ll build for that productivity piece and a consequence of that will be more secure.” (A, 19:44)

Data, Secrets, and AI Training Sets (21:43–24:52)

• Secrets Hidden in Data Sets:
  • As LLMs are trained on vast datasets—code, chat logs, etc.—leaked secrets are inevitably swept up.
  • “If you crack open those data sets and start looking for keys, it’s a very good indicator of where that data came from…” (A, 23:14)
  • Need for better mechanisms to clean training data of sensitive secrets while keeping datasets useful.

Password Cracking & The Cat and Mouse Game (24:52–27:12)

• AI and Password Cracking:
  • Language models could plausibly be used to generate smarter password-guessing rule sets by learning from an organization or individual’s likely behaviors.
  • “Maybe a language model could customize a rule set for a person or an organization.” (A, 25:15)
• Legacy Risk:
  • Old (pre-seed phrase) Bitcoin wallets—protected by passwords—are now targets for brute-forcing by those with significant compute power.
  • “You could hit a billion-dollar wallet…” (A, 26:17)

The Future: Integrating Security into the AI Stack (27:18–28:02)

• Cybersecurity Is Now A First-Class Citizen:
  • “There’s an OSI model equivalent for the AI world…includes cyber security. I really appreciated hearing [from a16z] that from day one.” (A, 27:18)
• Closing Thoughts:
  • Security must be designed alongside foundational technologies—no longer an afterthought.
  • “Now it’s doing real stuff and now it’s super meaningful and…is the future of the economy.” (C, 27:50)

---

Notable Quotes & Memorable Moments

• On the Real Problem Today:

  “It used to be the bottleneck was code development and now the bottleneck is secrets.” — Dylan Ary (00:00, 12:34)

• Secrets and Financial Risk:

  “The financial impact of these secrets is now more immediate…somebody says, okay, this is a problem. Today, the day that the secret leaked, I can see it’s a problem.” — Dylan Ary (00:00, 06:02)

• Secret Management Usability:

  “Every single secrets manager up until this point has taken 100% of the requirements from the security team…And the consequence is it has caused our developers to just not use the tool at all.” — Dylan Ary (17:33)

• On Security's Reluctance:

  “I had a conversation with the CISO as well…he had this thesis that if the secrets manager was really hard to use, it would make the developer think twice…They rolled their own, they just ended up manufacturing secrets and not storing them in the system.” — Dylan Ary (19:27)

• On AI Training Data:

  “If you crack open those data sets and you start looking for keys, it’s a very good indicator of where that data came from…real data from the real world because there’s real keys that can lock into real things.” — Dylan Ary (23:14)

• On the Future of Security in AI:

  “There’s an OSI model equivalent for the AI world…includes cyber security…whereas with the old OSI model, cyber wasn’t even a part of the story. Now we’re at least thinking about it before we’re building.” — Dylan Ary (27:18)

---

Important Timestamps

• 00:00–01:43 — Setting the stage: secrets as the new bottleneck
• 01:43–05:21 — Origin and evolution of TruffleHog
• 05:21–08:11 — Secrets leaks: financial impact and asymmetric risks
• 08:11–13:57 — AI agents’ use of secrets & rising leak trends
• 13:57–16:42 — Truffle’s product vision and future roadmap
• 16:42–21:43 — Cultural misalignment: Security vs Developer needs
• 21:43–24:52 — Data as a vector for secrets leaks (especially in AI training)
• 24:52–27:12 — Password cracking: brute-force, language models, and legacy wallets
• 27:12–28:02 — The future: Security’s place in the AI stack

---

Takeaways

• Secrets are moving from a niche concern to a central bottleneck for modern software and AI agent deployment.
• Leaks are increasing, not decreasing, and have immediate financial impact, especially in the cloud and AI context.
• Current secret management tools often sacrifice usability, pushing developers to insecure workarounds.
• Improvement requires building tools that serve both security and developer productivity.
• As data-hungry AI models ingest everything, including secrets, cleaning and managing data sets for security is an urgent challenge.
• The boundary between security, infrastructure, and AI is dissolving—security must be a foundational layer in the AI “stack.”

Transcript

[A]

It used to be the bottleneck was code development and now the bottleneck is secrets. It's literally the thing that makes developers slowest to get things in production. Right now the thing that's preventing us from moving as fast as the speed of the agent being able to write the code is the secrets. And that's what we're on mission to clear up and to solve. The financial impact of these secrets is now more immediate. It's definitely put some more immediate emphasis where somebody says, okay, this is a problem. Today, the day that the secret leaked, I can see it's a problem. I don't see it four years later when we finish argu, whether or not that is or is in our database.

[B]

If you can't robustly protect your secrets, you can't have reliable AI agents. Truffle Security is an A16Z backed cybersecurity startup with the goal of identifying and fixing exposed secrets such as passwords and API keys before they get picked up and exploited by hackers. Truffle's main product, Trufflehog, started out as an open Source project on GitHub before exploding in popularity and has now been downloaded over 15 million times. On this episode, Truffle's co founder and CEO Dylan Ary joins a16z partner Joel de La Garza to discuss the emergent security stack for AI agents, why leaks are actually getting worse and how Truffle evolved from an open source side project to a major VC backed startup. Let's get into it.

[C]

You know, we've been talking for the last couple weeks now that 2026 is really shaping up to be the year of the agents. We've been talking about the agent stack. What is an agent identity in agents? Security in agents? Observability. It's been agents nonstop and so really grateful to have you, Dylan, coming on the podcast today to talk about secrets and agents. And maybe as the CEO and founder and co founder of Truffle Security, you could start off by telling us what Truffle Security does and kind of what you focus on.

[A]

Yeah. So Truffle Security is a company built on top of a very popular open source cybersecurity tool called Truffle Hog. Truffle Hog is ran something like a quarter million times every day worldwide by security practitioners. It's been starred 23,000 times on GitHub. Within that is 8,000 different organizations that have mashed the like button. So it's ubiquitous within cybersecurity. If you're outside of cybersecurity is the most popular Tool for finding machine or non human secrets. What do I mean by that? Well, Joel is a human and he logs into Netflix with a password. A machine logs into a machine with a secret. And so that could take the form of a database password, it could take the form of a piece of cryptographic material, or it could take the form of just a random long token. For a very long time, they've been really difficult to use through no fault of developers, just from a lack of mature tooling, and that's caused them to leak out a hundred different ways. And So I wrote Troublehog as a passion project back in 2016, as a tool that's just meant to go out there and find those secrets in all the places where we don't want them. And every developer on the planet, myself included, has accidentally published a machine secret to a code repository. There are a lot of different ways it can happen. Truffle Hog is very good at finding when it happens. Validating that key is live, analyzing what that key can actually do, confirming that, yes, it does in fact have access to customer data. It does in fact have access to sensitive data. And if you're a defender helping to get that key revoked, if you're an attacker, there are no offensive capabilities within Truffle Hog. But everything I just described is usually enough information for an attacker to then go and build an additional step and take that key and do something malicious. So Truffle Hog for a long time has kind of sat that purple team, like red teamers can come in and use it to find the sensitive keys and then script up some bad things with it. Blue teams have used it to go in and find keys and then get them revoked. And that's been the legacy of Travelhog.

[C]

It's interesting too, because the rise of secrets as a problem is pretty much correlated directly to cloud, right? So I think in the olden days, everything was on prem. It was in your data center. You couldn't just telnet to it or connect to it or ssh or whatever. You had to actually go to the data center, plug into the local network and then get access to it. So people typically would use things like username and password, and that was sort of the original secrets. You know, the first transmission across the Internet in history was actually an attempt to log in. Right. So it's ingrained in the Internet. And then cloud came along and created these secrets, which you described. But it seems like as we've come further, like it's no longer just about getting access to things. Secrets are performing a lot of other functions.

[A]

It's a new problem and it's an old problem. And there's been huge like big named breaches. Like the Uber breach for example, where a bunch of teenagers found some Amazon key somewhere that they shouldn't have found it in a git repo and just directly talked to Amazon and downloaded a database backup that had every single driver and rider record. And so you're right in that cloud has like changed the game, but in a lot of ways the game has stayed the same. There are machines that need to talk to one another. There needs to be some piece of material to authenticate the machine. There's been a growing trend to rename machines as non humans, but it's the same repeat story. And then the question is, are we getting any better about preventing those keys from leaking out? And the answer is no. Like you could take Truffle hug today very easily. It's a very lightweight open source tool and you can point it at random code repositories on GitHub.com and you will 100% find keys that will give you access to data you're not supposed to have access to.

[C]

Well, the interesting thing with secrets, because I know we've been talking about this for years, obviously was a huge fan when you were releasing this as an open source passion project and still a huge fan as an investor to this day, it's been interesting because secrets like they weren't entirely appreciated outside of the security world. And then what started to happen was people came up with monetization strategies that could turn a secret very quickly into hard cash. And that could be spinning up AWS resources to mine crypto, which we saw a lot of, you know, back, you know, five, six years ago. It could now be stealing payment keys and issuing refunds or credits. Like what are you seeing at the forefront of kind of abuse of these secrets in the ecosystem?

[A]

You know what's tragic is a lot of times that students that see the worst of it. So what will happen is a large company will leak a key out, that an attack quickly spins up a seven figure mining operation behind. And this large account has all of the rate limits and exceptions approved and removed so that you can spin up that much compute in the first place. And then the cloud provider comes in and they're like, I'm so sorry this happened to you. It is your responsibility to make sure your keys don't leak out. But this one time will blow away the bill and this won't be a big deal. But on the other end of it when we look at the secrets that are leaking on GitHub, that accounts for maybe 20% of the leaks. The other 80% of the leaks are students who take their personal credit cards, go to Amazon for their personal projects, and create a personal account which do have limits on them. Like maybe you can only spend a thousand dollars in this account. But all the same, a crypto miner on a $1,000 account is a huge hit to a student. And you don't have a big account rep from Amazon coming in and waiving that. Instead you just have the student that's left picking up the pieces. So, yeah, I mean, I think, to answer your question, like, the financial impact of these secrets is now more immediate. And maybe a good example of that was when Dropbox got breached way back in the day. Not to say they did anything wrong. Again, just the tools are hard to use. That breach never financially impacted Dropbox. Right. Like you downloaded all of those usernames and passwords, the attacker did, posted them onto the public Internet. There was a long time of even figuring out whether or not it was those passwords. There was no litigation. There was no, like, financial impact on Dropbox. There was no cryptocurrency operation. But now, to your point, there is a direct and easy way to monetize directly from the company, not sort of indirectly through these password secondary markets. And that has changed the game a little bit. It's freed up a little bit of budget for people to want to tackle the problem. And it's, it's definitely put some more immediate emphasis where somebody says, okay, this is a problem. Today, the day that the secret leaked, I can see it's a problem. I don't see it four years later when we finish arguing about whether or not that is or is in our database.

[C]

Yeah, well, and arguably, I mean, I know it's not a solved problem and we're still making a lot of mistakes, but things have gotten better and we've made a lot of progress. And then I happened sort of like we have this as we were saying, you know, 2026 clearly shaping up to be the year of the agent. Every enterprise customer we talk to is either using form of agent in their business or building an agent to support their business. There isn't an existing agent stack, much less a security stack for agents. And it turns out secrets are still going to be really important, not only for these agents to operate, but also in the course of them developing things, they'll be using secrets. Just really curious, kind of like maybe to kind of break up the conversation. Let's maybe first start talking about, like, how are agents using these secrets? Like, to do their own thing, to spin up an AWS resource or something?

[A]

Yeah, I mean, so let's take a user story. I love using all of the CodeGen AI tools and in particular, like, some of them are getting really advanced. They can do a lot of things on your behalf. They can Talk directly to GitHub, for example, to answer your question, what can the agent do? Some of them could push code directly to GitHub on your behalf. How do they do that? Well, when you first sign up, you go through an OAuth flow and it says, would you like to grant access to your GitHub? If you say yes, a secret is manufactured behind the scenes that allows the agent to act on your behalf and the agent has access to that secret.

[C]

The OAuth token is of course a secret.

[A]

Well, the OAuth token is used to manufacture another secret. So, like, we're in the weeds a little bit. But yeah, if you go through an OAuth flow, there are a lot of secrets involved. And the reason why is what OAuth is. If you have a third party like GitHub and you're an agent services platform and I'm a developer, the way OAuth works is you go to GitHub and you say, hey, GitHub, I need you to go to Dylan and have Dylan give a thumbs up so that you feel comfortable giving me a token. And that whole exchange for you to talk to GitHub and GitHub to talk to me involves secrets every step of the way. So at the end of it, you, the agent, end up with a secret that you could talk to GitHub on my behalf. And then how do you manage that secret?

[C]

Absolutely. And I'm curious. I mean, I know you guys watch a lot of public repos, you watch a lot of private repos for secrets. Have you seen the number of secrets leaking out, trending in any particular direction?

[A]

We do watch large swaths of the public Internet for secrets that leak out, which includes coding platforms, that includes code testing platforms, that includes JavaScript. People just put JavaScript in a secret. Agents will just put job, you know, secrets in JavaScript. And we monitor common crawl, which is a scrape of the entire Internet. And I would say right now the trend is up, not down.

[C]

Yeah. And I guess the question, I mean, it seems like the coding practices and sort of the behaviors of some of these tools, it's just incredibly early. And it sounds like people are trying to solve it. Right. And I think there's probably some progress getting made there. And I'm sure it'll happen very quickly just because everything seems to be moving 10 times the speed. I think longer term is really interesting to me because with these agents, you see the promise of. I mean, you grew up in software. I've grown up in software. It's all I've ever known. Like, you know, first, my first precious toy was a Nintendo video game system. Right. Like, it's kind of just the way though, like, we've lived our lives in software and like, it just seems like we're getting to the point now where software is going to start escaping the machine. Where it's like these agents do start to look like things that move into the real world and start to do things on our behalf. Right. And you start with the easy use case, which is an agent that books me a restaurant reservation. And then you eventually end up with a use case with an agent that buys a birthday present for my kid. And so that presupposes shifting yet again to a very different type of secret where it may be a token to physically access a building. It might be a credit card that these agents are having to use on your behalf. And I'm kind of curious, like, longer term, how do you think that starts to develop? Because it seems like if you mishandle a credit card that's also pretty. Or the keys to your house, if I left the keys to my house on the street outside the building and people knew where I live, they'd have a very bad time.

[A]

I mean, you're going to see payment process providers restricting some of the capabilities you would otherwise be able to do with a credit card. And so what that might take the form of is maybe the agent has the ability to choose when to reorder your cat food, but maybe it can't go out and order 7,000 cat foods.

[C]

Spending limits.

[A]

And we need those gates in place. But what that means is we have to have these MCP servers, have to have their PayPal token or whatever, and then PayPal has to be set up to be able to put those limits in place. So those are changes that need to happen. But to your point about the software landscape changing, we both remember five years ago, let's say you just need to take all the like. Let's say you're a solutions architect and all you need to do is take all the customers that are new from the day out of Shopify and post them into Slack. You would spend hours and hours writing that script and testing it and all that stuff. Now that part of it takes six seconds. But then what do you need to do? Well, you need to go to your Shopify account, figure out how to get a secret out of it. Maybe that involves an IT ticket. If you don't have direct access, then you need to go to your Slack and you need to get a secret out of it. That probably does require an IT ticket, because usually that's more locked down. And then once you get those secrets, you have to figure out how to either give them access to your agent or how to give your agent access to your secrets manager. Both of those are unsolved problems. And that process takes hours. And so what's weird is like, it used to be the bottleneck was code development, and now the bottleneck is secrets. It's literally the thing that makes developers slowest to get things in production. Right now, the thing that's preventing us from moving as fast as the speed of the agent being able to write the code is the secrets. And that's what we're on a mission to clear up and to solve.

[C]

And as always, they're going to blame security for slowing them down. But I mean, it's interesting because you. It seems like you're going through a transition as a company yourself, right? Like, you've started effectively as a way to detect secrets, and then you became a way to validate secrets, and it sounds like you're moving more towards also managing those secrets. So it'd be great to maybe hear how you think about the company evolving as the technology landscapes and the requirements evolve.

[A]

Well, so we just raised a Series B. Thank you, Andreessen Horowitz.

[C]

Our pleasure.

[A]

Thank you, Intel Capital and all of the other countless investors that helped make that happen. And you're absolutely right. We started as a company that would just detect secrets, which was built from our open source. And then we have evolved that into the ability to actually integrate that secret into the provider, to check if it's live and see when it rotates. And then we evolved that into understanding the secret, which we call analyze. And that's like, does this thing have access to production data? Like, give me the whole story behind the secret. And so as an example, we just launched the ability to take a Google Cloud secret and see exactly what resources and what permissions within your Google Cloud organization that secret is able to do. We call that GCP analyze. And by the way, it's just a small aside because secrets are so freaking hard, it is very difficult to log into the Google Cloud interface and get an understanding of what that key can do.

[C]

It's the UX that shipped a thousand companies.

[A]

And so the fact that we were able to improve that story just make it very easy and quick to say, okay, that's what that thing can do. That is a huge win for our customers. But the number one question we started getting back is like, look, it's great that when a GCP key leaks, you can tell me what it can do, but what about my key that hasn't leaked? I still don't know what it can do. And what about my XYZ service that's reading in from an archaic secrets manager that's opaque, that I can't see? Like, I have no idea what that application could do because I can't see inside the secret manager? Wouldn't it be nice if you could classify all the keys that are in there, analyze them and tell me what, what that application can do as a proxy for us being able to actually crack open that Secrets manager and trying to understand it ourselves. And so that's a product that we're calling Inventory, which we'll be working on next year. So we'll be able to take people's existing secrets managers and see inside them and better understand them. And then there's a long tail of productivity tooling that we think we can bring to the world. And I think we'll probably start there. To your point is agents, like people are using agents to interact with, with secrets a myriad of different ways, and it's a pain in the butt. And so what we're going to do is we're going to make tooling, just make that way easier and way harder to do the wrong thing. It's going to save everybody a ton of time. And we're going to clear up that bottleneck so that it no longer takes you four hours to fight the secrets part and six seconds to write the script. Hopefully the whole thing will just take a few seconds and you can get up and running faster.

[C]

Awesome. Yeah, that's a really incredible roadmap and really looking forward to those capabilities. I think. It's funny, I mean, like, we all, I mean, we've been in security our whole careers. We know all the security people, we know how they can be pretty prickly. And I recall a conversation, I think probably 10 years ago with a CISO who is trying to make the argument that as you move to cloud, you end up essentially with two big problems, authorization and secrets. Right? And that tends to be like, where you fall down as A builder on top of cloud infrastructure. And I just remember very clearly the CISO being so certain that if you're building with secrets, fixed secrets, static secrets in your environment, you're an idiot, you're building wrong and that this is a problem that'd be solved in five years. So 10 years later, it seems like there's more secrets now. And I'm actually really curious because of where you sit in the ecosystem, like, do you think we have fewer secrets or more in 10 years?

[A]

Well, here's, I think the problem with secrets management up until this point, you have two main stakeholders that need to buy into a secrets management solution. You have the development workforce and you have the security team. Every single secrets manager up until this point has taken 100% of the requirements from the security team. What that means is the security team says I want short lived secrets that get rotated out every 90 days. And GitHub doesn't have an API for rotating secrets. So if you take the security team's requirement at their word there, what, you're expecting your developer every 90 days to have to log into this stupid thing and replace the secret. And so a consequence of that is your developers just don't use the secrets manager and then you have to take a step back and have that conversation with the security team a second time and say, look, we tried to lock them down with more role based access control, more encryption, shorter lived keys. And the consequence is it has caused our developers to just not use the tool at all. So is that a better place or should we maybe start taking some of our requirements from the developer and maybe start asking what would get the developer excited to use the secrets manager at the start of the process and not begrudgingly pulling it at the end. And I think if we take that approach, the data will be clear and truffle hog will show us from seeing how many keys leak out doing it this way to how many keys doing it that way, that we just need to start taking more requirements from developers. We just need to start listening to them more about what will unblock their processes and make them move faster and not what will try to lock these things down. And I had a conversation with the CISO as well, and this was a crazy conversation, but the CISO basically said I want the secrets manager to be as hard to use as possible. And the reason why, the reason why is the secrets that he had this thesis that if the secrets manager was really hard to use, it would make the developer think twice about whether they needed a secret in the first place.

[C]

It'S the punishment paradigm.

[A]

And do you want to know, do you want to guess what ended up happening? They rolled their own, they just ended up manufacturing secrets and not storing them in the system.

[C]

Because we pay our employees to get their work done, not to be secure, right? And so like anything you put in the, any obstacles you put in the way of the employee getting their job done, they're going to jump around.

[A]

Well, let's pay them to do both. So we'll build for that productivity piece and a consequence of that will be more secure.

[C]

Well, and it's even funnier because if you look, there have been a lot of vaults that have been built over the years, right? And the original vaults were the these things called HSMs. They were these hardware based security storage things that you put your keys into. You couldn't tamper with them. They have like little acid packets inside of them. If you crack the case open, the acid leaks out and it fries the memory, right? Like it's the most James Bond y nonsense you've ever seen in your life. Incredibly difficult to use, incredibly expensive. You like shift them the wrong way and they melt themselves. It's like horrible. And the secrets experience for a long time in the enterprise was so horrible that developers just went out and found their own solution, right? And that solution, without naming names, was a very simplistic check in, check out. Here's your secret vaulting type solution. And then the CISOs became the customers of that solution and that solution became more complicated and started to become hard to use. And so it's just really funny over time, to your point, like a lot of the problems are of our own making and we just haven't really leaned into that developer experience.

[A]

And even when they were simple, they still weren't built for productivity. You know, they were built to be key value source. They weren't self aware of what was inside of themselves and they didn't know what privilege the keys were that were getting loaded into them. So you couldn't answer questions like, you know, is this key being used in multiple places? What can that application, can it access? Shopify, can it access? Like these are just things that have been completely left out of story. And then when security entrenched all of their requirements, it made those questions harder to answer, not easier to answer. It made that secret manager harder to get access to. Which, you know, take that as a requirement from the CISO I mentioned before. But as a consequence, like the idea of being able to just understand some basic questions about what can my application do is just become intractable and it's way, way too hard. And we need to make that easier.

[C]

Absolutely. It's at the point where you're doing product gathering or product requirements gathering and you arrive at this thing needs to self destruct that you have to know about, even though you've kind of hit the end of the rabbit hole of requirements don't matter to the average user.

[A]

I mean, are you going to do incident response on that HSM that has ink all over it to try to figure out what happened.

[C]

And so one of the interesting things that's happening is that all sorts of data is valuable now. It used to be. I mean, I don't, you know, you remember probably five years ago in the enterprise, every CISO said I want to know where my most sensitive data is and I want to protect it. And so there was a whole litany of tools that were built to find sensitive data and people built programs around protecting it. There's fast forward five years and now it's like all data is valuable. Right? Like it's sort of the bitter lesson has been heard and enterprises are now very sensitive about all their data. And the gold rush is basically pre training models with data. And you know, the way that these things work is you need more and more tokens, more and more data to get better performance. And we're getting close to the end, perhaps, perhaps there's an argument that we're getting close to the end of having trained on all the data that's already out there. And one of the interesting things about this data is that it's collected from all different sorts of sources. It can come from code repos, it can come from open source code repos, it can come from proprietary code repos that have been licensed. And there's gotta be a lot of secrets in there. I'm curious, kind of what have you seen with some of these data sets and how have you seen secrets play out in that training and pre training world?

[A]

Yeah, I mean, so the reality is if you want a robot that behaves like a chatbot, you need chat data. And so that means going and harvesting a bunch of teams and Slack and IRC and any piece of chat data you can get your hands on. And then the question is, if the imperative is to move at the speed of light and get your thing trained up as quickly as possible and to use these culture of data scientists that are historically used to the academic world where everyone shares everything, what does that mean for all the teams and slack and chat data? And so we've done that exercise before. They just moved from LFS to a new system, but the data is the same. If you crack open those data sets and you start looking for keys, it's a very good indicator of where that data came from. Right. It's not manufactured from a model. It's real data from the real world because there's real keys that can lock into real things. And this is just another area where today the secrets maturity is not there. It's very difficult to keep those data sets helpful and usable, but clean and shareable. And it's. It's something that we need to work on making easier for people.

[C]

Absolutely. As you talk about having secrets in these next token prediction systems, the only thing I can think of is that they might. They must make wonderful password crackers.

[A]

Yeah, I don't know about that one.

[C]

I think, like, it's nowhere near efficient, I'm sure. And the cost of inference is so high.

[A]

Well, I think maybe the hypothesis you're thinking of is if we could train the models to think more like Joel. We get it to guess Joel's password.

[C]

Guarantee you can't. But yeah, no, I'm sure someone will respond to this video with some of my passwords. I'm sure they're out there, but it's an interesting question.

[A]

I mean, we could touch on the evolution of password correcting briefly, but it used to be we would just have these big rainbow tables of passwords. And then those rainbow tables got so big, it turned out it was easier to start with a base set of passwords and augment them with rules. And what are rules? Rules are like, you put an exclamation point at the end and you flip it backwards and you do all these things that people commonly do with their passwords. But then, you know, I think there's a really good argument to be had. Maybe a language model could customize a rule set for a person or an organization. When I used to do red teaming, we would break into a company, we would take that rule set and we would add, like the address of the company. We would add some things that are specific to the company. Yeah. Maybe if a language model were to just hammer on things that were specific to the thing that you're trying to, it could make that rule set really powerful. And you load that into your traditional cracker and probably get some more passwords out of it.

[C]

Yeah, absolutely. It's funny too, as we talk about these new gigawatt data centers, all I could think of is all the passwords you could crack.

[A]

With that. Right, all the bitcoin wallets. Yeah, exactly, because they used to be generated with passwords, now it's seeds. But back in the day, you just pick any password you want to encrypt your wallet.

[C]

And we would be using GeForce video cards to crack these things. Right. And now you got these black wallets.

[A]

It's funny, it's not the new wallets that you're cracking, it's the old ones still.

[C]

Yeah.

[A]

It's the ones that were manufactured in, like the first months and years of Bitcoin that didn't use seed phrases, they used seed passwords.

[C]

People are still going back in the actual phase space for that system is probably currently crackable with a data center.

[A]

That size to be able to guess all those old passwords. It's an ongoing cat and mouse game. But the stakes just keep getting higher and higher as the value of crypto keeps going up and up. So you crack one of those old wallets, you guess somebody's password back then, like. Yeah. I mean, you could hit a billion dollar wallet. Yeah, Yeah.

[C]

I mean, I bet you could probably find a way to find a wallet that's. That you could try to crack, understand the value in that wallet and then lease out enough GPUs to crack it.

[A]

You don't need the wallet because if you. All you need to do is you just need to generate seed phrases that match something that's funded on the chain.

[C]

On the chain. Yeah.

[A]

Yes. You don't need the wallet itself for those older ones. There are, you know, some of the newer ones, maybe you need the wallet and you need the decryption password and it's more advanced, but you just want to get the old ones. You just need to generate the right seed material, and as long as there's funds in it, you're, you know, good to run off to the Cayman Islands or whatever.

[C]

And this is why they don't let us play with GPUs.

[A]

This is why Joel is just going to be, you know, totally in another country one day. We're never going to know why.

[C]

Yeah, exactly. Awesome. This has been great. Thank you so much for coming by. Always a pleasure and congratulations on the round.

[A]

Thank you for all of your support. And look, I really liked Andreessen's philosophy on what the next couple of years are going to look like. And for those not familiar, the way it was explained to me by Joel and Martine and people smarter than me is there's like an OSI model equivalent for the AI world, and that includes energy. It includes data centers, it includes foundational models, but it also includes cyber security. And I really appreciated hearing seeing that from day one. Whereas with the old OSI model, cyber wasn't even a part of the story. Now we're at least thinking about it before we're building.

[C]

Well, now it's doing real stuff and now it's super meaningful and you know, it's the future of the economy. So thank you so much for coming on. This has been great. It's great to support you and really do appreciate all that you've done for us. Thank you.

[B]

Thanks for listening. If you enjoyed the episode, let us know by leaving a review. We've got more great conversations coming your way. See you next time. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com disclosures.