Podcast Summary: NVIDIA’s Agentic AI for Container Security with Amanda Saunders and Allan Enemark

Podcast Information:

• Title: Software Engineering Daily
• Host/Author: Software Engineering Daily
• Episode: NVIDIA’s Agentic AI for Container Security with Amanda Saunders and Allan Inemark
• Release Date: January 30, 2025
• Description: Technical interviews about software topics.

Introduction

In this episode of Software Engineering Daily, host Gregor Vand engages in an in-depth discussion with Amanda Saunders, Director of Enterprise Generative AI Software at NVIDIA, and Allan Inemark from NVIDIA’s Morpheus Cybersecurity SDK team. The conversation centers around NVIDIA's innovative "blueprints"—reference workflows leveraging agentic and generative AI for various applications, with a particular focus on vulnerability analysis for container security.

Guest Backgrounds

Gregor Vand introduces the guests, highlighting their extensive experience at NVIDIA. Amanda Saunders has been with NVIDIA for a decade, evolving with the company's transition from graphics to machine learning and now generative AI. Allan Inemark has spent seven years at NVIDIA, moving from data visualization to cybersecurity, utilizing his background in industrial design to contribute to the Morpheus team.

Understanding NVIDIA Blueprints

Amanda Saunders provides an overview of NVIDIA blueprints, describing them as composite packages that integrate NVIDIA’s libraries, SDKs, microservices, and open-source tools into customizable reference workflows. These blueprints serve as starting points for developers, enabling them to rapidly build applications tailored to their specific needs.

“They are reference workflows. They take all the libraries, the SDKs, the microservices, even things from the open source and from the ecosystem around AI and simulation and everything like that. And it packages it up into something that can be taken, composed, customized to meet the needs of whatever company is taking them on.”
— Amanda Saunders [05:00]

Blueprints Beyond Vulnerability Scanning

When asked about other blueprint topics, Amanda mentions that NVIDIA has developed around 14 blueprints, categorized into four main areas:

1. AI: Including customer service agents and digital humans.
2. Omniverse: For digital twin platforms and simulations.
3. Bionemo: Targeted at the healthcare marketplace to leverage AI in digital biology.
4. Isaac Groot: Focused on robotics platforms.

Deep Dive: Vulnerability Scanning Blueprint

The conversation shifts to the specific blueprint for vulnerability scanning in container security. Allan Inemark explains the challenges of managing the vast number of Common Vulnerabilities and Exposures (CVEs), which are continuously growing and pose significant threats to containerized applications. Traditional methods of vulnerability analysis are time-consuming and labor-intensive, prompting NVIDIA to develop an AI-driven solution.

“Right. So if you're releasing thousands of containers and then you have tens of thousands of CVEs and each one is a real difficult thing to mitigate. It's hard to put out secure software.”
— Allan Inemark [09:57]

Leveraging Morpheus SDK and Agentic AI

Allan elaborates on NVIDIA’s Morpheus SDK, a cybersecurity AI framework designed to handle large-scale data processing efficiently using GPU acceleration. The blueprint utilizes Morpheus to create an event-driven, streaming pipeline that automates the mitigation of CVEs by leveraging large language models (LLMs).

“Cybersecurity is a data problem. And you know, there's so much data that it's hard to get grips on. You can't store it all. And if you have stored it all, it's hard to kind of munch through it.”
— Allan Inemark [10:08]

Model Selection and Embeddings

The blueprint employs NVIDIA’s NIM (NVIDIA-Integrated Models) microservices, specifically utilizing the Llama 3.17B model for its balance between performance and capability. Amanda highlights the flexibility of blueprints in accommodating new models as they are released.

“The blueprint today built on Llama 3.170 B because we think that works really well. But three months down the line, six months down the line, maybe we'll update it with some of the later models.”
— Amanda Saunders [21:42]

Allan further explains the role of embeddings in translating documents into a format understandable by the model, enabling efficient querying and context understanding.

“The embedding space is sort of how you translate a document to something that the model can understand.”
— Allan Inemark [22:43]

Technical Architecture: Plan and Execute Pipeline

Gregor Vand probes into the architectural choices behind the blueprint, focusing on the "plan and execute" paradigm. Allan describes this structure as mirroring the workflow of security analysts, breaking down tasks into parallelizable segments handled by different agents powered by LLMs.

“You're making an application, you're more of an agentic system application. Right.”
— Allan Inemark [39:36]

Real-World Reception and Community Engagement

Both Amanda and Allan discuss the positive reception of the vulnerability scanning blueprint within NVIDIA and among external partners. The open-source nature of the blueprints encourages community contributions, enhancing the tools through collaborative refinement.

“Developers have been really excited to have a starting point... It's just a lot. And so I think just having that starting point for people to then say, okay, well, I followed the blueprint, I did these things, but actually for my use case, if I make these changes, it works better.”
— Amanda Saunders [31:09]

Future Directions and NVIDIA’s Role in Security

Addressing NVIDIA’s future in the security landscape, Amanda clarifies that NVIDIA does not aim to become a cybersecurity solution provider. Instead, the company focuses on empowering cybersecurity professionals with advanced AI tools.

“We are an AI company. We understand AI, we're excited about it, we've got a lot of experience in it. So what we want to do is bring AI to cybersecurity providers, to cybersecurity problems.”
— Amanda Saunders [40:45]

Allan adds that NVIDIA's approach is to offer versatile frameworks like Morpheus and blueprints that can be adapted to a wide range of cybersecurity challenges, fostering innovation across various domains.

Personal Reflections and Advice

Towards the end of the episode, Amanda and Allan share personal insights on career development. Amanda emphasizes the importance of continuous learning and adaptability, while Allan reflects on the necessity of staying curious and open to evolving technologies.

“You can learn anything. As somebody who doesn't have a traditional technology background... as long as you stay open and keep learning, there's no telling where you can go.”
— Amanda Saunders [42:23]

“What stuff is able to be done right now? So on some level, just be open to it and stay curious.”
— Allan Inemark [43:01]

Conclusion

Gregor wraps up the discussion by encouraging listeners to explore NVIDIA’s build.Nvidia.com platform to access and experiment with the blueprints. Both guests express enthusiasm for the ongoing development and community engagement surrounding NVIDIA's AI-driven cybersecurity tools.

“Go to build.Nvidia.com that's their starting point. We've got all the latest models on there. We've got these blueprints that you can go and explore, and then your journey just starts there.”
— Amanda Saunders [45:11]

Final Thoughts

This episode highlights NVIDIA’s commitment to advancing cybersecurity through AI, offering developers and security professionals robust tools to tackle complex vulnerability management challenges. By providing customizable blueprints and fostering an open-source community, NVIDIA empowers users to build secure, efficient, and scalable containerized applications.

Notable Quotes:

• Amanda Saunders [05:00]: “They are reference workflows. They take all the libraries, the SDKs, the microservices, even things from the open source and from the ecosystem around AI and simulation and everything like that. And it packages it up into something that can be taken, composed, customized to meet the needs of whatever company is taking them on.”

• Allan Inemark [10:08]: “Cybersecurity is a data problem. And you know, there's so much data that it's hard to get grips on. You can't store it all. And if you have stored it all, it's hard to kind of munch through it.”

• Amanda Saunders [21:42]: “The blueprint today built on Llama 3.170 B because we think that works really well. But three months down the line, six months down the line, maybe we'll update it with some of the later models.”

• Allan Inemark [22:43]: “The embedding space is sort of how you translate a document to something that the model can understand.”

• Amanda Saunders [31:09]: “Developers have been really excited to have a starting point... It's just a lot. And so I think just having that starting point for people to then say, okay, well, I followed the blueprint, I did these things, but actually for my use case, if I make these changes, it works better.”

• Amanda Saunders [40:45]: “We are an AI company. We understand AI, we're excited about it, we've got a lot of experience in it. So what we want to do is bring AI to cybersecurity providers, to cybersecurity problems.”

• Amanda Saunders [42:23]: “You can learn anything. As somebody who doesn't have a traditional technology background... as long as you stay open and keep learning, there's no telling where you can go.”

• Allan Inemark [43:01]: “What stuff is able to be done right now? So on some level, just be open to it and stay curious.”

• Amanda Saunders [45:11]: “Go to build.Nvidia.com that's their starting point. We've got all the latest models on there. We've got these blueprints that you can go and explore, and then your journey just starts there.”