Risky Business #801: AI Models Can Hack Well Now and It's Weirding Us Out
Release Date: August 6, 2025
Host: Patrick Gray
Guest: Adam Boileau
Featuring: Heather Adkins (Google), Amberly Jack (Producer)
Introduction
In this episode of Risky Business, host Patrick Gray delves into the evolving landscape of information security, particularly focusing on the intersection of artificial intelligence (AI) and cybersecurity. The episode features insightful discussions with Adam Boileau, Heather Adkins from Google, and Amberly Jack, providing a comprehensive overview of recent developments, potential threats, and innovative defenses in the cybersecurity realm.
AI and Vulnerability Discovery
Google's Project Zero and Big Sleep
The episode kicks off with a discussion about Google's recent announcement concerning the discovery of numerous bugs in widely-used open-source software. Patrick Gray interviews Heather Adkins, VP of Security Engineering at Google, who sheds light on Google's groundbreaking project combining AI with vulnerability research.
- Heather Adkins [02:47]: “This is a project we call Big Sleep... applying large language models, modern AI, to the problem of finding vulnerabilities.”
Heather explains that Big Sleep is a collaborative effort between Google DeepMind and Project Zero aimed at making zero-day vulnerabilities harder to exploit by leveraging AI for bug discovery. She emphasizes the significance of the bugs found, noting their potential impact:
- Heather Adkins [03:38]: “So these are big deal bugs... actively pursued by commercial surveillance vendors in order to cause chaos and mayhem.”
Adam Boileau's Perspective
Adam Boileau adds his thoughts on the matter, highlighting the credibility and expertise of Google's teams:
- Adam Boileau [01:33]: “Google Project Zero... have all of the compute in the world and all of the training data in the world. So, like, if anybody is going to do an amazing job of this, this is probably the crew that's going to do it.”
He acknowledges the initial skepticism around AI-driven bug hunting but concedes that Google's track record makes the findings credible.
Optimism for AI in Security
Heather expresses newfound optimism about AI's role in cybersecurity, contrasting it with her previous skepticism:
- Heather Adkins [04:50]: “I'm in the booster camp... making sure that the good guys have the same technology to find those bugs and fix them.”
She envisions AI as a force multiplier that can accelerate the patching and improvement of software, potentially transforming the security industry within the next five years.
AI in Penetration Testing
Crossbow: AI-Driven Pen Testing Startup
The conversation shifts to AI-powered penetration testing, focusing on startups like Crossbow, which has recently raised a Series B round of $75 million. Crossbow aims to lead the bug bounty landscape, currently topping HackerOne's leaderboard.
- Patrick Gray [09:25]: “Crossbow... it's actually leading the HackerOne leaderboard... AI is doing this at scale.”
Adam Boileau's Insights
Adam discusses how AI is automating the more mundane aspects of pen testing, allowing testers to focus on higher-level analysis:
- Adam Boileau [10:47]: “Most pen testing work isn't glamorous... finding them at scale is key.”
He acknowledges a natural hesitation ("bah humbug") toward AI in pen testing but recognizes the efficiency and scalability it brings.
Future of Pen Testing with AI
Patrick speculates on the future, imagining AI agents that can handle complex tasks autonomously:
- Patrick Gray [13:55]: “You could have a little AI James Kettle... you can just get the model to do it for you.”
Adam agrees, envisioning AI as sidekicks that enhance human capabilities rather than replace them:
- Adam Boileau [14:38]: “You just have AI do the stuff computers are good at and leave humans to do the bits that humans are good at.”
Geopolitical Cybersecurity Issues
China Accusations Against the US
The episode addresses China's stern accusations against the United States for hacking military R&D centers. Adam Boileau humorously critiques the Chinese cyberspace administration's claims:
- Patrick Gray [24:52]: “It's like they are accusing the United States of doing very tightly scoped and well-targeted legitimate intelligence collection that has military relevance.”
Adam relates this to potential Senate committee discussions on intelligence overreach, highlighting China's perception of US cyber activities.
Microsoft SharePoint Maintained by China
Renee Dudley's reporting reveals that SharePoint on-premises is maintained by Microsoft China, raising concerns about software supply chain security.
- Adam Boileau [26:41]: “Microsoft is an American company, but the reality is a little more messy... nationalizing software development is fraught with complexity.”
He underscores the importance of scrutinizing software supply chains beyond corporate headquarters, especially in a geopolitically tense environment.
US Legislation on Chip Backdoors
The discussion moves to proposed US legislation aimed at mandating backdoors in AI chips, allowing the government to disable them under certain conditions.
- Governor Adam Boileau [29:28]: “They are asking Nvidia and other manufacturers to have geolocation capabilities and the ability to turn them off if they are being used non-compliantly.”
Patrick raises concerns about the practicality and potential security implications of such measures. Adam likens it to failed copy protection schemes, emphasizing the technical challenges involved.
Recent Vulnerability Reports
SonicWall VPN Vulnerabilities
John Greig from The Record reports critical vulnerabilities in SonicWall's SSL VPN, prompting the company to advise users to restrict access or remove the devices from the internet.
- Adam Boileau [34:07]: “Somebody is ransomware-ing SonicWall customers... probably a SonicWall zero day in the wild.”
Patrick highlights SonicWall's mitigation advice, which includes enforcing multi-factor authentication, though Adam notes it's a reactive measure in the face of unknown vulnerabilities.
Dell Secure Enclave Bugs
Research from Cisco Talos uncovers significant bugs in Dell's secure enclave, affecting biometric authentication and potentially allowing attackers to bypass security measures.
- Adam Boileau [35:17]: “They found bugs in the Dell secure enclave... enabling attackers to install rogue certificates and gain persistent access.”
Patrick contrasts this with the robustness of Apple's Secure Enclave, attributing the difference to Apple's integrated security expertise.
ChatGPT Conversations Indexed by Google
Joe Cox from 404 reports that approximately 100,000 ChatGPT conversations marked as publicly accessible were scraped and indexed by Google, leading to unintended data exposure.
- Patrick Gray [38:00]: “Some chat GPT conversations... somehow got indexed by Google.”
Adam laments that attempts to delist the data came too late, resulting in the information being widely available.
Russia's Surveillance and Messaging App
Max Messenger: Russia's WeChat Rival
Amberly Jack explores Russia's development of Max Messenger, a government-backed national messaging service aiming to replicate the functionality of China's WeChat.
- Amberly Jack [46:25]: “It's designed to become a Super app where you can pay your taxes, sign government documents, talk to schools... everything, not unlike WeChat.”
Despite initial low adoption rates, with only 2 million registered users compared to WhatsApp's 100 million in Russia, the app's integration into government and banking services indicates a strategic push towards digital control.
- Adam Boileau [50:08]: “If you can do this, that becomes the playbook. In non-democracies, the government sponsors an everything app.”
Patrick and Adam discuss the potential for Max to become the sole platform for essential services, enhancing governmental surveillance and control.
Windows 10 End of Support: Sponsor Interview
Interview with Sean Ollerton of Device
The episode transitions to a sponsored segment featuring Sean Ollerton from Device, a company enhancing Microsoft's Intune platform to facilitate large-scale Windows upgrades. With Microsoft's end of support for Windows 10 looming in October, Sean discusses the challenges and solutions for enterprises.
- Sean Ollerton [55:17]: “Over 50% of enterprise devices are still running Windows 10... if you haven't started upgrading, you're likely too late.”
He outlines the upgrade process using Intune, emphasizing the importance of preparing hardware compatibility and orchestrating phased rollouts to minimize disruptions.
- Sean Ollerton [60:07]: “With Intune, once you're enrolled, you can use Windows update readiness reporting and create feature updates to roll out Windows 11 in stages.”
Sean also addresses licensing tiers, noting that Intune is broadly available across various Microsoft licenses, making it accessible for most organizations.
Conclusion
Patrick Gray wraps up the episode by thanking the guests and highlighting the transformative potential of AI in cybersecurity, the complexities of geopolitical cyber tensions, and the imminent challenges organizations face with the Windows 10 end of support. The episode underscores the need for robust security practices and the adoption of advanced tools to navigate the rapidly evolving digital threat landscape.
Notable Quotes:
-
Heather Adkins [04:50]: “I'm in the booster camp... making sure that the good guys have the same technology to find those bugs and fix them.”
-
Adam Boileau [14:38]: “You just have AI do the stuff computers are good at and leave humans to do the bits that humans are good at.”
-
Patrick Gray [38:00]: “Some chat GPT conversations... somehow got indexed by Google.”
-
Sean Ollerton [55:17]: “Over 50% of enterprise devices are still running Windows 10... if you haven't started upgrading, you're likely too late.”
This summary captures the essence of Risky Business #801, providing a comprehensive overview of the key discussions and insights shared by the guests. Whether you're an information security professional or simply interested in the latest cybersecurity trends, this episode offers valuable perspectives on the evolving interplay between AI, geopolitics, and cybersecurity.