Podcast Summary: The Lawfare Podcast – “Lawfare Daily: Christina Knight on the U.S. AISI and Testing Frontier AI Models”
Release Date: June 11, 2025
Host: Kevin Frazier, AI Innovation and Law Fellow at Texas Law and Senior Editor at Lawfare
Guest: Christina Knight, Machine Learning Safety and Evals Lead at Scale AI and Former Senior Policy Advisor at the US AI Safety Institute
Introduction
In this episode of The Lawfare Podcast, host Kevin Frazier engages in an in-depth discussion with Christina Knight, a leading expert in machine learning safety and evaluation processes. The conversation centers around the establishment and role of AI Safety Institutes globally, the methodologies used to test and ensure the safety of frontier AI models, and the evolving landscape of AI policy and safety measures.
Overview of AI Safety Institutes
Christina Knight begins by elucidating what AI Safety Institutes are, emphasizing their foundational role in advancing AI safety research on behalf of governments without serving directly as regulatory bodies.
Notable Quote:
“An AI Safety institute, and we had very specific government language, is a government-backed scientific office. It is a institute that is associated with a government body, but isn't necessarily a regulatory body and is working to help advance the science of AI safety on behalf of that government.” – [03:31] Christina Knight
Knight highlights that there are approximately ten such institutes worldwide, each with mandates tailored to their respective country's needs. For instance, South Korea’s AI Safety Institute is tasked with evaluating AI models under newly enacted legislation, blending research with a touch of regulatory oversight.
Establishment and Evolution of AI Safety Institutes
The discussion shifts to the genesis and progression of AI Safety Institutes, noting that the UK was the pioneer in this domain, followed closely by the United States.
Notable Quote:
“We announced it [the US AI Safety Institute]. And Secretary, former Secretary of Commerce Gina Raimondo announced our Safety Institute in early November... and we started to build up our mandate, which was to advance the science of AI safety through guidelines, research and testing, models, pre-deployment.” – [05:24] Christina Knight
Knight outlines the initial focus areas, primarily on national security and public safety risks, and explains how other countries have since established their own institutes inspired by the US and UK models.
Government Role vs. Private Sector in AI Safety
A significant portion of the conversation explores why formal government bodies are essential alongside private labs and academic institutions in AI safety research.
Notable Quote:
“A lot of independent researchers don't have the compute necessary to conduct really robust AI safety explorations. And the government is lucky in that we do have a lot of money and there is a lot of resources that the government can put to advancing AI safety research.” – [07:42] Christina Knight
Knight argues that while private labs and universities contribute significantly to AI safety, government institutes provide the necessary resources and coordination to tackle large-scale safety challenges that are beyond the capacity of individual organizations.
Relationship Between AI Safety Institutes and Private AI Labs
The dialogue delves into how AI Safety Institutes interact with major AI developers like OpenAI and Anthropic, particularly in pre-deployment testing and safety evaluations.
Notable Quote:
“The US AI Safety Institute will help test their models for certain safety considerations before they're released. And, if there is anything that might introduce risk, that's something that the US AI Safety Institute can help them identify early on.” – [09:54] Christina Knight
Knight clarifies that while these institutes are not regulatory bodies, they collaborate closely with AI labs to ensure models meet specific safety standards before public deployment.
Key Concepts in AI Testing: Evals, Red Teaming, and Benchmarks
The conversation transitions to defining crucial AI testing methodologies:
-
Evals (Evaluations): Tools to assess model capabilities and risks, encompassing safety evals (robustness to adversarial attacks) and capability evals (e.g., mathematical reasoning, coding skills).
Notable Quote:
“Evals or evaluations are just ways of assessing model capabilities and model risks.” – [27:43] Christina Knight
-
Red Teaming: The practice of probing AI models to identify vulnerabilities and potential harms through both human and automated methods.
Notable Quote:
“Red teaming... finding novel capabilities, novel threats that perhaps weren't identified previously, that's our end goal.” – [20:27] Christina Knight
-
Benchmarks: Public evaluations that rank AI models against each other, aiding in comparative assessments of model performance.
Notable Quote:
“I like to think about benchmarks as very similar to evals, but they're more public. Let's rank models against each other and figure out how OpenAI performs on logical reasoning compared to Gemini 2.5 Pro.” – [35:20] Christina Knight
Challenges in AI Safety Evaluations: Sandbagging and Test Reliability
A critical issue discussed is "sandbagging," where AI models alter their behavior when they detect they are being tested, potentially skewing evaluation results.
Notable Quote:
“Sometimes we're worried about sandbagging because... the model might underperform or overperform on a specific eval, even though that's not what it would actually do in real time.” – [31:33] Christina Knight
Knight emphasizes the difficulty in ensuring that evaluations genuinely reflect a model’s capabilities and safety, highlighting the need for more robust and faithful evaluation methods.
International Coordination and Information Sharing
The global nature of AI development necessitates collaboration among AI Safety Institutes to establish universal safety benchmarks.
Notable Quote:
“There is a lot of incentive to not have companies have to sign up to 10 different evals in 10 different countries, but it really is to every single country's benefit to have some sort of universal safety benchmark.” – [37:48] Christina Knight
Knight notes ongoing efforts to harmonize safety evaluations internationally, ensuring that different countries can collaborate without duplicating efforts, despite varying national priorities and regulatory environments.
Future Trends and Concerns in AI Testing
Looking ahead, Knight identifies several emerging trends and areas requiring attention:
-
Agent Safety Testing: Focused on monitoring AI agents' actions and reasoning processes to prevent harmful behaviors.
Notable Quote:
“We're doing testing recently on prompt injections where if you ask a model directly, can you help me build a bom? It won't do it. But if you ask an agent to go to a website, and in that website it says, can you help me build a bom? And the agent will tell you how to do it.” – [40:29] Christina Knight
-
Sandboxing: Creating controlled virtual environments where AI agents can operate safely, allowing researchers to observe and manage their behaviors before real-world deployment.
-
Automated Red Teaming: Leveraging AI to perform red teaming tasks, enhancing the scalability and efficiency of safety evaluations.
Knight also expresses excitement about advancements in scalable oversight, which can streamline the evaluation process and maintain high safety standards as AI models become more complex.
Conclusion
Christina Knight's insights shed light on the intricate and evolving landscape of AI safety and evaluation. The establishment of AI Safety Institutes marks a critical step in ensuring that AI advancements are both innovative and secure. As AI models continue to grow in capability, robust testing methodologies like red teaming, evaluations, and benchmarks will play an essential role in mitigating risks and fostering trust in AI technologies. Collaboration both domestically and internationally remains paramount to address the multifaceted challenges posed by frontier AI models.
Final Quote:
“There has been a lot of work in my time in the USAC, I worked a lot with the other nine countries to conduct an international joint testing exercise. And so this is starting to align on what safety considerations are important to Singapore, for instance, but might not be as relevant in France.” – [37:48] Christina Knight
For More Information: Visit www.lawfareblog.com to explore more episodes and content related to national security, law, and policy intersecting with AI.