Statecraft Podcast Summary: "How to Predict the Future" Featuring Jason Matheny

Published on June 25, 2025

Introduction

In this episode of Statecraft, host Santi Ruiz engages in a deep conversation with Jason Matheny, a distinguished figure in the Washington D.C. policy sphere. Matheny's illustrious career spans academia at Oxford and Princeton, leadership at the Intelligence Advanced Research Projects Activity (IARPA) from 2015 to 2018, a stint in the Biden White House, and currently serving as the head of the RAND Corporation. The discussion delves into the intricacies of the ARPA model, the importance of forecasting in policy-making, and the challenges of institutional design within government agencies.

1. The ARPA Model: Features and Uniqueness

Matheny outlines the foundational elements that distinguish the ARPA (Advanced Research Projects Agency) model from conventional research funding frameworks:

• Entrepreneurial Program Managers: These individuals are granted significant autonomy and budgets to pursue high-risk, high-reward research initiatives.

• Research Tournaments: Researchers compete in parallel, adhering to a common set of metrics that emphasize transformative potential.

• Defunding Unsuccessful Projects: Annually, projects that fail to meet predefined success criteria are defunded, ensuring resources are allocated to promising ventures.

Notable Quote:

"The three core elements are the program managers, the research tournament, and the defunding of research that isn't succeeding." [02:22]

2. Program Manager Qualities and Recruitment

Effective program managers are pivotal to the ARPA model's success. Matheny emphasizes the recruitment process, which seeks individuals with technical expertise, entrepreneurial spirit, creativity, and a high tolerance for risk. These managers undergo rigorous selection, including program pitches based on the Heilmeier questions, and must obtain security clearances—a requirement consistent across IARPA and DARPA.

Notable Quote:

"You structure your pitch in answers to the hallmark questions, but the goal is ultimately to persuade this group that you have an idea that could be transformative if successful." [04:36]

3. Balancing Risk in Research Projects

A critical aspect of ARPA's mission is identifying the right balance of risk:

• Too Risky: Projects that require violating fundamental laws of physics or solving unprecedented problems are deemed excessively risky.

• Not Risky Enough: Conversely, endeavors perceived as easily solvable or too incremental do not align with ARPA's high-risk, high-reward ethos.

Matheny introduces the concept of a "Goldilocks zone" for project risk, suggesting an optimal probability range for success that ensures transformative impact without untenable risk.

Notable Quote:

"There's a Goldilocks zone of appropriate probabilities—probably under 50% and over 5%." [06:06]

4. Enhancing the Heilmeier Questions and Strategic Analysis

While the Heilmeier questions provide a robust framework for evaluating research proposals, Matheny advocates for augmenting them with considerations of strategic impact and counterfactual scenarios. Specifically, he highlights the necessity of:

• Counterfactual Impact: Assessing whether the research would proceed without ARPA's involvement and its broader implications.

• Competitive Responses: Evaluating how adversaries might respond to the development and deployment of the technology.

Matheny suggests incorporating red teaming exercises to anticipate and mitigate potential vulnerabilities and misuse of technologies.

Notable Quote:

"We tend not to do a real rigorous assessment of... alternative efforts solving the problem." [11:43]

5. IARPA vs. DARPA: Differences and Intelligence Community Relationship

IARPA operates with a focus on the intelligence community—comprising 18 agencies tasked with information collection, analysis, and protection. Unlike DARPA, which caters to the Defense Department's broad needs, IARPA's projects are often more specialized and secretive, necessitating higher security clearances and fostering collaborations with smaller, security-focused contractors.

Notable Quote:

"The primary end user of IARPA's research is the intelligence community..." [26:40]

6. Focus on Human Judgment and Forecasting Accuracy

Matheny’s passion for improving human judgment stems from his engagement with Phil Tetlock's work on the limitations of expert predictions. At IARPA, initiatives like the ACE program focused on enhancing forecasting accuracy through crowdsourcing and identifying "superforecasters." He underscores the immense societal value of refining decision-making processes, particularly in high-stakes environments like national security.

Notable Quote:

"Improving the accuracy of those judgments could be worth billions of dollars per percentage point improvement." [05:58]

7. Institutional Challenges and Sustaining Innovative Organizations

Sustaining agencies like IARPA requires strategic hiring, protected autonomy, and congressional support. Matheny notes that the success of ARPA-like institutions largely hinges on:

• Hiring Exceptional Talent: Program managers who are creative and contrarian thinkers.

• Protecting Autonomy: Allowing for unconventional operations without bureaucratic interference.

• Securing Congressional Backing: Demonstrating impactful achievements to maintain funding and support.

Notable Quote:

"Almost every enterprise like 90% of the variance is determined by the people who are hired." [46:49]

8. Public vs. Secret Projects in IARPA

IARPA employs a rigorous cost-benefit analysis to decide which projects should remain classified. Factors influencing this decision include the availability of researchers, the necessity of secrecy for national security, and the feasibility of maintaining confidentiality without hindering project success.

Notable Quote:

"If you develop a technology that would actually create an asymmetric disadvantage if used against you... you should be especially thoughtful before developing cyber weapons." [54:44]

9. Asymmetric Technological Advantages for Democracies

Matheny explores how certain technologies can confer asymmetrical benefits to democratic societies compared to autocracies. Examples include:

• Encryption and Privacy Technologies: Strengthening democratic resilience against authoritarian surveillance.

• Large Language Models: While beneficial, they also pose challenges like disinformation, necessitating built-in safeguards.

• Immigration and Innovation: Democracies leverage diverse talent to spur technological advancements.

He emphasizes the need for proactive red teaming to foresee and mitigate potential misuses of emerging technologies.

Notable Quote:

"We really could have done a better job... in DNA synthesis by putting some form of security or safety into this technology from the start." [76:37]

10. Influence on Policy and Think Tank Best Practices

In discussing his experience founding CSET, Matheny highlights the importance of:

• Building Interpersonal Trust: Success in policy relies heavily on trusted relationships rather than measurable achievements.

• Hiring for Kindness and Objectivity: Recruiting individuals who are not only skilled but also compassionate and unbiased.

• Adapting Communication Styles: Tailoring outputs—ranging from one-pagers to interactive dashboards—to meet policymakers' diverse needs.

Notable Quote:

"The policy world is much more like West Wing or Madam Secretary than it is like House of Cards." [68:27]

11. Forecasting in National Security Decision Making

Matheny advocates for embedding robust forecasting mechanisms within key national security bodies like the National Security Council (NSC). He envisions tools that offer real-time, crowdsourced predictions to inform high-stakes decisions and preempt catastrophic errors.

Notable Quote:

"What you want is really some kind of EKG for the analytic community about a particular problem at any given moment." [80:56]

Conclusion

The episode underscores the critical role of innovative institutional models like ARPA in advancing national security and policy-making. Jason Matheny's insights reveal the delicate balance between fostering creative, high-risk research and ensuring strategic, secure implementation. Emphasizing the enhancement of human judgment and the importance of proactive strategic analysis, Matheny advocates for a future where forecasting and collaborative intelligence play pivotal roles in shaping effective policies.

To explore more interview transcripts, subscribe at www.statecraft.pub.