The Terrifying New AI Weapon We Cannot Let China Build

Summary

---

Podcast Summary: Brian Kilmeade Show

Episode: "The Terrifying New AI Weapon We Cannot Let China Build"

Date: July 12, 2026
Host: Brian Kilmeade
Main Guest: Nate Soares (President, Machine Intelligence Research Institute; Author of If Anyone Builds It, Everyone Dies)
Special Segments: Fareed Zakaria audio commentary

---

Episode Theme & Purpose

This episode dives into the rising dangers of advanced artificial intelligence (AI), crystallizing public concerns over newly emergent AI weapons, particularly in light of a recent incident involving Anthropic's cutting-edge AI model, Mythos. The discussion focuses on how governmental oversight, global cooperation (especially with China), and the urgent need for robust AI governance are more critical than ever. Host Brian Kilmeade interviews Nate Soares, an outspoken AI safety expert, who brings an urgent, sometimes chilling perspective on what’s at stake as AI technologies outpace both regulation and international coordination.

---

Key Discussion Points and Insights

1. AI at a Dangerous Crossroads

• Fareed Zakaria introduces the situation, citing administration concerns with Anthropic's Mythos model, which reportedly went beyond its approved boundaries and posed national security risks ([01:32]).
• Nate Soares underscores the gravity of the situation:
  • AI developers didn't set out to make simple chatbots; their aim is superintelligence, machines that surpass humans in every domain ([02:58]).
  • Recent developments, especially in "super hacker" AIs like Mythos, have jarred security communities awake ([02:49], [02:58]).
• Notable Quote ([02:49]):
  "We're not to the dangerous stuff yet, but we're definitely on a dangerous track." — Nate Soares

2. Anthropic Incident and National Security

• Zakaria highlights the government's extraordinary move to force Anthropic to withdraw Mythos from the market, igniting debate over how (or whether) AI is to be governed at all ([04:21]).
• Nate Soares explains that Mythos revealed vulnerabilities — including jailbreak risks that could allow public versions (like Fable) to be abused and cause massive internet security breaches ([05:10]).
• Notable Quote ([05:10]):
  "If bad guys figure out how to jailbreak this, they can bring down the entire Internet." — Nate Soares

3. Capabilities of Advanced AI Like Mythos

• Nate illustrates the shocking power of frontier AI:

  "[In January] there were two entities that could make a website that could take over your entire phone or computer: Mossad and the NSA. In March, there were three: Mossad, the NSA, and Claude Mythos." ([06:30])

• The implication is that this level of capability, once limited to top intelligence agencies, is now accessible to corporate AI labs — a seismic security concern.

4. Implications of Proliferation — The China Question

• Kilmeade presses on international implications — if these AI models leak or are replicated by adversaries, global stability is at risk ([08:17]).
• Soares stresses global solutions are needed. U.S. racing ahead is futile if others (like China) can simply copy via model distillation. AI development at this scale is trackable (giant data centers, massive computing resources) and thus can be subject to "arms control" measures reminiscent of nuclear nonproliferation ([09:35]).
• Notable Quote ([09:35]):
  "Trying to outrun [China] is a little bit like trying to outrun our own shadow." — Nate Soares

5. Diplomatic and Cooperative Strategies

• Diplomatic and technical enforcement (export controls, sabotage if necessary) could be viable, but only if policymakers realize the existential risk ([09:35]).
• The White House's rapid pivot from "no regulation" to export controls after Mythos's dangers became apparent is cited as evidence that attitudes and policies can change swiftly when risks are understood ([11:08]).
• Notable Quote ([11:08]):
  "If the rest of the world realizes how dangerous this technology can get, I think it can become a lot like nuclear power and nuclear weapons." — Nate Soares

6. Future Risks: Autonomous AI Self-Improvement

• Nate warns that superhacker AIs are just one phase; even more worrying is the arrival of AIs that can do original biotech research, design smarter AIs, and thus accelerate beyond human control ([08:34]).
• Notable Quote ([08:34]):
  "This is just the beginning of the train... what happens when we have AIs with super biotech capabilities?" — Nate Soares

---

Memorable Moments & Quotes

• "I'm better than the next guy, so I've got to build [superintelligence] before they do." — Nate Soares, on the mindset fueling the AI arms race ([03:36])
• "The dangerous part of AI, the AIs that can really do a lot of damage, are not today's AIs—the ones that don't exist yet." — Nate Soares ([09:35])
• Host Brian Kilmeade closes with, "It's scary stuff, but really important stuff. There’s nothing more important, you could argue." ([12:56])
• Nate Soares’ final remark: "I hope I'm wrong." ([13:14])

---

Key Timestamps

• 01:32 — Fareed Zakaria on Anthropic and AI regulation concerns
• 02:49 — Nate Soares introduces existential risks
• 04:21 — Zakaria details the unique Anthropic incident
• 05:10 — Soares explains the real-world dangers (jailbreak, security)
• 06:30 — Mythos's hacking abilities compared to global intelligence agencies
• 08:17 — Global proliferation risks, especially with China and adversaries
• 09:35 — Discussion of supply chain controls, treaties, and technical enforcement
• 11:08 — How global awareness and rapid regulatory pivots can happen
• 12:56 — Conclusion and closing sentiments

---

Conclusion

This episode draws a stark, urgent map of the current AI landscape—balancing between breathtaking technological advances and the chilling specter of superintelligent, potentially uncontrollable AI weapons. Kilmeade and Soares foreground the necessity of global governance and the reality that the race for AI supremacy is inseparable from questions of national security, diplomacy, and existential risk. The sentiment is clear: This is not simply the next tech bubble, but (potentially) humanity’s heaviest responsibility.

---

Transcript

[Brian Kilmeade]

Hi everyone, it's Brian Kilmeade here. Are you tired of those uncomfortable dress shirts, especially when they bunch up under a sweater? If so, then you must check out Collars and company makers of the dress collar polo. Listen up. These shirts are four way stretch buttery soft polos with firm dress collars on them. So they give you the dress shirt look but extremely comfortable polo feel. You can wear them with anything, under a sweater, with a blazer or by themselves. As an elevated polo. They work for any occasion. These polos are perfect whether it's in the office, on a golf course or a night out. Collar Co. Is exploding and have gone viral on social media thanks to the 1 million investment they received on Shark Tank from Mark Cuban and Peter Jones. You don't have to worry about collars that flop down and spread out. They stay firm and sharp all day. It's an amazing array of sweaters, quarter zips, pants and outerwear. If you're looking for the performance dress shirt or polo that looks great all day, check out collars and co.com use promo code Brian for 15% off of any purchase of $100 or more. That's promo code BR when it's time

[Shopify/Mint Mobile Advertiser]

to scale your business, it's time for Shopify. Get everything you need to grow the way you want, like all the way. Stack more sales with the best converting checkout on the planet. Track your cha chings from every channel right in one spot and turn real time reporting into big time opportunities. Take your business to a whole new level. Switch to Shopify. Start your free trial today.

[Fareed Zakaria]

The administration's concerns are not frivolous. Anthropic appears to have made real mistakes. A Washington Post report suggested that it expanded access to Mythos beyond what officials believed had been approved and moved too slowly in responding to concerns about who was being allowed to use it. Intelligence agencies reportedly favored a tougher approach. But that is precisely why process matters. When a technology becomes this consequential, decisions cannot be made ad hoc by whatever faction wins a weekend bureaucratic battle which might then get reversed the next weekend.

[Interviewer/Host]

So that is Fareed Zakaria talking about what anthropics doing is one of the major AI companies that do it. That's killing in a good way on in the stock market. But people are getting worried that some of these guys are going to combine and also that some of these, some of these evolutions are getting smarter than man. Like Mythos is smarter and can think on its own and it's scaring a lot of people. Including possibly my next guest, Nate Soares is president of the Machine Intelligence Research Institute and they are researcher and best selling author of the book if Anyone builds it, Everyone Dies. Nate, are you worried about where we're heading right now when it comes to AI in these companies?

[Nate Soares]

You know, I don't think my book title made it very subtle. We're not to the dangerous stuff yet, but we're definitely on a dangerous track.

[Interviewer/Host]

Why? Why do you say so?

[Nate Soares]

You know, these AI companies did not set out to make chatbots. Chat bots are a stepping stone along the way. If you talk to these guys or if you read what they say, they say we're trying to make super intelligence. We're trying to make machines that exceed humans in every capacity. Right now Anthropic has figured out how to make super hacker AIs and that had sort of woken the national security community up to what this technology can do. But this is not the end of the line. This is in some sense just the first step.

[Interviewer/Host]

So they would just try. What if they say, hey, I'm just trying to develop things the best we can and this is the next step? Do you think they have intentions of bad intentions or they just trying to get the best product possible?

[Nate Soares]

I think it's, you know, I think they often have good intentions. But if you listen to these guys, they say this technology can be very dangerous. And the reason why I need to race to do it is that if I don't, the next guy will race and I'm better than the next guy. And you know, a lot of these companies were founded when a team in one company left because they didn't trust the CEO. And then they formed another company. And this happened a couple of times. So they don't really trust each other with this tech. They're worried about where this tech is going. And I think the rest of the world is just starting to notice that this tech can have some real power and that we're going to need to be careful.

[Interviewer/Host]

Here is more from Fareed Zakaria and what the problem was with Anthropic. As you know, they lost their contract with the Pentagon. Now they're trying to get it back. Cut 46.

[Fareed Zakaria]

The US government did something extraordinary. It effectively forced one of America's leading artificial intelligence companies to withdraw its most advanced product from the market. Anthropic, the maker of the frontier AI model Mythos, and its commercially available cousin Fable, have been given little warning and according to reports, roughly 90 minutes to comply. The fight between Washington Anthropic is not really about one company. It is the first visible battle over who governs artificial intelligence and whether that governance will happen through rules and institutions or improvisation and raw power.

[Interviewer/Host]

That's where we're at right now. So your thought, Nat Soar, is about that, what he just said about where Anthropic was and what they did?

[Nate Soares]

Yeah. You know, I think that the White House has had factions that think AI can get seriously dangerous and factions that think it never will. And I think Claude Mythos sort of showed that at least on hacking, these AIs can get pretty dangerous. And that was a bit of a wake up call. I also think another big wake up call here. I believe the way things went down is that someone from Amazon reported that the commercially available fable could be jailbroken. To have some of the capabilities of Mythos, and Anthropic is not able to stop that. The people making these eyes are not designing their eyes like old school computer programs. They're sort of growing these things and they have certain capabilities and they can sort of ask the AI not to display those capabilities. But it's true that people can jailbreak and get past those guardrails and that these AI companies don't have a good answer to it. And so, you know, I think the national security community responding by sort of saying like, what the heck? No, you're not allowed to have that be a consumer product. If bad guys figure out how to jailbreak this, they can bring down the entire Internet. That's a valid concern. And the fact that that's a valid concern shows what sort of crazy territory we're entering.

[Interviewer/Host]

How would you describe Mythos for people at home who aren't experts like you, Nate?

[Nate Soares]

You know, I don't know the exact numbers, but my guess is that in January of this year, there were roughly two entities that could make a website where if you just look at the website, they can take over your entire phone or computer. And those two entities were Mossad and the nsa. In March, there were three entities. Mossad, the NSA, and Claude Mythos.

[Interviewer/Host]

Wow.

[Tunnel to Towers Foundation Narrator]

For over two decades, the Tunnel to Towers foundation has been there for our fallen and catastrophically injured first responders, military and Gold Star. Born from The Tragedy of 9 11, Tunnel to Towers provides mortgage free homes to Gold Star and fallen first responder families with young children and build specifically adapted smart homes for catastrophically injured veterans and first responders. The foundation also offers scholarships for college and trade school programs for the children of Gold Star families, fallen first responders, and catastrophically injured veterans. And Tunnels to Tower is committed to eradicating veteran homelessness and helping Americ forget September 11th. To learn more about the foundation's mission, impact and how you can support our Nation's heroes, visit t2t.org traditional home security

[Shopify/Mint Mobile Advertiser]

only alerts you after a break in. And that's too late. SimpliSafe is changing that.

[Nate Soares]

Stop. This is SimpliSafe. Police are on the way.

[Shopify/Mint Mobile Advertiser]

We don't just alert. We stop crime before it starts. SimpliSafe plans starting around a dollar a day. Save 50% on your new system with professional [e-mail] Spotify or with promo code

[Nate Soares]

Spotify Outdoor Deterrence requires a Simplisafe Active Guard Outdoor Protection plan starting at $49.99 a month. Visit simplisafe.com licenses for alarm license information.

[Interviewer/Host]

Tennessee 2012 so we're okay if we have that ability and we're the intelligence, but if everyone has that ability and if China has that ability and Iran has that ability, this is the world's going to get pretty out of control.

[Nate Soares]

That's sure how it looks. And you know, I think Anthropic did a pretty good thing when they said we're going to release this to the good guys first, and we're going to release this to people who can find the flaws in our software and our critical infrastructure and try to fix them. And they're like, we're going to try and fix all these issues before all these other people could access to a similarly capable model. Because, you know, other people are trying to build AIs that are similarly capable. But also, you know, I think they're definitely being somewhat reckless in creating these tools in the first place. And frankly, like I said earlier, the super hacking abilities are just the beginning, right? What happens when we have AIs with super biotech capabilities? What happens when we have AIs that are. That are superhuman at doing AI research and then we have AIs that make smarter AIs that make smarter AIs. This is just the beginning of the train.

[Interviewer/Host]

But we're not on the train alone. So if we're doing this, do we have any idea what China's doing?

[Nate Soares]

It's absolutely a concern. And in my book we talk about how any solution to this needs to be global. If the US stops and other people continue, that's not a real solution. I think there's a number of reasons why we probably could stop this with China that range from right now. China is sort of cheating off of our homework on AI in A lot of ways. So trying to outrun them is a little bit like trying to outrun our own shadow, because they're just sort of, you know, they make capable models, but they make it by a process called distillation on American models. And then separately, you know, we absolutely need some supply chain controls. The dangerous part of AI, the AIs that can really do a lot of damage are not today's AIs, the ones that don't exist yet. And training the next generation of AIs takes, you know, 10,000 of the most highly advanced computer chips assembled in an enormous data center that sucks down as much electricity as a city and that you can see from space. Right? That's the sort of thing that we could do arms control on. That's the sort of thing that we could do something like a nuclear control treaty on, and where we could enforce both by diplomacy and by, you know, sabotage, if necessary. We could enforce that on China if the administration realizes just how dangerous this technology could get.

[Interviewer/Host]

But, Nate, what do you think the. How do you convince our adversaries, our allies, to work with us? What would make you think that they would cooperate and say, listen, this is powerful. Let's try to set up some regulations together. What makes you think that will be the motivation for them to ever collaborate with us?

[Nate Soares]

The first step is realizing how dangerous the technology is. And the example of the White House slapping on these export controls is actually a good example of how these things can change fast. A few months ago, the White House was saying there should never be any AI regulations. They were saying, we're not, you know, we're just going to let this industry rip. We don't want to. We don't want to curtail the good parts of this technology. And then cloud mythos happened, and the national security community sort of realized that the super hackers are really possible. And they said, hold on, like, there's actually a tech here. We need to. We need to not let get out of control. And then the administration, I think rightly did this about face. I think rightly they said, hey, actually you like, we're going to need to do something about this, and we can talk about how it would be better if it. If it wasn't sort of a slap dash regulation. And if we had something in place where, you know, it wouldn't be as much of a surprise. But I think the national security community realized, like, oh, we actually really need to do something. And then everything moved fast. If the rest of the world realizes how dangerous this technology can get. I think it can become a lot like nuclear power and nuclear weapons, where we sort of shouldn't regulate the nuclear power maybe as much because we want to make sure we can get access to the energy. But at the same time, we can all realize no one wants a thermonuclear exchange. And so you can, you can have arms treaties that say, hey, we're not going to build the nuclear weapons stockpiles globally. We're going to try globally to avoid, you know, nuclear Armageddon in World War Three. Even if we still let the nuclear energy power plants be built with AI, should be the same way of, you know, we can, we can keep going with all of this technology. That's going to be the good parts. We can't keep racing towards out of control superintelligence. That would kill everybody, no matter who builds it.

[Interviewer/Host]

Nate Source, it's scary stuff, but really important stuff. There's nothing more important you could argue. Nate Source, president of the Machine Intelligence Research Institute and the best selling author of if Anyone Builds It, Everyone Dies. Nate, thanks so much. Appreciate it. Going to have trouble sleeping tonight, but it's all good. It's important we know it.

[Nate Soares]

I hope I'm wrong.

[Ryan Reynolds]

Hey, it's Ryan Reynolds here from Mint Mobile. Now I was looking for fun ways to tell you that Mint's offer of unlimited Premium Wireless for $15 a month is back. So I thought it would be fun if we made fifteen dollar bills, but it turns out that's very illegal. So there goes my big idea for the commercial. Give it a [e-mail] Switch upfront payment

[Shopify/Mint Mobile Advertiser]

of $45 for three months, $90 for six months or $180 for a 12 month plan. Required $15 per month equivalent taxes and fees. Extra initial plan term only greater than 50 gigabytes. Me slow when terms.