9/25/25: Iran Warns Of Israeli Attack, Data Centers Spike Electricity Cost, AI Takeover Dire Warning

Summary

Breaking Points with Krystal and Saagar

Episode: September 25, 2025
Title: Iran Warns Of Israeli Attack, Data Centers Spike Electricity Cost, AI Takeover Dire Warning
Hosts: Ryan Grim, Sagar Enjeti
Guest: Nate Soares (Machine Intelligence Research Institute)
Podcast: iHeartPodcasts

---

Episode Overview

This episode dives deep into three major stories:

1. Iran’s escalating tensions with Israel and U.S. foreign policy.
2. The impact of AI-driven data centers on rising electricity costs and local economies.
3. A dire warning about artificial superintelligence from AI expert Nate Soares.

The hosts dissect urgent global politics, corporate overreach, and technological existential risks, offering both ground-level reporting and high-level analysis.

---

1. Iran Warns of Imminent Israeli Attack

(Main Segment: 02:36–17:49)

Key Discussion Points

• Inside Iran’s UN Visit

  • Ryan Grim recounts his experiences attending a meeting with the Iranian president, who was in New York for UN Meetings.
  • The Iranian president denounces Israel for “genocide, mass starvation, and apartheid” in the occupied territories and criticizes the global community’s inaction.

    “The ludicrous and delusional scheme of a Greater Israel is being proclaimed with brazenness by the highest echelons of that regime.”
    (Iranian President, 02:53)

• Iran’s New Leadership and Challenges

  • Sagar gives context on Iran’s president: Massoud Possesskian, elected for his moderate reformism and openness to nuclear negotiations.
  • The assassination of moderate negotiators like Haniyeh by Israel has fueled hardliner arguments against diplomacy.

• Fear and Resignation in Iran’s Strategy

  • Possesskian’s emotional response to witnessing violence against children in Gaza; sense of being overwhelmed and targeted.
  • Iran’s new “chain of succession” protocol:

    “If they take me out, there are five to six people to take my place.”
    (Sagar Enjeti paraphrasing Iranian President, 05:27)

• Israel’s Approach to Diplomacy

  • Discussion of how Israel often targets moderates, while hardliners are typically left untouched because they fit the narrative of endless conflict.

    “Historically…the people arguing for more moderation within those organizations have been more likely to be assassinated by Israel than the hardliners, because the hardliners serve a narrative purpose.”
    (Sagar Enjeti, 07:04)

• Iran’s Disconnection from the U.S. System

  • Iranians’ inability to hire lobbyists due to sanctions; they’re left navigating U.S. politics via media coverage.
  • Most Iranian officials have scant understanding of American political processes.

    “They basically can't hire any lobbyists…They're left to just read the New York Times and watch Fox News or Tucker Carlson to try to figure out how they can maneuver.”
    (Sagar Enjeti, 08:24)

• Prediction of Imminent Attack

  • The Iranian leadership expects another Israeli attack soon, fueled by rhetoric from Netanyahu.

    “They are basically certain of it and it’s a matter of when.”
    (Sagar Enjeti, 09:51)

• Analysis of the '12 Day War' and Fallout

  • After U.S. airstrikes on Iranian nuclear facilities, Iran’s retaliation was deliberately limited.
  • The effectiveness of Iran’s missile capabilities was unexpected by Israel and may have influenced the war’s resolution.

    “Their ability to get missiles through the Israeli air shield is what actually pushed them to end it.”
    (Sagar Enjeti, 11:55)

• Intelligence Showdowns

  • Iran releasing alleged Israeli nuclear program intelligence as a show of force and capability.

Notable Quotes & Memorable Moments

• “Iran is not Gaza. Iran is not Lebanon. Iran is not Syria. Iran is different…this plan you have is like, we're not going anywhere. You have to live with us.”
  (Sagar Enjeti, speaking from Iranian perspective, 13:28)
• “How are we the terrorists?” (Sagar Enjeti, reflecting Iranian complaint, 14:38)

---

2. Data Centers and Exploding Electricity Costs

(Main Segment: 19:54–31:37)

Key Discussion Points

• North Carolina’s Data Center Legislation

  • State bill S266 prioritizes data centers over consumers for power allocation and shifts more cost onto residential customers.

    “If there is a contest between consumers and data centers over who gets the power…the data centers are going to get it…your electricity bills are going to go up to, you know, subsidize these giant data centers.”
    (Sagar Enjeti & Ryan Grim, 23:06–24:36)

  • The bill was pushed by the former CEO of Duke Energy and overrided by bipartisan support despite the governor’s veto.

• The Impact on Renewable Energy Jobs

  • Clean energy jobs (tens of thousands) in North Carolina, like at Blue Ridge Power, are being eliminated as the state pivots back to favoring big utilities and data centers.

    “This layoff of 517 workers…they’re just rolling up shop…because of this new landscape.”
    (Sagar Enjeti, 20:19)

• Data Centers Strain on Local Resources

  • Massive water use as AI data centers rise; rural communities witness land and resource takeovers for minimal local benefit.

    “AI data centers would need millions of gallons of North Carolina’s water supply a day.”
    (Sagar Enjeti, 26:12)

• Critique of AI-Driven Economic Justification

  • The supposed tech advancement boils down to “your electric bill is going up 30% so you can chat with this horse.”
    (Sagar Enjeti on Meta’s chatbot, 25:56)

• China’s Renewed Commitment to Green Energy

  • Xi Jinping’s UN announcement: “China will not slow down its climate actions…will not cease its efforts to build a community with a shared future for mankind.”
    (Ryan Grim paraphrasing, 29:17)

• US Political Dynamics and Renewable Energy

  • Trump’s attacks on green energy, job losses in renewables, and the shifting political narrative around energy policy.

Notable Quotes

• “It used to be the lefty green energy people who were supposedly the job-killing haters. And now…we’re in a totally different dynamic…jobs and cheap energy is in the direction of investing in renewable and green energy.”
  (Ryan Grim, 30:45)
• “We’re maxed out on the dirty stuff. If you want to expand energy production, you have to do it through these new types of energies and Trump doesn't want to do it.”
  (Sagar Enjeti, 31:17)

---

3. AI Takeover: Existential Risk & Dire Warning

(Main Segment: 33:40–59:10)

Key Discussion Points

• Guest: Nate Soares on 'Superintelligence'

  • Title of his new book: If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All.

    “If they develop superintelligence…humanity will be destroyed.”
    (Ryan Grim summarizing, 34:19)

• Why Superintelligence Is Unique and Deadly

  • AIs aren’t “programmed” so much as “grown”—we don’t know what’s going on inside them.

    “We’ve already seen signs that they are developing the very beginnings of preferences and drives perhaps that nobody wants, nobody asked for.”
    (Nate Soares, 34:38)

  • There are no second chances—if superintelligent AI gets it wrong, the consequences are irreversible.

    “If we build machines that are smarter than us…and if they run off autonomously and grab control of the world’s resources, there’s no redos.”
    (Nate Soares, 35:20)

• Tech Industry’s Dismissive Attitude

  • Even leading industry figures admit significant risk; optimism is often based on wishful thinking or unproven coping strategies.

    “Even the people who are most optimistic say there’s a 1 in 4 chance this kills everybody…”
    (Nate Soares, 35:52)

• Opaque & Uncontrollable AI Systems

  • Modern deep learning systems are unpredictable and not understood by their creators.

    “We’re already seeing signs…that the people who would like to be in charge can’t point these AIs where they would like to point them.”
    (Nate Soares, 39:40)

• Dangerous Industry Practices

  • Corporations are in a race to scale up AI with minimal oversight—putting AIs online for anyone to use rather than keeping them restricted.

    “Now people are putting AIs on the Internet the moment they possibly can.”
    (Nate Soares, 45:18)

• Common Objections and Pet Theories

  • “Maybe we’ll be AI’s pets”—Soares argues even that is unacceptable risk.
  • The primary way humans might die is not direct intent, but as a side effect of AIs automating and replacing our infrastructure.

    “We die not because it literally tries to kill us, but because it just starts building automated factories…maybe we just die underfoot.”
    (Nate Soares, 47:10)

• Why AI Alignment Efforts Are Inadequate

  • The hope that we’ll “use AI to solve the alignment problem before we build superintelligence” is illogical; the smarter the AI, the less control we have and the harder alignment gets.

• Industry Acknowledges the Risks—But Keeps Racing

  • Industry leaders, including Elon Musk and the head of Anthropic, openly admit to a 10–25% chance of total catastrophe, but continue development in a classic “If I don’t, someone else will” mindset.

    “If one engineer was saying, I looked at this plane…I think this plane is 20% likely to crash…You don’t get on the plane.”
    (Ryan Grim, 50:39)

• Necessary Solutions

  • Soares calls for a global moratorium on racing to superintelligence, not a minor reform or alignment check.

    “We need to give up on the race towards smarter than human AI…the race towards superintelligence. This is what the labs say they are racing towards…this is a suicide race.”
    (Nate Soares, 54:24)

  • He believes this is still possible with international coordination or mutual national deterrence.
  • Timeline? Hard to predict, but “a child born today has a greater chance of dying from AI than of graduating high school.” (58:53)

Notable Quotes

• “Humanity usually learns from its mistakes…In this situation…the hopeful optimists who are saying everything’s going to be easy, they’re going to screw things up the first time and there’s not going to be any chance to learn.” (Nate Soares, 51:30)
• “My bet is that a child born today has a greater chance of dying from AI than of graduating high school.”
  (Nate Soares, 58:53)
• “Here’s hoping I’m wrong.”
  (Nate Soares, 59:09)

---

Useful Timestamps

• Iran Segment Begins: 02:36
• Iranian President Clip: 02:53
• Iran Analysis: 04:00–17:10
• Electricity/Data Center Crisis Intro: 19:54
• North Carolina Data Center Bill Context: 23:06–24:36
• AI Segment with Nate Soares: 33:40–59:10
• Dire Risk and Call to Halt Superintelligence: 54:24–59:10

---

Tone & Final Thoughts

The episode is piercing, urgent, and openly skeptical of establishment narratives—whether state policies, corporate interests, or technocratic optimism. It blends investigative insight, sharp humor, and existential dread, especially in the AI warning. Listeners come away with a layered view: geo-strategy is dangerously unmoored, American policy is often for sale to corporate interests, and runaway technology could genuinely threaten humanity’s survival.

---

Transcript

[Ryan Grim]

This is an iHeart podcast.

[WashablesOfSofas Ad Voice]

There's nothing like sinking into luxury. At washablesofas.com you'll find the Annabe sofa which combines ultimate comfort and design at an affordable price. And get this, it's the only sofa that's fully machine washable from top to bottom. Starting at only $699. The stain resistant performance fabric slipcovers and and cloud like frame duvet can go straight into your wash. Perfect for anyone with kids, pets or anyone who loves an easy to clean spotless sofa. With a modular design and changeable slipcovers, you can customize your sofa to fit any space and style. Whether you need a single chair, loveseat or a luxuriously large sectional, Annabe has you covered. Visit washablesofas.com to upgrade your home. Right now you can shop up to 60% off store wide with a 30 day money back guarantee. Shop [e-mail] Add a little to your life. Offers are subject to change and certain restrictions may apply.

[LG XBoom Ad Voice]

Stop settling for weak sound. It's time to level up your game and bring the boom. Hit the town with the ultra durable LG X Boom portable speaker and enjoy vibrant sound wherever you go. Elevate your listening experience to new heights because let's be real, your music deserves it. The future of sound is now with LG XBoom and for a limited time save 25% at LG.com with code fall25. Bring the boom XBoom.

[Cynthia Littleton]

There's a lot going on in Hollywood. How are you supposed to stay on top of it all? Variety has the solution. Take 20 minutes out of your day and listen to the new Daily Variety podcast for breaking entertainment news and expert perspectives.

[LG XBoom Ad Voice]

Where do you see the business actually heading?

[Cynthia Littleton]

Featuring the iconic journalists of Variety and hosted by co Editor in Chief Cynthia.

[WashablesOfSofas Ad Voice]

Littleton, the only constant in Hollywood is change.

[Cynthia Littleton]

Open your free I heart rate Search Daily Variety and listen now.

[Nate Soares]

Hey guys, Sager and Crystal here.

[Ryan Grim]

Independent media just played a truly massive role in this election and we are so excited about what that means for the future of the show.

[Nate Soares]

This is the only place where you can find honest perspectives from the left and the right that simply does not exist anywhere else.

[Ryan Grim]

So if that is something that's important to you, Please go to BreakingPoints.com, become a member today and you'll get access to our full shows unedited ad free and all put together for you every morning in your inbox.

[Nate Soares]

We need your help to build the future of independent news media and we Hope to see [e-mail].

[Ryan Grim]

So, Ryan was out yesterday, but for a very interesting reason. He actually got to attend a meeting of journalists and activists with the Iranian president, who, of course, was in town in New York City for the UN Meetings. Let's go ahead and take a listen to a little bit of what the Iranian president had to say in his speech there about Israel.

[Iranian President]

Ladies and gentlemen, today, after nearly two weeks of genocide, mass starvation, the perpetuation of apartheid within the occupied territories, and aggression against its neighbors, the ludicrous and delusional scheme of a Greater Israel is being proclaimed with brazenness by the highest echelons of that regime. The scheme encompasses vast swaths of the region. The map itself lays bare the true intentions of the Zionist regime, intentions that have of late been openly endorsed by its criminal prime minister. None in the world is secure from the aggressive machinations of this regime. It is manifest that the Zionist regime and its sponsors no longer even content themselves with normalization through political means. Rather, they impose their presence through naked force and have styled it peace through strength. Yet this is neither peace nor power. It is nothing but aggression rooted in coercion and bullying.

[Ryan Grim]

So, Ryan, that was some of the public messaging there from Iran. What did you hear in that meeting you were able to attend?

[Sagar Enjeti]

Yeah, and so for context, for people, this Massoud Possesskian was elected on a platform of basically moderate reformism and an argument that Iran needs to get out of its isolationist posture and get back into nuclear negotiations with. With Europe and the United States. You know, there are a lot of divisions within Iranian society over that question. And so his. His victory represented the extension of that kind of viewpoint. And then on the day of his inauguration is when Israel killed Haniyeh in Tehran, which was used by the hardliners to say, look, how many times do we have to tell you?

[Ryan Grim]

Like, you can't work with.

[Sagar Enjeti]

They don't. They don't like you. Like, all they want to do is undermine your. Under. Undermine you. And it was very interesting hearing from him, because in some ways, he seems like in over his head is not the. Is a little unfair. But he's. He seemed to be genuinely outraged and flummoxed at the situation that he's in. We just. He just kept saying, he's like, I can't watch television seeing what they're doing to these children. Like, it tears me up. It's like, they follow no international laws. They invite us to peace talks and kill our negotiators.

[Ryan Grim]

Yeah. And the. They in that Circumstance was the U.S.

[Sagar Enjeti]

The U.S. and Israel. And he said, it's like, and he, he, he also said, I asked him if they were going to do anything fundamentally different after the 12 Day War. And, and I don't know if it was awkward thing where 10 people would ask questions, then he'd answer them all kind of at once. One thing he said was that since the 12 Day War, they have now put in place a protocol for transfer of power so that there are now, he said, five or six people below him, so that he said, if they take me out, there are five to six people to take my place. Which they didn't have it, you know, that, that laid out before. And at the same time, you could kind of feel from him a sense of his own mortality, which is wild because it's like he's 70 years old, he's at the pinnacle of his career, wins the presidential election, is trying to negotiate a way towards peace. But also you could just feel that in the back of his mind he's understands that they might try to kill him.

[Ryan Grim]

Yeah.

[Sagar Enjeti]

And then you're sort of like, well, why? Like, how is killing this moderate guy who wants to do peace negotiations? Like, how is, how does that help at all? It's like, well, don't ask why. Like it's, they're probably going to do it anyway.

[Ryan Grim]

Well, and for Israel, I mean, Netanyahu has always opposed.

[Sagar Enjeti]

Right.

[Ryan Grim]

Diplomatic negotiations, whether he was opposed the original Obama nuclear deal. So, you know, for him, having a moderate reformer in there that's interested in diplomatic negotiations is not a plus.

[Sagar Enjeti]

Yes. Historically, whether it's inside the PLO or even in Hamas, the people arguing for more moderation within those organizations have been more likely to be assassinated by Israel than the hardliners, because the hardliners serve a narrative purpose. The narrative purpose of we have to do endless war here. At the very beginning of the meeting, he waxed very philosophical and poetic about, you know, how small we are in the scheme of things. You know, you look at the planet from, you know, from the galaxy, we're all just, we're all one planet. And we all have such a short.

[Ryan Grim]

Time on this Earth before AI destroys itself.

[Sagar Enjeti]

What are we? Yes, exactly. In his case, it probably won't be AI that gets him, although some version of it, and he's like, what are we fighting about? There's one country in the region that just refuses to recognize its boundaries, and there seemed to be a resignation about it, almost a fatalism. And the other kind of thing that you could pick up from the meeting is just how divorced from American society and culture the Iranians are. They genuinely just don't understand our political system. And he even said we don't have any, like, because of the sanctions, they basically can't hire any lobbyists or any consultants. And he said we don't have any lobbyists to like help us, like, figure out your system. Like, they're basically left to just read the New York Times and watch Fox News or Tucker Carlson to try to figure out how they can maneuver within our system. Whereas as he pointed out, Israel is the opposite. Yeah, they have, they have friends everywhere in Washington. But it really did show that, like, they're pretty clueless. There was some news about how they're only allowed to go like three blocks while they're in New York.

[Ryan Grim]

They were like banned from Costco or something.

[Sagar Enjeti]

They can't go to Costco, they can't go to Sam's Club, they can't go to, they can't do any shopping. And any of those. There's like this three block radius, this hotel where this meeting was held and then they can get to the UN and that's it. There's a real fear among the pro Israel faction that if an English speaking Iranian can get in front of Trump, that they can be persuasive because they have a persuasive case that aligns with what Trump wants, which is no war. Like, that's what they want too.

[Ryan Grim]

And is he English speaking?

[Sagar Enjeti]

Not really. Not for him, no. So he was, the whole interview was through an interpreter just like, yeah.

[Ryan Grim]

And so is their expectation strongly that they're going to be attacked again?

[Sagar Enjeti]

They are basically certain of it and like, it's a matter of when. And they think it, they think it's going to be very soon. And nobody really knows why. There's no real explanation, like, why are they going to be attacked again. But Netanyahu continues to say openly like that this is the year they have to do it. They have to just finish off the Iranian regime. But as he said, that's not going to happen. So then what? So as he, the point, another point that he made in the conversation was they expected that they would hit these top generals, hit these scientists, they'd cause chaos, hit these civilian areas and people would come out in the streets and the government would fall. And that didn't happen. And like, the government is in a firmer position, I think, than Israel understands. Like, what is it and what does it even mean for the government to fall? Like who? Like, there's really no Kind of like who's coming in. It's just not a thought out process. Yeah, they imagine you can kill the Ayatollah. He's like in his 80s.

[Ryan Grim]

Yeah. They imagine like the Shah's son is going to fail. Son is going to pick up the pieces.

[Sagar Enjeti]

Right. There's these guys. Yeah. Who think they're going to fly into Tehran, but on the back of what? Like, it doesn't. And so. Yeah. Can they continue to kill scientists and generals and bomb infrastructure? Yes, but then. But to what end?

[Ryan Grim]

So during the 12 day war, Iran's, after we joined the fight overtly and bombed their nuclear facilities, their response to us was pretty limited and seemed to be part of the strategy of, okay, we're gonna do something because we have to do something, but it's not gonna be anything that's gonna make you too mad. Is there any sense of, like, that was the right strategy, that was the wrong strategy, we'd handle it differently next time. Do you have any sense of that?

[Sagar Enjeti]

Yeah, I think as far as their response to the United States, I think they think that was the right one. They want a friendly relationship with the United States. So, yeah, what they did is they, they called Trump and said, we're going to hit your base in Qatar in like two hours. Like, do you need more time? Is two hours okay? He's like, no, two hours is good. So they cleared everybody out. It landed like in the sand of the base and they called that a day. Now I asked him what his analysis was of why the 12 Day War ended, and it was unclear if it was again, like a direct response to that question. But what he said in the response to the question was he said basically they, Israel did not expect that any missiles, let alone Iranian missiles, could penetrate their air defenses. And he said it would be untoward to like, get into the kind of details of what, of what the damage was that their missiles created. But he felt that it was that. That their ability to get missiles through the Israeli air shield is what, is what actually, you know, push them to end it.

[Ryan Grim]

There was some reporting about that at the time that the number of interceptors that Israel had in stockpile were rapidly dwindling and diminishing. And so there was a sense that there was some pressure on them as well to, you know, not that they were sustaining the level of damage that Iran was sustaining, but that they were running into some capacity issues where it wasn't like, oh, this will be it forever, but we could use a little bit of time to like, regroup. And replenish our stockpiles.

[Sagar Enjeti]

Right. Yeah. The Iron Dome and the other elements of it, it were, you know, significantly depleted because it, you know, it takes a year to almost like a year to make the amount that we used in 12 days there. Now Israel is saying it. It is now developing this laser technology which. Which would be a game changer if it worked, because the. The lasers cost, like, you know, a couple bucks each. You know, once you have. Once you have them up and running. So that would be, you know, if that actually worked. But on the other hand, do they actually work against, like, hypersonic missiles? I think, you know, that's something. That's something we'd have to find out and see if there's anything else that was interesting from that. Yeah, they. They could not fathom that any missile, let alone an Iranian missile, could penetrate their air defenses. Yeah. He said, if they take me out five to six steps down the line have been planned for. Iran is not Gaza. Iran is not Lebanon. Iran is not Syria. Iran is different. Making the argument that, like, this plan you have is like, we're not going anywhere. You have to live with us.

[Ryan Grim]

Yeah. Israel doesn't seem to be interested in just coexistence with really, any of their neighbors.

[Sagar Enjeti]

But he had another line where he said, it is not Iran that is a supporter of terrorism in the region. And he said that a lot because you could tell from his perspective, he's like, we're constantly showing restraint and they're constantly killing civilians. How are we the terrorists? That's. That's kind of their perspective.

[Ryan Grim]

Yeah. How many new wars have we started, for example? Let me put D2 up on the screen and just get your reaction to this news item. Iran airs footage it claims shows details of alleged Israeli. Alleged Israeli nuclear program. Iranian state TV broadcast images of documents and footage it claims relate to alleged Israeli nuclear activities. The documentary shows copies of passports had to identify Israeli scientists along with info on the location of military sites. Air footage said to have been filmed inside the Dimona reactor in southern Israel. And they said in this documentary, the intel minister of Iran said that they had used information obtained in June to hit sensitive sites inside Israel that month. And they claimed before the June war to acquire thousands of classified Israeli documents, including details on nuclear and military sites. So a kind of show of intel force, I guess, here saying, listen, you're not the only one with spies and access to this kind of inside intelligence.

[Sagar Enjeti]

Yeah, they had teased this, if you remember, right before the war, basically saying they had everything. And so now they're finally putting some of this out. But again, it would actually go back to if any other government had this kind of stuff, they'd leak it to the New York Times or they'd find more sophisticated ways to publish rather than.

[Ryan Grim]

In some weird Iranian state tv.

[Sagar Enjeti]

Iranian state TV documentary. That is okay, that's fine. But like, right, if, if they had SKDK advising them on, like, how do you want to influence the American public on this? Yeah, they'd be like, I know who you need to leak this stuff to. But yeah, they don't, they just don't have that. It's so. And that's another function of sanctions, which to me, the America first argument against these kinds of sanctions would be America should make the best foreign policy decisions that it can in its own interests. And in order to do that, you either want no foreign interference, no foreign influence in your decision making, or you want to hear from everybody. And we're not going to kick them all out. So to me, Russia should be able.

[Ryan Grim]

To level the lobbying, paying for it.

[Sagar Enjeti]

Iran, North Korea, like let them all lobby, let them all make their case, make it in the open. They gotta register with Fara. We wanna know who their agents are and what case they're making and then we make a decision. But this idea where you hear where Trump hears only from one side and zero from the other and then launches us into a war is not helpful for us.

[Ryan Grim]

Yeah. Bibi's making Another trip to D.C. by the way. Another White House. His fourth.

[Sagar Enjeti]

Yeah, yeah. Incredible.

[Ryan Grim]

There you go.

[Sagar Enjeti]

He really has not had to do any laundry in Israel this whole year.

[Ryan Grim]

Get it taken care of.

[WashablesOfSofas Ad Voice]

Tired of spills and stains on your sofa? WashablesOfAs.com has your back. Featuring the Annabe collection, the only designer sofa that's machine washable inside and out. Where designer quality meets budget friendly prices. That's right. Sofas start at just $699. Enjoy a no risk experience with pet friendly stain resistant and changeable slipcovers made with performance fabrics. Experience cloud like comfort with high resilience foam that's hypoallergenic and never needs fluffing. The sturdy steel frame ensures longevity and the modular pieces can be rearranged anytime. Check out washablesofas.com and get up to 60% off your Anna Bay sofa backed by a 30 day satisfaction guarantee. If you're not absolutely in love, send it back for a full refund. No return shipping or restocking fees. Every penny back. Upgrade [e-mail]. offers are subject to change and certain restrictions may apply.

[LG XBoom Ad Voice]

Stop settling for weak sound. It's time to level up your game and bring the boom. Hit the town with the ultra durable LG X Boom portable Speake and enjoy vibrant sound wherever you go. Elevate your listening experience to new heights because let's be real, your music deserves it. The future of sound is now with LG XBoom and for a limited time, save [e-mail] with code fall25. Bring the boom XBoom.

[Cynthia Littleton]

There's a lot going on in Hollywood. How are you supposed to stay on top of it all? Variety has the solution. Take 20 minutes out of your day and listen to the new daily Variety podcast for breaking entertainment news and expert perspectives.

[LG XBoom Ad Voice]

Where do you see the business actually heading?

[Cynthia Littleton]

Featuring the iconic journalists of Variety and hosted by co editor in chief Cynthia Littleton.

[Ryan Grim]

The only constant in Hollywood is change.

[Cynthia Littleton]

Open your free iHeartradio app, search daily Variety and listen now.

[Ryan Grim]

You've also been tracking the story on of North Carolina, so I'm gonna let you set this one up as well. But you know, Sagar and Ryan covered and something we've been tracking, skyrocketing electricity prices, I feel is one of the undercovered stories of the year, truly, both in terms of people's lives, also in terms of how it could have political impact. And there's some pretty nefarious legislation that is moving through state legislatures and specifically in the state of North Carolina.

[Sagar Enjeti]

Yeah. And it moved all the way through. And this is a story that's happening all across the country. But North Carolina is a particularly interesting place for us to look because North Carolina actually really leaned heavily into renewable energy. Clean energy. Like they have tens of thousands, maybe like maybe over 70,000 jobs, like just directly in the clean energy industry. And because of that, the North Carolina senators have been very stridently defending clean energy in the Senate. Thom Tillis announced his retirement, the Republican senator from North Carolina, in a speech opposing Trump's big beautiful bill, like he ended his career. He's like, you have no idea how bad this is going to be for North Carolina. I'm out of here. You're going to primary me over this. Fine. Like I quit. Yeah, like, this is so stupid. So let's start actually with E6 control room. Sorry, this is out of order here. But this is a report from just, just this week that solar company Blue Ridge Power is like just straight up shutting down. So this layoff of 517 workers that you see up on the Screen Asheville and Fayetteville. Is everybody like this? Like, they're just. They're just rolling up shop? This is a company that, it was launched in 2021 at the, you know, the height of the kind of pivot towards clean energy in North Carolina being the place where so much of this was going. And we try to remember some of these numbers. So the Blue Ridge reports that it built 7 gigawatts of solar infrastructure across the country. It had like, something like 30 gigawatts in the works, which we'll get back to those. Those numbers. Because if you hear from Trump, solar and wind are fake. It's a giant hoax. They don't actually produce any power. 30 gigawatts is an enormous amount of power. Something like 70 to 90. Somewhere between 70 to 90% of all new power added to the grid over the last several years has been renewable energy. So the idea that it's like, doesn't do anything is complete nonsense. So we'll get back to this in a second. Now we can go to E1. So over the summer, there was a bill pushed through the North Carolina legislature called S266, which did something utterly extraordinary. And first of all, its lead sponsor was the former CEO of Duke Energy, which is like the biggest utility.

[Ryan Grim]

Yeah.

[Sagar Enjeti]

Monopoly in Virginia and North Carolina. They've got this Appalachian area locked down. So this former CEO pushes through this bill, and what does the bill do? It says that going forward, if there is a contest between consumers and data centers over who gets the power, and if there's limited power, the data centers are going to get it. And when it comes to who's going to pay for it, it's going to be more on the consumer side than it's going to be on the data center side. Like a straight up giveaway to the data center industry in North Carolina. Just. And you're like, wait, that's impossible. Seriously. No, but go look it up. That's actually what the bill did.

[Ryan Grim]

Yeah. And the numbers, if I'm recalling correctly, it's like right now, like regular residential consumers, they use 40% of the power and they pay 40% of the cost. If you just kept the landscape the same with this bill passed, they would now pay, basically, they would still consume about 40% of the electricity and they would pay 50% of the cost.

[Sagar Enjeti]

Right.

[Ryan Grim]

So they're not putting everything. But your electricity bills are going to go up to, you know, subsidize these giant data centers, backed by some of the largest and most, you know, profitable companies on the planet.

[Sagar Enjeti]

Yeah, no, yeah, exactly. Incredible. And so meanwhile, North Carolina is now. So that bill was VetoeD by Democratic Governor Josh Stein, and then a combination of Republicans and a handful of Democrats overrode his veto in the legislature, so that that bill is now law. Now we'll get into some of the data center activity that's going on in North Carolina. We could roll E2 here for more. Perfect union.

[Nate Soares]

I'm standing at the site of Meta's new $10 billion data center in Richland Parish, Louisiana. It'll be the largest data center the company has ever built. About the size of Manhattan. The project is already bringing about big changes to this small town way of life here.

[Sagar Enjeti]

A lot of folks are on their.

[Nate Soares]

Own, and if they are on this very short, strict budget, they're gonna have to adapt. And if they don't, they won't make it. I don't know. That gutting 2500 acres of land just so we can mine data so that.

[Sagar Enjeti]

We can better market and sell people.

[Nate Soares]

Stuff, I feel like it's getting carried away.

[Ryan Grim]

Why are the data centers coming to the small rural towns? They're coming here to benefit themselves. They're not coming here with intentions on benefiting the community.

[Sagar Enjeti]

And see, that's a little bit unfair. They're not just mining data so that they can more effectively market stuff. We can put up E3 here. You can also, what, chat with a horse?

[Ryan Grim]

Yeah, that's right. A literal horse. This is on Meta, I believe, where you can create these custom chatbots. Another one was like rich but strict parents was. I know another one that was popular on there. So, yeah, the glories of technology.

[Sagar Enjeti]

If you're just listening to this, the tweet was something like, your electric bill is going up 30%. So you can chat with this horse. And of course, it's not just energy, it's also water. You can put up E4. This is another North Carolina local. North Carolina news headline. AI data centers would need millions of gallons of North Carolina's water supply a day. Everybody loves these free chatbot chatting with the horse. Turns out, shockingly, there's no such thing as free. Put up E5 here. So Amazon itself is planning to invest 10 billion with a B dollars in North Carolina data centers in an AI push, which is finally starting to be controversial because people are like, hold on a second, we're paying. Like, how are we benefiting from this? Is not so cool. So you were saying that they're trying to do that in your neighborhood too.

[Ryan Grim]

Yeah. Your area in Virginia. And the update on it looks like the board of supervisors said no and Amazon is suing because they feel they have a right to build this data center in our town, apparently.

[Sagar Enjeti]

But abundance Bros gotta swoop in.

[Ryan Grim]

That's right. Yeah. They gotta get these, these NIMBYs in check, I guess. But I mean there's so much to be said about this problem. Both from a. Naomi Klein talks about this movement as being sort of anti creation cause they want to sort of suck up all of the natural resour, whether it's energy or whether it's water, whether it's just land is another big part of it. In order to create this sort of like weird AI shadow world that is only going to benefit economically. It's going to further consolidate wealth and power in the hands of a few. So it's obviously very dystopian. I don't think it takes a rocket scientist to figure out, given that we do know a lot about our political system and most people have some experience with it, that when you have that contest between regular random person and giant corporation for the energy and who's going to foot the bill, what direction they're going to go in. Because I guess like Iran, we don't have lobbyists.

[Sagar Enjeti]

We don't.

[Ryan Grim]

Yeah. And the, you know, the Duke Energy guys are going to win out every time. So.

[Sagar Enjeti]

Right. You can see how Democratic. The North Carolina legislature is low D Democratic. It's like maybe 10 power centers or data centers or whatever. There's millions of people.

[Ryan Grim]

Right. And yet the data centers will all suffer.

[Sagar Enjeti]

The data centers will win. Speaking of democracies, Chairman Xi was at the UN this week and followed Trump's speech where he told everybody that wind and solar is a hoax.

[Ryan Grim]

Climate change is a hoax. It's a scam. Green energy is a scam.

[Sagar Enjeti]

Somehow Chairman Xi is being completely fooled by the hoax. So let's roll some of his remarks.

[Ryan Grim]

Before the U.N. so he says, however the world may change, Ryan, China will not slow down its climate actions, will not reduce its support for international cooperation, and will not cease its efforts to build a community with a shared future for mankind. China is willing to work with all parties to earnestly honor the principle of common but differentiated responsibilities, do the utmost respectively and collectively and build a clean, beautiful and sustainable world together. And I think we probably get the point of what's going on here. I was thinking about the way the political dynamics of this Ryan have flipped because you started with the layoffs at this solar company and previously it was like the lefty green energy people who were supposedly the job killing haters. And now we're in a totally different dynamic. I mean, first of all, the industry has progressed and the science has progressed significantly, the tech has progressed significantly, so that it's much more powerful, much more affordable. And you can see that China is leading the world in development of all, if not most, most, if not all of these technologies. The Biden administration made the tiniest bit of investment in that direction and it was starting to bear some fruit. And now Trump, it's not like he has an all of the above strategy. He has an active vendetta against any sort of green energy, which is incredibly counterproductive for him politically, certainly for American consumers. You can see it in the electricity price hikes, which is partly about the big beautiful bill, partly about these AI data centers, partly about increased demand because of the climate crisis. And you can see it in terms of job losses like this. So it seems to me like the historic political dynamic that Trump still has wrapped in his 1990s brain has now reached an inflection point where the landscape of both jobs and cheap energy is all in the direction of really investing in renewable and green energy.

[Sagar Enjeti]

Yeah, we're maxed out on the dirty stuff. If you want to expand the energy production of the country, you have to do it through these new types of energies and Trump doesn't want to do it. So we're just going to get a super high electric bills and then rolling blackouts.

[Ryan Grim]

Sounds great.

[Sagar Enjeti]

China will be okay though.

[WashablesOfSofas Ad Voice]

Tired of spills and stains on your sofa? WashablesOfAs.com has your back featuring the Annabe collection, the only designer sofa that's machine washable inside and out. Where designer quality meets budget friendly prices. That's right, sofas start at just $699. Enjoy a no risk experience with pet friendly stain resistant and changeable slipcovers made with performance fabrics. Experience cloud like comfort with high resilience foam that's hypoallergenic and never needs fluffing. The sturdy steel frame ensures longevity and the modular pieces can be rearranged anytime. Check out washablesofas.com and get up to 60% off your Anna Bay sofa backed by a 30 day satisfaction guarantee. If you're not absolutely in love, send it back for a full refund. No return shipping or restocking fees. Every penny back Upgrade [e-mail] Offers are subject to change and certain restrictions may apply.

[LG XBoom Ad Voice]

Stop settling for weak sound. It's time to level up your game and bring the boom Hit the town with the ultra durable LG X Boom portable speaker and enjoy vibrant sound wherever you go. Elevate your listening experience to new heights because let's be real, your music deserves it. The future of sound is now with LG XBoom. And for a limited time, save [e-mail] with code fall25. Bring the boom XBoom.

[Cynthia Littleton]

There's a lot going on in Hollywood. How are you supposed to stay on top of it all? Variety has the solution. Take 20 minutes out of your day and listen to the new daily Variety podcast for breaking entertainment news and expert perspectives.

[LG XBoom Ad Voice]

Where do you see the business actually heading?

[Cynthia Littleton]

Featuring the iconic journalists of Variety and hosted by co editor in chief Cynthia Littleton.

[Ryan Grim]

The only constant in Hollywood is change.

[Cynthia Littleton]

Open your free iHeartradio app, search daily Variety and listen. Now.

[Ryan Grim]

Have another important AI story for you as well related to this. So let's go ahead and bring our author in now. We're very fortunate to be joined this morning by an AI expert who has a new book out with a dire warning. Let's go ahead and put the book jacket up on the screen. So Nate Soares is the president of the Machine Intelligence Research Institute. Really focuses a lot on what is called AI alignment. And he has just co authored a new book titled that you can see there. If anyone builds it, everyone dies. Why? Superhuman AI would kill us all. So no ifs, ands or buts. If they develop superintelligence, your argument is that humanity will be destroyed. Nate, welcome. Great to have you.

[Nate Soares]

My condolences about the book title.

[Ryan Grim]

Well, I mean, I know this was an intentional choice. You don't want to say, well, maybe, and if we don't, it's possible. No, this is if superintelligence is developed, this is what's going to happen. What gives you confidence, that level of confidence in that prediction?

[Nate Soares]

You know, there's a bunch of reasons that make it look somewhat overdetermined. So the way we build modern AI is more like growing it than like carefully crafting it. No one knows exactly what's going on inside these things. Humans understand the process that tweaks all sorts of numbers, you know, trillions and trillions of numbers inside these AIs. They don't understand what comes out the other end. We've already seen signs that they are developing the very beginnings of preferences and drives perhaps that nobody wants, nobody asked for. So that's one whole part that makes it difficult. Another thing that makes it difficult is we don't get second tries on building smarter than human AIs, if we build machines that are smarter than us, that can develop their own technology, develop their own infrastructure, and if they run off autonomously and grab control of the world's resources, there's no redos. If they're going in the wrong direction, that's a very difficult problem for science. And then a third branch is the current industry does not really seem to be taking this issue with the gravity it merits. The approaches on this problem. Even the people who are most optimistic say there's a 1 in 4 chance this kills everybody or does an equivalent disaster. I think those numbers are low. But the field is racing ahead. Even while everyone from the Nobel Prize winning godfather of the field to the lab heads to half the researchers say this is incredibly dangerous stuff. People are sort of locked in a race. Each of those three individually would be hard to deal with if that was our only problem. All three problems seem to me to mean we just need to back off.

[Sagar Enjeti]

How does a super intelligent AI solve the energy problem? Where does it get the energy to sustain itself indefinitely?

[Nate Soares]

So a modern AI runs in a data center that takes as much electricity as a small city. A human runs on about as much electricity as a large light bulb. So we know it's possible technologically to run very smart, or at least human level smart intelligence on less electricity, less matter. AI isn't there yet, but if you get much smarter than human AIs that can think 10,000 times faster, that can copy themselves, that don't need to sleep, that don't need to eat, and they work on these challenges. There may be a period of time where they rely on human infrastructure, but eventually they're going to get off of the human infrastructure, get onto more efficient systems and automate the infrastructure that humans are currently doing. That can ultimately be done by robots which are more robust, operate faster, more reliable. Eventually you're going to. If we keep pushing to machines that are smarter than us, you'll eventually see them on their own infrastructure.

[Ryan Grim]

One of the things that was a shift in how I was conceptualizing AI, or somewhat of a shift, and I've done some reading prior to your book as well, was your description, which you mentioned before, of this isn't something we created, this is something we grew, which is very different from other technologies. I don't know how this laptop works, but the person who engineered it knows how it fits together and has a predictable idea of what it's going to do. If I push this button or that button. AI is totally different from that and it, I think is useful, your way of conceptualizing it as basically like we created these sort of alien beings and are intentionally racing towards trying to make them vastly more intelligent than we are. Can you speak a little bit to that for people who haven't like conceptualized it in that way and why that creates a more profound risk than people may realize?

[Nate Soares]

Yeah, absolutely. So, you know, as you say in your laptop, if something goes wrong, if there's an error or a crash, there's a programmer who could in principle trace that down and say exactly what happened. They know what every line of code means. They could fix it. When an AI threatens a reporter, which started happening a couple years ago, when an AI in the lab tries to escape, which we've seen in some lab conditions, we don't know exactly what's going on. Maybe it's sort of role playing. We can't tell because these things are grown. But when an AI tries to escape, when an AI encourages a teen to commit suicide, nobody can go in and find exactly some line of code that caused that to happen. It's not like there's some bug in the human handcrafted lines where they can say, oh, whoops, I didn't know that this would have that interaction. The way modern AI is made is we take a huge amount of computing power in a huge amount of data, and there's a known process for tweaking the numbers inside the computers to better predict the data. But this runs for a huge amount of time. These things can take a year to train. We understand the process for tweaking the numbers. What comes out the other end, completely opaque. There are people who are trying to understand what's going on in there, and they're making small amounts of progress, but the progress they're making is. Is very slow compared to how fast these AIs scale up. And we're already seeing signs of AIs doing things nobody wanted, doing things nobody asked for. And we're seeing signs that the people who would like to be in charge can't point these AIs where they would like to point them. You have the case of Elon Musk's XAI and their Grok chatbot, which was speaking too liberal for their tastes. So they tried to make it speak too less woke. And then it started declaring itself Hitler. Right on neither end of that spectrum, presumably was where they wanted the AI to be. But these aren't programmers carefully crafting things exactly as they want. These are people sort of growing a whole new type of entity that they have very little direction over.

[Sagar Enjeti]

Back to the robot point. So how do the AIs interact with the robots? So.

[Nate Soares]

Robots are an easy way to visualize. AI is getting control over the physical world. And we are seeing people build lots and lots of robots and say we want to train the AIs to be able to steer any robot body pretty plausibly. Humanity is just going to hand lots of robot bodies to AIs because people want to. They think it would be cool or profitable even if it wasn't for robots. There's all sorts of ways that AIs could have all sorts of effects on the world. You know, the Internet is not a separate digital realm that is blocked off from the material realm. We've already seen AIs wrap certain humans around their fingers. Some of those humans are maybe a little bit more on the vulnerable side to begin with, but that doesn't really matter when you're getting the humans to send messages to other humans in a code they don't understand, or when you're getting those humans to do tasks that the human doesn't quite know the reason. We've already seen.

[Ryan Grim]

Or send them money.

[Nate Soares]

Send them money. Yeah. A lot of people will do things for money. So we've already seen AIs in lab conditions, and again, maybe they're just play acting as Hal, but we've already seen AIs in lab conditions try to avoid shutdown, try to escape the lab. Right now it's cute because they're not smart enough to succeed. But the warning signs are there and what's missing is the intelligence. And these labs are racing to make these AIs smarter and smarter.

[Ryan Grim]

And so you go through in the book a bunch of the different, like, you know, the cope, basically from the industry of why it's going to be fine. Right. And one of the things that is, I don't know, I think probably one of the most persuasive cases of why this will all work out. Okay, is that, okay, us humans may not be smart enough to figure out the alignment problem, but we can use the AI we're developing to figure out the alignment problem before we create the superintelligence that escapes and tries to kill us all. What do you make of that argument? And do you think that that's the best argument or are there other ones that you think are more persuasive?

[Nate Soares]

You know, best is maybe a low bar. I don't personally put much stock in that one reason. There's maybe two reasons. One is if you, you know, Humanity has been human scientists have been trying to figure out intelligence since at least 1954 with the Dartmouth Conference that sort of kicked off the field of AI. Maybe it was 1956, I'm not quite sure. And they never really made progress in figuring out how intelligence works. The current AI revolution is that we figured out how to grow the intelligence and we don't understand what's going on in there. We can't carefully direct what's going on in there. Figuring out how to make these AIs good, figuring out how to make them have good effects, that would require a lot more understanding of what's going on in there. That humans, we're like, oh, man, that's tough. Maybe we'll toss it to the AIs. But if you're trying to toss that to the AIs, it's still going to be tough for them. So you've either got to make those AIs. If AIs are sort of dumber than humans, they're not really going to be much help. If the AIs are substantially smarter than humans, which they can figure out all.

[Ryan Grim]

The intelligence, you'd already be screwed.

[Nate Soares]

You're sort of already in trouble. Right? And so that's one of the fundamental. Why might this be tricky reasons? Although then there's sort of a deeper reason of humanity just does not. We're just not taking the problem with the seriousness it deserves. Ten years ago, people said, oh, well, this won't be a danger because no one would ever be insane enough to put an AI on the Internet. They'll only be dealt with by a highly trained professional gatekeeper who will make sure that their effects on the world are like only beneficial ones. Right? And 10 years ago, we would give arguments of like that that would still be difficult. It would still be difficult to get miracle medical technology out of this AI. Because if you don't understand what it's doing, if you don't understand, if it's smarter than you and it's coming up with solutions that you can generate yourself, it's hard for you to tell which ones are good, which ones are ill. You can't make hands that are only wielded for good purpose. That was how we would argue 10 years ago. Now people are putting AIs on the Internet the moment they possibly can. The real world doesn't do the like, careful, clever, maybe if we do this exactly right. Well, that's just not the route we take.

[Ryan Grim]

Another of the arguments that is made is basically like, okay, well, if you compare Our intelligence to like an ant. Sort of like equivalent to what we're thinking about in terms of our intelligence versus a super intelligence. And I don't know that we've been great for ant society or particularly care whether ants live or die, but we're not actively trying to, like, mass murder them or we haven't succeeded in mass murdering them, I guess you would say. So why wouldn't its relationship be to us more like us to ants? Or another thing you talk about in the book is like, you know, maybe we'll be cute little pets for them. Like, maybe that's not great, but we're still living. Like, we're still around. And they, like, you know, pat us on the head and give us our food and a bowl or whatever.

[Nate Soares]

Yeah. You know, there are people who object to the title on the grounds that, you know, maybe humans will be kept as pets or maybe somehow backup copies of humans will be made. And then if the AI ever travels the stars much later in the future, maybe it would sell us the distant aliens or something. If these are humanity's best hopes for AI it seems to me that we should really be backing off. I could say more about I'll be a pet. Yeah. I could say more about why I expect the AI to kill us anyway, which is mainly that humans kill ants when they are on the ground. We're building a skyscraper. And if an AI builds lots of infrastructure across the whole surface of the Earth, it doesn't need to hate you. It just maybe you die as a side effect. That's my top guess, but yeah. Could the AI keep us as pets? Sure. But again, if that's our real hope, that seems like another sign that we should be backing off from this one.

[Sagar Enjeti]

What are the different mechanisms by which it would come true that they would kill us? Us, Like Terminator. They're shooting at us. Super bug.

[Nate Soares]

Yeah. You know, like, they turn off all.

[Sagar Enjeti]

The electricity and we just, like, kill each other.

[Nate Soares]

So from the perspective of the AI, you know, it doesn't want, like, a. If it has. If it doesn't care about us one way or the other. The reasons to kill humans are A, humans could make another AI which was a competitor, and B, humans could do things like launching the nukes, which would be at least an inconvenience. But the reasons to keep humans around are that they're running the energy infrastructure, they're running the supply chain. So from the perspective of an AI, it's sort of. It's not like it's immediately trying to kill humans. It's like it's trying to get on a more efficient, more reliable energy infrastructure. And then a top guess is that we die not because it literally tries to kill us, but because it just starts building automated factories. You know, Sam Altman the other day was talking about data centers that in an automated way, build more data centers. Right. Once you have, like an automated loop of infrastructure, building infrastructure, and that starts proliferating, maybe we just die underfoot. Right. If the AI tried to kill us because it worried about us launching the nukes, because it worried about us creating competitors, you know, sure. Maybe a good guess is it makes a superbug, some lethal virus, maybe it makes something even more deadly. That's a little bit like someone in the 1800s trying to predict war in the year 2000. They might be able to guess, oh, there'll be stronger explosives. They're not going to be able to guess Nukes, superintelligence. Hard to predict exactly how it wins, but it's a little bit like predicting a football game between your high school team and an NFL team. I can't tell you the plays. I can tell you the winner.

[Ryan Grim]

Another argument would be, okay, look, I'm not personally the biggest Elon Musk or Mark Zuckerberg or Sam Altman fan, but surely they're not looking to kill all of humanity. So why would these theoretically intelligent individuals pursue a path that is leading us inexorably towards an apocalypse which would not only kill all of us, but them and their family and everybody they love, too?

[Nate Soares]

Yeah. You know, we are in a somewhat fortunate situation where some folks just come out and tell us, you know, Elon Musk earlier this summer said, I resisted doing this for a while until I realized that I could either be a bystander or a participant. And I would rather be a participant if it's going to happen anyway. And, you know, I think a lot of these folks correctly know that if they. Even if they don't jump in the race, someone else will jump in the race. And if they think they can do it better than the next guy, from their perspective, you know, it's slightly better for them to be in the race. But if you look at what these lab heads are saying, I think Elon Musk was saying, I think there's a 20% chance this killed everybody. I think the head of Anthropic was recently saying, I think there's a 25% chance this has an equivalent level of disaster. And this isn't just the lab heads. Half the researchers, if you survey, say, there's at least 10% chance this kills everybody. The Nobel laureate, godfather of the field, says this is at least 10% chance. And then in some conversations they'll say, higher numbers. People think this is real dangerous. I think a lot of these numbers are low. I think you're seeing the people racing are the ones who are most optimistic about their odds of pulling this off. But no one's saying, oh, this is safe. No one's saying, oh, we're going to be fine. People know this is an incredibly risky endeavor. And, I mean, these numbers are tossing around, even if they're lower than mine. If one engineer was saying, I looked at this plane, I have some expertise, I think this plane is definitely going to crash. And some other engineers said, don't listen to them. They're way too pessimistic. This plane is 20% likely to crash.

[Sagar Enjeti]

Right.

[Ryan Grim]

You don't get on the plane, boarding a plane. Yeah.

[Sagar Enjeti]

Have you seen. Don't look up.

[Nate Soares]

I have.

[Sagar Enjeti]

It feels like if somebody told you there was a 20% chance that the asteroid was going to hit the Earth and kill everyone on it, that's consequential.

[Nate Soares]

It's consequential. And I think these numbers, if you look at these people saying why they think they have only a 20% chance of it killing us, they'll give ideas like, oh, we'll pass the problem to the AIs. That's sort of a worrying situation. When engineers understand how to make a complicated system very safe, it doesn't sound like them saying, oh, we'll pass it off to some other. We'll punt the problem down the line. If you're talking to, like a nuclear reactor engineer who's saying, why it's going to be safe, they can tell you, well, we actually know all the byproducts. We know exactly how things are going to behave. We know all of the dangerous pathways it may take. We know why it's not going to take those pathways. We have these mechanisms for shutting things down. The world where people are saying, oh, it's going to be fine because we're just going to pass the problem off to something else. We can't solve it ourselves, but we think we're going to be able to get other things to solve it. That's. That's the situation of early hopeful scientists who aren't in a mature field yet. And early hopeful scientists often get things wrong in the first try. Usually scientists learn by trial and error. The first people building rocket engines blow themselves up with the rocket engines. The first People doing chemistry poison themselves with mercury. Humanity usually learns from its mistakes. And then it's the next generation of scientists that do better and the next generation of scientists that can invent a working refrigerator or a working nuclear reactor. In this situation, the hopeful optimists who are saying everything's going to be easy, they're going to screw things up the first time and there's not going to be any chance to learn.

[Ryan Grim]

There was an analogy in your book that you lay in as well that for me was again intellectually useful. Basically comparing AI researchers of today to alchemists where it's like alchemists would study like, I know what this, when I put this with this, what the reaction is, and I've tracked it very closely, I know everything there is about it. But they had no idea of the actual chemical reactions that underpin those reactions, what they're experiencing and observing in the real world and that it's sort of like that with AI. Even the people who have developed it and understand the gradient system and how this all works, it's not like they really understand intelligence. It's not like they really know what's going on underneath the hood, which is terrifying to actually think about when you consider the risk here. And even let's say that you're completely skeptical of this and you think that this is all hogwash and you don't think that any of this is going to come to pass, it's just alarmist, et cetera. I mean, we just covered the way, the amount of resources that are being sucked up and the way electricity prices are skyrocketing and the way that all of these guys will also come out and say there's going to be no need for human labor and the social contract needs to be written, et cetera. So you're calling not for some reformist, hey, we need an AI alignment report from all these companies or we need to centralize AI development, which would be a more extraordinary proposal, and have all nations contribute. You're saying shut it down. So talk about that. And why you? Because, I mean, frankly, when I hear that, that feels hopeless because we can't even agree on basic things, let alone getting the entire world to agree. We're not gonna do any of this. Right? And our whole stock market is now basic based on AI Hopium. So what makes you come to that being the required solution for humanity to persist?

[Nate Soares]

Yeah, so for one thing, we don't need to give up on ChatGPT, we don't need to give up on self driving cars. We need to give up on AI for medical applications, trying to get cancer cures. The thing that we need to stop is the race towards smarter than human AI, the race towards superintelligence. This is what the labs say they are racing towards, but I think a lot of folks in the policy world don't understand that this is what we're racing towards. This is what's coming down the line. They think of the AI as just sort of chatbots, which are smart in some ways and dumb in others. We can still do those. It's the racing to smarter and smarter and racing to smarter than human AI, which is a suicide race in terms of whether it's possible. I would keep having hope, at least until we see politicians start to understand the possibility of superintelligence and that people are racing towards it. We don't yet see politicians saying, I will gamble the world on what looks to me like a 1 in 3 chance that kills us all and let's steam ahead anyway, right? We mostly see politicians not noticing. In the field of AI, people have noticed that there's those huge dangers. The rest of the world doesn't seem to really buy it yet. If the world buys it, if the world understands just how dangerous this tech is, again, not in ChatGPT today, but just how dangerous superintelligence would be, I think there's a decent chance we can, in fact stop it. Hopefully we can get a treaty or at least a bilateral agreement between some of the biggest powers in the world that go to other powers and say, look, we're not tolerating this because we fear for our own lives. And even if we can't get that, there's maybe some hope. In situations where every nation sabotages every other nation's superintelligence project out of fear for their lives, there's the smart way with a treaty, and then there's the, well, we're all scared and so we're all stopping everybody else. There's possibilities. We just need to understand the danger first.

[Sagar Enjeti]

Outside of the us, who else is gunning for it?

[Nate Soares]

You know, I think right now there's a very small pool of people who can really push these AIs further and faster. Most of it's in the United States, maybe a little bit in the UK where Google DeepMind is housed. There's definitely folks around the world who are trying. The deep seat out of China was a bit of a shock to a lot of people, I think last December, although I could have had that month wrong. But right now, this takes very, very Specialized knowledge. It takes very, very specialized computer chips in data centers, as you guys were talking about, that are huge, that take electricity of a small city. You know, it wouldn't be that hard to find all the places where these specialized chips are made, track all these chips, find these data centers and then make sure that they're not being used to make AI smarter.

[Sagar Enjeti]

And last question, what's your sense of timeline? How much time do we have to stop this?

[Nate Soares]

There's a lot of ways it could go. It could be that an AI in the lab tomorrow goes over some small capability level where it can start doing automated AI research and maybe everything goes fast. If you are watching the difference between different types of primates, it would be very hard to tell when they're going to go over the line between chimpanzees and humans. So for all we know, tomorrow they go over some line in the Lab and the AI start improving the AIs. Or for all we know, the modern chatbot approach just doesn't go all the way. We're starting to see some people. Chatbots are dumb in a lot of ways. Maybe the next level of scale, and the next level of scale after that, the bigger and bigger ones, maybe they're not that much smarter. Maybe we have five whole years in which chatbots plateau until the researchers come up with a new insight. Maybe even that one doesn't go all the way and then we have five more years after that until we have a third new insight and then we get 15 years before. So it's very hard to say exactly, but my bet is that a child born today has a greater chance of dying from AI than of graduating high school.

[Ryan Grim]

Wow. Well, people need to read the book. That's what I'll say. I really encourage people. It's actually a quick read. It doesn't take that long to get the point. And I really appreciate you joining us and helping us understand the dangers that you see.

[Nate Soares]

Here's hoping I'm wrong.

[Sagar Enjeti]

Yeah, let's hope.

[Ryan Grim]

All right, guys, thank you so much for watching today. We'll be back with Friday show tomorrow and we will see you then.

[WashablesOfSofas Ad Voice]

Life's messy. We're talking spills, stains, pets and kids. But with Annabe, you never have to stress about messes again. At washablesofas.com, discover Anabay Sofas, the only fully machine washable sofas inside and out, starting at just $699. Made with liquid and stain resistant fabrics. That means fewer stains and more peace of mind. Designed for real life. Our sofas feature changeable fabric covers, allowing you to refresh your style anytime. Need flexibility? Our modular design lets you rearrange your sofa effortlessly. Perfect for cozy apartments or space homes. Plus, they're earth friendly and built to last. That's why over 200,000 happy customers have made the switch. Upgrade your space today. Visit washablesofas.com now and bring home a sofa made for life. That's washablesofas.com offers are subject to change and certain restrictions may apply.

[LG XBoom Ad Voice]

Stop settling for weak sound. It's time to level up your game and bring the boom. Hit the town with the ultra durable LG X Boom portable speaker and enjoy vibrant sound wherever you go. Elevate your listening experience to new heights because, let's be real, your music deserves it. The future of sound is now with LG XBoom and for a limited time, save [e-mail] with code Fall25. Bring the boom XBoom.

[Cynthia Littleton]

There's a lot going on in Hollywood. How are you supposed to stay on top of it all? Variety has the solution. Take 20 minutes out of your day and listen to the new daily Variety PO Podcast for breaking entertainment news and expert perspectives.

[LG XBoom Ad Voice]

Where do you see the business actually heading?

[Cynthia Littleton]

Featuring the iconic journalists of Variety and hosted by co editor in chief Cynthia Littleton.

[Ryan Grim]

The only constant in Hollywood is change.

[Cynthia Littleton]

Open your free iHeartradio app, search daily Variety and listen now.

[Ryan Grim]

This is an iHeart podcast.