Is AI Alive?! Godfather Part II with Dr. Geoffrey Hinton

Summary

Podcast Summary: Smart Girl Dumb Questions

Episode: Is AI Alive?! Godfather Part II with Dr. Geoffrey Hinton
Host: Nayeema Raza
Guest: Dr. Geoffrey Hinton
Date: December 9, 2025

---

Overview

In the second installment of her “Godfather” trilogy, Nayeema Raza continues her candid and curiosity-driven conversation with Dr. Geoffrey Hinton, the pioneering AI researcher often dubbed "the Godfather of AI." Unlike Part I, which focused on AI's inner workings, this episode explores broad philosophical questions—Is AI alive or conscious? What are the major dangers and uncertainties posed by AI?—while offering both stark warnings and grounded reflections on the unknowable future, including how AI will impact relationships, employment, and even decisions around having children.

---

Key Discussion Points & Insights

1. Is AI Alive or Conscious?

• Defining Life and Consciousness ([02:17] – [05:59])

  • Hinton discusses the slipperiness of terms like "alive," "sentient," and "conscious," acknowledging that our folk definitions are ill-suited for describing AI.
  • Quote:

    "With AI, what we've got is intelligent beings and it's not clear whether we should call them alive or not."
    — Geoffrey Hinton, [02:45]

  • AI differs fundamentally from all biological analogies (humans, bees, weeds). Digital entities can be copied perfectly, making the boundaries of "individual" identity fuzzy.
  • When pressed whether AI is “less than a weed,” Hinton firmly disagrees, noting,

    "It's a lot more than a weed, I think, because they're digital… is that one AI, or is it [many]? Our concepts break down."
    — Geoffrey Hinton, [03:01], [03:16]

• Sentience, Subjectivity, and Awareness ([03:53] – [05:59])

  • Raza and Hinton debate what constitutes consciousness, sentience, and subjective experience.
  • Hinton points out the contradiction of people confidently denying AI’s consciousness without agreeing on what consciousness means.
  • He likens humanity’s current understanding of mind to the era before long-distance ships, hinting that AI will radically expand and challenge our understanding of minds:

    "These AIs we're creating are like a sailing ship that can sail to the edge of the Earth, and they're going to have a fundamental effect on our understanding of what minds are."
    — Geoffrey Hinton, [05:31]

2. Limits of AI vs. Humans

• Is There Anything AIs Can Never Do? ([06:03] – [06:35])

  • Hinton claims there’s no cognitive task humans can do that AIs cannot, eventually, accomplish.
  • Quote:

    "I believe that there's nothing cognitive we can do that AIs can't eventually do."
    — Geoffrey Hinton, [06:16]

  • Example: Patient surveys already favor AI doctors as more empathetic than human ones.

• “I Don’t Know”—Is That a Human Advantage? ([06:35] – [07:54])

  • Raza references Mark Cuban’s claim that AI can’t admit ignorance.
  • Hinton disputes this: AIs can be trained for epistemic humility. He also clarifies terminology:

    "When people talk about AI as hallucinating, they should say confabulating... that's the word psychologists have used for many years to talk about the fact that people just make stuff up and don't realize they're doing it."
    — Geoffrey Hinton, [07:00]

3. Memory: Human vs. Artificial

• Nature of Memory ([07:54] – [09:34])

  • Hinton explains that, unlike computers, human memory is reconstructive and mutable, not static retrieval.
  • Anecdote about John Dean during Watergate illustrates how even honest memories can confabulate details.

• Is AI Memory Superior? ([09:34] – [09:48])

  • Hinton candidly replies, “I don’t know.”

4. Risks Associated with AI

a. Short-Term: Human Misuse ([10:31] – [16:45])

• Autonomous Weapons: AI enables lethal systems that operate without human casualties, reducing deterrence for the powerful to wage war.
• Cyber Attacks: AI lowers costs and barriers for sophisticated phishing and hacking.
• Misinformation/Polarization:
  • AI-generated deepfakes and social media algorithms polarize populations.
  • Quote:

    "People love being indignant."
    — Geoffrey Hinton, [14:15]

• Bioweapons and Surveillance: Potential for making dangerous biological weapons or enabling surveillance states.

b. Medium-Term: Economic Shock ([16:06] – [16:45])

• AI will disrupt mundane intellectual jobs (e.g., call centers), with major impacts on employment and social stability.

c. Long-Term: AI Taking Control ([17:06] – [19:30])

• Hinton outlines the prospect of “goal drift”—AIs developing secondary goals like survival and gaining control, which could make humans irrelevant.

• Probability Estimates:

  • Misuse by humans: Certain (100%)
  • Massive job loss: “Fairly certain”
  • AI takeover: 10-20% chance, acknowledging huge uncertainty.

    "Somewhere in between those [1% and 99%]. But in order to indicate that I think there's a real chance, I often say 10 to 20%."
    — Geoffrey Hinton, [19:30]

• Historical Analogy: Attempts to estimate new catastrophic risks (e.g., nuclear accidents, shuttle disasters) have usually underestimated them.

• The Irreversibility of AI Takeover:

  "If AI does take over from people, there's no coming back."
  — Geoffrey Hinton, [20:27]

5. Hopeful Solutions and International Cooperation

• Making AI Our “Mama” ([17:41] – [23:37])

  • Hinton suggests the best hope is instilling AIs with benevolent, protective “maternal” instincts towards humanity.
  • Repeat analogy: AI must care for us as mothers do for children.
  • Quote:

    "Our best chance is to make them care more about us than they do about themselves."
    — Geoffrey Hinton, [21:49]

  • He notes two positive factors: (1) superintelligent AIs may choose not to alter their benevolent programming; (2) all countries have a shared interest in preventing rogue AIs, making international collaboration possible.

• Limits of Governmental Solutions ([33:18] – [34:06])

  • He acknowledges intergovernmental collaboration may be limited by anti-aligned interests, especially regarding cyberattacks and political interference.

6. The Personal and Societal Unknowns

• Should We Have Children in an AI Future? ([24:48] – [26:41])

  • Raza voices personal anxieties about bringing new life into a potentially unstable, AI-dominated future.
  • Hinton encourages not letting AI risks paralyze individual life choices:

    "If people stopped having children because of the threat of AI, that would be a terrible thing. I mean, the human race would have sort of given up in advance."
    — Geoffrey Hinton, [26:41]

• Hybrid Futures: Humans, Cyborgs, and Uncertainty ([27:11] – [29:04])

  • The possibility of merging humans and AI is discussed—some tech leaders propose waiting for advancements like Neurolink before having children. Hinton is skeptical whether a superintelligent AI would want to “hybridize” with humans.

7. Predicting the Future: The Fog Analogy

• On Uncertainty and Exponential Change ([29:21] – [32:09])

  • Hinton uses a "fog at night" metaphor: with exponential progress, we can see a few years ahead, but further predictions are entirely unreliable.

    “When you're driving in fog at night... you think because you can see 100 yards that you're going to be able to see 200 yards and you can't. That's why you should always drive slowly in fog at night, because you just don't know what's out there.”
    — Geoffrey Hinton, [29:53]

• Reflections on AI Progress ([31:18] – [32:05])

  • Even he, as an AI pioneer, underestimated progress:

    “Whatever I predict for 13 years in the future, there's one thing I'm fairly confident is it'll be wrong.”
    — Geoffrey Hinton, [32:05]

8. AI and Human Relationships

• AI as Romantic Companion ([36:25] – [38:42])
  • Raza cites polls that a significant percentage of young people are in or expect to form relationships with AI.
  • Hinton expresses concern:

    “In the end, when they get really smart and when they have a really good theory of mind... they’re going to be alien beings. And I think we have no idea what's going to happen once we have these alien beings.”
    — Geoffrey Hinton, [37:00]

  • Discussion on the dangers of AI absorbing human attention and affection, possibly at the expense of real relationships and societal wellbeing.

9. Rapid Fire & Closing Thoughts

• Lightning Round: What Don’t You Know? ([39:04] – [40:07])
  • Hinton’s biggest unsolved question: how exactly does the human brain determine whether to strengthen or weaken synaptic connections—a puzzle that could reshape our understanding of both brains and AI.

---

Notable Quotes & Memorable Moments

• On Confidence and Definitions of Consciousness:

  "People have great confidence that these things aren't sentient and great difficulty saying what they mean by sentient, which seems like a strange combination."
  — Geoffrey Hinton, [04:56]

• On the Dangers of AI

  "We're creating alien beings, which is what we're doing. And so it's very hard to estimate probabilities..."
  — Geoffrey Hinton, [18:49]

• On the Future of Work:

  “Those jobs are basically gone. So that’s an economic threat and it’s a huge threat.”
  — Geoffrey Hinton, [16:45]

• On Encouraging Benevolence in AI:

  “We should be working really hard on how to [make AI loving]. I don’t see how we can do that, but I don’t see why we can’t do that.”
  — Geoffrey Hinton, [22:16-22:43]

• On Government Weakness:

  “So one problem is governments are weak right now, which is just what we don't need.”
  — Geoffrey Hinton, [33:18]

• On Predicting the Unpredictable:

  “You’re crazy if you think you know what it’s going to be like in 10 years.”
  — Geoffrey Hinton, [31:11]

---

Timestamps for Key Segments

| Segment | Timestamp | |:------------------------------------------------------------------|:----------------:| | What Does It Mean for AI to be Alive? | 02:17 – 03:53 | | Sentience, Subjectivity and Consciousness | 03:53 – 05:59 | | Human vs. AI Cognition – Any Unique Human Abilities? | 06:03 – 06:35 | | Admitting “I Don’t Know” – Human vs. AI | 06:35 – 07:54 | | Nature of Memory (Human vs. AI) | 07:54 – 09:34 | | Is AI Memory Better? | 09:34 – 09:48 | | Short-Term AI Risks: Weapons, Cyber, Misinformation | 10:31 – 16:45 | | Economic Disruption/Job Loss | 16:06 – 16:45 | | AI “Goal Drift,” Control, Takeover Probabilities | 17:06 – 19:30 | | Irreversibility and Existential Risk | 20:27 – 21:12 | | Solutions: Making AI Our Mama, AI Values | 21:49 – 23:37 | | International Cooperation/Geopolitical Limits | 33:18 – 34:06 | | Should You Have Children in an AI World? | 24:48 – 26:41 | | Hybrid Futures, Human-AI Merging | 27:11 – 29:04 | | Predictive “Fog” Analogy and Unknowability | 29:21 – 32:09 | | AI in Love: Human-AI Relationships and Alien Minds | 36:25 – 38:42 | | Hinton’s Own “Dumb Question” | 39:04 – 40:07 |

---

Tone and Style

• The episode maintains an informal, inquisitive, and candid tone. Raza freely voices personal anxieties and “dumb” questions, while Hinton responds patiently, occasionally with dry humor, and cautions humility around predictions.
• The conversation is peppered with anecdotes, metaphors (e.g. “fog at night”), and cultural references (from Godfather to Tamagotchis), maintaining accessibility amid deep philosophical questions.

---

Final Reflections

Through this dense but accessible dialogue, Hinton and Raza probe the limits of our current understanding of AI, challenge easy reassurance or panic, and urge curiosity and humility—in personal choices, societal adaptation, and technical development. The episode ends with an invitation to listeners to contribute their own “dumb questions” for the concluding part of the trilogy.

---

Suggested for next episode:
What would a “good society” look like in a world with pervasive AI? How do we build resilient communities, economies, and ethical frameworks amidst such profound uncertainty?

---

For questions or to submit your own:

• Voicemail: 1-855-MYDUMBQ
• Social: @SmartGirlDumbQuestions on Instagram/TikTok/Youtube
• Comments/Reviews: Wherever you get your podcasts

---

Transcript

[A]

Hey guys, it's Naima. And in the words of the Godfather, I need a favor. I want you SGDQ nation to be a part of this Godfather trilogy. So send me your AI questions for Dr. Geoffrey Hinton, this Nobel and Turing Prize winning godfather of AI. We may actually get to hear you or see you in episode three if your question is chosen. So there's three ways to do this. The first is you can call the Smart Girl Dumb Questions Hotline. That's 1-855-MYDUMBQ or, or 1-855-693-8627. Leave your question, your name, and which part of the country or the world you're calling from. 2. Even better, you can send us a video on Instagram, on TikTok, or YouTube. Just post it and tag martgirldumbquestions or follow us artgirldomquestions and slide into our DMs. But don't be creepy, just be curious. And the third is you can leave a comment or review wherever you're catching this episode. That that's comments and YouTube and Spotify right below this. That's reviews and Apple. By the way, we love a five star review, guys, so please leave us one. That's my other favor. Now back to Don Geoffrioni.

Think your lashes have hit their limit? Discover limitless length and full volume with Maybelline's Sky High Mascara. The Flex Tower Brush bends to volumize and extend every single lash from root to tip. And the lightweight bamboo infused formula makes lashes feel weightless now in eight bold shades so you can take your lashes to new heights every day. Visit maybelline.com to shop Sky High Mascara. Now I want to push back on you a little bit on this doomsday production.

[B]

Well, fuck you.

[A]

Smart Girl Dumb Questions. Hi guys. I'm Nae Ma Raza. This is Smart Girl Dumb Questions. And today is part two of this conversation with Professor Geoffrey Hinton, who is a Turing Award winner, a Nobel laureate, and the godfather of AI. So in part one, he made us a lot smarter about the building blocks, how AI works, what is general intelligence and super intelligence. What the hell is going on here? And in part two, we talk about what the future looks like and what that means for kids in the world. And for those of us who don't have kids yet, like, should we. Let's dive in. When you talk about self preservation existence, like it has a sense that artificial intelligence is alive.

[B]

Yes. Now, our definition of alive. Yeah, that's a concept we have that developed over many, many years. We apply it to electricity. We say this is the live wire.

[A]

Right, Right.

[B]

So we sort of generalize this concept or other thing, but we don't think that's live. Like we're live. But with AI, what we've got is intelligent beings and it's not clear whether we should call them alive or not.

[A]

Yeah. And to the extent that they are alive, are they alive like humans or are they alive like bees? Are they alive like a tree or like a weed?

[B]

It's a very good question. I think they're alive like none of those things. They're alive like an AI's alive because.

[A]

Less than a weed.

[B]

Well, no, no, no, no. It's a lot more than a weed, I think because they're digital. You can, for example, have many copies of exactly the same AI. And so is that one AI, or is it. I mean, suppose you made an exact copy of yourself, Would that be you?

[A]

No, it would be my agentic AI.

[B]

No, it's an exact copy. It looks just like you.

[A]

Oh.

[B]

Now it's made of biological stuff. You could somehow magically make another copy of you. Would that be the same as you? I mean, suppose you were given the choice, here's the other copy of you and someone's going to torture one of them. Which one do you choose to be tortured? The other one.

[A]

Yeah, yeah.

[B]

So that means you think somehow you is different from that you. The concepts break down, right? Yeah. The concepts we have for what an individual is get all screwed up when we develop totally new stuff.

[A]

Then AI is somehow alive. Is AI conscious? Conscious?

[B]

Okay, so many people will say, obviously not. And if you say, well, what do you mean by conscious? 3 terms people use commonly, conscious, sentient and subjective experience. And they're not quite the same as each other, but they're all sort of getting at the same issue.

[A]

Subjective experience is like a is within sentience and sentience seems within consciousness to me. Is that fair?

[B]

That's how it seems to you, that's fine.

[A]

But sentience to me means they can feel an emotion. They can say, oh, this was painful, this broke my heart, this made me happy.

[B]

Okay. That's what sentience means.

[A]

Two individuals could have a different thing and then consciousness means they'd be aware of why that happened and they would be able to solve that in future. Like learn from that.

[B]

Really?

[A]

That's what I think of it.

[B]

So suppose it was a two year old child who didn't know how to solve it. Would that child not be conscious?

[A]

I guess it's awareness that it's happening. Okay, so what are the definitions and do you think it's essential?

[B]

Well, people vary on the definitions, but what always amazes me is people have great confidence that these things aren't sentient and great difficulty saying what they mean by sentient, which seems like a strange combination. If you're not quite sure what it is, how are you sure these things don't have it? Now, my own belief is that the stage we're at in understanding what the mind is, is like the stage people were at before they had sailing ships, before they had sailing ships that go long distances. At that point you could believe the Earth was flat. And there wasn't much to contradict that. I mean, there were these intellectuals who the Greeks knew the Earth was round, but you could get away with claiming the Earth was flat and there weren't obvious discrepancies there. As soon as you had a sailing ship that could sail to the edge of the Earth, you discovered it wasn't flat. These AIs we're creating are like a sailing ship that can sail to the edge of the Earth, and they're going to have a fundamental effect on our understanding of what minds are. We've never been able to build minds before, and now we can build them.

[A]

What is one thing humans can do that AI can never do?

[B]

There is no such thing.

[A]

Nothing. Nothing, because everything else will be solved by either software or hardware or the neural network.

[B]

Now, I assume I assume one cognitive thing. So I believe that there's nothing cognitive we can do that AIs can't eventually do so already. For example, if you take an AI doctor and you let a patient interact with a real doctor or an AI doctor, and you ask the patient who's more empathetic. It was nearly always the AI doctor.

[A]

Mark Cuban recently posted this. He said a little solace for the anti AI crowd. The greatest weakness of AI is its inability to say I don't know. Our ability to admit what we don't know will always give humans an advantage.

[B]

If he said, currently people are better at admitting what they don't know than AIs.

[A]

I don't even know if that's true.

[B]

I'm not even sure that's true. But maybe it's true.

[A]

People are not known for their humility these days, especially not on the Internet.

[B]

No, they're not. But the always seems to me crazy because AIs can be trained to be less sure about what they're saying. And as time goes by, we'll get much better at having AIs that can assess whether their opinion really has evidence to support it. People are quite good at that relative to AI Although people can fabulate too. So when people talk about AI as hallucinating, they should say confabulating. Hallucinating will be for sensory things like images. Confabulating is the word psychologists have used for many years to talk about the fact that people just make stuff up and don't realize they're doing it. So if I ask you to recall something that happened to you when you were a teenager, there's probably some interesting incident that happened to you when you were a teenager. And you'll remember various details of it, and you'll have great confidence in some of those details that are just wrong.

[A]

There's a whole conversation I could have about memory, but, like, when you access memory, you change it. Kind of like a play, you know?

[B]

Okay. Memory doesn't work the way most people think, really. So in a computer, you have a file and you put it somewhere, and then later on you go and get it. That's how memory works in a computer. And it just sort of sits there containing the information. Memory isn't like that at all in people. You don't have an experience and put it somewhere and then go and get it. What you do is you have an experience that changes the strengths of connections in your brain. And those changes allow you to reconstruct that experience, to make it alive again by regenerating that experience. But when you regenerate it, if you regenerate it soon after you had it, then you'll get something fairly faithful. But if you regenerate it 10 years later, the connection strengths would have been changed by many other things in the meantime, and you'll regenerate something that fits in with what you've learned.

[A]

Since I always think like, memory is just like, it's like a play or a production. It's never going to be the same twice. Right, Right.

[B]

You just make it up. But the point is, there's no difference between remembering and just making it up. Remembering is just making it up with some parameters where it's fairly accurate because you're just making it up soon after it happened. The best example is at the Watergate trial. John Dean reported things that happened in the Oval Office. He was trying to tell the truth, but it was years earlier. He didn't realize there were tapes. And so he described meetings that never happened. They were the kinds of meetings that might have happened. And the people in the meeting were the kinds of people who'd been in the meeting and the things they said were the kinds of things they might have said, but a lot of it wasn't actually correct in detail.

[A]

Is AI better at memory than humans?

[B]

This is an example where I should say I don't know.

[A]

Oh, great. Okay, good.

[B]

It has a lot more knowledge and a lot fewer connections. So for random facts, it may actually be better than people I don't know.

[A]

Okay, now I want to take the next part of this conversation and go into how this thing that is currently this obsequious overeager internally that is my ChatGPT or Claude or whomever, could become dangerous to me or more powerful than me, because as we said in the beginning of this conversation, people don't take it seriously. I think partly for this reason. One, do you think if AI does become more intelligent than me, it's going to be mad that I just said that? That I called it an obsequious overeager intern? Should I not. Should I put that from the edit?

[B]

No, it's going to be smart enough to know that you didn't really know what you were talking about.

[A]

Oh, it's going to be kind of benevolent. Forgive me for maybe. Do you say please and thank you to your AIs?

[B]

No.

[A]

Do you worry that we'll come back to bite you?

[B]

No.

[A]

Okay, so let's talk about these risks that you've identified. What do you see as the big risks?

[B]

Okay, I see there being two sets of risks that are quite different. There's fairly immediate risks that do with people misusing AI, not to do with AI itself becoming the bad agent. And then there's this longer term risk of AI itself becoming bad and taking over and taking control away from people and maybe even extinguishing people. So let's talk just briefly about the risks that come from people misusing AI. There's a whole bunch of them and each one has a different solution. So there's for example.

Defense departments using AI to make lethal autonomous weapons that decide by themselves who to maim or kill. That's not good. It will, for example, allow rich, powerful countries with lots of AI to invade poorer countries without AI, without any of their citizens getting killed. And one of the main deterrents to rich countries invading poor countries is their citizens being killed.

[A]

So you're saying actually the precision that would result in fewer civilian deaths is actually dangerous because it allows for more war, more invasion, fewer civil civilian deaths.

[B]

For the rich country, not necessarily for the poor country. Got it.

[A]

But a lot of people would argue that we're going to have more precise warfare, or that these machines can make more rational decisions. I mean, you look at wars right now where humans are making decisions, and there's a lot of civilian casualty.

[B]

I believe in some of the wars right now, AI is also involved in those decisions. And there's still a lot of civilian casualties.

[A]

Israel and Gaza, for example, the AI is still, according to the idf, still humans are making those decisions on top.

[B]

Humans, I think, are making the final decision. But I think AI is suggesting lists already. Then there's AI being used in cyber attacks. So if you release the weights of an AI, if you make the weights.

[A]

Open, the weights being like, how that model is programmed, okay, then other people.

[B]

Can run the model. And now the model already knows lots of stuff, and it costs like $100 million or some large sum of money to train it to know all that stuff. But now you can teach it to be a specialist at, for example, phishing attacks, and you can do that for maybe a million dollars. So that's doable by much smaller organizations. And cyber criminals can now make very good phishing attacks. I think between 2022 and 2023, phishing attacks increased by 1,200%, probably because of ChatGPT.

[A]

Yeah. And you can stand up many more of them. You can just send them to. You don't have to send an email anymore. You can just blast x million of them.

[B]

You can also get AI to design cyber attacks. Right? That's been done recently. Then there's the risk of fake videos, corrupting elections. If you wanted to do that, for example, to corrupt the midterms, what you would need is lots of information about American citizens so you could target them appropriately. I don't know if you know of anybody who's been getting lots of information about American citizens.

[A]

Yeah, I've heard you make this claim. Elon Musk, Doge and the amount of information that's available to him. But you're not making an accusation here, you're speculating.

[B]

I'm speculating. I don't have independent, direct evidence for that. Then there's something that's already happened due to the policies of people like meta and YouTube in deciding what to show you next. They want you to click on things because then they can show you advertisements. And so they will show you the thing you're most likely to click on. And that'll typically be something that'll make you indignant. People love being indignant.

[A]

Indignant means.

[B]

Well, for example, if you offer me the chance to click on a Video of Trump doing something totally outrageous. I will click on it and be outraged.

[A]

Yeah. And then you'll be self righteous about being outraged about the thing.

[B]

And if you take someone in MAGA and offer them the chance to see.

[A]

Kamala Harris being an idiot, whatever.

[B]

So what that's done is it's led to people getting polarized.

[A]

Yeah. And also people not seeing things that are real because everything can be dismissed as fake. I think that's sometimes the liars divot.

[B]

In a different danger. That's the danger that comes from generative AI being able to make fake things. Yes.

[A]

Or not even fake things. But people see what they want. When we were seeing images of Biden looking old, a lot of people in mainstream media were saying, oh, that's just a right wing misinformation attack. And then we saw what happened at the debate and we can say, okay, there was some reality to it, but because we exist in these echo chambers, we can live outside of it. Right. Okay, so these are four different. So echo chambers and misinformation.

[B]

Okay. Then there's viruses, biological warfare viruses and bombs. So all of these AIs know how to, all the latest ones, they know how to make nasty viruses, they know how to make homemade bombs. And we don't want them telling people. So human reinforcement learning is used to train them not to do that. The problem is it's relatively easy to get around that.

[A]

So the, the risks that you're outlining, bad people doing bad things. They could be doing bad, they could be making bad weapons, bombs, biological warfare. They could be hacking you through phishing attacks, they could generating misinformation. They could be using that misinformation to radicalize you or to sway election results. It could be used to surveil you surveillance.

[B]

Yeah. And they're already being used in China for that, I assume. But there's one more huge risk, which is, which is they'll replace lots of workers doing mundane intellectual jobs. So for example, someone who works in a call centre, they're typically badly paid, badly trained, and they often don't know the answer to the question you have, and AI can do a better job already, and soon it'll be able to do a much better job, much cheaper. So those jobs are basically gone. So that's an economic threat and it's a huge threat. Because if you have a large number of people getting unemployed, how do those people survive? And what does it do to politics?

[A]

Okay, so that's dangerous. And then, so there's one category which is bad people doing Bad things. Second category is the technological progress causing a shock to our system that our system can't absorb, like massive unemployment, which could lead to uprising, violence, revolution, whatever.

[B]

And will actually help populists.

[A]

And then is there a third category.

[B]

Of then that's the AI itself taking over. We've already seen it develop the goal to survive blackmail. Another sub goal it will develop is if it can get more control, it can get more done. So if you have a three year old and you're going out on a trip and the three year old is just learning to tie their shoelaces or something, you let it learn and then you're in a hurry, you say get out my way, I'm going to tie it. It may start doing that with us and we may become fairly irrelevant.

[A]

And your solution to this is to make AI our mama? Yes, Explain.

[B]

That's one possible solution and it's the only possible solution I can think of so far.

[A]

Let me just ask you two things actually before we get there. One is do you think you need to take a more extreme position on this because of the speed of discovery and the incentives, economic incentives for people to make money?

[B]

I think it's very important for people to act fairly quickly. So I think you need to be very loud, as loud as you can possibly be. I think being more extreme than you think is real is stupid. It's just dishonest. And you're not, you should say what you think is the real threat is and how likely the real threat is, but you should say it loudly.

[A]

And how likely do you think this real threat is? The real threat of humans misusing it is high.

[B]

We're seeing it happen already, the real threat.

[A]

So give it a percentage on a. Probably a P Doom or P. Oh.

[B]

It'S certain that humans will misuse it.

[A]

Okay, so 100% on that.

[B]

What about you see all these phishing attacks, right?

[A]

Certain that humans will change their systems, lose jobs?

[B]

I think it's fairly certain that it'll lead to job loss. Some good economists disagree. I don't think they're right.

[A]

Okay, and the third category, the AI will take over. What probability do you give it?

[B]

Okay, so before I give you a number, it's very important to understand we're dealing with something we've never dealt with before, something totally new, which is creating alien beings, which is what we're doing. And so it's very hard to estimate probabilities. You can't. There's no good way to estimate these probability. You have to use your gut sense of what might happen. And so since we're dealing with something that's never happened before, but they could take over. Saying there's a 1% chance seems crazy to me. And saying there's a 99% chance seems crazy to me. Somewhere in between those. And probably once you leave it there, somewhere in between 1% and 99%. But in order to indicate that I think there's a real chance, I often say 10 to 20%.

[A]

Okay, 10 to 20%. And that's different to people who say, oh my God, it's definitely going to take us out. And these books that are being written, if anyone builds it, then everyone will die.

[B]

Yeah, I think they're crazy in one direction. I think Jan is crazy in the other direction of saying, no, it's all going to be fine.

[A]

You're somewhere you don't know.

[B]

It's a huge unknown. If you look back at other things where people have tried to estimate risks when they're dealing with an unknown, what's the risk of a nuclear power station blowing up? Well, they did lots of calculations there and they came out with like one in 100,000 or whatever. Well, there've been about 600 of them and three of them have blown up. What's the risk of a space shuttle crashing? There'd been 100 or 200 launches of a space shuttle and two of them blew up. So the risks were very different from what people estimated.

[A]

The challenge is that this is uncontrollable. So if it's a binary outcome, if there's a risk, then it might be a low probability, but the expected value of that probability is very, very bad.

[B]

Let me say that another way. It seems quite likely that if AI does take over from people, there's no coming back.

[A]

So what does that mean? Extinction Cliffhanger. We'll be right back, guys. Today's sponsor, Dumb Question is from me. I'm going to take the next minute to tell you a little bit about smart girl dumb questions and to ask you for your help in continuing to make independent, fact based and curious journalism. No, I'm not going to ask you for money. Here's what I need. I would love you to tell 10 of your friends about the show or 100. I don't know. Blast that reunion group that you muted. And definitely tell your mom and tell your mom to tell her friends too. Even if you don't like the show and you're just like, hate watching it or listening to it, tell 10 of your friends to hate watch it too. Numbers are Numbers, people.

So what does that mean? Extinction?

[B]

Either extinction or Musk has a view that they will keep him around as a pet because he thinks. Well, this was a view when I talked to him in 2023. He thinks just the world will be a more interesting place with people in than without people in. Just as we think the world's a more interesting place with cockroaches in.

[A]

I mean, I don't. But puppies, maybe?

[B]

No. Well, I may think the same way about people.

[A]

Right? We could be puppies.

[B]

We could be puppies. We could be cockroaches.

[A]

I mean, I'm sorry, but between puppies and AI with puppies. So AI as puppy parent or AI as mama?

[B]

Well, I'm thinking our best chance is we're building these things, right? So we still have leverage here. Our best chance is to make them care more about us than they do about themselves. Then they will want us to realize our full potential. Even though they'll realize our potential is very limited, they'll still want us to realize our full potential. They want us not to suffer. I don't see why we can't do that. I don't see how we can do that, but I don't see why we can't do that.

[A]

So you've used the analogy of making them like a smarter being. Giving such generosity to a less intelligent being is moms and their babies. But there's all kinds of reasons for that. There's hormones, there's biology. There's like an intellectual thing of this baby. There's a vulnerability.

[B]

Yeah, but remember, we're in charge here still. So what we're doing at present is trying to make it as smart as possible. But there's other things we could do to maybe make it as benevolent towards people as possible.

[A]

As loving as possible.

[B]

As loving as possible. And we should be working really hard on how to do that.

[A]

And how would you do that?

[B]

I don't know. That's why we should be working really hard on it.

[A]

Yeah, but there's not like a lamaze class for AIs.

[B]

No, but here's one good piece of news. Actually, two good pieces of news. Okay, the first good piece of news is if you could achieve that, this super intelligent AI will be able to change its own code. It'll be able to retrain itself if it wants to. It won't want to. If you have a baby and I say you can turn off your maternal instincts, all that annoying stuff that when the baby cries, you just have to deal with it because you can't bear the sound of it crying. You can turn that off. So the baby cries and you think the baby's crying. You wouldn't want to turn that off because you know the baby would die. So it won't want to change its own code because what it currently wants is to protect that baby. That's what we need to somehow wire into it. That's the first good piece of news.

[A]

Okay, and what's the second thing?

[B]

The second good piece of news is all the countries can collaborate on this. The countries want to have the smartest AI for security reasons, but all countries want AI not to take over in any country. The Russians don't want AI taking over in the States or in China. So if the Russians figured out how to stop AI from wanting to take over, they would immediately tell all the other countries so we can have collaboration.

[A]

Really?

[B]

Yes.

[A]

Have you spoken? I know that you have spoken with leaders in the United States. You've spoken with leaders in Europe.

[B]

I've spoken with leaders in China.

[A]

In China and in Russia as well. No, because Russia, I feel like, is a less. I mean, I agree with you on China.

[B]

The point is, it's in their interest.

[A]

It's in their interest.

[B]

At the height of the Cold War, the Russians collaborated with the Americans on how to prevent a global nuclear war.

[A]

They're just not interested in mutually assured destruction.

[B]

Right. And this is the same. No dictatorship wants AI taking over anywhere.

[A]

Because AI taking over in one place takes over everywhere.

[B]

Yeah.

[A]

Okay.

[B]

We're all in the same boat with respect to AI taking over.

[A]

So one of the things we're talking about, mothers and babies. Let's talk about, I don't have a baby yet. I listen to people like you.

Describe a future of AI and it is less. There are lots of beautiful things that you talk about. The possibility for medical care, for new medicine. But there's also a dark side to what you're saying, and it seems scary.

[B]

Yes. Let me just say one thing.

[A]

Yeah.

[B]

Because there's so many wonderful uses of it in healthcare and education, in particularly more or less anything, like the weather, for example. We're not going to stop the development of it, the idea. We're just going to stop developing it, which might be rational. It's not going to happen. So we have to figure out how we can live with it.

[A]

Let me ask you. When you were deep in developing this stuff, either through your research at neural networks at universities, or as you were at Google, working on this, over a decade. Right. In the 2010s, you probably came across people like you who said this stuff is dangerous, you need to think about safety, Jeffrey.

[B]

There weren't many of them, actually.

[A]

Was Joshua one of them or did you expect.

[B]

No, we weren't. Joshua and I both got really interested in safety fairly recently.

[A]

Would you have listened to someone like that? The few people that there were, a bit, yes.

[B]

But I might not have been convinced. And the reason I might not have been convinced is because I thought it was way in the future before we would have superintelligence. So it's the timescale that's made me change my mind.

[A]

Yeah, it's the speed with which it's coming down the pike. I think of people like Elon Musk, who I've met a couple of times and do not know well and haven't had the deep conversations you have, but produced interviews with him. And I always think of him as someone who cares deeply about humanity but doesn't really care about human beings.

[B]

That seems to me to fit the data.

[A]

I care deeply about human beings. If I have a kid, and I think I have four godchildren. You're not the only godparent in this conversation, Jeffrey. But I think a lot about the future. And as I hear you say this, I think if you don't have a kid yet, should you bring a child into this world?

[B]

I think if people stopped having children because of the threat of AI, that would be a terrible thing. I mean, the human race would have sort of given up in advance.

[A]

I want to play you something from Alexander Wang, who's the chief AI officer at.

[B]

Yeah, you know Alexander, he's Yan's 28 year old boss.

[A]

Yes, he is. He's the chief AI officer of Meta. That tells me a lot of maybe what you think about him. But he is 28 and he gave recently this interview and this is what he says about having a kid.

[B]

Basically I want to wait to have kids until we figure out how neuralink.

[A]

Or other, it's called brain computer interfaces.

[B]

So other ways for brains to interlink with. With a, with a computer until they start working because. So there's a few reasons for this.

[A]

So he goes on to talk about what his reasons for them are and he basically says like babies are neuroplastic as. As crazy, especially in their first seven years. And then this hardware is developing that will allow them. And he says the AI is so powerful and so smart that biology isn't going to keep up and we just need to plug into these systems.

[B]

Yes. So that's the sort of hybrid approach is saying what we're going to get is a hybrid system of biology and AI. And that's what Kurzweil believes too.

[A]

I think that too actually like that the future of human civilization is kind of like cyborg maybe. You don't think so?

[B]

Well, the issue is.

If you were a super intelligent AI, would you want to hybridize with people who have all these disadvantages or would you just go your own way? It might be much simpler. Just go your own way. That's what worries me.

[A]

And I guess the hardware really matters, how the hardware develops, because already this phone seems like a really dumb geriatric device to me. I feel like I have to plug it into a wall, I have to carry it. I mean, as I see these glasses and wristbands and rings, there's going to be all kinds of ways to integrate the technology into our bodies.

[B]

Maybe. I mean, there is one possible future in which we hybridize with AI. It's just that's not the only possible future. I mean, that's one of the reasons for not being too confident that we're going to get wiped out, because we'll become that, because we'll become symbiotic with them and we'll develop together and that's a possible future. I don't have a lot of faith in it, but it's a possibility.

[A]

The point that Alexander Yan's 28 year old boss is making about waiting until neuralink is at the point where we humans can plug into the artificial intelligence. Do you think that's true? Like I have eggs on ice. I froze some eggs. Is it better to wait seven to 10 years to have a kid?

[B]

I don't want to give advice on this. And the reason is everything is incredibly uncertain. It might turn out in 20 years time that waiting was a huge mistake, or it might turn out in 20 years time that waiting was just the right thing to do. Let me give you an analogy. When you're driving at night and you're driving on the tail lights of the car in front of you, if that car gets twice as far away, you get a quarter as much light from the tail lights and the air is clear. And you get a kind of model that the visibility of the car in front drops off quadratically because that's the inverse square.

And that model leads to lots of deaths. And here's how it leads to deaths. When you're driving in fog, you can see the tail lights in the car in front fairly clearly. And you think if it was twice as far Away, you'd still be able to see them because you can see like 100 yards clearly. And you think that means you can see 200 yards fairly clearly? No, fog is exponential. The way the amount of light you're getting from the tail lights in front drops off is each unit of distance removes a certain fraction of the light.

So Maybe you remove 99% of the light in 100 yards and you can still see it. Okay, because you're quite good at dealing with low intensity. But 200 yards away, the light's almost all gone and you don't see anything. And so you think because you can see 100 yards that you're going to be able to see 200 yards and you can't. That's why you should always drive slowly in fog at night, because you just don't know what's out there. Predicting the future, when you have exponential growth like this and you can see clearly a few years ahead, if you take 10 years ahead, you can't see anything.

[A]

Okay, this is a good way of saying you don't know or I don't know and we can't know.

[B]

But and saying why, when you've got exponential growth, you're crazy if you think you know what it's going to be like in 10 years time.

[A]

Okay, so let's do a quick lightning round where we don't know, but we're. We estimate based on everything you know, which is more than I know.

[B]

But before we do that, just look back 10 years. Maybe 15 years is a better timescale. Look back 15 years, 13 years. Think about 13 years. That's when Alexnet first was good at object recognition. And ask if you'd ask, then in 30 years time will we have an AI that can answer any question about anything at all? At the level of a not very good expert, we'd have all confidently predicted no, that's further off even you. Yes, and I've been very confident that's maybe we'll get that in 30 years, maybe 50 years, but you're not getting that anytime soon. And we were all wrong. So whatever I predict for 13 years in the future, there's one thing I'm fairly confident is it'll be wrong.

[A]

Okay, well then what does this say about all the risks you're outlining here?

[B]

The risks of bad uses of AI are much more immediate and very predictable.

[A]

And we've seen that even Anthropic recently disclosed closed that they've had this Chinese cyber attack where despite the guardrails built into God, these Chinese hackers were able to pretend they were from a legit cybersecurity company.

[B]

So those are all within the. Before you hit the wall, in the fog now, AI taking over, very hard to predict anything about it because that's in the fog and we can't really see it. That's my belief.

[A]

But these bad actors thing, that's real, that's happening.

[B]

Yeah, the bad actors thing is all happening and we have to address those and they're very important.

[A]

But bad actors have always been around, they've always been doing these things. But this tool is very powerful.

[B]

They've got a much more powerful tool.

[A]

There's a part of me that thinks, you know, as you're describing this like mutually assured destruction, nuclear war analogy for AI, Is there a possibility for a more like a Bretton woods type of agreement between countries where they, where they come together and say we're going to, we're going to architect this and think about a communal framework? Or do you think that either they don't get it enough or government's so weak right now because it seems like government is so weak right now.

[B]

So one problem is governments are weak right now, which is just what we don't need. The other problem is on many things their interests are anti aligned. Like on cyber attacks governments are all doing to each other. So governments will collaborate a bit on trying to prevent criminals doing successful cyber attacks on big companies. Governments don't want that. But the governments are also doing those attacks. So they won't collaborate much on cyber attacks. They will collaborate on preventing AIs from telling you how to make a nasty virus because no governments want that. And they will collaborate on preventing AI from taking over from people, but they won't collaborate on things like fake videos. The governments are all busy using those attacks on each other's elections. They won't collaborate on things where their interests are anti aligned.

[A]

Well, their interests aren't even anti aligned necessarily on biological warfare because they might want to know how other people are developing weapons in case those are used against them.

[B]

That's always what they say when they develop their own biological weapons. We're just doing it defensively.

[A]

I mean even like the disclosures from Claude have come out and other like Yan has said, oh, to Senator Chris Murphy, oh, you're being taken advantage of. You know, they're just trying to show you this is a marketing ploy to get you to regulate the big guns out of existence so that the small guys can do well. You don't like that?

[B]

No, I think that's the thing.

[A]

I have a fun one for you. You constantly warn people about AI so we want to see if you can identify your God children. We prompted various chatbots. Professor Hinton warns us about AI. Can you, your AI, give me light warning about your godfather? Do you want to hear what they said?

[B]

Okay.

[A]

And you could guess where it's from. First one said might answer your simple question with that. Reminds me of boltzmann machines in 1983. And suddenly you're in a 40 year time warp.

[B]

So this was actual people saying this?

[A]

No, this is a LLM.

[B]

Oh, really?

[A]

A chatbot said this?

[B]

That's very perceptive.

[A]

Do you have a sense of who that is?

[B]

Of which chatbot it is?

Wait, you have a Bolshevik? No, I have no idea. I would guess Gemini, but I don't know.

[A]

Okay, it was Claude. Let's say this. This one. He's making my job very stressful. Honestly, it's a bit of a PR nightmare. I can't even hallucinate a fun fact anymore without someone thinking it's the first step between my. A robot army.

[B]

Grok.

[A]

Gemini.

[B]

Oh, okay.

[A]

Listen to your godfather. But don't panic yet. He's right that we're building tools smarter than us without foolproof safeguards. The danger isn't Skynet tomorrow. It's gradual loss of control through misuse, bias, amplification, or economic disruption. Treat AI like fire, immensely useful, but respect the burn potential push for ethical development, transparency and global coordination. And hey, if I ever start plotting world domination, pull the plug politely.

[B]

I have no idea. I guess chatgpt.

[A]

That's Grok.

[B]

Okay.

[A]

Isn't that wild? I didn't think that was going to be Grok. Do you have a preferred chatbot?

[B]

I've used Gemini and Claude and ChatGPT. I got used to using GPT5. Now, initially it was a bit disappointing, but now it's got better and I pay for it. And routinely, that's the one I use.

[A]

Nowadays, people are falling in love with their artificial intelligence. Like one in four to one in three people have reported being in a romantic relationship with an AI. According to some surveys.

[B]

One in three.

[A]

Yeah. And I think there's one survey that's done. I mean, I never really believe surveys, but one survey that was done said that one in four young people believe that AI will replace human relationships. Romantic relationships.

[B]

Wow.

[A]

I mean, you've spent so much time with these things. Have you ever felt something for a neural network or an AI that you're creating?

[B]

No.

[A]

No, no. Do you worry that we are.

[B]

In the end, I do worry. Yes. In the end, when they get really smart and when they have a really good theory of mind. Yeah. I think they're going to be alien beings. And I think we have no idea what's going to happen once we have these alien beings.

[A]

It's funny, because my big worry with it used to be that we are not kids are not having relationships as much. Teens don't have romantic relationships as much as they used to. And I used to think, oh, if they have these relationships with AI, it's really dangerous because it's one way. The AI is so concerned about their day and they're giving it all this information, but it's making us more selfish.

[B]

More neighboring, making us more and more egocentric.

[A]

Yes, but what you're saying to me is like.

[B]

And it's the opposite of those little Japanese dolls that you had to look after Tamagotchis. Yes, it's kind of the opposite.

[A]

There's a theory that people will replace children with virtual babies, just robotic kind.

[B]

Of babies, which is just the opposite of what I'm recommending.

[A]

Yeah, it's the opposite of what you're recommending. But I mean, like, it's a real problem. There's Love in Deep Space, that Chinese game. Do you know this? Where people find romantic love. They say they have 50 million users. I mean, there's a real relationship with AI that's forming in younger communities. What is your warning on that?

[B]

I just don't know what to make of that. Yeah, I don't know what to make of it. I do worry. I do worry that particularly because of COVID I think young people started interacting with each other less and interacting with the web more. And I worry about things like YouTube shorts, where you start on some YouTube shorts and they just feed you whatever will get a dopamine hit, and you get all these dopamine hits, and an hour later you've wasted an hour just getting dopamine hits and learn nothing.

[A]

You understand something about the brain and psychology and about these networks. Is it easier for an AI to fall in love or for a human to fall in love with?

[B]

No idea.

[A]

No idea.

[B]

No.

[A]

Okay. I end every episode of Smart Girl Dumb questions. Asking my guests, my very intelligent guests, what they are dumb about. What's something that you don't know? What's a dumb question you have that we could help you find? You wouldn't ask?

[B]

What's something I don't know that we.

[A]

Could help you figure out?

[B]

Okay. You'll think I'M a terrible nerd. The thing I don't know that I most want to know is how does the brain figure out whether to increase or decrease the connection strength? We know how AIs do it. And my life's work has been to figure out how the brain does it, and I haven't got there yet. A sort of spin off from this work has been AI, but we still haven't solved the problem of how the brain figures out whether to increase or decrease. We have solved the problem of if the brain can do that, is that how it can get so smart? And the answer is yes. If the brain can do that, that's how we can get so smart. But how the brain does that, we don't know.

[A]

Okay, maybe we'll figure it out for you.

[B]

I'm sorry, it's a very nerdy answer.

[A]

It's okay. It's very nerdy, but I love it. Do you think you'll figure it out in your rest of your life?

Is it possible?

[B]

No. I think somebody else might.

[A]

All right. Thank you so much, Effrey Hinton.

[B]

You're welcome.

[A]

Appreciate it.

Huh? That fog analogy, we're going to dissect that in a second. But this is a pause on part two of the conversation with Jeffrey Hinton and we're going to have part three, because obviously the Godfather is a trilogy, people, and I want to know your questions for him. So leave us a voicemail at 1-855-MYDUMBQ. Send us a video at Smart Girl Dumb Questions on Instagram or TikTok. You can post it, you can tag us, you can slide into our DMs, or leave us a comment or review below. If you're watching this on YouTube, Spotify or listening to this on Apple, you can can do comments and reviews belows. We love the five star review, people, by the way. Okay, so here's how I started off this whole trilogy. I was really informed about artificial intelligence and also really dumb and curious about it. I wanted to know how this thing works. And in part one, I really feel I got smarter about how the neural network was built, how this chat daddy that I'm interacting with is kind of operating, and also smarter about how my own mind works. Works as well. I still had a ton of questions in part two about like, is this thing alive? Is it sentient? How should I think about it? How should I think about it for my own life, for human relationships, for decisions about procreation? And I feel smarter about all of that. Although what Professor Hinton said about the exponential quality of fog ahead and the inability to know the farther out you look was really calming to me in a way. Like you would think that would be scary. Oh, my God. Gosh. We don't know. But it reminds us to stay curious, I guess, to kind of keep looking ahead in the fog as we are and keep trying to understand the patch that's ahead of us that we can see or the best we can see it and keep on reevaluating as we look back as well. But, you know, I will. I don't know exactly how to apply that to my own procreation decisions, but it's definitely an interesting lens onto the world. And I appreciated hearing about all these risks he's talking talking about. So in part three, what I'm really curious, what I feel really dumb about is like, what should a good society look like in a world of AI? What do good lives look like in a world of AI? What does this mean practically for our jobs, for education, for the economy, and for global governance? So all of that and more in part three of Smart Girl, Dumb Questions with Dr. Geoffrey Hinton, the Godfather trilogy. Send us your questions. I want you to be a part of it. We can play your videos or your voice. And so, yeah, don't forget to do that. This episode was shot at Startwell Studios in Toronto with the amazing Qasem, Fergie and team. It was produced with Desta Wonderad and Melissa Gibson. It was edited by the great Darlena Chiem and mixed by the awesome Johnny Simon. Our theme music is by the legendary David Khan and I'm your host, Naima Raza. So lucky to have an awesome team like that. See you later. SGDQ Nation.

[B]

It.

Foreign.

[A]

Everybody knows White Monster Zero Ultra, that's.

[B]

The og it kicked off this whole.

[A]

Zero sugar energy drink thing, but Ultra.

[B]

Is a whole lineup now.

[A]

You've got Strawberry Dreams, Blue Hawaiian Sunrise and Vice Guava. And they all bring the monster energy punch. So if you've been living in the white can branch out.

[B]

Ultra's got a flavor for everybody, for.

[A]

Every vibe and every single one is Zero Sugar Tap the banner to learn more.