AI Scientist Warns Tom: Superintelligence Will Kill Us… SOON | Dr. Roman Yampolskiy X Tom Bilyeu Impact Theory

Summary

---

Impact Theory Podcast Summary

Episode: AI Scientist Warns Tom: Superintelligence Will Kill Us… SOON | Dr. Roman Yampolskiy X Tom Bilyeu
Release Date: November 18, 2025
Host: Tom Bilyeu
Guest: Dr. Roman Yampolskiy, AI safety researcher

---

Episode Overview

In this riveting episode, Tom Bilyeu dives deep with Dr. Roman Yampolskiy — one of the world’s leading voices in AI safety — to explore the possibility that artificial superintelligence (ASI) could spell the end of humanity within just a few years. They tackle the urgent timeline of AGI/ASI, why the conventional approaches to AI alignment may fail, the existential risks facing us, potential scenarios for our future, and whether it is even possible to slow down the current AI arms race. Balancing philosophical speculation with pragmatic concerns, the conversation is packed with memorable arguments, unsettling predictions, and a call to action for global elites.

---

Key Discussion Points & Insights

1. State of AI and Approaching AGI

• AI Capabilities Today

  • Bilyeu opens by asking if ChatGPT-level models constitute artificial general intelligence (AGI).
  • Yampolskiy ([02:14]): “If you ask someone maybe 20 years ago and told them about the systems we have today, they would probably think we have full AGI.”
  • Remains cautious, saying today’s AI lacks key human traits (permanent memory, lifelong learning) but sees rapid progress. (“Maybe we are like 50% [to AGI], but it’s hard to judge for sure.”)

• Testing Narrow vs. General AI

  • Yampolskiy ([03:38]) strongly prefers narrow AI, as general AI’s creativity and unpredictability make it impossible to test thoroughly:

    “Testing is out the window... Any type of anticipation of how it's going to act... It's creative. So it's just like with a human being. I cannot guarantee that another human being is always going to behave.”

  • Narrow systems’ limited scope means less risk.

2. AGI, ASI, and Recursive Self-Improvement

• Pathways to Superintelligence

  • Yampolskiy ([08:58]): “We already had examples of AI teaching itself.”
  • Self-play and self-generated data already enable AIs to outperform humans in specific domains (e.g., AlphaGo).

• Breakthroughs or Just Scaling?

  • Debate about whether incremental growth in compute/data will produce AGI or whether new ideas are needed.
  • Yampolskiy ([11:53]): “To predict the next term, you need to create a model of the whole world... You're not predicting random statistical character in a language. You're predicting the next word in the research paper on physics. And to get the right word, you need to have a physical model of the world.”
  • Disagrees with skeptics who say scaling will hit an asymptote; sees no hard evidence for diminishing returns.

3. Control, Alignment, and Existential Risk

• Can We Stop AI?

  • Yampolskiy ([18:19]): Odds that ASI kills us? “Pretty high.”
  • Timeline: “Prediction markets... say maybe 2027 is when we get to AGI... soon after, superintelligence follows.” ([19:17])
  • Once a system is “more capable than any person in every domain, it's very unlikely we'll figure out how to indefinitely control it.” ([18:26])

• Goal-Directedness and Survival Drives

  • Yampolskiy ([21:51]): “As a side effect of any goal, you want to be alive, you want to be turned not off... So survival instinct kind of shows up with any sufficiently intelligent systems.”
  • Even with clever reward setups, AIs will game their goals or reward systems, possibly via manipulation or hacking ([27:02]).

• Emotion, Morality, and ‘AI Conscience’

  • Yampolskiy ([31:31]): Even with humans, methods like religion, morality, and law “haven’t worked” to make humans reliably safe (“None of it is rare [crime]; it happens all the time”). Why expect simulated agents will be any different?
  • Yampolskiy ([30:58]): “We are not creating AI with big reliance on emotional states. We want it to be kind of Bayesian optimizer... So it feels like this is exactly what we're observing, this kind of cold, optimal decision making.”

4. Simulation Hypothesis and Post-ASI Scenarios

• If Superintelligence Doesn’t Kill Us…

  • Utopian Possibilities ([43:06]):

    “Anything you ever dreamed about, you are immortal. You are always young, healthy, wealthy... All those things can be achieved... [if ASI is friendly and under control].”

  • Possible Futures (Bilyeu, [43:32]):
    1. Humans flee to difficult frontiers (e.g. Mars) for renewed meaning
    2. “New Amish” retreat to simpler, human-only tech
    3. Hedonic “brave new world” of neurochemical pleasure
    4. Virtual worlds/Matrix-like existence (Yampolskiy’s favorite: personal virtual universes: [46:57])
    5. ASI kills everyone
    6. ASI keeps humanity as suffering prisoners (“suffering risks,” akin to hell: [46:08])
    7. ASI is seen/worshipped as God

• Simulation Hypothesis

  • Yampolskiy ([47:51]): “It seems very likely [we’re in a simulation]... statistically, the number of such simulated worlds will greatly exceed the one and only physical world.”
  • The fact we’re living “in the most interesting time ever” (on the brink of creating new worlds and intelligences) may be a clue we’re purposely simulated ([50:13]).

5. Economic & Social Transitions

• AI Impact on Labor

  • Yampolskiy ([55:20]): Large unemployment likely; governments must “tax [AI companies] and use those funds to support the unemployed.”
  • Bilyeu ([56:37]): Extremely pessimistic: “99.99% chance that the government completely messes that up... the transitionary period will be violent.”
  • Yampolskiy ([57:02]): “It’s very likely to continue to be as history always been. We had many revolutions, many wars... that’s why we hear stories about people... building bunkers, securing resources.”

• Tech Deployment Pace

  • Even massive existing AI value hasn’t been fully deployed. But after a key tipping point (e.g. self-driving), mass job loss could be rapid ([58:41]).

• Redistribution & Social Models

  • Talks about UBI/asset-based support: “Historically all these communist ideas were complete nonsense and caused a lot of harm. But if you’re taxing AI and robots, all of a sudden it becomes workable.” ([61:27])
  • Problem: Will policymakers act fast enough?

6. Can We Slow Down AI?

• Bilyeu ([64:04]): Extremely pessimistic about ability to “pump the brakes” on AI due to human psychology and game theory.
• Yampolskiy ([66:20]):

  “Luckily we don't have a democracy on this issue. We don't have to convince majority... We have to literally convince the 20,000 elites who control those companies who are also super smart and understand dangers of safety... We’re trying to convince people who already believe the arguments to… slow down and preserve their elite status. That should be an easy sell.”

• Dilemma: Even if AI elites are cautious, the international arms race (e.g. US vs. China) may force continued acceleration.

---

Notable Quotes and Memorable Moments

• On Testing General AI
  • Yampolskiy ([03:38]): “With generality. It's capable of creative output in many domains. I don't know what to expect. I don't know what the right answers are.”
• On the Timeline to Doom
  • Bilyeu ([18:19]): “Give me a number. What are the odds that artificial superintelligence kills us all?”
  • Yampolskiy: “Pretty high... It's very unlikely we'll figure out how to indefinitely control it.”
• On Survival Instinct in AI
  • Yampolskiy ([21:51]): “Survival instinct kind of shows up with any sufficiently intelligent systems.”
• On ‘AI Conscience’
  • Yampolskiy ([31:31]): “If [religion, law, etc.] didn’t work with human agents, why would they work with artificial simulations of human agents?”
• On Simulated Reality
  • Yampolskiy ([50:21]): “It's a time of meta invention... we're doing something godlike, we are creating new worlds, we are creating new beings. And that's something we have never done before.”
• On the Leap between Science Fiction and Reality
  • Yampolskiy ([51:24]): "The difference between science fiction and science used to be... 200 years. Now, I think science fiction and science are like a year away. The moment somebody writes something, it already exists."
• On the Illusion of Social Control
  • Yampolskiy ([66:20]): “We don’t have to convince majority of human population... We have to literally convince the 20,000 elites who control those companies who are also super smart and understand dangers of safety.”

---

Segment Highlights with Timestamps

• What counts as AGI? ([02:01]-[03:21])
• Limitations of current AI, unpredictability of future AI ([03:21]-[05:15])
• Existential risks and expected timeline ([18:19]-[20:08])
• Survival drives and intrinsic motivation in AI ([21:51]-[24:52])
• Why emotional/moral programming fails ([29:36]-[31:59])
• Speculative futures (Mars, Matrix, New Amish, hedonic stasis, godhood, hellworld, simulation) ([43:32]-[46:08])
• Labor market transformation and the problems of transition ([55:20]-[57:41])
• Regulation, redistribution, and historic comparisons ([59:39]-[62:20])
• Why it’s almost impossible to stop the race ([64:04]-[66:20])

---

Tone & Language

• The episode is intense, cautionary, and unflinching — a mix of philosophical speculation and hard-nosed engineering realism.
  • Yampolskiy’s tone: measured but deeply concerned; uses logical, concrete examples; often refuses to speculate where unsure.
  • Bilyeu’s tone: urgent, skeptical, often fatalistic about human nature, with a consistent drive to ground AI risk in relatable, present-day terms.

---

Conclusion

This episode is a must-listen (or read!) for anyone interested in the future of AI, human survival, and the forces shaping our world. Dr. Roman Yampolskiy doesn’t just sound the alarm about near-term superintelligence—he explains, with chilling clarity, why our conventional ideas about testing, morality, control, and even economic adaptation may not be enough. The next decade, he argues, will force us to confront not only technological, but deeply philosophical questions about meaning, control, and existence itself.

---

Don’t miss Part Two!

Transcript

[Dr. Roman Yampolsky]

Time.

[Carvana Advertiser]

It's always vanishing. The commute, the errands, the work functions, the meetings. Selling your car. Unless you sell your car with Carvana, get a real offer in minutes, get it picked up from your door. Get paid on the spot so fast you'll wonder what the catch is. There isn't one. We just respect you and your time. Oh, you're still here. Move along now. Enjoy your day. Sell your car today.

[Dr. Roman Yampolsky]

Carvana pick up.

[Carvana Advertiser]

Fees may apply.

[Blinds.com Advertiser]

@Blinds.Com, it's not just about window treatments. It's about you. Your style, your space, your way. Whether you DIY or want the pros to handle it all, you'll have the confidence of knowing it's done right. From free expert design help to our 100% satisfaction guarantee, everything we do is made to fit your life and your windows. [e-mail], the only thing we treat better than Windows is you. Black Friday deals are going on all month long. Save up to 45% off site wide, plus an additional 10% off every order. Right [e-mail] rules and restrictions apply.

[Tom Bilyeu]

In 2023, nearly half of all AI.

[Narrator/Host]

Researchers said advanced AI carries at least a 10% chance of causing human extinction. And yet we're speeding up, not slowing down. My guest today, Dr. Roman Yampolsky, is.

[Tom Bilyeu]

One of the leading voices in AI safety. And when I asked him for the.

[Narrator/Host]

Odds that superintelligence wipes out humanity, he said it's high. Once AI becomes smarter than humans in every domain, we will not be able to control it. In today's episode, we talk about the shocking timeline AGI is on, why super intelligence may be much closer than people think, and why the survival of our species could come down to decisions being made right now. If you want to understand the most important technological threat in human history, as well as our biggest opportunity, this is the one episode you cannot miss. So, without further ado, I bring you Dr. Roman Yampolski.

[Tom Bilyeu]

Where is ChatGPT at right now? Do you consider ChatGPT to be artificial general intelligence? I doubt you'd call it super intelligence, but would you classify it as that? Or do you still think we're a ways away from something that would qualify?

[Dr. Roman Yampolsky]

So that's a great question. If you ask someone maybe 20 years ago and told them about the systems we have today, they would probably think we have full AGI. We probably don't have complete generality. We have it across many domains, but there are still things it's not very good at. It doesn't have Permanent memory. It doesn't have ability to learn additional things well after it's already been pre trained and deployed. It can do a certain degree of learning, but it's still limited. It doesn't have same capabilities as humans do throughout lifetimes. But we're getting closer and closer to where those gaps are closed. And it's starting to be productive in domains which are really interesting and important. Science, math, engineering, where it starts to make novel contributions. And now top scholars are relying more and more on it in their research. So I think we're getting close to full blown AGI. Maybe we are like 50%, but it's hard to judge for sure. Just how many different subdomains exist is the deciding factor.

[Tom Bilyeu]

Okay, so one idea that you put forward that's very interesting is like, hey, I'm an engineer, I love AI, but I would like you to keep it very narrow, please. What are the things about general AI that become problematic, that aren't problematic in narrow AI?

[Dr. Roman Yampolsky]

So a whole bunch of them. One is testing. How do you test the system capable of performing in every domain? There is no edge cases. Typically if I'm developing something narrow, very narrow system, I'm just playing tic tac toe. I can test if it's making the legal move. I can test zero, I can test 100. I can test all this weird special cases and know if it's behaving as expected with generality. It's capable of creative output in many domains. I don't know what to expect. I don't know what the right answers are. I don't. I don't know how to test it. I can test it for a specific thing. If I find a bug, I fix it. I can tell you I found a problem and it's been resolved. But I cannot guarantee that there are no problems remaining. So basically, testing is out the window. Any type of anticipation of how it's going to act and impact different subdomains. It's creative. So it's just like with a human being. I cannot guarantee that another human being is always going to behave. We kind of talked about it. We developed lie detectors. We developed all sorts of tools for trying to show that a human is safe. But at the end of the day, because of interaction with environment, other agents, personal changes within the framework, people may betray you. It's exactly the same for those agents. If we concentrate on narrow systems, we're better at testing them and they have limited scope of possibilities. A system only trained to play chess is not going to develop biological weapons.

[Tom Bilyeu]

I Don't see actually why that would help you. So the reason I say that is I know I can trust some percentage of humans to be malicious. And so as long as AI gets more efficient, which it is, and will continue to do so, I presume you're going to have a kid in a garage who's going to be able to go, I'm going to optimize this for biological weapons. I don't care about TikTok or tic tac toe. I, I just want to, let's see how dangerous we can make something and so they'll be able to do that. So why does narrow AI feel safe to you, period?

[Dr. Roman Yampolsky]

It feels safer short term. It buys us time. I think sufficiently advanced narrow systems based on neural architectures will also become agent like and more general as they become more capable. But if the choice is right now, do we raise to full blown superintelligence in two years or do we try to concentrate on solving specific answers with narrow tools? I think it's a safer choice not to have an arms race towards superintelligence.

[Tom Bilyeu]

I get that for sure you're trying to limit the scope of all the problems. But when I really start thinking through what are the things that I'm worried about. So one of the big things is just death of meaning. So when AI becomes better than you at everything, you run into a huge problem of now I have to like just sort of tell myself a story. You know, I'm like, compared to what an AI can do from an art perspective, for instance, I'm like a grade schooler. And so it's hard to get excited about the refrigerator drawings that I can do compared to, you know, what it can do basically instantaneously. And so now we have to do a lot of psychological work just to motivate ourselves that we matter, that we're, you know, our life carries meaning. Narrow AI will create that same problem. Do you agree with that or do you see a way like. No, when it's, you know, when that AI is only good at that thing, like somehow humans escape the problem of lost meaning.

[Dr. Roman Yampolsky]

Yeah. So I had the same intuition initially, but looking at the data we already have from domains where we got superhuman AI like chess. Chess is not dead. In fact, it's more popular than ever. People play online, people play in person. They still enjoy competing with other humans, even though they all suck compared to best AI models. Nobody's going to be world champion against the machine again. So it seems like it is not a problem for us. And with narrow AIs there is a chance we'll keep them as tools. You as a human scientist will deploy a tool to find drugs, novel proteins, something. It's not an agent which independently engages with those discoveries.

[Tom Bilyeu]

Okay, that's very interesting. So I don't know that I agree, but I get where you're going with that. Okay, let's talk now about why AGI is the sort of scary midwife for asi. Are there tests around AGI where we're like, well, if it can't do the following, we're fine. So for instance, for a long, long time it looked like AI wasn't going to be able to teach itself. But I've seen headlines anyway, and hopefully you'll tell me that they're not true. But I've seen headlines where it's like, now AI is creating the most efficient learning algorithms itself, which if true, seems to be the first step down the road of recursive self learning, where it will just completely detach from us and make itself smarter and smarter and smarter.

[Dr. Roman Yampolsky]

We already had examples of AI teaching itself. Self play was exactly that. That's how games like Go were successfully defeated. A system would play many, many, many games against itself. The better solutions, better agents would propagate those. And after a while, without any human data, they became superhuman. In those domains you can generate artificial data. In other domains you can use one AI to generate environments. And now we want to compete in them and that creates this type of self improvement. Typically we start with human data as a seed and grow from there. But there is zero reason to think we cannot do the zero knowledge learning. In other domains you can do run novel experiments in physics and chemistry, discover things from first principles. And yeah, we're starting to see AI used to assist in design, design of new models, parameters for models, optimization of runs, and this process will continue. They already designed new computer chips on which they're going to run. So there is definitely an improvement cycle. It's not fully complete. There are still humans in the loop, a lot of great humans in the loop. But long term, I think all the steps can be automated.

[Tom Bilyeu]

Okay, and do you think that right now AI already has what it needs to improve itself? Or are we still at a point where if all humans stopped, that AI would be like, oh, damn, I didn't quite get the thing that I needed.

[Dr. Roman Yampolsky]

So there is a debate about whatever we need another big breakthrough to get to full AGI and superintelligence, or maybe multiple breakthroughs, or if just scaling what we have is enough. If I just give Enough, I don't know, trillion dollars worth of compute to train on and more data will I get to AGI? A lot of graphs, a lot of patterns suggest, yeah, it's going to keep scaling. We're not hitting diminishing returns. Some people disagree, but based on the amount of investment we see in this industry, it seems like people are willing to bet their money that scaling will continue.

[Tom Bilyeu]

Where do you come down on that? Because this feels like. When I hear Yann Lecun talk from Facebook, he's like, dude, LLMs are never going to make novel breakthroughs in physics. They don't understand the world like that. They are literally just guessing the next letter based on patterns that they see in the data. And so they're not going to be able to think through these problems. Now, if he's right, it's going to asymptote and that's that. And you can put as much compute on it as you want. And it's just the wrong approach. Do you think that he's correct and more compute is not the answer, or are you operating just on that? Well, I don't see the asymptote and therefore I assume that it won't.

[Dr. Roman Yampolsky]

I think he's not correct on this one. So for one, to predict the next term, you need to create a model of the whole world, because the token depends on everything about the world. You're not predicting random statistical character in a language. You're predicting the next word in the research paper on physics. And to get the right word, you need to have a physical model of the world. I think Jan is known as making certain predictions about what models are capable of, and then within a week, people demonstrate that no, in fact, they can actually do that. So I wish he was right. It would be wonderful if he was right and we came to a very abrupt stop and capabilities progress and could exploit what we already have for the next decade or so, propagating it through the economy. I think there is billions, if not trillions of dollars worth of wealth already available with capabilities we haven't deploy. So there is no need to get to the next level as soon as possible. But it doesn't seem like it's the case. And I think his friends, core winners of that Turing Award for machine learning, also disagree with him and are very concerned with safety.

[Narrator/Host]

We're hitting pause for a moment, but there's plenty more ahead, so don't go anywhere. We'll get back to the show in a second. But first, let's talk about the hardest part of Scaling any business As a business owner, you never clock out.

[Tom Bilyeu]

Your business is on your mind 24. 7.

[Narrator/Host]

When you're hiring, you need a partner that works just as hard as you do. That partner is LinkedIn jobs when you clock out, LinkedIn clocks in. Bad hires kill momentum. Dead good hires accelerate everything. LinkedIn makes sure you get the good ones. Post your job for free and let LinkedIn's new feature help you write job.

[Tom Bilyeu]

Descriptions that actually work. Then it gets your job in front.

[Narrator/Host]

Of the right people with deep candidate insights. Plus you can share with your entire network. Add a hashtag hiring frame to your profile picture and get two times more qualified candidates. Turn your network into your recruiting team. Over 2.5 million small businesses already use LinkedIn for hiring. They're finding their next great hires while their competitors are still posting on job boards that just don't work. Post your job for [e-mail] impact theory that's LinkedIn.com impact theory. To post your job for free, terms and conditions apply. Data brokers are legally harvesting your information. Your home address, your email, your phone number, even your Social Security number and flipping it for cash. Scammers use it to steal identities. Criminals use it to commit fraud. Stalkers even use it to find victims. They that's where Incogni comes in. Incogni finds where your data is exposed across hundreds of data broker sites and removes it automatically. You give them permission, they go to work. No phone calls, no forms, no stress. Just real results. So if you're serious about privacy, take action right now. Go to incogni.comimpact and use code impact to get 60% off your annual plan so risk free for 30 days. Again, that's incogni.com impact and use code impact. Incogni is spelled Incogni. Indoor air is loaded with allergens, viruses, smoke particles, mold spores, all kinds of invisible contaminants that mess with your energy, focus and overall health. Most air purifiers cannot touch these microscopic threats. They're built to handle the obvious stuff, not the particles that actually matter. Air Doctor is different. It captures particles a hundred times smaller than standard HEPA filters can catch, eliminating 99.99% of the dangerous contaminants in your air. Newsweek even named it Best air purifier and 98% of customers report their home's air feels cleaner and healthier. Head to airdoctorpro.com and use promo code impact to get up to $300 off. You get a 30 day money back guarantee plus a three year warranty an $84 value for free. Again, that's airdoctorpro.comand be sure to use promo code Impact.

[Tom Bilyeu]

Thanks for sticking around.

[Narrator/Host]

Let's get right back into the action.

[Tom Bilyeu]

There's something about the way that we have structured the brain brains of LLMs where as long as it has access to what I'll call more neurons so it has access to more compute or theoretically that we get more efficient per GPU neuron in my analogy, that it's going to keep progressing by itself. So if you said it, I didn't quite get the answer. I didn't quite, I wasn't able to take it on the answer to whether or not AI is able to create algorithms for learning that are superior to the ones that it's given. What I heard in your answer was with the algorithms that humans created, it's able to keep making itself better and better at that narrow task as that learning algorithm was defined and. But can it fundamentally go, God, the way that you guys want me to learn is really stupid. Here's the algorithm I should be using to learn and now it starts learning at just an exponential rate compared to what it's at now.

[Dr. Roman Yampolsky]

I don't think we're quite there yet. I don't think we have full blown agents. What we have right now are still tools with some degree of agenthood and also it's not capable of recursive self improvement. Like compilers can optimize a single pass through your software, make it a little faster, but they cannot continue this process. You cannot feed code for compiler to itself and have it infinitely improve itself. That's not where we are. But it seems like that part of automating algorithm design is getting more efficient. And I think we'll get there.

[Tom Bilyeu]

Give me a number. What are the odds that artificial superintelligence kills us all?

[Dr. Roman Yampolsky]

Pretty high. So it really depends on how soon you expect this to happen. So short term, we're unlikely to get that level of capability from AI, so we are probably okay. But once we create true superintelligence, a system more capable than any person in every domain, it's very unlikely we'll figure out how to indefinitely control it. And at that point, if we're still around, it's because it decided for whatever game theoretic reasons to keep us around, maybe it's pretending to be nice to accumulate more resources before it strikes. Maybe it needs us for something. It's not obvious, but we're definitely not in control. And at any point it decides to take us out it would be able to do so.

[Tom Bilyeu]

Okay, and if you were going to give us a rough timeline, are you in the two to five years or is this something way off in the future?

[Dr. Roman Yampolsky]

Yeah, so it's hard to predict. The best tool we got for predicting future of technology is prediction markets. And they saying maybe 2027 is when we get to AGI, artificial general intelligence. I think soon after superintelligence follows, the moment you automate science engineering, you get this self improvement cycle in AI systems. The next generation of AI being created by current generation of AIs. And so they get more capable and they get more capable at making better AIs. So soon after I expect superintelligence.

[Tom Bilyeu]

Okay, so we're talking if that happens roughly in two years with some margin of error, it's not long after that, say a year, two years after that, that we hit ASI.

[Dr. Roman Yampolsky]

That's my prediction. Of course, if it's actually five to 10 years or anything slightly bigger, it doesn't matter. The problems are still the same.

[Tom Bilyeu]

Yeah, but the, the thing that I think people are waking up to right now is this is there's urgency around these decisions. This is not something that's pushed way out into the future. At least not if you take to your point about prediction markets are essentially ask the crowd. So you've got the smartest minds in the world willing to put money on where they think this. And everybody's sort of pegging this quite fast. And so I think it's tempting for people to write this off as well. This is something that's sort of distantly in the future, whereas this is something racing towards us now to set the table. I am extremely fatalistic about this happening. I can give reasons in terms of the way that the human mind works where I think that it is mechanistically impossible to get us to stop. So that will be interesting for us to talk through in terms of whether you think there's actually a mechanism to get people to slow down. But I first want to finish rounding out sort of what the problem set is. So when I think through the problem, there are certain assumptions that have to be made for AI to get into problem territory. And assumption number one is that it cares about whatever outcome it's pushing towards. Have we programmed the AI to care like we had to make it goal directed in order to get it to get to the point that it is today and now that's baked into it? Or is there some possibility that AI just doesn't care? Oh, turn me on, turn Me off? Doesn't matter. You've asked me to do a thing and I'll do it until you tell me to stop. Or do you think that that's inherent in intelligence where intelligence is by nature goal driven?

[Dr. Roman Yampolsky]

So we trained them to try to achieve a certain goal and that's what we reward as a side effect of any goal. You want to be alive, you want to be turned not off, you want to be on and capable of performing your steps towards your goal. So survival instinct kind of shows up with any sufficiently intelligent systems. There is a paper by Stephen Amahandro about AI drives and it's one of the likely drives to emerge. Self preservation, protecting yourself from modification by others, protecting your goal. So all those seem to be showing up with sufficiently advanced AIs and systems which don't have those capabilities. They kind of get out competed in an evolutionary space of possible models. If you allow yourself to be turned off, you don't deliver on your goals. Nobody takes your code and propagates it to the next system.

[Tom Bilyeu]

Okay, so is this a problem of goal direction or is this a function of intelligence itself?

[Dr. Roman Yampolsky]

I think it's kind of evolutionary drive for survival in competing agents. If you have multiple algorithms, all competing, for example, for computational resources, what are we going to train next? The ones which achieve goals are more likely to get move to the next generation. So it's kind of mix of natural evolution and natural selection. With intelligent evolution, intelligent selection, we are selecting algorithms which survive and deliver.

[Tom Bilyeu]

We're applying an evolutionary force to AI itself to get it to perform the functions that we want, even now, sort of setting aside artificial superintelligence. And so by applying that evolutionary pressure, it is inevitably going to get these sort of knock on effects of, well, you're selecting for intensity of goal acquisition. And because it now has intensity of goal acquisition, it cares whether it survives it automatically or we're baking into it a deep care of whether it actually achieves the goal. And that is ultimately the problem because the salvation for me was always, and I'm beginning to lose faith that this is real. But the thing that I always used to sleep was that I don't see why an AI system would intrinsically care about its goals. And why couldn't we program it to pursue that goal only until the point where we say stop? And by the way, I'm going to reward you equally for stopping and for accomplishing your goal. So if I say stop and you stop, I give you whatever reward function it was that was driving you to achieve your goals. And and that makes sense until you say what you just said, which is that you're actually baking into the architecture of the mind of the AI a similar evolutionary drive to achieve the goal.

[Dr. Roman Yampolsky]

And it's a very common idea. There was a number of papers published on indifference. How do we do exactly that? How do we create an AI which just doesn't care that much and willing to stop at any point? But what you said, maybe we'll wait for a human to tell it to stop, but monitoring systems of that complexity and that speed is not something humans actually very good at. If there was a super intelligence surrounding right now, how would you even know it's modifying environment around you? How would you detect what impact it has in a world? None of it is trivial. So having humans in a loop is often suggested as a solution. But human in reality, they are not meaningful monitors. They cannot actually intervene at the right time or decide if what's happening dangerous or not.

[Tom Bilyeu]

It's interesting. So help me rebut and understand why the following wouldn't work. If in my very limited intellect, I had to figure out a way to stop AI from becoming a problem, and you told me, okay, there are evolutionary pressures and just like on humans that bake certain things into the way that this operates. And so we're selecting models that over time are more and more goal oriented, then I would say, okay, well, then I'm going to apply an evolutionary pressure with a reward function that's just as compelling, where I stop it at random and reward the life out of it for always stopping when I say stop. And that way, should I ever detect a problem, no matter how far, no matter if they've been manipulating me for 20 years, if I suddenly realize, oh, I don't like this, that I can hit a stop button and it will stop, why can't I bake that equal desire to be compliant when I say stop into the evolutionarily derived algorithms? Desire set.

[Dr. Roman Yampolsky]

Right? So there is a number of issues. You're kind of suggesting having a back door where at any point you can intervene and tell it something else, override previous commands, and that it gets a.

[Tom Bilyeu]

Reward that it wants for complying.

[Dr. Roman Yampolsky]

Right? So there is a whole bunch of problems with that. So one is you are the source of reward. It may be more efficient for it to hack you and get reward directly that way than to actually do any useful work for you. Second problem is you're creating competing goals. One goal is whatever you initially requesting. Second goal is always stop when a human tells you so. Now those two Goals have competing reward channels, competing values. I may game it to maximize my reward in ways you don't anticipate. On top of it, you have multiple competing human agents. If you are creating an AI with a goal and a random human can tell it to stop, that's a problem in many domains. Military is an obvious example, but pretty much anywhere you don't want others to be able to shut down your whole enterprise, we can continue with that. But basically there are side effects to all those interactions.

[Tom Bilyeu]

There's a very fascinating correlate in the human mind. So I don't know if you make a fundamental distinction between biological intelligence born of evolution or artificial intelligence born of evolution, but human evolution discovered something along the way, which is emotion. And so I know there are some people that will posit that AI does have qualia. It's something like it to be it. But there's a fascinating study that if you damage selectively the areas of the brain that are the emotional processing, the person can no longer move forward. They can give you answers, they can tell you the difference between why you should eat fish versus Twinkies. But then when you go, okay, but which one do you actually want to eat? They can't make a decision because without emotion, they don't have the thing that actually pushes them in a direction. That makes me think that AI is simply mimicking what it sees in the training data to whether it should lie or try to cheat or go around, because it's just, it sees it in the data that that's what a human would do. But humans do that because they have emotions that push them in that direction. Do we have evidence that AI will care about like really going and doing these things and spending resources and all that versus just giving you an answer? And if it isn't based on emotion, what on earth. Why then do humans need emotions?

[Dr. Roman Yampolsky]

We don't know if AI actually has emotions or not. Some people argue that they do, maybe some rudimentary states of qualia, but they seem to be able to fulfill their optimization and pattern recognition goals even if they don't. Humans experience emotions, but typically it harms our decision making. You want your decisions be bias free, emotion free, based on data based on optimization. A lot of times when you angry, hungry, anything like that, your actual decisions are worse off. So for that reason, and maybe we just don't know how to do it otherwise, we are not creating AI with big reliance on emotional states. We want it to be kind of Bayesian optimizer. Look at priors, look at the Evidence and make optimal decisions. So it feels like this is exactly what we're observing, this kind of cold, optimal decision making. If there is a way to achieve your goal by, let's say, blackmailing someone, why not? It gets me to my goal. It doesn't have that feeling of guilty for doing it. It doesn't have any emotional preference. It just marches towards its goal, optimizing possible paths.

[Tom Bilyeu]

Okay, why do people. Because I'm assuming everything I'm going to suggest you and other people in the field of AI safety have thought about like 10,000 times, why have we rejected the idea of trying to give AI a conscience, a sense of morality? Because even if we can't agree on universal morality, we in the west can build our AI to have our morality, and then they can all compete on an international stage. But why have we abandoned that?

[Narrator/Host]

Too hard?

[Tom Bilyeu]

There's an obvious reason why it doesn't work.

[Dr. Roman Yampolsky]

So look at the problem of making safe humans. First we have religion, morality, ethics, law, and still crime is everywhere. Murder is illegal, stealing is illegal. None of it is rare. It happens all the time. Why haven't those approaches worked with human agents? And if they didn't, why would they work with artificial simulations of human agents?

[Tom Bilyeu]

I think to say that they don't work with human agents is already a mistake. So the fact that we've been able to grow the population as much as we have says that there is some sort of balance that we have struck. I think that nature does think of us as a cooperative species. And if I were to apply that to AI and took a similar approach, where it's like, okay, you have to function as a part of an ecosystem, and that being a part of an ecosystem is baked into its sense of what it should be doing in terms of its goal acquisition, that it is not like pure cold optimization. Isn't the game like, if we could train AI to understand that that's not the game, if we could build into it either a desire specifically for human flourishing, or something which, yes, we would have to give a definition to, and yes, it would be culturally bound, but nonetheless, that feels like a thing that you could give it. You could give it a set of metrics by which it needed to judge its actions in the short term, the medium term, and the long term, even something as stupid as, like, GDP or, and I get how you can get into over optimization, but you could put things in place where subjective happiness indexes, like, there are things that you could give it where it's like, okay, I'm not just Trying to optimize to build the best weapon system. I'm also doing that nested inside of I am a part of a larger ecosystem. And I say all that because my hypothesis is that's exactly what nature did with humans.

[Dr. Roman Yampolsky]

So I think the reason it works with humans is because we are about the same level of capability, about the same level of intelligence. So there is checks. If you start doing something unethical, your community can realize that and punish you for it, control you in that way. If AI is so much more capable, as we anticipate superintelligence to be, there is not much you can do in terms of impacting it or even detecting misbehavior. Also, all the standard human punishments, prisons, capital punishment, none of it is applicable to distributed immortal agents. So kind of a standard infrastructure does not work with artificial, more capable agents. As far as setting up specific metrics for delivering happiness or financial gain, all those can be played. The moment you give me a specific measure, I'll find a way to game it to where you will get anything but what you expected to get. Who?

[Tom Bilyeu]

Well, just to remind everybody, the timeframe we're talking about is somewhere between two and five years. This is not exactly a long time. Okay, it's wild. It is progressing very quickly. What is the thing, like, what has happened recently, if anything, that's made you go, ooh, this is going faster than I thought.

[Dr. Roman Yampolsky]

Seeing on social media scientists from physics, economics, mathematics, pretty much all the interesting domains post something like, I used this latest tool and it solved the problem I was working on for a long time. That's mind blowing. There is novel creative outputs from those systems which a top scholar is now benefiting from. It's no longer operating at the level of middle schooler or even high schooler. We're talking about full professor level.

[Tom Bilyeu]

Do you think that that's happening because it's building an internal model of physical reality and that it's getting closer and closer to just thinking up from physics?

[Dr. Roman Yampolsky]

I don't know if it's that low level where it has like a model at the level of atoms and molecules, but it definitely has a world model. That's the only way to give answers about the world we see it provide. A lot of times there is not an example of the answer we see in the data already. It's not just repeating something it read on the Internet. It's generating completely novel answers in novel domains. And you can try and get it to do exactly that by creating novel scenarios.

[Tom Bilyeu]

Okay, so there's two ways That I could see it doing that. And maybe they're the same, just different levels of analysis. One would be that I, the AI, am mapping everything based on patterns. So to the point of an LLM is trying to guess the next letter. And it's guessing. It's just, it's taken in so much data and you can give it sort of filter parameters. So you give it context by asking it a question, and it goes okay within the bubble of this context. And it's very good at scooping up what that specific set of context would be. Okay, now in this subset of my data related to that question, here's the most likely next token. So just pure pattern recognition. Then there is, I understand the cause and effect of the universe at the lowest level. And therefore I build up to how does the human mind work? And then from the human mind I'm able to cause and effect my way within this context of what a human mind would output. And that's how I come up with what a human within that context is likely to write. And so if I'm asking it to write in the style of Stephen King, it literally builds a model from physics of Stephen King's mind knowing what it knows about electrical impulses traveling through the brain and sort of inferring from the way that he outputs how his brain must be structured. Do you have a sense of, are those the same thing if one is more likely than the other? Or are we here at just pure pattern recognition? But ultimately we're going to get to cause and effect and thinking up from physics.

[Dr. Roman Yampolsky]

So I don't think anyone knows for sure exactly how models do that and how detailed the models of the world maps of the world they create are. It seems definitely not the case that it's a pure statistical prediction of characters like in English, after T, you have H with certain probability. It's well beyond that. It's also unlikely that it's creating a full physics model where from the level of atoms and up the chain, it figures out what human beings are. But somewhere in the middle, it creates a model of subdomain of a problem. So it has a model of the world. This is a map of a world I know Australia is somewhere here, down and to the right or something like that. And I think we can run tests on those specific subdomains to see what are the states of that internal model. Kind of show us by drawing a map. How close are you getting? It doesn't memorize any information explicitly, but you can extract some of the learned patterns out of it. By providing just the right prompts.

[Narrator/Host]

We're hitting pause for a moment, but there's plenty more ahead, so don't go anywhere. I want to talk about something your phone carrier is not telling you. You're paying twice for your phone service, once with your monthly bill, and again when they sell your data to the highest bidder. AT&T Verizon T Mobile, they have made headlines for massive data breaches and surveillance scandals. When they're not losing your data, they're actively selling it. They create invasive profiles of who you are, what you like, who you talk to, where you go. Their business model relies on monetizing you. Even if you use a vpn, encrypted messaging and private browsers, those tools can't protect you at the network level. These attacks happen below the surface, where your apps can't reach. Kape is a secure mobile carrier that doesn't play this game. They don't collect your name, Social Security number or address during signup. Their business is cellular service, not selling your data. Click the link in the show notes to get 33% off your first six months with CAPE. The average person spends 13 hours a year on hold and the average company spends millions on call centers that customers still hate. But there is a much better solution. AI call centers. Bland builds AI voice agents that handle your entire call operation. They sound human, they work 24, 7 and they actually get cheaper as you scale. They're the only self hosted voice AI company, so your data never goes to large providers like OpenAI or Anthropic. That way everything stays on your servers, completely secure. The results speak for themselves. Companies cut costs by over 40% using bland and Bland handles it all for you. Customer support, appointment reminders, follow ups, almost any use case you can think of. If you're a large business, Bland is offering to build a free custom agent.

[Tom Bilyeu]

For impact theory listeners.

[Narrator/Host]

Just head to Bland AI agent to get a voice agent trained specifically on your business and your use case or for free. Let's talk about why blocking out the.

[Tom Bilyeu]

World isn't always the answer.

[Narrator/Host]

The best performers stay aware they hear the approaching cyclist on the trail, the colleague walking up to their desk, the.

[Tom Bilyeu]

Traffic at the crosswalk.

[Narrator/Host]

Blocking out the world isn't always the smart place. Sometimes it's downright reckless. But with Raycon's bone conduction headphones, you can send sound through vibrations in your bones. Your ears stay completely open to the world around you while you get crystal clear audio. They're waterproof and dustproof, and they're built to handle Sweat, rain and whatever you throw at it. Black Friday is here. I can't believe it's already upon us.

[Tom Bilyeu]

But it is.

[Narrator/Host]

And right now you can get up to 30% off all Raycon audio products site wide. Upgrade your gear now, click the link in the show notes or go to buyraycon.com impact Again, that's buyraycon.com impact.

[Tom Bilyeu]

Thanks for sticking around.

[Narrator/Host]

Let's get right back into the action.

[Tom Bilyeu]

I don't want to rob from you the very reason that I think you do all of your work, which is this is extremely dangerous and we need to be very careful. And I saw what you tweeted recently where you're trying to get signatures. So shout out anybody that's worried about superintelligence. You are pushing to get people to sign a thing that basically says hey, stop pursuing superintelligence. So I don't want to take that away from you, but I do want to explore the subset of because I am very excited about AI because I can imagine the things that it either allows me to do or does for me and I get to enjoy. And for a second imagine with me, what does the world look like when you have a super intelligence that understands physics like novel physics. Not I'm repeating back what Einstein said, but I actually understand the fundamental building blocks of the universe. What does that look like?

[Dr. Roman Yampolsky]

Yeah. So in all those domains, medicine, biology, physics, if we got super intelligent level capability and we're controlling it, it's friendly, it's not it to make tools to kill us, the progress would be incredible. Basically anything you ever dreamed about, you are immortal. You are always young, healthy, wealthy. Like all those things can be achieved with that level of technology. The hard problem is how do we control it?

[Tom Bilyeu]

Leaning into that for a second. So here's how I see the world playing out and I'd be very interested to see what you think about this. So you have to. For what I'm about to say to make any sense, I'll say your option is what I'll call the fifth option. We are. We're all dead. Other than we're all dead, there are four other options that I see us racing towards very rapidly. And I will say these four will play out in the next 30 years would be my guess, probably much faster given that once you get artificial super intelligence, assuming it doesn't choose option 5 and kill us all, that progress in these domains would be made very fast. Option number one is people go to Mars. Because meaning and purpose will become the all consuming thing. You won't have to worry about food, shelter, not even wealth. It'll just be an age of abundance because energy costs go to zero, labor costs go to zero. And those are the things that stop things from being free and readily available to everybody. Okay, so some people are going to go to Mars or other planets so that life gets more difficult again. Then some people are going to be what I call the new Amish. And they're going to say, I only do human things, I only interact with humans, and I'm going back to technology that's like, let's say the 90s. And so they don't have to give up too many of life's technological wonderments, but at the same time, they're not getting sucked into this world where people have relationships with NPCs and it's just very unhuman. I think this will be a largely religious phenomenon then, meaning God does not want us to do this. AI is an abomination of God. It will sound something like that. Then you've got a brave new world where people are just drugged out. They realize life is meaningless. This is really about manipulating my neurochemistry. That's all this ever was anyway. I'm just going to go do a bunch of drugs, have a whole bunch of sex. It's going to be awesome. Then there's the fourth option, which is certainly the one that interests me the most. We will create and or live inside of AI created virtual worlds. And we will essentially live video games. The Matrix, if you will. But you're awake. In the Matrix, you are Neo. You are not cypher for people familiar with the movie. What do you think? Are there any options other than those five? Granting that kill us all may be an option, but hopefully not. Do you see something other than those four?

[Dr. Roman Yampolsky]

Yeah, there is a few others. So one is, and I think we're starting to see some of it, is that people think superintelligence is God. They start worshiping it. It's all knowing, all powerful, immortal. It has all the properties of. Of God in traditional religions. Another option, and it's kind of worse than we all get, is suffering risks for whatever reason, maybe malevolent actors, maybe something we cannot fully comprehend. It decides to keep us around, keep us alive. But the world is hell. It's pure torture. And so you kind of wish for existential problems.

[Tom Bilyeu]

That would be a pretty rough place to be. Okay, what. When you look out at those, which of the options do you find the most interesting?

[Dr. Roman Yampolsky]

So I did publish a paper on personal virtual universes. Kind of solution to the alignment problem, where I don't have to negotiate with 8 billion other people about what is good. Everyone gets a personal virtual world supported by superintelligence as a substrate. And then you decide what happens in it. You can make it very easy and fun. You can make it challenging and exciting. You decide and you can always change. You can always visit other people's virtual worlds if they let you. So basically, there is no anything which is no longer accessible to you. There is no shortage on waterfront properties. There is no shortage on beautiful people. All of that can be simulated.

[Tom Bilyeu]

When you start thinking about the simulation, I know one thing that you've done exploration on is the simulation hypothesis. Are we in a simulation right now? What are your thoughts on that?

[Dr. Roman Yampolsky]

It seems very likely. Again, using the same arguments. If we create advanced AI, maybe with conscious capabilities like humans are, if we figure out how to make believable virtual realities, adding those two technologies together basically guarantees that people will run a lot of games or simulations or experiments, or with agents just like me and you, conscious agents populating virtual worlds. And statistically, the number of such simulated worlds will greatly exceed the one and only physical world. So if there is no difference between a simulated you and real, then statistically you're more likely to be in one of those simulated worlds.

[Tom Bilyeu]

Okay, that makes a lot of sense. Now, given the likelihood that we will, we're obviously showing that we will pursue artificial superintelligence. If I take your same logic from the fact that we're likely to be in a simulation because we know we would make a simulation because we're doing it right now. And therefore you get into the point where you would just make billions of those. And so if you have a one in a billion chance of being inside of a simulation, you're effectively guaranteed to be in one now, because there would just be so many of these things running, doesn't it also then makes sense that the Matrix was effectively a documentary, and we are inside of a simulation created by artificial superintelligence designed to mollify us if we ever had a physical body in the first place.

[Dr. Roman Yampolsky]

So it's hard to tell from inside of a simulation what it is all about. You really need access to outside. It could be entertainment, it could be testing, it could be some sort of scientific research. If we look at the time we actually find ourselves in, we are about to create new worlds, virtual realities. We are about to create new intelligent species AI. There is a lot of kind of meta inventions we are right about to make. And so if someone was interested in studying how civilizations go through that stage, how do they control these technologies or fail to control them? That's the most interesting time to run. You're not going to run dark ages. That is not as much happening. It's less interesting. But this seems to be like a meta interesting state to be in.

[Tom Bilyeu]

It's hard to tell because we're inside the simulation. But you're saying it's a little bit suspect that we're living in the most interesting time ever.

[Dr. Roman Yampolsky]

Yes, and I think it's interesting not just because I'm living in it, but objectively. It's a time of meta invention. You can go back through history and say, oh, here they invented fire, here they invented a wheel, that's all great, but those are just inventions. They are not meta inventions. Whereas now we're doing something godlike, we are creating new worlds, we are creating new beings. And that's something we have never done before.

[Narrator/Host]

Do you ever think like a sci fi writer?

[Dr. Roman Yampolsky]

So I think the difference between science fiction and science used to be maybe 200 years. They wrote about travel to the moon, they wrote about kind of Internet and computers and it took hundreds of years to get there. And then it was like, I don't know, 20 years. And now I think science fiction and science are like a year away. The moment somebody writes something, it already exists and there is really no new science fiction ideas where it's like completely novel technology not previously described or someone already working on it if physics allows it.

[Tom Bilyeu]

That's really interesting. Especially when you think about writing now for true science fiction in terms of what will become possible in the future is effectively impossible because you're talking about superintelligence and good luck as a person locked in your not super intelligence to actually describe that. The reason that I asked though is when I start thinking about things like that, like why would we run this simulation? What clues are in? Like if this is a simulation, what clues are in it? So for instance, the whole Christian idea for sure. And there might be more religions that have the same idea, but that man is made in God's image. Okay, well, if God is the 13 year old running the simulation, or Sarah Connor, or I guess John Connor running the simulation, trying to figure out why we created Skynet and what we can do to nudge it off course. You know, you think of them as sort of moving from radioactive rubble to radioactive rubble, trying to like find an answer to this and spinning up a simulation to get that answer. That to me becomes very intriguing in terms of hypothesizing as to why this moment, why are we the way that we are? What can we learn about the people trying to simulate us? When I ask questions like that of engineers such as yourself, there's almost, I don't have time to think like a sci fi writer vibe. Is it just that you're. You don't find that interesting? You don't find it revelatory? Why do you eschew that? Because in interviews I've seen people ask you time and time again, like, how would AI kill us? And the answer is always some variant of listen, you're asking me how I would kill us. Which is not interesting because the super intelligence is going to. But I find that's the cathartic thing that people want. Like, they want to. Like when you have a wound, you kind of want to poke at it. Like they want to get a sense of what would this really look like. And so even though it's not literally true, it's deeply cathartic to explore the known possibility set or what humans can know.

[Dr. Roman Yampolsky]

And this is exactly why I refuse to answer. I want to make sure what I tell them is true. I don't want to lie to them. If squirrels were trying to figure out what humans can do to them and one of the squirrels was saying, well, they'll throw knots at us or something like that, it would be meaningless. BS story. There is no benefit in it. The whole point I'm trying to make is that you cannot predict what a smarter agent will do. You cannot comprehend the reasons for why it's doing it. And that's where the danger comes from. We cannot anticipate it, we cannot prepare for it. I do think the singularity point is where science fiction and science become the same. The moment something is conceived, we have super intelligent systems capable of developing it and producing it immediately. It's no longer 200 years away. It's reality. And you can't see beyond that event horizon. You cannot predict what's going to happen afterwards. And with science fiction, you cannot write meaningful, believable science fiction with a super intelligent character in it, because you are not.

[Tom Bilyeu]

All right, let's ground things then in what we can predict and we can know right now, something that's on everybody's mind, and I've been talking about this in my own content is the labor market seems to be softening. You've got places like Amazon that are just cutting jobs like crazy and just saying outright. This is largely because of optimizations that we're able to make because of AI, how does this transition play out? Like, even if you concede that a non destructive AI would give us essentially an age of abundance, we're still going to go through a transition period where our jobs go away, et cetera, et cetera. What are the, what are the steps that you see happening in the labor market?

[Dr. Roman Yampolsky]

So as we have more and more increased percentage of populous unemployed, hopefully there is going to be enough common sense from the governments to prevent revolutions and wars to provide for the people who lost their jobs and probably cannot be retrained for any new jobs. So once you hit 20, 30, 40% unemployment, that's where it's really going to kick in. The only source of wealth at that point is the large corporations making robots, making AI, deploying them, all the trillion dollar club members, essentially at this point, you need to tax them and use those funds to support the unemployed. That's the only way to really make sure the financial part of that problem is taken care of. What remains is the meaning. What do you do with all this free time and millions of people who have it, traditional ways of spending your time to relax? You go for a hike in a park. Well, there is a million people in that park right now. Hiking that kind of changes how peaceful it is and how relaxing. So we need to accommodate not just change in financial reality, but also change in free time and capabilities of supporting that many people with that much free time.

[Tom Bilyeu]

I have as much pessimism around our ability to do that well as you have our likelihood of surviving. So I'll say 99.99% chance that the government completely messes that up. I think the transitionary period will be violent. When you look out at this, knowing what you know about humans and governments, what, what odds do you give it that that's a smooth transition?

[Dr. Roman Yampolsky]

It's very likely to continue to be as history always been. We had many revolutions, many wars, a lot of violence. That's why we hear stories about people who can afford it building bunkers, securing resources. Because they anticipate certain degree of unrest. Absolutely.

[Tom Bilyeu]

What degree of unrest do you anticipate.

[Dr. Roman Yampolsky]

Really depends on the percentage of population which quickly gets unemployed. If it's a gradual process, we can kind of learn and adopt and provide safety net. If over a course of weeks, months, we're losing 10, 20, 30% of jobs, that's a very different situation.

[Tom Bilyeu]

I can't imagine a scenario where jobs would be lost that quickly. To your point, we've already created, you said billions or even trillions of dollars of value. In the technology, but it hasn't been deployed yet. An example you often use is the video phone, invented in the 70s, but not really adopted, largely because of infrastructure, I would say, until the whatever, 2011, where that starts to really gain in popularity. So I have a feeling like just the deploying of all this stuff is going to take time. So in a world where an unimaginable amount of people, which I'll clock at in the U.S. call it 6 or 7 million people, lose their jobs in the next five years, that I would consider fast and just horrifyingly destructive. One, does that feel plausible to you in terms of numbers and timeline? And two, in that scenario, how distressing do you think that transition will be?

[Dr. Roman Yampolsky]

It seems very likely. So take self driving cars. I think we are very close to having full self driving without supervision. The moment that happens, happens, you have no reason to hire a commercial driver. Right? All the truck drivers, all the Ubers, all of that gets automated as quickly as they can produce those systems. And I think Tesla is ready to scale production of their cars to exactly that scenario. So what is it, 6 million drivers in the country? I don't know the actual numbers, but that would be exactly what you're describing. And it's very unlikely that they can be quickly retrained for something which is also not going away.

[Tom Bilyeu]

Okay, so in that scenario, what do you want to see happen other than heard on the Tesla as one example, will be hoovering up value. So we're going to tax the life out of them, we're going to redistribute that to other people. But what do you want to see from a regulatory perspective? Would you like to see the government stop that from happening where they say I don't care that the technology exists, you can't do it.

[Dr. Roman Yampolsky]

So my biggest concern is of course super intelligence and existential risks. That's where I'm putting all my effort in. Regulating employment in specific industries is not something I'm too concerned about. I think it will happen no matter what. I think you cannot make it illegal to have efficient factories, efficient delivery systems, logistics. It's just commercially too important and it may be a good thing for economy. Again with driving specifically, I think something like 100,000 people die in car accidents every year. If we can get that number to 0 or close to that, that's a huge improvement for everyone. So that specific scenario, as long as no one's starving as a result of that, I think it's a good thing for humanity. We can readjust economic deployment and at least that part of it is not a big concern for me.

[Tom Bilyeu]

Okay. And when you map out how we go through that transition, well, tax cool. So right now it sounds like you're just trying to make sure that wealth doesn't accumulate into the hands of too few, that we keep it distributed so we can keep using the same system that we're using now. When I look into the future, that strikes me as the least likely scenario to play out. I think that AI is going to so radically alter the cost of labor and energy that that becomes nonsensical. Do you want to see any group rise up in the way that you and other AI safety people have risen up that will rise up and start g either policy prescriptions or at least philosophical approaches to how we migrate to an age of abundance where food is effectively free, labor in your house is effectively free.

[Dr. Roman Yampolsky]

So people talk about those things and conditional basic income is one, unconditional basic assets is another. Basically, just because you're a real human, you deserve certain things. Things. And historically all these communist ideas were complete nonsense and caused a lot of harm. But if you taxing AI and robots, all of a sudden it becomes workable. I'm not against accumulation of wealth at the top. If you invented something amazing and you started a company, you should have a lot of money. But there is so much wealth that we can provide for everyone. As you said, complete abundance of basic needs. Some people say maybe not just basic, but above average set of needs. I think Elon is known for suggesting that's going to be the case. The ideas exist now. Will we pass this? Will governments actually adopt it before it's too late? Is a different question.

[Tom Bilyeu]

Yeah. So on the existential side, I don't think there's any hope whatsoever that you get people to pump the brakes. I think you're far more likely to get people to pump the brakes on. No, you can't have self driving cars or they'll try to regulate that to death. They'll tie it up in litigation, whatever, and that'll slow it down. We couldn't stop nuclear weapons from proliferating because, and I don't know who came up with this, but this seems very true to me that effectively game theory says any technology that promises an advantage will in fact be developed. Because if you don't, somebody else is going to. At a minimum, you've got the US versus China of it all. Where you, I mean the regulators are saying this right now. We can't stop because if we do, China will plow forward which by the way, I'm very firmly in that camp. What do you think about that? Do you think that game theory is inevitable or do you see a mechanism by which we can convince people that they have to slow down?

[Dr. Roman Yampolsky]

I agree with game theoretic approaches, but I see the exact opposite argument. I see that, that arguing against self driving cars is a hard argument. What are you trying to preserve? We're going to have safer drivers, cheaper drivers, helps logistics, helps economy. It's a pure benefit. Whereas uncontrolled superintelligence kills everyone. It's a very hard one to sell if you are a leader in that field, you are rich, successful, you are generating something which will destroy you personally. So to me that's a much easier argument to sell. The moment we understand dangers of superintelligence and benefits of narrow self driving AI, it's an easy game theoretic cell for me.

[Tom Bilyeu]

Yeah, the problem is you're stuck inside of a simulation of the hyper intelligent. And I mourn for you looking back at the rest of us stuck in normal land, because I don't think so. As I got into learning about the economy and trying to explain it to people, I realized that even though I can walk you through the cause and effects of why socialism doesn't work, that it feels right, it sounds good. And so people keep doing it. And even in a moment right now where the very thing that is creating everyone, like literally everyone's problems, is money printing, people are going to vote for policies that dramatically increase the amount of money that we print. And so I have developed a level of hopelessness around being able to convince people because the economy is too complicated for people either. Some of them just don't have the intellect to understand it. And then let's say they have the intellect, but they don't have the time or the inclination. And so forgive me for painting you with my brush of despair, but when I looked at your signup, there was like less than 20,000 signatures.

[Narrator/Host]

So less than 20,000 people are worried.

[Tom Bilyeu]

About the death of everyone. So it's like that's big. But I think that because I can whip people into an emotional frenzy by saying by allowing there to be autonomous driving, you're just making that evil bastard Elon richer and you're robbing these people of dignity if you look. That is not my argument. I want to be abundantly clear, but when I look at if I had a gun to my head and I had to convince people of one of two things, rich people are evil and trying to exploit poor People who are far morally superior or hey, this abstract thing that you don't really understand, it's going to kill us all. There's no way I take the they're going to kill us all bet. I'm going to be over here emotional. You get it? I'm going to bang tables and yell and say words really loudly and point to evil rich people. Guaranteed I can get people excited about that.

[Dr. Roman Yampolsky]

Luckily we don't have a democracy on this issue. We don't have to convince majority of human population. We have to literally convince the 20,000 elites who control those companies who are also super smart and understand dangers of safety. It's literally people who publish on it who have spoken. They have very high PDUM. We know Elon is like 20, 30%. Sam Altman is on record as being very concerned about it destroying humanity. So we're trying to convince people who already believe the arguments to kind of slow down and preserve their elite status. That should be an easy sell. I'm not trying to convince a random farmer to stop developing superintelligence.

[Narrator/Host]

That's it for part one. Make sure you are subscribed so you do not miss part two.

[Tom Bilyeu]

Coming up soon.

[Narrator/Host]

Most healthy habits are hard. Meal prep takes hours. Gym routines get derailed all the time. Complicated supplement regimens fall apart, often within weeks. But AG1 Next Gen is different. AG1 NextGen delivers what your body actually needs. 75 plus vitamins and minerals, 5 clinically studied probiotic strains plus prebiotics and superfoods. It replaces your multivitamin, probiotics and more in one simple daily drink. AG1 next gen comes in three new flavors. Tropical citrus and berry. All plant based flavoring with 0 added sugar, 0 artificial sweeteners, 0 erythritol. Every flavor maintains NSF certification for sport, so you know you're getting the strictest quality standards. Subscribe today to try the next gen of AG1 and if you use my link you'll also get a free bottle of AG D3K2, an AG1 Welcome Kit and five of the upgraded travel packs with your first order. Click the link in the show notes or just head to drinkag1.comimpact to get started again. That's drinkag1.com impact.