53: Zhengdong Wang - Feeling the Wave

Summary

Dialectic Episode 53: Feeling the Wave

Guest: Zhengdong Wang (AI researcher)
Host: Jackson Dahl
Release Date: July 29, 2026

---

Episode Overview

This episode of Dialectic features a conversation between host Jackson Dahl and Zhengdong Wang, a prominent London-based AI researcher known for his annual letters, especially his influential 2025 "letter" that discusses the so-called "compute theory of everything." Their dialogue explores how fast AI is progressing, what it means to "feel the wave" of transformative technological change, the evolving definitions of AGI (Artificial General Intelligence), the personal and philosophical ramifications of living through such times, and the power—and necessity—of mythmaking in science and society.

---

Key Discussion Points & Insights

1. Feeling the Wave: Personal and Societal Perception of Change

• What is ‘Feeling the Wave’?
  • Wang describes how becoming convinced of AGI's real possibility was a slow process, even as an AI specialist. He says it “took me five years of full time being an AI researcher for me to become this AGI pilled” [00:00], suggesting that personal transformation often lags behind technological change.
• The Problem of Incremental Experience:
  • Noticing AI’s progress is hard because changes feel incremental if you use the latest models daily. It’s easy to normalize rapid shifts instead of truly “feeling the wave crash over you.”
• Need for Personal Tests:
  • True conviction often comes from posing yourself “test problems” that you know intimately and observing AI confound your predictions over time. “You need to pick your own test…” [34:47].
• Difficulty of Collective Realization:
  • It's unlikely there will be one macro-event (like COVID-19) that wakes everyone up. More probable: a series of impactful but fragmentary advances (“It has to be like, repeated instances… each time it’s something new” [36:35]).

2. Research as an Activity: Boredom, Curiosity, and Obsession

• Why Research versus Engineering?
  • Wang describes: “I just want to acquire information, and that makes me happy” [05:11]. For him and many others, research is about an intrinsic drive to answer questions nobody has answered before.
• Personality Types in Research:
  • Researchers are often broadly curious or “depth-first” and obsessive—both archetypes have value.
• Prioritization and Taste:
  • Great research means knowing “what to skip and what not to skip,” balancing thoroughness with the field’s relentless pace [08:28].

3. Being in and out of the Conversation

• The Need to ‘Do Both’:
  • To make meaningful progress and avoid herd mentality, you must participate in core conversations and remain removed enough to generate new, original insights. “You kind of need to both be in the conversation enough to not be missing obvious things, and … to have some new ideas” [11:56].

4. AGI as a Moving Target & ‘Model Does the Eval’

• Redefining Progress – ‘Model Does the Eval’:
  • AGI definitions will always shift. Wang prefers viewing progress as whether a model can do whatever is evaluated (“the model does the eval”), arguing that if “people find it useful” then the eval and thus the progress is real [13:24].
• On Emergent Abilities:
  • Models often acquire surprising skills simply as a result of general scaling—such as being good therapists while learning to code [13:24].
• Limits of Eval-based Progress:
  • “We will achieve every evaluation we can state. … We will fall short on every capability we struggle to state” [18:19]. Epistemic limits matter, especially for messy or subjective domains (e.g., “how much joy did you bring to your colleagues?” [21:16]).

5. Automating Research & Self-improvement

• Recursive Self-Improvement (RSI):
  • RSI gets easier as models can increasingly handle the research process. But as Wang notes, “you’re kind of procrastinating the question of what makes a good eval” [25:56].
• Human + AI Collaboration:
  • Advances have always been about “adding more barrels to the gun”—be they human or model. There's still “infinite demand” for attention and experimentation [27:00].

6. Exponential Curves, Timelines, and Preparing for What’s Next

• The Exponential Question:
  • The key distinction is whether we are on the exponential or not. “Are you on the exponential or not? … Tie your ship to the exponential” [47:15].
• Coping with Uncertainty:
  • For most, knowing that world-changing events loom brings a mix of urgency, existential anxiety, and clarity about what matters.
• Personal Impact:
  • Wang suggests that coming to terms with transformative change allows us to care less about petty concerns and focus more on what we value or relationships [48:15, 50:52].

7. The Role and Making of Myths

• Science Needs Myths:
  • Mythmaking isn’t secondary—it gives context, motivation, and shapes identity. “If research engineers continue to see myth making as a chore… they will keep working in a world under myth” [54:32].
• Best Myths So Far:
  • The mission-driven myth (e.g., “curing cancer, eliminating poverty”) or “you are Oppenheimer on the Manhattan Project” powerfully motivates researchers, but may be insufficiently pluralistic for the full public [55:23].
• To Policymakers:
  • Effective messaging to governments involves framing AI as a continuation—“we are keeping a promise that we previously made” (e.g., more abundance, but not breaking the social contract) and "volatility is good" [59:33].

8. Decentralization, Competition, and the Dangers of Concentrated Power

• AI’s ‘French Revolutionary Tendencies’:
  • Wang worries we risk “ignoring our inheritance to remake the world from abstract rational principles” [62:22]. He ultimately trusts decentralization and competition to mitigate risks but notes “both are true at the same time” (progress and risk) [63:24].

9. Risks, Doom, and Wang’s Non-doomerism

• Risks to Focus On:
  • Wang’s main worries involve malicious actors, not runaway paperclip scenarios. “Most people are good, most people are not malicious” [67:39].
• Balance between Progress and Liberty:
  • Restricting liberty is a greater risk to Wang than progress itself [69:24].

10. Personal Meaning, Production, and the Changing Nature of Work

• Work, Meaning, and AI:
  • With AI automating more tasks, meaning will increasingly be found in the personal, relational, and self-imposed quests. “The personal stuff is always, like, your domain is going to be new…your friends want to know what you think” [75:45, 79:52].
• ‘Production’ vs. ‘Consumption’:
  • Research is productive; but a privileged life involves intentionally consuming as well—experiences, ideas, and events [92:09]. “You have to produce so they let you consume” (credit: Tyler Cowen) [93:26].
• On Rest and Guilt in Fast-moving Fields:
  • “Now is better than ever … now is [in fact] the last time” [96:47]. “You do matter. And you don’t matter” [98:07].
• Creating Meaning:
  • Even if AIs can surpass human output, meaning can be reclaimed by valuing the process because it was wrought by a person [98:50].

11. Pluralism, Optimism, and the Importance of Marginalia

• Pluralism as North Star:
  • No single value, myth, or narrative can suffice—embrace the diversity of stories, methods, and existential responses. Influence from Isaiah Berlin and the “Hedgehog and the Fox” parable is prominent [101:27].
• Marginalia, Personal Writing, and Living Well:
  • Wang advocates for annual letters as a means of increasing awareness, synthesis, and personal growth [107:57]. “It’s the best antidote to the temptation to time box research, writing or enjoying life.”

12. Quickfire & Miscellaneous

• On London:
  • “The best pre-AGI city and the best post-AGI city” [117:46], praised for its diversity and pluralist tradition.
• Why Container Ships?
  • “They're just so efficient,” a symbol of productive global standardization [119:24].
• Advice on Productivity:
  • Toggling between “do more, you can have free lunch across the Pareto front” and the need for honest prioritization [121:13].
• Obituaries & Books:
  • Love for The Economist obituaries—concise, diverse, and personal [112:37].
  • Favorite book: One Hundred Years of Solitude—“a book about everything… nothing feels out of place” [114:43].

---

Notable Quotes & Moments

On Perception and Adaptation:

• “If you see this and you're not convinced, I'm not sure what would convince you. How many doublings do you think we need before this total physical transformation happens? Either this AI investment is going to exceed world GDP ... or we're going to have to grow world GDP because of the AI. We will know the answer in the next five years.”
  — Zhengdong Wang [00:00]

• “You kind of need to both be in the conversation enough to not be missing obvious things, and… be removed enough to be able to have some new ideas that aren't in the core stream.”
  — Jackson Dahl [11:56]

On Evaluations and Progress:

• “We will achieve every evaluation we can state... We will fall short on every capability we struggle to state.”
  — Jackson Dahl paraphrasing Wang [18:19]

On the Emotional Impact and Mythmaking:

• “My only contribution to the literature is this: Are you having fun?”
  — Zhengdong Wang [01:09 and 89:28]

• “If research engineers continue to see myth making as a chore… they will keep working in a world under myth they keep complaining is an inferior.”
  — Wang, on the necessity of myth [54:32]

On Liberty, Risks, and Decentralization:

• “Most people are good, most people are not malicious. … The non-liberty is a bigger risk.”
  — Wang [69:24]

On Living Through Transformative Times:

• “We get to make our meaning. We get to choose.”
  — Dahl (summarizing Wang, agreeing) [100:40]

• “Pluralism is just like, the highly most highly prized value to me.”
  — Wang [101:27]

---

Timestamps for Key Segments

• What it Took to “Feel the Wave”: [00:00], [34:47]
• Researcher Motivation & Disposition: [05:11] to [07:53]
• Exploring AGI and ‘Model Does the Eval’: [13:24], [18:19]
• RSI and Automating Research: [24:35]
• Personal Impact & Exponentials: [47:01]
• On Mythmaking & Policy: [52:08], [54:32], [59:33]
• Risk, Doom, Decentralization: [66:53], [69:24]
• Purpose and Meaning in Work: [75:45], [79:52], [81:57]
• Pluralism, Marginalia, Optimism: [101:27]
• Annual Letters & Personal Reflection: [107:57]
• Obituaries & Book Recommendations: [112:37], [114:43]

---

Tone & Language

Throughout, Wang is thoughtful, measured, and pluralistic—constantly balancing between optimism and skepticism, determinism and agency. Dahl matches with intellectual curiosity and occasional wry humor, pushing for synthesis and clarity.

The dialogue is marked by a willingness to step between big theoretical abstractions and the pragmatic, often personal, implications of rapid technological change. The episode’s feel is both philosophical and practical—full of “finger feel” about the historical moment combined with humility before the unknown.

---

Suggested Next Steps for Listeners

• **Reflect on your own “wave tests”—what would convince you that society is at an inflection point?
• Explore Wang’s annual letters for deeper dives on these topics.
• Re-examine the role of myth and narrative in shaping not only your own professional or intellectual journey, but in building broader societal consensus (or resilience) around transformative technology.
• Read widely: Isaiah Berlin, The Economist obituaries, One Hundred Years of Solitude, and books like When We Cease to Understand the World are all recommended.
• Write an annual letter (even just for yourself).
• Stay pluralist, optimistic, and open to marginalia.

---

Closing

This episode is a dense, wide-ranging meditation on what it’s like to live—and do research—at the crest of a technological “wave,” and on what it will take—intellectually, emotionally, and narratively—for more people to join in “feeling” that reality. Wang’s blend of technical insight and philosophical reflection is an invitation to remain both ambitious and humble, and to keep both “producing” and “consuming” in a world defined by both accelerating change and enduring questions.

Transcript

[A]

So it actually took me five years of full time being an AI researcher for me to become this AGI pill, so to say. I felt like it was so hard for me to feel the wave crash over me. I really feel this difficulty. If you just look at 10 years ago to today, which I wrote about the Transformer paper, not even out yet. And then just last year, you know, the Pope named himself Leo because of artificial intelligence. And just that level of change going from a small academic niche no one's heard about to, you know, the world spending like close to a trillion dollars on investment. If you see this and you're not convinced, I'm not sure what would convince you. How many doublings do you think we need before this total physical transformation happens? Either this AI investment is going to exceed world gdp, which means it's going to have to crash, or we're going to have to grow world GDP because of the AI. We will know the answer in the next five years.

[B]

It will either implode or we will enter a new evil.

[A]

All of us are wondering what's going to happen. It's sort of like knowing like a pandemic level event is on the horizon. We just don't know if it will be very good or bad. Totally valid question. What am I supposed to do about that? Right? But maybe one thing is just like, it just becomes a lot easier to stop caring about the petty things. I wish I didn't care so much about what other people thought of me, or I wish I spent more time with my friends or something like that quote from you.

[B]

My only contribution to the literature is this. Are you having fun? My question to you is, are you having fun?

[A]

I am having a blast. Are you?

[B]

Welcome to Dialectic Episode 53 with Zhang Dong Wong Zhang Dong is an AI researcher based here in London, and I got to know him by way of his annual letters, particularly his 2025 letter at the end of last year. In it, he wrote about what he calls the compute theory of everything, about how scaling laws have brought us an incredibly long way in AI and will carry us far into the future. It was one of the first things that I read that really made me believe in where we're going. Obviously you use AI models, you look back at the progress, but as he articulates in the piece, it can be hard to really feel the wave, to really feel the AGI, in part because you're just comparing it to what you most recently used. And the progress is, while fast, fairly incremental. One of the things I focused the conversation on is how the rest of the world, outside of a small number of people primarily working at AI labs, can not only think about and prepare for, but feel the future that is coming or frankly feel the future that is already here. I talked to him about what it means to be a researcher AGI as a term, and why a better frame for AI may simply just be the model does the eval, and about this notion that he uses in the piece to describe the feeling of the wave crashing over you as you understand what being on an exponential means when it comes to artificial intelligence. And importantly, Zhangnong also is able to talk about why, despite some concerns and despite the fact that it certainly is going to be different and there is going to be change, AI could be one of the great transformers technologies not only of our time, but of all times. You can also find the transcript and all the links for the topic we discuss at the link in the Description@Dectic FM Zangdong. I hope you enjoy the conversation and before we get into things, I would like to thank Notion Dialectics Presenting Partner Notion is a collaborative workspace for your life's work that has rebuilt itself from the ground up for the AI era. One of my favorite articulations of Notion comes from an idea from Jeffrey Litt, who is episode 21 on the podcast and shortly after I interviewed him actually joined Notion. And that is malleable software. Jeffrey argues that software should feel like any environment a craftsperson might use, a kitchen, a workshop, in that the user can shape their tools and their environment to fit them and their needs. Increasingly, software is easier than ever to create. You can ask an LLM to create nearly anything and and yet being able to have living, adaptable, flexible software that integrates with every other tool you use, that is collaborative, that can be shaped and formed to whatever you and your team need, remains remarkably powerful. In Notion's case, this not only applies to you personally, but your entire team and now the agents and AI that you work with as well. And Notion's ultimate goal, as stated by founder and CEO Ivan Zhao, is that it help us think together in the long run as we have more and more capability. The goal of doing things, building things and thinking together is what enables human ingenuity. You can learn [e-mail] dialectic with that, here's my conversation with Zhengdong Wang, Zhangdong Wong. Thank you for joining me. This is a long time coming. I'm really excited to talk about, great to be here, all kinds of things. We are going to start with Research. Research is, as you say, at its core, a human activity. Why? Either in this context explicitly, or even going back farther, back to college or high school, like why. Why was research something that. It's also relatively unique for at least software and kind of like Silicon Valley technology. The word research is like, outside of AI, it's like not. Not very common. Why not just go be a software engineer? Like, what. What about re. The orientation of research was.

[A]

So, yeah, yeah, I didn't know this when I started college and sort of had to discover it too. But I really think it is just something like you have a question and you don't know the answer and you really, really want to find the answer. Maybe for a lot of other people with different preferences, they get just as much satisfaction out of here's a system and you optimize it or, you know, it impacts a lot of people and that gives you the same sort of satisfaction. Maybe I could just say, like, my preference is I want to know the answer to things. Like, I just want to acquire information, and that makes me happy. I just want to know. So when I was doing internships or doing research assistantships or all kinds of things you try out in college, I just found that actually when I did the software engineering internship, I would go through the summer and do a lot of stuff and it would be very satisfying. But I could sense that I was getting bored even at the end of two or three months, because I could sort of see where it was going. And I imagined like, oh, this, this thing will get bigger, it will scale up, more people will use it and it will become even faster. And all of that was great, but because I could see the end of suddenly was less interesting to me. Maybe different personality types. And mine is like, I just want to acquire information. And so this aspect of something being unknown, whether it's like a really big fact or a really small thing, just the satisfaction of being able to be paid to go answer questions that I have with incredible resources is really interesting.

[B]

Are you easily bored?

[A]

Yeah, I think so. Unfortunately, yeah. I'm one of those people who goes about different projects and should probably finish more of them.

[B]

Do you think that's common amongst researchers?

[A]

I think so. I think there's probably a bunch of different ways to be a good researcher, and this is one of them. You're more curious about a broad range of things. But I also feel like a really great research archetype is you just can't move on into until you know the answer. Right. And so you're just Incredibly obsessive and you just do like a depth first search and you just keep going and asking, why is this weird? Or in the process you see a bunch of weird things and you just chase down every single one of them. And that's what gives you a better mental model of the whole thing and lets you see patterns or make connections that you wouldn't have known to plan ahead of time just because you notice something weird and you're not satisfied. Just like, like leaving that be so you can go explore something else.

[B]

I think of on your blog you recommend Laura Deming's the Rage of Research. Yeah, it's like captures that, that spirit

[A]

that just like I have to, I, I really wish I could, I could channel that more because I, I don't often feel very enraged as a person, so I just try to think of rage as like, I just really want to know or like deep interest or something.

[B]

Aside from disposition, what do you think either in the general sense or at least for you personally, makes for a good researcher and maybe more specifically for a good research engineer.

[A]

Knowing what to draw as an abstraction is a kind of just like prioritization. I often find myself when I go between different projects, just sort of going back and forth between two extremes before finding a dialectic as you know, where it's like, well, the last project I went too fast and I skipped over some things or I had to backtrack and redo a lot of things. And so this time I'm going to be really careful and I'm going to do things very, very slowly but surely. But then you do that project and then you realize, well, the field of AI moves very fast and that was definitely the wrong choice and you just need to be better at having better taste, having better prioritization and knowing what to skip and what not to skip. And then you maybe do a project on that extreme and you go too fast again. And I think finding balance is just something that you need to get a lot of reps in to improve at.

[B]

Maybe there's a case to be made that research, perhaps more than almost anything else, is really about holding tension between explorer.

[A]

Yeah, I think, I think that's fair. And then I am, I am all about, maybe, maybe this is a cop out to answer this sort of question, but I am all about like, you just have to do both at the same time. And the idea of like, oh, just one is better, or that you should just focus on one thing at a time. I think that maybe for a Y

[B]

combinator startup is right but maybe. Right.

[A]

But I think. I think that is just too easily conceding that you aren't able to do everything at once. You know, there's also this quote of like, you just need to do everything and then you will win the LBJ Caro thing. But I feel like.

[B]

Has that been true in general for AI research, do you think?

[A]

No, I don't think it's possible to do everything, basically. And even as the field just grows a lot, there's just way, way too much to do. And so I think there's going to be infinite demand for.

[B]

Are those who have done. Sorry to interrupt. Are those who have done the best, generally over the last five, seven years, the ones who have on the margin, done more, or are the ones who have chosen better? Kind of a ridiculous question.

[A]

I think it's chosen better just because of the amount of different things you could try all the time. And I think even if you're not doing research and you're just scrolling through Twitter or looking at all these papers, you defer a lot of curation to people telling you what to read, or even if it's not worth reading, the fact that everybody else has read it means this is something that's. That you should know about. Right. Even if it's just to know what the conversation is. One important aspect of research is that you just need to be part of the conversation. Whatever that conversation it is that's going on. It might be really stupid, it might be just totally wrong. And history will later show that this is a totally wrong direction. But especially if you're a researcher looking to join the field and you're not already established in it, I think you might have your own, maybe even very correct ideas of what you should be doing, what the field should be doing. But that just like getting yourself into the conversation, wherever that might be, even if it's now just being on Twitter instead of publishing journal articles or things like that, is like an important first step.

[B]

It's kind of another instance of you have to do both, which is one problem certainly Silicon Valley has these days is it can be remarkably mimetic. And I guess what I'm hearing you saying is you kind of need to both be in the conversation enough to not be missing obvious things, and you need to be removed enough to be able to hopefully have some new ideas that aren't, like, in the core stream of what everybody else is trying.

[A]

Yeah, yeah. So just do both. Answer again. Yeah, just do both. But both are true at the same time.

[B]

Yes. Yes. Hedgehogs and foxes all over again. AGI, you kind of continually come back to this theme that AGI is a moving target. I think this is you in 2023. To be clear, I'm not saying that building something most people would call an AGI is impossible. Instead, I more and more see the where we build a machine that people agree is AGI before we write some words that people agree defines AGI. And you actually have a different frame in your that you spend like Most of your 2024 letter on, which is the model does the eval as kind of this continuous frame to think about AI progress rather than like, are we at AGI yet? You say, you might even say that the only time AI researchers are doing AI research is when they choose the evaluation. The rest of the time they're just optimizing a number. And then at the most abstract level, the model does the eval, describes the entire field of researchers pursuing general intelligence. Maybe we can start simple, or perhaps not so simple. What makes a good eval?

[A]

I just think if it's something people find useful. So if you take one extreme of the position I'm giving, it's just very empirical, right? Just totally ignoring anything predictive that you might want to say about AGI. Just that in the process of going through human life, we have all of these things that we want to accomplish. Some of them might be like graying food, some of them might be folding your laundry. And then there's vaguer stuff like we pay each other to do some cognitive work that's harder to define, that maybe once a year we do a performance review of people on. And then at the very end, there's some philosophical topics that people have been working on for thousands of years and haven't found the answer to. I just think that anything that people find either useful in itself or just useful instrumentally, but still very useful would make a good eval. So when the models couldn't just do basic language things, then that's useful just as a proof of are we making progress towards something that isn't useful yet, but is definitely we know or we're pretty sure is better than the other stuff. And now we have these large bundles of evals that do math, code a bunch of different things. And then I think there is a lot of generalization that's very surprising. I don't think there's a lot of optimizing going on about how good of a therapist it is, for example. But as the models get better at coding, they also happen to be good therapists. And so as you get closer to the real thing you want, then you can set your eval to just be a very specific, like just can you can grow my food, can you wash my clothes? And all of that is just very useful, even if it's not general. And people care about the definition of general because they want to draw a distinction of what is it that makes us human, what distinguishes us from non intelligence or possible other intelligences. And so in drawing this definition, you might want to draw the distinction or the boundaries very clearly and so come up with Eavas for that. But I think that is also a bigger, more long term question and possibly, possibly hopeless, right? Because we've had a lot of human intelligence writing books about what is a distinction, what are useful distinctions for a long time that's not scaled up super intelligent super fast, of course, but I think there has been some progress in that. Philosophers would know better than me, but I think there's been non zero progress at the very least. It's been interesting for a lot of people over many, many years to think about and talk about with each other, to make each other better thinkers. And so those evals are hard to define, useful in different ways. Maybe one day we'll get the LLMs to do them, but maybe it's not useful because they're mainly for humans.

[B]

Man. I have a bunch of questions. This is sort of abstract, but is all AI research science?

[A]

So one time David ha tweeted, AI research is just applied philosophy. And I just really like that tweet. And it's a shame he doesn't tweet anymore. But when you ask if it's all science, I think AI research also appealed to me a lot in college because it could be everything. Research engineering is like licking all possible things you could do in research and engineering. It really is like AI research has this arrogance that it could be everything, right? And now there's these post AGI teams popping up across all of the labs and it's like you do have a remit to talk about economics, right? Because it of course will affect that of all of the humanities, philosophy, what does it mean to live a good life, to find meaning in work, in religion, in relationships? It just accelerates all of the things that people would be thinking about if their material needs and emotional needs were taken care of. You just see straight to the end of what you would be doing with your time. When people ask what will you be doing in a post AGI world? It is, I would say, very similar to asking the question what would you be doing if you had all your financial things figured out? If you had your relationships and everything figured out? Maybe you would be creating art. Maybe you would be asking about these philosophical questions. Maybe you would be pushing yourself in some athletic event subject to constraints that you totally set upon yourself. Nothing is really that different in that way. And AI or AI research covers everything, has the arrogance to think it could cover everything, but it's just very broad and accelerates what you would be doing without AI.

[B]

Anyway, you have a point in that piece where you talk about the two conclusions of if you take the model does the eval to the end. First you say the first awesome conclusion of the model does the eval is that we will achieve every evaluation we can state. You wanted to say, is there any limit to the model does the eval? Its second awesome conclusion is that we will fall short on every capability we struggle to state.

[A]

State.

[B]

I think this is interesting in the context of something you said earlier, which is there are actually all these emergent properties of AI too. Like the evolution of evols evals from the outside looking in at least went from like, do you know this information? Can you solve this math problem? To increasingly stuff today, like can you do this economically viable activity? And increasingly also we seem to keep finding in large part due to scaling just like even either more good stuff keeps falling in. And so I'm almost wondering like, are we already at the point where there's still plenty of evals we could presumably create? But like, it feels almost more like saying recognizing after the fact, after the model has does something. It's more like RLHF or something like that does that distinction making?

[A]

Maybe one practical thing that I think would fall under this that's like hard to state. Maybe one day we'll be able to state it is just a lot of jobs that potentially the tasks that make up the job could all be automated, but that even still it's really hard to know before the fact how good an employee is going to be. Maybe jobs in the modern sense have only existed for not very long, but it's still long enough. I don't know when the first job performance review ever was, but maybe even when they were building the pyramids in Egypt is hard to specify or maybe easier. You're just like, how many rocks did you place? Yeah, but that has been tried very hard, right? And now even if you could say, well, as part of this job of a cognitive work, I produce this spreadsheet in this case that's subject to these Very specific parameters. Right. But then once you have a tool that does that, just like the computer was a tool or the spreadsheet is a tool that automated a lot of humans doing computers, then the job totally changes. And then what defines, like you being an employee at this company, maybe you just accelerate straight to like just, just how much revenue did this employee bring in? Right. Like as a company, Right.

[B]

We've actually designed a lot. At least large parts of society is designed pretty heavily around a meritocratic ish orientation around markets. Like that's your worth. And granted it's abstracted because it's actually what does your boss think or whatever. But that is a template, for example, for how model evals have like Claude's value as a reference point based on how much revenue it creates. The world opens up a can of worms for how Claude is going to evolve, right?

[A]

Yeah. And you're like, okay, so the actual stuff that we thought we could say is actually pretty vague. So let's just go up some level of abstraction, like abstract the exact tasks in the bundle and let's just say how much revenue, Right. Or like a possibly related thing is just how much joy did you bring to your colleagues? Maybe we live in a world where everyone just needs to have a job for some societal reason that we haven't closely examined. You want to go to work, you want to feel like you're contributing to your community, have a sense of purpose and then if you're a personality hire non derogatory, then it's really hard to specify, but you'd be doing a good job. If you are charismatic and your colleagues enjoy working with you and you raise the spirits of everyone and everyone just feels, feels like they live a more fulfilled life. I think this is also hard to specify and maybe your performance review at the end of the day justifiably is just like, how happy did you make everyone? How much does everybody like you? And I think that's fine too. But yeah, so going back to your original question of model just the eval, what can we specify? What can we not specify? I think a very near term practical thing that's relevant for AI research is these jobs where a lot of the tasks, tasks we will surely automate. What does that mean for the job? The job being an abstraction that bundles these tasks currently but surely will change and what is even a job, it's also on the spectrum of purpose in life to a specific set of tasks that needs to be done. That's very fuzzy and unclear, but the awesome conclusions of you just keep looking around for all of these tasks.

[B]

Tasks.

[A]

A bunch of humans are trying very hard at this right now. More and more humans will join the field of specifying things. Just like for all of history we've been trying to specify things. Soon there will be more AI agents joining in this task of looking at all the things and trying to specify them better. And maybe in the end we'll just have these impossible to specify things like what is consciousness or something.

[B]

And when you say things to specify, you're kind of of that's the model. Does the eval package inside of that?

[A]

If I'm understanding. Yeah. So like you want to specify things because once you do that and then you have this very general algorithm that

[B]

can, the galaxy brain can war machine towards it.

[A]

Yeah.

[B]

When it comes to the future of AI research, especially that last bit, which is like we will struggle to hit evals, we fail to specify a lot of this is speculative, but especially as we move towards the world where of a lot like we are seemingly approaching the automated AI researcher, how do you expect AI research to actually happen? Are we going to offload most of these evals to the models themselves? This also gets a little into like I was just rereading the Leopold PDF and he's, he gets to the super intelligence harness part and he's kind of just like, I think we're going to come up with something kind of like RLHF, but like tbd. And one answer would be to go back to it is like we want models that are good at creating economic value and we can kind of measure that based on the hive mind AI of capitalism, but we don't have many other things like that for other types of values.

[A]

Okay, so I'm going to give like a overview of like my current thinking on this related to rsi, but it's very lightly held. So RSI is even easier than people make it out to be. And then the RSI is recursive self improvement. Yeah. So like getting the AI to specify the evals for itself and things like that and the artifact that comes out of it, this AGI that does very well on all of these evals, will also be easier than people think, but that once we have that, it won't be what it's cracked up to be. So recently Jack Clark wrote this post that he thinks based on public information that 60% chance of achieving RSI, so achieving this totally end to end frontier research, totally automated by the end of 2028. But I think that if we think of all of recursive self improvement as some kind of spectrum as well. We're well on the way on this spectrum and if we just think of it as this is a way to get any eval, like any single number or any bundle of evals up, then it's just like some kind of meta algorithm, some kind of metasearch. And while you're working on that this, you're kind of procrastinating the question of what makes a good eval.

[B]

Totally.

[A]

Yeah. So like sort of like alpha zero

[B]

even like it's, it's sort of just like yeah, we, we will surely develop the war machine out. I don't mean in the violence sense, but like the super algorithm to solve to any endpoint.

[A]

Yeah, yeah, like just like this super laser that once you focus it on something it will, it will be solved. We don't know how to define consciousness. There's all these philosophical questions that we've have a hard time specifying. But there's all this like low hanging fruit of like now researchers spend a lot of time like cop pasting paths or generating these plots by hand, changing the colors of the lines and things like that. And all of this obviously could be automated or just is very, very well within sight if not done already. So Andrej Karpathy's auto research is a very basic version of this.

[B]

On some level it sort of sounds like you're saying we will have laser beams and for the most part it will be people pointing the laser beams at something. But the laser beams are just getting better, stronger, they last longer, whatever. And you're, and you're, it sounds like you're a little bit skeptical in the near term of like the AIs knowing where to point the laser beams themselves.

[A]

I think, I think it will happen. I just, I just don't know when. And maybe compared to like the optimistic AI researcher, the 2028 country of geniuses

[B]

in a data center take off all.

[A]

So I mean on that it will be like everyone has a country of geniuses in their pocket as well. But that's the difference. Yeah, but on the laser beams, yes. I think we have these very high powered laser beams. I think right now since forever have had these laser beams that are just like human expert AI researchers. Right. Like 10 years ago when it was an academic niche, we had a few very high powered human laser beams that were copy pasting the checkpoint paths themselves and doing this kind of search themselves. And then because the models are general, because they really do replace tasks that are very useful to AI research, and they're hugely complementary to humans, and that's really all you need. We're expanding the number of barrels on the gun, and there's just so much to do, there's just so many things to try that even with the incredible pace that we're building out, compute that talent is joining the field and we're doubling the number of barrels on the gun, it still won't be enough. Or now we just expand what we want to do. Right. We look at all these fields of cognitive work and soon physical work, and we'll be like, okay, so now we need to optimize this for this particular task, for this particular job. And you might think in the end, well, none of this is very general. Right. You're just sort of brute forcing.

[B]

Yeah, it's the AlphaGo creativity thing.

[A]

Yeah. You're searching all.

[B]

You're brute forcing creativity.

[A]

Yeah. And at the end of that, you think like, like move 37 is like the obviously best move. And without having exhaustively done that search, it looks like genius to you. But if you had looked at all the millions of moves, then it's sort of obvious or something. But you do this for all the spreadsheeting jobs or something like that, or eventually for invention.

[B]

Again, maybe I'm thinking about this wrong, but part of the way I'm almost thinking this laser beam metaphor is at some point, point will have big enough lasers that you can point the lasers at cancer. And you just say, like, run the search. And granted that's not actually one giant tractor beam. It's like a million different laser beams of so many different permutations. But if you scale it enough.

[A]

Yeah, no, no, I think it is a good analogy because maybe one thing I'm surprised by, I'm not sure how much other people are surprised by, maybe they are too, is like how easy language is. Or I actually do think a lot of other people will be surprised by this, where maybe like an early research agenda. I think OpenAI published it. I don't remember. They had all these bets early on in the company, including rl, including robotics and things like that. And it wasn't so clear that that language would be like the first solved, maybe it would be the last solved. And the thing that we're doing right now, talking to each other in this space of language, it's surprising or could be a little depressing just how predictable all of this is, right. That we could do so much of language, like not the best poems, not the best novels, but so much of what we do with language is actually extremely statistically predictable. Or just like there's these connections or we can interpolate between the spaces and maybe you could say the same about a lot of biology. Or like this, our human laser beams are just doing very little jaunts into the. This big unknown. And there's so much we don't know in the universe. There's so, so much we don't know. And this very powerful statistical tool like the thing Demis Hasavis says in his Nobel lecture, his challenge is like I'm sure we'll find a link but like a classical algorithm will be able to solve all of these problems.

[B]

Yes.

[A]

Yeah.

[B]

If you have a big enough. If you have a galaxy size calculator. Most complex, complex problems are just a matter of time and search.

[A]

Yeah. And, and, and we, we, we could just be surprised by, by how simple it is relative to.

[B]

And we're almost conflating that for this word we call intelligence. Perhaps that would be like a slightly strange but plausible view on what AI is which is actually there's no, there's no threshold of intelligence and it's just a. Again big enough search, big, big enough data.

[A]

Yeah, yeah. So, so I, I just think the, the whole alpha, alpha, alphago alphafold. Like, like this analogy is super, super useful. And it kind of really quickly speed ran through. There's pre training, there's rl it's this very clean what counts as winning a game, what counts as a value or something like that. And you can just map it on these much fuzzier spaces that humans do. And then maybe at the end of the day we find that all of the things that we humans do that we think are very complicated, it's actually like an extremely low dimensional statistical thing in the entire universe. And so we have these power things that it could just find all the patterns really fast interval all the spaces really fast.

[B]

What I'm sort of hearing you imply is that it's very plausible that most of the things we know a lot about and our intelligence itself might not actually be that complicated and the universe might still actually be wildly, far more complicated than you can possibly imagine. Like it's possible AI speedruns us and it still hardly knows anything.

[A]

Yeah, yeah. And I think Demos does a really good job of like keeping us focused on these stakes of like, you know, this, this Silicon Valley like techno singularity kind of where we're going to have Dyson spheres, we're going to have extreme abundance. Like even, even that is like so small or just like doesn't pales in comparison to true nature of reality, which is like, like you, we have no idea what's going on in this whole universe. And we have this chance of like, like understanding like a, a tiny bit like seeing some patterns or like, you know, touching the fabric of reality of the. The. The whole thing. And so yeah, we, we could, we could be more ambitious or that's like even weirder. It's like hard to. Hard to.

[B]

It's humbling and inspiring at the time. It's June 2026. I don't know when we'll put this out, but probably around. Around now. And I generally try not to have conversations that are super like of the most moment. But I think it's worth grounding the time in part because I want to talk about the main theme of your 2025 letter, which is I would say it was the. Probably the first thing I read where I at least very strongly secondhand felt the wave crash over me. Probably using some of these tools lately. And you basically lay out what you would call your compute theory of everything you say, however hard I try, I don't think my descriptions will even move you much. You need to pick your own test. It should be a problem, you know. Well, one you've worked for years and one you're supposed to be an expert in. You need to predict the result year after year. Then you need to watch AI confound those predictions anyway. Kind of a version of what we were just talking about. Like actually maybe language isn't that hard. I think we are in a different part. We're in a different time than we were even in December when you wrote this. The US government has started to pay more attention, so on and so forth, and yet most people clearly still have haven't felt the wave. Do you think for every individual person it's just going to be this kind of thing? Like do you think there is a macro event that would convince people? Is it getting easier for you to convince people? Maybe a different way of asking the question, to use a metaphor used in the piece is like will there be like a March Covid moment where like in January and February it was available for people to see, but like the world didn't wake up.

[A]

Yeah, this is a great question. This is something I keep thinking about all the time too. So I wanted to write this particular thing for the letter because I felt like it was so hard for me to feel the wave crash over me. And as I mentioned, I maybe have longer timelines than the optimistic AI researcher So it actually took me five years of full time being an AI researcher for me to become this AGI pilled, so to say. And for this to happen, I had to see so much evidence. Like just evidence across time. Yeah.

[B]

Which is that first quote is kind of, that's the critical part of it is like, I tried this now. I saw it be not that good. Two years passed. Oh my gosh.

[A]

So evidence across time, like, like repeated instances and tests that I set for myself. So it's not like you're telling me what this test is. You're telling me it's really impressive. It's like, I know this problem. I know it would be impressive.

[B]

You have finger feel in that problem.

[A]

Right, right, right. And like, I, I know like you, you can't, you can't cheat on this in, in, in, in ways that I would something. And that's just like such an expensive way to AGI bill something to learn something. Like, you can't expect everyone to spend five years of their life full time

[B]

or shut down all the schools. In the COVID metaphor.

[A]

Yeah, yeah. So there must be some accelerating way of this. Oh, when you're talking about the COVID analogy, I think one thing that will do would be like robots or like, that's like maybe my upper bound on robots walking around.

[B]

Yeah, yeah, but, but not, not a perfect metaphor. But like, I remember the first time I got into Waymo and I was like videoing the Waymo, and I'm like, this is the craziest thing that's ever happened to me. I got in the Waymo. 60 seconds later I was on my phone and I had forgotten.

[A]

Yeah, so that's a great point. So, so that also needs to be like repeated instances where you're like, okay, so, so this week Waymo showed up, and then the next week there's these automated like house building robots that like started building data centers in my backyard.

[B]

You have to be thrust into moving history, perhaps on a repeat. Like, it can't just be one instance of moving history. It has to be like, yeah, right, right.

[A]

And I think that countries that have developed really fast in the last few decades, my parents generation, like feel this very like deep in their bones where they grew up in China and when they were kids there was like rationing. They had like meat once a month or something like that. And now they live in this big suburban house in the US and it's

[B]

really great, not to mention so many parts of China. China.

[A]

Yeah.

[B]

Jason sun had this line somebody sent in about like the Skyscrapers popping up like mushrooms.

[A]

Yeah. And there's this picture of Shenzhen 20 years ago, and now it's like, just like a field. And then 20 years later, there's skyscrapers. And Matt Iglesias tweeted, like, I bet this really ruined the character of the neighborhood or something like that. But that. Just, like living through that over a long period of time, seeing so many changes like that, that makes you feel it. Right. But, but so, so that's. That's the upper bound of.

[B]

It is like a fundamental characteristic of humans that we. Like, you don't see your kid's baby for three months, and it seems like they've grown, like they've doubled in size, yet the parent doesn't notice it. This is like a. For better or for worse, we are so good. We might hate change, but we're so good at normalizing change, which makes this jolting hard.

[A]

Yeah. And so I really have to credit people who are, like, way more rational than me or, you know, community, maybe that. That was like, rationalist to Jason or calls themselves, they were early to the pandemic. They were early to AI. They deserve huge credit for that. And I just can't get myself to do that. And I had to see so much evidence. And so I really feel this difficulty of, like, I feel very, very privileged also to be surrounded by people who are, like, very optimistic about AI telling me all the time, trying to. Trying to convince me of things. And then finally I was like, okay, I've got it. And now I'm like, some people say, like, off the deep end or something.

[B]

Is that intellectual or more. I don't know. Yeah, metaphysical, spiritual.

[A]

It definitely has to be a combination. So, so, so there's these, like, piece of evidence I see before myself. But then you still need someone to handhold you through the rational, like, look at the bigger picture. Look. Look broadly. And then I also tried to do this in the opening to my letter. And I keep trying new ways of doing this. So maybe I'll try now, which is like 10 years ago. It's like the Transformer paper, not even out yet. Fifteen years ago. That was about when deep learning started working. The AI could start recognizing cats and dogs. And then just last year, the Pope named himself Leo because of artificial intelligence. And he's like, this is a revolution that deserves the same kind of shepherd that the first industrial revolution went through. And that's why I'm going to name myself Leo. And also there's imo, gold medals, which are not even impressive. Now, six months into the year and just that level of change, or going from a small academic niche no one's heard about to the world spending close to a trillion dollars on investment earns, propping up the economy and things like that. And now in more recent, still unresolved political news, the federal government realizing some of models are really cybersecurity dangerous and this thing still not being resolved. Right. I'm just like, if you see this and you're not convinced, I'm not sure what would convince you. That's one aspect of it. And looking forward, it's like, okay, so maybe you've just started reading the news, just woken up from a coma. You're like, okay, I can see that AI is a big thing, but I

[B]

would argue if you had woken up from a coma, it would be easier.

[A]

Yeah, yeah, yeah.

[B]

This is the thing is that like we're, we're not, I'm not referencing, I don't know, it just came out that OpenAI 5.6 or GBT 5.6, the government I think is going to be like hand selecting who gets access to it.

[A]

Right? Yeah. Every individual or something.

[B]

If you showed me that relative to two years ago, I would have been like, oh my gosh. But they just banned. Or whatever.

[A]

They banned that, that, that, that's super fair. So, so I did try to go like imagine 10 years ago and then now, but, but, but, but say you, you see everything now and you're like, okay, I gr. Big deal now. But, but still, you know, maybe, maybe like bigger than the microwave or something, but like not like millennium defining technology,

[B]

which, or even like the Internet was a big deal. Like I think there's like we're starting to get to the point or even the Industrial Revolution. I have a, I have a sort of idea in my head that like the Industrial Revolution was a big deal. Granted, if I went back and lived it and lived through the change, I would have. So these things are hard in the abstract, which is part of why I asked the question. Is it purely an intellectual thing or

[A]

something like, oh yeah, so, so yeah, by, by the way on this. I think, I think people should all be history majors. And this helps a bit where as part of the intellectual thing you go back and you read stuff and 100 years ago people still felt like the end of the world was coming. Chemical weapons in World War I. And there's motor cars everywhere and societal issues looming on the horizon that you don't know how to solve. And America is only 250 years old. It's like people have been just trying to make it work the whole time and abundance is a new thing. But on the rational aspect, then if you look just a few years out in the future and you grant that the AI becoming a bigger deal has been going on longer than you expected. So maybe one world is like, you see ChatGPT, you see GPT4 and you think, okay, this is obviously very impressive, obviously beat my expectations, expectations, But I'm not ready to sort of just go full straight line up and to the right on the graph because you're just like more of an empiricist. You haven't seen enough data points. Then you just think, okay, maybe it will keep improving for a few generations, but if it plateaued, right, it wouldn't be that weird. And so you're just like waiting for more evidence. If you grant that now, you could say there's either five years or there's 10 years, or 15 years, or 70 years of continual dots on the plot that keep going up into them, right? Then the rational thing that I'm trying out to pill people is like, how many more doublings do you think we need before this total physical transformation happens or bigger kinds of transformation happens? And so even if you think the current level of investment is unsustainable, like amount of capex invested or revenue or something like that, we'll just know the answer in the next five to ten years. Right now we're in this sort of compute crunch because, because we need to build a lot of infrastructure. But even if the rate of growth slows, if it just continues, either AI investment is going to exceed world gdp, which means it's going to have to crash, or we're going to have to grow world GDP because of the AI, or we'll reach some crazy form of super intelligence and all this crazy stuff will happen. Or there will be some, hopefully not, but, but some terrible like world historical geopolitical conflict tragedy or something.

[B]

That's kind of the point you make at the beginning of the piece though, around the, the, the Peter Till quote about Elon. For the people who haven't read it, like Teal's point is that Elon's talking about we're gonna have billion robots and we're gonna have all of these national debt problems. And, and Thiel is critiquing those two things aren't mutually exclusive along the lines of what you just said, granted, your point is that perhaps we did get the robots and the debt.

[A]

Yeah, but, but the, basically the, the intellectual side of the equation looking Forward is like, I'm trying to tell you, like, it's gotten so big now that we can rest easy. We will know the answer in the next five years.

[B]

It will either implode or we will enter a new evo.

[A]

All of us are wondering what's going to happen. Or there are these people who are coping with AI is big, but it's going to stay this big. And things don't change very much or history has ended. So things just generally. Yeah, nothing ever happens. Things just generally stay the way way things are. And I'm like the Financial Times chart where one line is going up and one line is going down. We'll just know the answer. It's gotten big enough where this will happen very soon. And so it's very exciting. And I don't think it takes that much of a leap of. Once you accept that AI is a real thing today and that it has changed a lot in the last few years, that, um, yeah, we'll, we'll just know what happens. It's sort of like knowing like a, like a pandemic level event is on the horizon. We just don't know if it will be very good or bad.

[B]

I don't think that's gonna inspire much confidence in people. That last line.

[A]

Right. But. But maybe this exercise is just to. Just to get people like, like convinced about the moving history rather than.

[B]

Yeah, well, so there's one line in the piece that I think is. Is really poignant. You say it works one point that the curve is exponential. So if it happens, it doesn't matter when. And then you go on to say, what does it matter if it's two or 20 years away? The only change that's mattered to my AI timelines is that I used to think it wasn't going to happen in my lifetime, and now I think it is. Can you talk about how that has most. That view has most changed your research the way you think that cut on what you just said would say, we're not going to have the debt implosion thing or we're not going to have the economic implosion thing in the next five years. We are going to like it's going to happen. We're moving up the exponential. If we are moving up the exponential, exponential at all. Like we're going to clear the jump. Maybe. But why. Why is that framing of it? I mean, obviously on one sense, it's just fundamental and that you're going to get to live to experience it. But I think there's a subtler frame on it, which is that like if we're on the exponential, it's, it's not slowing down.

[A]

Like yeah, maybe I would think about it and this is like extremely relevant for like everyone's like personal life. Whether or not you're in tech, whether or not you're in a. I'm just

[B]

like, granted the two or 20 matters. The two are the 20 years. Like that is a big difference.

[A]

It definitely matters. But, but, but overwhelmingly, most important of all is like both. Are you on the exponential or not? Do, do you recognize that? Or like if you can just do something to you know, like, like tie your ship to the exponential versus just, just like, just like knowing it. It's like you, you, you're just probably broader, make like better decisions or, or something like that.

[B]

Well, so sorry to interrupt but like that is, I think that's quite an interesting question. Which is one, one other meta reason why people aren't able to feel the, or aren't feeling the wave crash over them is even if I do look at the math, which is just look at the scaling laws, like the, the root point you make over this whole piece is that we're on this like long journey, maybe arguably back to Moore's law or even further, which is just like do the math. One reason perhaps that we don't feel it is I'm like, what the heck am I supposed to do with the fact that I'm. What does. Hitching your, your wagon to that, hitching yourself to that wagon. What does that even mean? Like, go work in AI does it?

[A]

Yeah, I, I think like any of these specific things are, are probably going to be wrong. And I was talking to a friend about, you know, like there, there's one term 19 uncertainty, but I like better the Donald Rumsfeld. He goes up to the podium and he's like, there are no knowns. There are known unknowns and there's unknown unknowns. And then in the unknown, unknown situation, you really don't know what the optimal strategy is based on just some deep preference you have. You might be extra greedy or extra risk averse, but both of those seem just as good to me. I just think that, yeah, maybe within AI worlds where we're convinced that this is going to be a big deal, we spend a lot of time talking about is it going to be two or 20 years, which like you said, it's super important. It will totally change how, how society is able to adapt well to it or not. But then for someone else maybe kind of like the defaults haven't really thought about it. Reaction is just, okay, I just really hope this doesn't happen or something like that. But maybe if you look at all the beliefs you have or which conclusion you draw, if you just look at the evidence and apply your own priors to how you would weigh the evidence, you might come to that uncomfortable conclusion and that of this will happen in the next 20 years, in the next 50 years, just like sometime in your lifetime. Right. And it definitely matters when. But I think it does give a lot of clarity to things like I was mentioning. It just accelerates you asking the question of what do you value? And so now you could say, okay, if you're like a young person, I need to get a job, I need to save enough money, I need to, to figure out a bunch of things in my life, right? And then when you're young, you think you're going to live forever and you never really think about the longer term things. But I think the fact that this phenomenon exists, even if it hasn't really physically affected your life at all, you're totally far away from AI worlds or something. This is sort of like telling you there's going to be pandemic level of change in your life, lifetime. Totally valid question. What am I supposed to do about that? Right? But maybe one thing is just like, it just becomes a lot easier to stop caring about the petty things because you're like, yeah, I've just received this prognosis of like, yeah, things are just going to be very different in five years. You're going to look back and you're going to be like, I wish I didn't care so much about what other people thought of, of me or I wish I, you know, spent more time with my friends or something like that.

[B]

It does feel on something the more I think about this. It does feel in some level like if I told you you were going to die in a year or anyone, they were going to die in a year, especially if they were under the age of 80. Yeah, I suspect they would really change their behavior. If I told you you're going to die in three weeks, like, but if I told you you're going to die in seven years, like, yes, intellectually I think I would versus 40.

[A]

Yeah.

[B]

But like there is, we're really bad letting abstract remotely far away things. Let letting the wave crash over us.

[A]

Perhaps a question would be, and unfortunately I think, I think this is a really good analogy and I'm just like really desperately trying to, to find like the positive versions of These analogies that there's like you, you receive a terminal death. Yeah, you, you receive a terminal diagnosis. Slash, like the whole of society goes through a pandemic. But I'm like, but, but, but, but, you know, I, I, I really do believe that AI is going to go well, or I think it's way more likely to go well than not. And so while these analogies work really well because they really hit on some sense of urgency or make you feel like the stakes, that sort of thing, the kind of like imagine, you know, you will win the lottery in next year or something like that. Or like, yes, but even that, by

[B]

the way, if I was going to win the lottery in three, three weeks, my behavior would be very different if

[A]

I knew I was going to lie, right? And so the key to finding such an analogy as an individual winning the lottery or society as a whole winning a society wide lottery, this kind of, how to make that more compelling or how to communicate that. I think the whole field is also struggling with this, where the bleak things just hit home more in some way, unfortunately. But yeah, whatever you would do there, it's like, yeah, imagine you're going to be fabulously wealthy in a few years, like you said, you would be acting very differently or you would receive some kind of new clarity about what you care, about what you value, or you would just know, like, oh, I need to really quickly start thinking about what I value, what makes me human, things like that. I think the labs have achieved this. That's why they're starting all these organizations thinking about all of this, the transition and also after the technology is more stable.

[B]

One kind of major implication of this is that the people who do know this, who have the secret or not so secret, people working in AI whatever need to be better at talking about it. It's clear that that's something you think a lot about, both in your personal writing as well as what you kind of urge people on. There are two quotes that I picked out on this that I, I liked. The first, you say some people are quick to disavow themselves from doomers or accelerationists, but, well, what else is on offer? Not much. If research engineers continue to see myth making as a chore second class, a lesser use of their time to quote unquote, real technical work, they will keep working in a world under myth. They keep complaining is an inferior and then slightly more fun. You quote Tolkien as he says, fantasy is a natural human activity. It certainly does not destroy or even insult reason. And it does not either blunt the Appetite for nor obscure the perception of scientific verity. On the contrary, the keener and clearer is the reason the better fantasy will make. What makes for good myth maybe to start.

[A]

Yeah, I think there is a really good myth specifically for people who are likely to become AI researchers.

[B]

Right.

[A]

Like all these labs, they have lots of very talented people working for them and they could, you know, I don't know, go into finance and make a lot more money or something like that. But partly because it's like the other mission driven people. But like the mission and the mission of whether it's touching the fabric of reality, understanding the true nature of reality, or it's impacting a lot of people's lives, like curing cancer and eliminating poverty, hopefully eliminating inequality, all of these. That's like a myth that's like, in a good way, a mission that really drives people that would make it.

[B]

It's such a strong myth, perhaps in part, that it could be part of why these people aren't thinking about that much myth making otherwise. Because it's just like so intrinsic and obvious to us. It's so catnip to a certain kind of person that they're just like, how could, how could you care about anything else? This is the most important thing.

[A]

Yeah, yeah, exactly. And also, you know, the myth of like, oh, you are on the Manhattan Project, you are Oppenheimer, you are contributing to this like world historical event individually. That also works. So I think in that way the myth making is going quite well. It's just that this is definitely too simplistic of a view, that there's not enough understanding of the diversity of the fantasies that would appeal to more people or that more people have. Maybe it's something like that. Maybe it's something like change is just really uncomfortable because uncertainty is really uncomfortable, comfortable. And just generally people don't really like uncertainty. So maybe the kind of find the perfect myth or find the Steve Jobs for AI is kind of just like a really hard problem or really hard to do. That could be part of it. I think it's definitely worth working on and thinking about though. And that right now the best myths are directed at the people most likely to receive it. And then you receive it, you might join AI as well. As AI becomes bigger, it becomes more inclusive, more pluralist, more normal, so to say.

[B]

Two questions. Number one would be in that quote, you reference doomers and acceleration risks and what else is on offer. I don't think you're a doomer. I feel pretty confident on that. And maybe you're an Accelerationist by some definition, certainly to a normal person, but I don't get the sense that you're like a pure accelerationist. I think you're working on this in part because you think it's the most important thing. Do you have a sense of other narratives that are on offer?

[A]

Yeah, so. So this quote was from a couple of years ago, right? Yeah, yeah. So I think. I think that that was the year where maybe like E. ACC was coined or something like that. And then. And then in my mind, that was the year, like newspaper starting started to use the word doomer. And so I chose those two as, like, these are like the most prominent myths of the time, but they are kind of. I mean, yeah, they're like, definitely ingrained in the water. Still, I think there are new myths, more complicated ones being developed. I would agree with you. I think if I had to choose between these two ends, I would say I'm more of an accelerationist because I think it will go. Go well. And in the same way, I'm trying to find an analogy that's positive. I think you should really focus on the solving cancer 10 years earlier kind of thing being really good, rather than the kind of trying to avoid some bad thing that you think will happen earlier. But these myths have also gotten more inclusive, and so there's different strains of accelerationism out.

[B]

Right. The second part of my question was going to be, you've also spent time, as I understand it, with a number of people who are closer for better, have a better way of putting it closer to the policy side of the world. I think certainly in the UK and maybe also in the US do you have a sense of what types of myths or stories or even just simple messaging have been or will be most resonant or clarifying for those people? I mean, it feels more relevant than ever, at least in the US and some of it's about persuasion, but some of it's just actually about, like, I'll let you decide how you want to react to this. You might be anti, but I need you to feel the wave. You. You make some point somewhere. And granted, I don't think this is that recent, but it wasn't that long ago either that, like, there's a few people in the Trump administration, a few people in Saudi, and maybe like a couple other senior politicians in the world who have felt the wave.

[A]

Yeah, not. Not my point, but I was quoting some.

[B]

I'm sorry.

[A]

Yeah. So I think that there's two maybe reassuring things. One is this is not really change in the thing I wrote where I cited this person, you should frame it as like, we are keeping a promise that we previously made, and the way we're keeping it or the technology that we use to keep it is different. But broadly, we have a promise that is the social contract, and we will uphold it by distributing the benefits of this technology to everyone, make sure no one's left behind. But fundamentally, the important stuff has not changed. Like, you're living your life, you get a new house, but everyone is still going to get a house. The change will not be so disruptive or so uncertain that you need to really worry deeply. And the second thing that could be reassuring is volatility is good. We've always had change, and change has always been good. And this is like, you grow up in a country with 10% GDP growth over decades, and you just watch where you live turn into a field, into skyscrapers, and all of a sudden you have these really fast bullet trains or something, something like that. It was like, yes, there's a lot of change, but broadly, overall, it's, it's, it's good for you. And I, I think it works for America, too. Like, America, only 250 years old. You, you look at like some guy who's alive today, and they're like, you know, just like three generations back was like ninth president of the United States or something like that. It's just not, not that much time. And most of, for most of American history, so much change, right? It's like so much, much happened. For most of American history, America was not like the global superpower, right? And so that's a lot of change. And telling this myth where you are empowered to make this change or benefit from this change. We would hate it if we had an extremely classist society where nothing ever changed and you're just kind of stuck in whichever social standing that you have. But in a world you're part of, a country of very dynamic people who are starting businesses and doing all this cool stuff, and you're making the change yourself. And then you will benefit from this change, as has happened for all of history. And in fact, it's part of your national identity that as a person of this country, you're really good at adapting to change, and change has always been really good for you. I think that could be reassuring currently.

[B]

It certainly seems that there is something structural about AI and maybe just modern kind of techno capitalism that is creating a smaller and smaller group of really powerful actors. Maybe a different cut on this would be like, that's one Justifiable concern. There's a different take you have somewhere, and I think you're kind of referencing somebody else's idea, but it's the person who's talking about the French Revolution versus the replacement of the English monarch. And it's almost like it's definitely a little Machiavellian or something, but it is an interesting sort of way to think for whoever's trying to change the world. You best keep some of these promises. You say, I think this is you. The AI industry has all kinds of French revolutionary tendencies and our own self conception. We're bold, inevitable, and on the right side of history. At our worst, we're ignoring our inheritance to remake the world world from abstract rational principles, dismissive of accumulated experience and impatient that no one else is keeping up. I think we're meeting a resistance to that impulse that is earned.

[A]

Yeah. So if I could go back to my classic cop out answer of like, both have to be true at the same time. It's like, well, I really believe in like competition being good. And if you want to prevent, you know, like really serious, like, power concentration, there just needs to be a lot of competition. So a lot of unpopular decisions that labs have have made people complain about them. Maybe they made these various decisions because they feel like they're in a position of strength and they're like, we think this would be best to do with the models and if we do this, the market can't do very much about it, so we're just going to do it. But then because there's competition, maybe that gets walked back in the end. Maybe labs would do other things that they're not doing, such as, we're just going to develop this RSI AGI thing by ourselves and just keep all the innovations to ourselves. But because there's a market that's not going to happen, or there's a lot of competition between different models. So one company's idea of what is moral and good and how much to defer to the user versus how much input the model should have to the user. The model telling you something is wrong or something. All of this, I think competition is generally good. But then of course that means maybe this pulls the future closer to us a bit faster. And you're like, wait, but wait, that's bad. Both kind of need to be true and happen at the same time.

[B]

I appreciate you muddling through this with me. I mean, part of the meta thing that I think I'm feeling as I've been thinking about this and throughout this conversation is Just like it's all further case to be made that like the more we can make more a wide, a plurality of different types of people, Americans, whatever, politicians, whatever, to feel the wave and thus really deeply engage with this. There's a tension there of course, because one response to that might just be like we need to stop the AI. But the best possible future to me seems roughly like as many, many influential and regular people as possible care about this stuff. They have the like optimistic understanding of like what the technology is, but they also aren't just putting it off as this pie in the sky thing to say like we have two to 20 years to solve some of these problems and it's going to take the whole gang to like really?

[A]

Yeah, no, but, but, but we should focus on it. It will take the whole gang like you know, all, all the labs I, I think think are extremely sincere and well intentioned of like what we know is that it's going to be nuts. And you know, we're not professional economists. Well some of us are not as many as like the whole field of economics or the whole field of philosophy or the whole field of you know, whatever would make to go well for like all these different areas of human inquiry. And we're just kind of like creating these things to, to these organizations to think about these questions because they are important questions. But yeah, some kind of main goal is to get more people into this project and it will take the whole gang. And if you don't like that big tech is deciding all these things, then

[B]

get involved.

[A]

Yeah, and you can't consistently hold that this AI thing is a scam and also like oh please stop disrupting my field. Yes, yes,

[B]

maybe just briefly, I know we briefly touched on it, but why aren't you a doomer?

[A]

So I think I am still more of an empiricist and therefore there are a lot of risks I do worry about. And maybe this requires a definition of doomer. But yeah, a lot of the risks are very like humans are very involved in them. Like malicious humans using the AI for bad purposes I think generally characterizes the risks I'm worried about rather than the

[B]

paperclipping, whatever we lose control scenario.

[A]

And some other general assumptions I'm making are like there will be competition or there is just going to be a lot of degrees of freedom for a lot of different things that you can't predict ahead of time to happen or avenues of these threat vectors. And so given that we live in this very decentralized world that we've made choices in society to support the benefits of the decentralized world, then you really need to lean into it. And so if it is a spectrum of accelerating versus slowing down, then I think most people are good, most people are not malicious. And of course there's offense, defense, balance. But the kind of solution that what I imagine a vague definition of doomer would support, I feel like would be counterproductive. I'm not at all saying the risks aren't real. I'm saying I put a much greater weight on these more human being involved kind of risks and that the best solution to them, like I wrote in the piece, responding to Dara's piece, is, you know, very decentralized, very like a lot of good people, like in a, like a very decentralized way thinking about solutions to this.

[B]

Yeah, yeah, there's a, there's, there's a cut on this that's sort of like the cost of progress and the cost of liberty is some risk. And we have take, we have, we have like taken that trade for a long time and it has paid off really well.

[A]

And, and, and I would add that the non liberty is, is a bigger risk.

[B]

Yes, yes. One last thing on this, a quote I liked and maybe just another opportunity for the wave to crash over people a little bit on like compute and scaling. You say the biggest mistake people make when they make the case for AI is that they say it's different this time. It's not different this time because it's always been different. There hasn't been any constant normal trend ever. And all we've done is be optimistic that we'll muddle through. Nothing is truly inevitable, certainly not progress. And progress too might stop tomorrow. All things considered, though it would be stranger if it did than it, if, than if it didn't. Perhaps you even make this point. But I, I couldn't help reading this thinking about like, not only does this apply to scaling laws or possibly even computing and Moore's law, but like maybe all of human progress, maybe even the trend of complexity in this little corner of the universe that we're in. You also noted your frustrations with the foundation series. Speaking of determinism, how do you think, given maybe a quote like that, which is holding a lot, how do you think about on one hand, the sense of inevitability or determinism and also will or agency?

[A]

Yeah, so both are true at the same time. But more seriously, it's like, well, when you look at the trend of 2% GDP growth for since we invented invention or something like that and if you zoom in really closely and you look really closely, it's like all these people trying very hard and doing very different things every time and very weird things every time. And then only when you zoom out, it looks like this.

[B]

We call it a law.

[A]

Yeah, yeah, yeah. And so also, if you believe that AI is general in some way or is extremely complementary to humans in some way, it's like these human laser beams that we're focusing on problems. And as like a general law or a general rule, we know that if we focus this human laser beam on this problem for some amount of time, we're going to get something out of it. And of course it's not guaranteed. It's like probably you're just like rolling dice or something. But probably you put enough laser beams humans on something, you get.

[B]

We even see this with things like the space race in the 60s or Covid vaccines. Like there is something about human. We put the tractor.

[A]

And individually too. Like, you know, will to power as a researcher or as a creative, you're like, oh, damn, am I really going to be able to think of a next piece? But if you just block aside 10 hours, you're going to get something out of that set of conventions. Yeah, it's going to be a bit more than you expected, a bit less than you expected. But in that way, everything is different. Everything is like, it could definitely be zero. So that aspect is extremely contingent, not determined. And then when you zoom out, it looks more determined, but then, you know, it's both determined and also contingent. The thing about the foundation series is I just think there's no character development in there because there's these hundreds of years of time jumps. And so you only get a character for a chapter or something like that. And I think the idea is very cool and very important. And a science fiction book should really lean into its one idea. Right. I think it's hugely impactful and it may be captured something about the time it was written in as well that people imagine, oh, we have Newton's laws. We're discovering all sorts of laws. Maybe we'll discover laws to everything. Right. And that's also a very old idea.

[B]

Amazing point.

[A]

And to have a work of fiction that really explores that is really great. So I'm not anti foundation just because it has that idea. I think that relies on the reader to not read that and totally go off the deep end on that end of you read that and you're like, okay, well, now I'm going to discover the true law theory of Nature. Yeah, but it's like a data point that you take in your own inner balanced, everything is true sort of idea of. Yeah, that's just another character in the pluralistic universe.

[B]

I want to talk about personal implications of where we might be going in the era to come. First of all, similar note to what we just spoke about. I wanted to read two excerpts. First, from your 2025 letter. You say, later you'll think, who could possibly compete? How could your cleverness be worth anything more than a hill of beans against an artifact that cost millions and concentrates within it the cleverness of billions of humans? How arrogant to think clever enough to outpace a factor of a thousand, then another thousand, then another thousand. This is what they mean when they say general purpose technology. Perhaps a slightly wordy, but I think very good articulation of the super laser beam. Meanwhile, in your 2022 letter, the first one at least available to me, there's a slightly different tone. I think you say some people think that there are few if any scientific breakthroughs remaining. They think research progress is hard to measure. Ideas are getting harder to find. Maybe all the good ones have already been had. Maybe some extant thing is all you need. I wouldn't be so sure. Marvin Minsky, luminary of our field, predicted, quote, within a generation, the problem of creating artificial intelligence will, will substantially be solved, end quote. That was in the late 60s. He joins a distinguished class, Lord Kelvin in 1897. Quote, There is nothing new to be discovered in physics now. All that remains is more and more precise measurement, end quote. Cicero reported that Aristotle, he had just about completed philosophy and that it would surely be completed a short time after his death. Don't be them here, actually take the long view. We're coming back to it over and over again. But there is a paradox there, and one is empowering to the human spirit and the other is like deeply, deeply humbling, if not kind of disappointing or even sad. Maybe the best way to ask the question would be, do you still believe in the Zhengdong of 2022?

[A]

Yeah, I can see the tension there. But I think it is consistent in the reason why you are trying to be clever when you do the research and when you're trying to compete against the models, where if you're just doing it for some instrumental purpose, like we do in a lot of the things we do in daily life and errands and for our job, it's like we need to get this thing done and here are the things that we do to get there and in A job you might think like, oh, I'm just having a blast every day doing this thing. Right. And that can all be consistent and all be great in that if you want the end goal itself, then you can use a tool to accelerate the middle part that maybe you don't care so much about and reach the end goal. But let's say in terms of software engineering, you also like the process itself, you like the puzzle solving aspect aspect of the engineering. Maybe it is a bit disheartening that you don't find it as useful to other humans when there's this tool that could do it, but that one, you can still explore this kind of puzzle solving aspect yourself in the same way that athletes do or game players do autotallic. Yeah, games being a very self imposed limitations kind of can I solve these problems within these limitations kind of. Or as an artist and now you can do software engineering as an artist or something like that. And the other aspect that I think is new that's more consistent with the earlier, more human empowering thing is both that there's just so much stuff to do and that part of my earlier answer about getting AGI will not be what it's cracked up to be is that, yeah, if you just think of the amount of things we will demand or the amount of things that we will explore, just how big the universe is and how weird it is, there will always be something to do. And scarcity in the definition of will the cost be zero? Will we really have infinite of it? We won't reach that. We'll just do everything faster. We'll do more things, get even more niches and write even more fiction or something like that and create even more universes for ourself. The second way that there will always be stuff to do is just something being very personal. So everything about you, you maybe like to do puzzles or you like to create this sort of art, just the fact that you are doing it yourself is also very new. So whether you do it with AI or you do it by yourself, or people are fans of your work just because it is you. Right? Right now, friends will read your stuff because they want to know what you think. They want to know your favorite, favorite flavor of ice cream, not the LLM's objective optimal best flavor of ice cream. And many people have written about that this as well. And so the personal stuff is always like your domain is going to be new. And Scott Alexander I think has written a very good post or that makes you feel this particular wave crashing over you of Imagine in the future if there are people in other galaxies and they think they read all the history of what was the creation of powerful AI like, and they're like every single character that was like, like, even marginally related is like a celebrity, like, oh, this was their favorite form of ice cream. This was like a blog post they wrote. And it reminds me of in the Bible, the first European convert to Christianity. Her name is Lydia, and she's just remembered forever because she's living her life and is part of this world historical story. So the personal is new. There's just infinite demand for everything. There's only some particular things that maybe because they're accelerated by tools or something, you would do it differently. But I just see everything as like expansion of options available.

[B]

I agree with much of that. And yet part of the implication in that 2022 quote, I think is about like doing things of subject substance. And granted, I think relational life is of substance, but in the abstract sense or in the grand sense, it would be like discovery research, like truly getting closer to know. In some sense there is like a humbling, in a positive way the idea that actually, like we know 0.00001% of what there is to know, we can know so much more. But that isn't really something you, you mentioned. And then I guess at a more local level, most people aren't going to discover scientific theorems or whatever. I do think there's just a sense that for many people, it's hard to have meaning without some kind of meaningful work.

[A]

Yeah.

[B]

So I'm curious how you would take those two on top of what you said recently.

[A]

Rebecca Lowe wrote this blog post, she's a philosopher about some broader definition of work where this idea of replacing jobs. Jobs, but what even is a job? And I think she. And I would agree with you that everyone needs to have some kind of purpose. Maybe it's just relationships, but maybe it's not just relationships. And you just need to feel like you are doing something that you find meaningful or you are contributing to something larger than yourself, a community or a bigger project or something like that. But I think that can be very broadly defined. And I'm curious if you, if you think that people's idea of that can change or not. So maybe with the invention of some tools for agriculture or for making clothes or something like that, previously our culture would place much higher weight on being able to hunt or being able to farm, or there's something real about waking up early in the morning and doing this work that's real work. And maybe even today we idealize this idea of real work instead of spreadsheets or something.

[B]

I was at dinner last night, one of those open kitchens, and I was like, wow, it'd be kind of cool to work in a kitchen for a little while.

[A]

So they were building a apartment building next to the office a few years ago, and colleague and I would joke, like, you know, we're just like, coding, but if you want to see any real engineering happen, just, like, look out that window.

[B]

I think, I think, I think I agree. And yet, like, go back, take one single thread. Throughout all of human history, to my knowledge, certainly all of, like, known own history, there has been a frontier to explore. Perhaps, if anything, like the last 30, 40 years has been like an anomaly where it's like the frontier was the Internet. Now obviously it's extended to space. But, like, some people will be totally fine in relational context. Some people will be totally fine doing. Making good pants, making music, whatever. Like, I. I make podcasts. But there is a sense that there is some deep human thing that I. It's hard for me to imagine going away, at least as a species of our desire to, like, have quests and grand adventures and to be of consequence. And perhaps that last bit is the part that, like, was made up all along or where we were deceiving ourselves all along. And maybe this is. This kind of goes hand in hand with a broader question of, like, at some. On some time, or least at. And do we get superseded in the local context of what intelligence means over here?

[A]

Maybe. Correct me if I'm wrong, but I would say an assumption that your question is there's this space of questing that your quest could do this, that all this unknown and because of AI, AI is just taking up a lot of this space of Quest. It's like 90% of all the questing could be done.

[B]

That could be theoretically 100% of the globally relevant questing. Yeah, maybe we'll still have locally relevant questing.

[A]

Right? Yeah, so.

[B]

Or maybe we never had globally relevant.

[A]

So I guess, like, like, yeah, maybe could. Could you talk about this assumption a bit? Because my assumption would be the. The idea of like 100% of the globally relevant questing would itself need some justification. Or it's like your burden of proof to show because as long as everything isn't instantly completed instantly at zero cost or something, let's say your quest is like, oh, I want to go to that galaxy over there and I want to explore everything, and I'm taking all these Robots and starships with me. And when you play a video game like everything is, is done for you, right? You, you still, you still make these like vague decisions. And, and your, your, your choice of like, this is who I want to be, this is my identity is like still your choice. There's, there's still like choices that literally only you could make that.

[B]

Is that true for like science? Maybe, maybe put it a different way. Well, a few thoughts. Number one, the quote I read, which is like, you're clever. Cleverness. What did you, what did you say? Later you'll think, who could possibly compete how your cleverness could be worth anything more than a hill of beans against an artifact that costs millions in concept. Thousands and thousands. Thousands. You have an example, you give a different example of the idea. We talked about it. Alpha Fold or alphago being. Is it actually creative? Well, it doesn't really matter. A third thing I would say is like you are someone, I feel quite confident who could just go enjoy themselves all, all the time, right? And eat food and visit art. And you do a lot of that fortunately. And yet you are also like, no, I have to work on this. And maybe part of it is like, there's only a few more years, but we want to matter. We want to matter, right?

[A]

Maybe it is true that there is some kind of objective definition of what morality is or just objective answer to everything or when you discover true nature of reality, there's this theory of everything, there's one equation or something like that. That is the case. And maybe to take what you're saying really, really seriously, I think that if we were to find such a thing and we were to all very deeply be convinced by that, that would be concerning or that would open us up to exactly the kind of worry that you bring of like, everything is determined, there is nothing else that's really mattered. We've done 100% of the things that really mattered. You could think of it like that. You could also think of it as like, this is the completion of the game. Like the Minecraft you've killed the ender dragon. Credits start scrolling and there's this nice poem written there and that would be the completion of the game of the universe. And in Demis biography with Sebastian Malady, Malaby he's like, once I know that, then I will shuffle off my mortal coil. Or he says something like that. Maybe that's one way to salvage it. I agree that in that situation though that would open this up to your concerns. But I think we are just so far away from that and that even if our lifespans end up being much, much longer, I think a lot of it will just look like, okay, I've decided to take my starships and go to that galaxy and I'm going to make a bunch of choices along the. That are really, I find quite meaningful.

[B]

I think that's right. I think that's right. I think on some level there is a fear and a plausible reality that we stop being the most significant actors in the discovery of knowledge and the exploration of the universe.

[A]

I think this is all just relative to what previously culture or what we grew up in and maybe find it hard to change about our own preferences. If we grew up, you know, finding a lot of meaning in like, fixing up our house or like tending to, you know, our livestock or something like that, then I think, I mean, every single generation thinks like, ah, this is like the kids these days. Like, everything's getting worse and changing. And now another way in which it's not different this time, because it's always been different, is that these kinds of changes are just happening faster and faster. Where, yes, the short term can be a lifetime, but also it was great when the short term was longer than a lifetime or longer than the characteristic length of a career. But now if you need to transition in a space that's shorter than the average human career, then that's a step change in new problems. If you only had to do it once because it's in the middle of your career, career, fine. And so in that way it's different. Maybe now you're like, okay, this uncertainty is really worrying. I'm going to have to change my life to adapt to this uncertainty. And then three years pass and you're like, I did it. I found new sources of meaning, found new things that I find matter just as much as what I previously did. Then bam, the AI can do that now and you have to just do it again. Yeah. So maybe another just like general personal thing of like, being a lot more okay with. With change in like a very. In like a meta way.

[B]

A quote from you. In my time as a research engineer so far, I have enjoyed many, too many to count meditations on research taste. My only contribution to the literature is this. Are you having fun? My question to you is, are you? You having fun?

[A]

I am having a blast. Are you?

[B]

I am. I am. I think part of this future we're talking about is like, how. How much fun do you have or can you have by surfing the ways of change? And there are times where I feel Afraid. And there are times where I feel. Yeah. Confused. Or like, if you ever respond to that with a desire for any inaction, I think you can spiral in the negative way in some weird sense. It relates a little bit to, like the classic Dan Nabil, like, do more and you'll get more energy thing.

[A]

Yeah.

[B]

And so I think there are ways to respond to this, all of this, and be paralyzed. And there are ways to respond and be like, as you said, there's so many galaxies to explore.

[A]

Yeah. The galaxies I, I think are still quite far. But the. Having fun is like, there's, there's always this, like, type two fun of like, you know, people say you go on a hiking trip or there's like some experience, you have a lot of uncertainty and then you look back and there's some nostalgia or like you were glad to have gone through the experience. Of course the experience has to go well. Right. Like, of course you can't ignore the. Well, if, if you're hugely uncertain in the future because a lot of the things that you relied on that your life depends on are, like, now in flight, that is really bad. But I think it is a privilege to live through interesting times.

[B]

Are you having type 2 fun or type 1 fun? I was recently asked this.

[A]

I think it is a lot of type 2 fun with very short feedback where you reach the end where you look.

[B]

Lots of hills. Yeah. You're climbing lots of hills.

[A]

Yeah. Like, wow. Even now, I think to a few years ago when ChatGPT came out and I thought, imagine the first time I tried ChatGPT and before that, the first time I tried some kind of language model that was not chatgpt or imagining before that, trying image models like Dall E, just how different everything was then. It's like, wow, did I really live through this and see it happen and talk to people at the time it was happening. That all feels so far away, even though it wasn't that. That close.

[B]

Why do you. I. What do you mean by. And why do you identify as a consumer?

[A]

Yeah, so I, I was thinking of this when you were asking me if, If I'm having fun. And by, by consumer, I, on one hand mean like the, the. The character you read about in your economics textbook. You know, there's producers and there's consumers. I'm just like, I'm the consumer. I'm just like, consuming this stuff. And then a broader definition of. I really mean everything. Not just a nice restaurant, a good movie, but also rarer forms of consumption. I feel extremely privileged to Know some of the much better AI researchers than me, like big characters in the fields, the people who will map onto Oppenheimer and Rutherford and all these things aliens

[B]

in the future galaxy will be reading about.

[A]

Yeah, yeah. Or later you read history textbooks. And these people who are like characters and to know how they feel about certain things, to have been able to get their thoughts on something that's just extremely rare form of consumption that I feel very privileged to be able to partake in.

[B]

Is research a type of consumption?

[A]

No, I think it's production. You can get satisfaction and fun in discovering a fact and keeping it to yourself, and that can be a form of consumption. But producing is like you've discovered this fact and it's very cheap to share it, so you should share it. One thing Tyler Cowan has said before, I think it was him and I. I've just stolen it. I don't know if he still holds himself to this, but he's like, I just produce so that they let me consume. And we're both selfish in this particular way.

[B]

I was going to say. So at least you're producing a little bit. Most people's problem is that they would lean over too much into consumption. Perhaps in your milieu, and certainly in San Francisco, it's the inverse. People only work. But.

[A]

Yeah.

[B]

How do you structure those.

[A]

Yeah, yeah, yeah. There is not a perfect balance because I definitely have this Protestant work ethic sort of thing, or I need to deserve being alive every day and that I should probably just be producing 100% of the time just to deserve being alive and deserve being as lucky as I am. Part of that is selfish in the way of they won't let me talk to these really cool AI researchers. If I don't produce anything. I'm just not going to be.

[B]

Consumption is unlocked.

[A]

Yeah, yeah, yeah. You've interviewed President Bush's personal assistant. Right. That's like extremely rarefied consumption. Or I so look up to that. Or like, must be so cool. You get to try out on Air Force One, like, what kind of special meal they have that day or something. You get the little napkin with the presidential seal on it. But they don't let anybody do that. Right. You've gotta be producing something. Either like writing as a White House correspondent or guarding the president or be the president. That's how you get to consume that.

[B]

I guess that's kind of what I do here. I'm producing just a little bit, so I get to be in the room.

[A]

Yeah, yeah. And it's super fun. And maybe this is how the market works. Or it's like, you want to consume this thing, well, then you have to produce the thing that's like, this is the price of consuming this is like producing this thing. I also think one should not be totally selfish, right. And production is good. Otherwise we would not have all of the wonderful things.

[B]

The automated laser beams are going to be doing it all for us.

[A]

And for most of the time, people wrote all that code by hand. That was a lot of production. And I bet even if they enjoyed it, they might have been writing like slightly different code that wasn't all for one goal and purpose. Of being able to. Yeah, of being able to consume better.

[B]

So what do you say to probably people who are more likely to be your peers, who are like, feel that there's no way they could possibly take time off or take a vacation in these pressing times? Well, maybe to take the other side of that question.

[A]

Yeah, a bunch of things. So because we're on an exponential now is better than ever. Now is in fact the last time. Second, all of the normal reasons why it's good. Like, you know, humans still operate on a certain timescale. And if we are, you know, packed end to end every day, we really aren't doing any thinking. And we're probably just like, inundated by all the noise and information. And it will literally be worse for you. The whole thing, I could say about like, mental health and that being the most important thing. And so if you're thinking about this, you should just probably take a break. And it's just like, win, win across the board for everything practically. You could vacation for a year and come back and your $20 a month subscription to AI will just be like 10 times more effective.

[B]

But what about the people who are working at these places? I mean, I've definitely talked to a number of friends in AI who are like, I literally can't imagine taking a day off.

[A]

If you're having fun, that's great. I'm just saying, like, I don't think the cost would really be that much. And yes, I'm sure everybody is like, contributing their own subjectivity and like, the whole project is very slightly different, but

[B]

you don't matter that much.

[A]

But you do in the way that if you take a break, the economy without AI will still grow at 2%. Right. And both are true. You do matter. And you don't matter.

[B]

I think that's right. Quote from you. I think it's possible his 2025 letter or one of the more recent essays Jonathan Malik Malasik in May reminded us that AI cannot teach us how we want to live. He writes of the humanities, I will sacrifice some length of my days to add depth to another person's experience of the rest of the theirs. Many did this for me. The work is slow. Its results often go unseen for years, but it is no gimmick. I think we've hit this point probably plenty, but do you have any advice for people on how to live?

[A]

Yeah, I think that's a beautiful quote by him and I quoted it at length and it's a great piece. When he wrote that post, he maybe laments AI a lot more than I do. And so I wanted to include it as. As like maybe to contrast some of the more optimistic about AI stuff. But when I read that, I think of it as like, that is sort of the choice of making something matter. In like the thought experiment you gave me earlier, where there was this writing thing, maybe you were doing it so that you could write things that other people would enjoy reading. And now there's this machine that can write things that other people enjoy reading more than what you were doing. And so that would feel like a loss of something that matters to you. At the same time, you and all your audience and everyone who's in this project of writing and reading things together can decide, okay, but it's only going to matter if we wrote it right and we're going to make this choice. And it matters because it's hard. Hard. It's not easy for me to write this thing, and it would be super easy for me. And other people would know it would be easy for me if I just prompted the machine and then I output the machine thing. So the value comes from the fact that I spent a part of my short and precious life to write this thing for you, even if by some eval it's worse. And then you've created meaning, you've created real work that matters as well. And I don't think it matters any less just because all the humans involved are eating food grown by robots or something like that, because they've all decided that's fine. That's not what we care about.

[B]

We get to make our meaning. We get to choose.

[A]

Yeah. And maybe later on at a different level of abstraction, farther into the future, it's like death. I'm going to send my starships out to these galaxies. And the work is hard to click this button or something like that.

[B]

You're playing Starcraft?

[A]

Yeah,

[B]

just a handful of additional miscellanea. One of My favorite parts of that recent letter is you talking about pluralism, marginalia and optimism. I think it's worth people go reading. But one the of the parts that really anchors it so well is Andor in season two of Andor. I love Andor, but I'll open up to you. Why are those three values so kind of highly prized to you?

[A]

Yeah, I think maybe pluralism is just like the highly most highly prized value to me, even though I recognize the contradiction of like, oh, what do you mean? The most highly prized value is pluralism or something like that. And the other two I just wrote as sort of like themes to personal life of the year, where I got to travel to a lot of different countries and I got to see a lot of tiny bits of bureaucracy. And then I got to see a lot of people being so optimistic. And then for that letter, I wrote about Isaiah Berlin a lot. And I really imagine him as both intellectually a role model, where I think I agree with him about a lot of what he says about pluralism. Not being able to rank order your values as a lot of philosophies related to AI are very prone to doing the hedgehog and the fox, as you know, but also a sort of role model as a consumer producer, where in his life he basically knew everybody who was alive at the time or got to talk to them and lived to this very full.

[B]

It was a good hang.

[A]

Yeah, exactly, exactly, exactly. And he also produced stuff and he also spoke to the general public or a lot of his lectures or his essays were very helpful to outside academia. Just explaining ideology in the 20th century. And one of my aspirations will be to just do a tiny bit to help explain technology, which I think is the thing of the 21st century. It probably has always been. But he has also a role model in that way. So talking about Berlin a lot, pluralism being an important value. And the other two are mainly just things that I felt like tied together. The other stuff that happened in the

[B]

year Marginality came up even in our conversation. It's the list of names, the people on the edges. What idea of Berlin's or piece of Berlin's do you think would be most impactful for people to know about or familiarize themselves with? Would it be hedgehog and the foxes something else or maybe not a piece, but just an idea.

[A]

Just the idea of the hedgehog and the fox I found really fun. And people have mentioned there's just something about animals imagining the animals.

[B]

I liked your little short story, by the way.

[A]

Oh, thanks. Yeah. So I Think even just the idea and reading a few pages of the essay. So it's quite a long essay and most of it is about 12 Tolstoy, actually. So I found it super interesting. Yeah, it is maybe a bit separate from just the idea where he wants to write an essay about Tolstoy, but he's like, okay, but how do I introduce this essay? Tolstoy doesn't fit in these two categories perfectly in the way that all these other people in history. Let's just begin with this thought experiment. This is what a hedgehog is. This is what a fox is. Who's a hedgehog? Who's a fox. A hedgehog knows one big thing, and a fox knows many things. So you can think of it as like a generalist or like someone who is obsessed with something. And then if you take in some kind of like, consilience, value, pluralism, everyone should really be both at the same time.

[B]

Always. It's always both. Just to wonder, optimistically, briefly, for a moment, how might things be if all of this really works? How might things be really great? What are you excited about?

[A]

Okay, so also to connect it to things not really being different because they're different all the time. I think that in the wealthy world, we basically have already reached this, right? Like, people who are very wealthy in the world today, probably a lot of the people listening to this are never have to worry about food or water or things like that, have a lot of choice in what they can do with their time, have a lot of choice in how they decide their own identity and what they value. Of course there are limitations. And as humans, maybe we focus too much on the limitations of, like, I have to do this. I really wish I didn't have to do that. And I think that this will become available to a lot more people. We have to deal with inequality as an issue. But just making everybody wealthier works when making people wealthier is basically free. But not only that, but second, that people who were already able to do this will sort of come face to face with the fact that you should be doing this earlier, you should be doing this now. You could evolve. Yeah, you could have always been doing this. The thing about AI as a thing, just like pandemic as a thing, or all these serious kind of near term thought experiments you could do about your own life as a thing will really make you come face to face with it. Maybe people already are and that's why some people turn to religion, something like that. And I think we will just be engaging with this question a Lot more. I also really am inspired by demises. Maybe we will really find the true nature of reality in some way. That's very convincing to us as well.

[B]

You write annual letters largely inspired, I think, by Dan Wong. Yes, there was a bit where you're quoting Dan advocating for the letters. First, Dan says, I don't understand why more people aren't writing them. It's not just about sharing your thoughts and recommendations with the rest of the world. Having this vessel that you're motivated to fill encourages being more observant and analytical in daily life too. And then you say, he's right. This letter, the only deadline I give myself every year, is an immensely powerful nudge to do more interesting things during the year, if only subconsciously. It's the best antidote to the temptation to time box research. Writing or enjoying life? How has writing these letters changed you? How has it improved your life?

[A]

Yeah, I think really a lot of it is subconscious where you think, okay, well, life is long or short, however much, much you say it. When you think of a whole year. And I don't know if people usually think in the Future in like 12 years, five years. Right.

[B]

You wrote somewhere, by the way, about like a 12 year plan.

[A]

Yeah, yeah, because it's like so, so divisible, you know, but yeah, in the same way that you, you can't really plan ahead and then, and then therefore you should really plan ahead. Right. Like, yeah, this is a tangent, but. But because the near future is so uncertain, then if you were to think like, who am I going to be 12 years ahead? You really focus on what doesn't change. Right. Or what do I really value that sort of thing. And that's also very clarifying. But yeah, subconsciously I think through the year you just think like, oh, this will be good for the bit.

[B]

Are you writing throughout the year?

[A]

No. So I have a Big Apple note that I just put bullets in. And this could be thoughts or this could be, I have to probably fit in this important event in AI at some point or just to not forget it. Or the model releases are so fast or so frequent, I'm like, did that model come out this year or last year? It's just one Big Apple note.

[B]

Is there anyone you really wish would write an annual letter?

[A]

I think all my friends should write one.

[B]

It's hard. These are like 12,000 words, but you

[A]

only have to do it once, once a year. I don't think I'm very productive as a person. There are people who are churning out 3,000 words a week. It's harder when it needs to be some kind of synthesis or you're reflecting on something instead of maybe just journaling. And there's no set boundaries of how far back you have to go or what you have to cover. But it's easier in that it is personal. And it just so happens that personal stuff is scarce, and personal stuff the AI will never be able to automate. Right. But, like, writing about personal stuff is a lot easier than I think. Than, like, okay, you have to reinvent philosophy and come up with a new theory.

[B]

You do a little bit of both. But how have you become a better writer over the course of these? You've done four of them now.

[A]

Yeah, so every year I write these. I also keep a little list of like, okay, these are things I have to remember about writing the next time around. Some of them are just things that everyone knows but are still really hard to do. Just getting words on the page is kind of the most important thing. And you have to get through the bad words to get to the good words. I think in the same way, in the rest of my life, I rewrite lists all the time. It's sort of like you have a row of your ducks and you're patting all the ducks just like, yes, you're still there. I've done the next sequential thing on the list. I think that often rewriting end to end is helpful. So even if you're writing the exact same words, it is sort of like you're reading it as a reader would read it. And by the time you're at the end of writing a longer piece and you've spent a lot of time with it, you're lost in the sauce. So for my last letter, just the opening few paragraphs, paragraphs of how to sort of just with words, try to AI pill somebody. It doesn't work on me anymore. Right. Because I've read it so many times. But maybe you can salvage that a little bit by. You start from a blank page, and you just write sequentially exactly as the reader would read it. And then in a very short context, you load it all into the memory of like, okay, so the reader knows this fact now. And now I've introduced this proper noun, and so I'm loading the reader's context as they would read it. Then I would get to a point of like, oh, this doesn't make any sense, because I'm lost in the sauce. And so I've mentioned this concept before even introducing it, or something like that.

[B]

Wow, it sounds exhausting. But probably good advice, probably effective.

[A]

It doesn't take that long, actually. I think, at least for me, main bottleneck of writing is just like, am I writing directionally Correct. Right. So. So all the time that I spent retyping the exact same words, it's not the bottleneck. And it can also add some momentum. Yeah, yeah, yeah, yeah.

[B]

Get the pedals going. Why do you love the Economist's obituaries?

[A]

Oh, yeah. So one is there's only 50 a year. Two is they pick subjects that are not, you know, just like, famous people. So, like, if the queen dies, the queen gets an obituary. But a lot of them are people most people will not have heard of. So someone who was just a really important member of this community, like this tiny island off the coast of Scotland, and there's this guy who's really important to that community, for example, or the last speaker of a language, or someone who started a school for disadvantaged children in this particular part of London, but something like that. So. So I just love learning about this or acquiring this new information that I never would have known otherwise. So a lot of it is curation. I think the curation is excellent. I think that the fact that they're written from the perspective of the person in a way that's more. It's not fictional, but there's more flourishes that are associated with trying to put you physically in the place. Maybe there wasn't an ocean breeze, but who does it hurt to pretend there was a breeze on this important day in this person's life? And all Economist pieces are quite concise, and so it's just wonderful.

[B]

One or two that you would very specifically recommend come to mind?

[A]

Yes. Okay. So Pasha Lee and Albert woodfox, both from 2022. And then I also want to say from that year, Thich Nhat Hanh. I emailed Enro, the editor, saying she should do an obituary for him. And she replied, it's like, I'm on it. So I don't know if I actually made a difference, but that's pretty.

[B]

It would be quite cor. I don't know. Correlation, causation. That's pretty cool. Wow. 100 years of solitude. Why is it your favorite?

[A]

It's just really beautiful. And I think it's a book about everything, so everything is in it, and maybe Even more than 100% of everything is in it because you've got people just suddenly floating off to heaven and gypsies visiting your village, showing off new technologies that never existed. So I think there's a way of Doing magical realism or trying to write a book about everything that just doesn't work. And this maybe doesn't answer your question, but I just feel like everything fits together so well. Or nothing feels out of place, or nothing feels. This is just an extraneous detail or something. I guess somehow everything just fits in super well. And generally in books and films and things, I like ensemble casts or like many generations, it's just so rich and Magnolia. Yes, yes, I. I love that one as well.

[B]

A book I love. When We Cease to Understand the World. You. This is you. I won't presume to tell you what you should think after reading this book, but surely everyone who reads this book will agree. Any scientist who reads this book and also thinks they're worth their work is worth a damn should think something. I. I don't. I don't have a question.

[A]

I just.

[B]

I wanted to read that. That was. That was good. I assume you've read the Maniac as well. Yes, I still haven't, but I'm curious if which book feels more resonant for the time. Is it still.

[A]

So I think when we cease to understand the world is more evergreen. And then for me, the last third of the maniac, which was about DeepMind and AlphaGo, I think it goes over a lot of the same ground of the documentary. So watching the documentary is good, but in general, I just think the author, Labratot, so he's apparently friends with Dumas as Jasmine. Yeah. And I think he is also really good at keeping in mind the sort of weirdness or the grandness of the scale. He would also be someone I would look towards whenever I'm like, oh, is all we have have really this techno singularity where at the end of it we get flying cars. No, we can be much more ambitious than that. We can face the whole scale of all we don't know in the universe and then that's really maybe the most ambitious we could be.

[B]

There's a quote from him, I think, in the interview with Jasmine that I'm sure I'm going to butcher. But it was something along the lines of that's the thing about humans. We're. We're far better at being than we are at knowing.

[A]

Yeah, every. Everyone should watch that interview with Jasmine.

[B]

It's really good.

[A]

Yeah, really good.

[B]

We're in London. You seem to have a lot of love for this place. You've called it the best pre AGI city and the best post AGI City.

[A]

That's right.

[B]

What do you love about it

[A]

other than the long litany of just like practical things like parks, best airport, court. I will not be elaborating food, all of these things. I just think there is. It's a great representation of pluralism. The fact that there has been 697 Lord Mayors of London or something like that's existed for a long time. There must be something about the city existing for that long, having such a diversity of anything that you want to do, you would be able to find a scene for it. The fact that people are so reasonable and so funny.

[B]

A good place to be a consumer.

[A]

Yes, for sure. The best place.

[B]

Do you think you'll ever make a game?

[A]

Yeah, I think so. And it's getting easier every year. So in terms of. Should you take a break or something? Yeah, I think I've always wanted to make like a Chinese history inspired Game of Thrones kind of thing. Yeah, I think people should write a series like this too. I've tried Ken Liu's Dandelion Dynasty. Unfortunately I couldn't get into it as much, but I think there's a lot of room for that.

[B]

What about container ships? Why. Why are you so into them? Maybe you're not so into them, but

[A]

I am so into them. They're just so efficient or like, yeah, we don't wish for world government or anything. But the fact that everybody has agreed on this kind of standardization, I think the benefits of how much it's improved our lives is just hard to comprehend. Just, you know how like, like people, they ship trash to a different country to be sorted and then shipped back. That turns out to be the most efficient way to do it. Maybe.

[B]

Okay, so there are few ways that top down, total control or at least total collective decision making can be pretty good.

[A]

I think just agreeing on some standards, which doesn't seem like it should be that high stakes, just could have like a lot of benefits. Yeah,

[B]

you. I think I referenced it earlier. You often cite this Nabil Qureshi and Dan Wong kind of advice on productivity, which is just do more. I think you say you can have free lunch across the Pareto front. You also somewhere else. Forgive me, because I don't. I don't have the date written down, so it's possible this was two years ago. But this is in one of the letters. You say the problem is more general than exercise though. If I want to read, play music, practice Chinese, pick up new hobbies. And it isn't happening more by now. What makes me think it has a better chance of happening later? It's time to either change or quit. I was optimistic last year about a big virtuous cycle where doing everything makes everything else easier. I'm going to take the opposite view this year that I should be honest about the actual trade offs I face either way. It sounds trite but I'll figure it out one day. Obviously I don't think these things are fully mutually exclusive but I think one of the things we kind of gradually learn sometimes it hits you in the face is that there isn't that much time.

[A]

Yeah.

[B]

And there you really do have to choose. I think the doing more thing is also true. But I, I was just curious how that. How it's going.

[A]

Yeah. I think the year after that I sort of even said slightly the opposite.

[B]

Yeah.

[A]

Where I'm going to like do more. And so yeah I think a lot like a lot of things it's you like, like, like when we started off talking about, about research, you take a position that is maybe slightly too much to the extreme and you learn something about it and then you just develop a better taste or prioritization so that in the future you are sort of doing the balance better.

[B]

Do you ever let your computers idle overnight?

[A]

So I think this depends on compute allocation for the team where it's shared across the team. So I rest assured that the computers are never idle.

[B]

Well, I feel like including many non AI researchers. Most of the people I know are paranoid and freaked out to ever leave Claude not running on their computer.

[A]

Yeah but that's also no way to live. I think there are bigger costs to that where yes you're letting your computer or Claude idle but if your production function which I really think all humans are, you just need time to think and one should not be concerning themselves with tiny things like optimizing their usage so that you can focus on bigger problems. Maybe you should hire somebody to use

[B]

your

[A]

quota efficiently but you just need to be focused on the big problems.

[B]

Right.

[A]

And so there will be bigger costs that come with if you micromanage your

[B]

quota, your time, maybe leave the optimization to the machines.

[A]

Sounds good.

[B]

Uh, I worth shouting her out because I'm gonna, I'm having dinner with her tonight. Um, but I know you love it. Why do you resonate so much with everything's a scam.

[A]

Yeah. I think it's just a great reminder like maybe maybe in, in the same kind of of thing where every time you read a work of fiction say foundation and it just makes one point. Super super. Well just like this is a great reminder, a lot of the things rivates does is a great reminder. The fact that she made this a song and it's on Spotify it's very much you can just do things. Everything is a scam opens up your ability to do things. I'm also very insp. Inspired by her opening a toy store in London. A lot of other schemes that she's running and scam isn't so bad. It's just the rules are changeable or the rules are made up or not Totally set in stone indeed.

[B]

My last thing is a quote from Burke that you quote in one of your letters. This felt to me fitting because it seems like a case for the long journey of research into something discovery and adventure. I'm quoting now by slow but well sustained progress the effect of each step is watched the good or ill success. The first gives light to us in the second and so from light to light we are conducted with safety through the whole series we compensate, we reconcile, we balance. We are enabled to unite in a consistent whole the various anomalies and contending principles that are found in the minds, in effect, affairs of men. From hence arises not an excellence in simplicity, but one far superior, an excellence in composition.

[A]

I think it just encapsulates the point I was trying to make there, but also what we talked about of trying to make the transition to powerful AI go well, there is just a lot of talk about this time is different, about step changes, about distinction. Right. And that's very useful in shaking people and getting them to feel the wave crashing over them. But once you get there, or sort of be careful what you wish for, maybe you don't want the prize on offer. And really the more effective or the better way to do it, it is there are these promises that we've made before and we're going to keep them. And the important stuff is not going to change. The important stuff has always been there. AI just makes you face the questions that you should have been facing all along a lot sooner or at the right time. And it's great that something that Burke wrote so long ago is still relevant and a great point for him as well indeed.

[B]

Anything else you want to talk about? That's all I got. Zhangdong, thank you so much. This was wonderful.

[A]

Thank you. It's been wonderful.

[B]

Thank you for listening to my conversation with Zhangdong. You can share the episode with a friend if you want to help out. That certainly means the most. You can also rate it, give it a review on Spotify or wherever you're watching or listening, and of course subscribe or follow too if you want more more episodes. And once again I'd like to thank Notion for making the show possible. Like I talked about with Zhengdong, the capabilities and the speed of improvement in AI is very hard to keep track of, and the way Notion integrates AI and agents makes it easy to have a single place where you can actually coordinate everything and have access to the latest models the day they come out, whether that be OpenAI, anthropic, Google, open source models or otherwise. I think the way the future of work is going is that you and your collaborators will manage and work with a wide range of agents doing all kinds of different work and use AI to give you leverage on the work that you don't want to automate. Notion is the ideal hub or even operating system to run your company, your team, your project from because you have access to the work itself, all of your collaborators, and these agents that give everyone superpowers. Thanks again to Notion and you can learn [e-mail] com dialectic thank you for listening and supporting the show and I will see you next time.