OpenAI cofounder Greg Brockman on the scaling hypothesis and refactoring as a killer AI use case

Summary

Cheeky Pint Podcast Summary

Episode: OpenAI cofounder Greg Brockman on the scaling hypothesis and refactoring as a killer AI use case
Date: June 18, 2025
Host: Stripe (co-founder John Collison, with Lex Fridman as interviewer substitute)
Guest: Greg Brockman (OpenAI cofounder, ex-Stripe CTO)

---

Episode Overview

This episode features a candid conversation with Greg Brockman, cofounder of OpenAI, exploring the origins and trajectory of OpenAI’s approach to building Artificial General Intelligence (AGI). The discussion delves deeply into the “scaling hypothesis,” breakthroughs in deep learning, lessons from the Dota 2 AI project, shifting benchmarks in AI, and core product decisions at OpenAI. The dialogue also covers the evolving landscape of AI use cases, bottlenecks, and the unique organizational and personal perspectives that have shaped OpenAI's journey.

---

Key Discussion Points & Insights

1. The "Scaling Hypothesis" and OpenAI’s Origin Story

• Reverse Startup Philosophy
  • OpenAI did not start with a well-defined problem, contrary to traditional startup advice.
    • “This is totally backwards from how you're supposed to do a startup, right?... You're supposed to have a problem and we had no idea what the problem was.” — Greg Brockman [00:00]
• Early Signals in AI Progress (2012-2014)
  • Exponential improvements in deep learning for computer vision and translation signaled a new paradigm.
    • “Suddenly you're getting great results in machine transl. And I think this pattern was applied in subfield after subfield.” — Greg Brockman [01:04]

2. Why Did Deep Learning Suddenly Take Off?

• Critical Mass of Compute Power
  • Advances in hardware enabled previously theoretical algorithms to work.
    • “If you look at the number of orders of magnitude of compute that we've gone through from 1940 to today, I mean, it's just astounding.” — Greg Brockman [02:32]
• Scalable Algorithms & Observing the Scaling Law
  • OpenAI’s Dota 2 project showed that simply scaling up compute repeatedly improved performance.
    • “They'd scale up by another 2x and we had 2x performance. And just so clear, you just need to keep going. Like, where does this thing peter out? And it just never did.” — Greg Brockman [03:14]

3. The Dota 2 AI Project: Lessons in Management and Machine Learning

• Outcome vs Process-Driven Management
  • Attempts to set outcome-based milestones failed. Focusing on experimental inputs yielded progress.
    • “You cannot set outcome based milestones. What you can do is you can control the inputs...” — Greg Brockman [05:19]
• Unexpected Behaviors and Magic in Deep Learning
  • AI agents began inventing novel strategies (e.g., baiting), reflecting deep learning’s non-linear creativity.
    • “It had learned a baiting strategy...that bot was just undefeatable and we played against this number one player and won. And to me, this is like the story of how deep learning works, right?” — Greg Brockman [06:53]

4. Rethinking the Turing Test and AI Milestones

• Beyond the Traditional Turing Test
  • Real progress is measured by economic value and daily impact, not just indistinguishability from humans.
    • “If you look at the strict version of the Turing Test, I would actually claim we haven't done it yet...But I think that the right question...is like, well, what is the milestone that we should be chasing in terms of capability?” — Greg Brockman [08:11]
• Personalization as the Next Frontier
  • AI tools will become more valuable as they gain persistent memory and can offer personalized interactions.
    • “Now my usage has totally reversed. I want ChatGPT to know, to remember everything. I want it to remember all of my interactions because it's useful.” — Greg Brockman [09:26]

5. Productization at OpenAI: Following the Technology

• API First, Use-Cases Followed
  • They released GPT-3 as an API with no specific product, seeing what the world would do with it.
    • “We wrote down a list of like 100 different products. Right. We could do a medical thing...You give up on the G in AGI...So someone had the idea of saying, well, why don't we just make an API and let people figure it out.” — Greg Brockman [10:11]
• Unexpected Early Traction
  • First real traction and revenue came from AI Dungeon, a text-based adventure game—far from their initial aspirations for medicine and education.
    • “AI Dungeon was a text based adventure game...I believe they were our first paying user.” — Greg Brockman [11:51]

6. AI’s Real-World Impact: Medicine, Life Coaching, and Education

• Barriers are Lower Than Expected
  • Applications in medicine took off quicker than expected because the existing alternatives were poor.
    • “Medicine is an example of one where I kind of thought it was going to be one of the last domains...but it turns out that the bar is so low and you just need to exceed WebMD.” — Greg Brockman [13:42]
• Other Breakout Use Cases
  • Life coaching and education (“Bloom 2 Sigma effect”) show measurable improvement and huge promise.
    • “Life advice kind of application...education is another area...there are studies coming out now that actually show that people are able to learn better through the use of these tools.” — Greg Brockman [14:07]

7. Bottlenecks: Operating Systems, Energy, and Data

• Hardware and Interface Limitations
  • Current AI adoption is stymied by OS-level restrictions; as capability grows, convenience will catch up.
    • “There's capability and convenience. What you're referring to is the convenience, Right. It's like, pretty inconvenient to do the screenshot and paste it. But the thing is, if the capability is good enough, you are willing to accept any sort of inconvenience.” — Greg Brockman [17:00]
• Compute, Energy, and the “Wall”
  • Compute and energy will eventually present harder bottlenecks than data.
    • “It really should be that it's energy manufactured into intelligence and that that's your only bottleneck.” — Greg Brockman [20:28]

8. AI as Scientific and Coding Collaborator

• Can AI Produce Groundbreaking Discoveries?
  • Greg anticipates AI solving Millennium Prize problems within two to five years, given enough compute.
    • “Just wait...I would put two to five years as the right number. And I think ultimately this comes back to the question of benchmarks...” — Greg Brockman [18:29]
• Vibe Coding and Refactoring Are the Future
  • Next generations of coding AI will excel at unglamorous but vital tasks, like massive codebase refactoring.
    • “Maybe the killer enterprise features is refactors. It's like rewriting your Coppola app or changing your Facebook.” — Greg Brockman [26:17]
  • AIs managing teams, not just acting as software engineers, are entirely plausible—and potentially transformative.
    • “Is there a world where the AI becomes the manager and that gives you ideas and gives you some tasks to do. And that's something that again is just like totally backwards in terms of how we think about it.” — Greg Brockman [24:58]

9. Organization and Personal Background

• Blurred Lines Between Research and Product
  • OpenAI aims to avoid siloing product and research, favoring collaboration and agility.
    • “We really want to blur the lines and have people cross, collaborate. And so it's very different mindsets from how you would traditionally build a product. Versus how you do research.” — Greg Brockman [10:11]
• Product Strategy: Focus and Synergies
  • Core model is the “asset”; applications are built based on ease, impact, and synergy.
    • “Maybe an analogy is to a company like Disney where you make one core asset like Little Mermaid...And then you productize it in all these different ways.” — Greg Brockman [27:41]

10. Personal Anecdotes: Growing Up in North Dakota

• Value of Unstructured, Supportive Environments
  • Early academic freedom and lack of distraction enabled self-driven learning and eventual technical excellence.
    • “Our doors didn't even have working locks...I had a lot of freedom academically.” — Greg Brockman [29:00]

11. Reflection on AGI Predictions

• Surprising Paths to Progress
  • Original timelines for AGI were optimistic but missing nuance; progress has been unpredictable but magical.
    • “AI is surprising. I think that that is like the single most consistent theme. Is that the thing we were picturing? We got something different, but we got something better, more magical, something that is more helpful.” — Greg Brockman [30:53]
  • OpenAI’s internal metric: at least one step-function breakthrough per year.
    • “One goal of OpenAI that we have successfully achieved is every year to have at least one result that just feels like a step function better than anything before.” — Greg Brockman [31:20]

---

Notable Quotes & Moments

• On the Scaling Law:
  “You just need to keep going. Like, where does this thing peter out? And it just never did.” — Greg Brockman [03:14]

• On Counterintuitive Startups:
  “API and let people figure it out...this is totally backwards from how you're supposed to do a startup.” — Greg Brockman [10:11]

• On AI’s Biggest Early Product Success:
  “AI Dungeon was a text based adventure game...I believe they were our first paying user.” — Greg Brockman [11:51]

• On OpenAI’s Guiding Principle:
  “Every year to have at least one result that just feels like a step function better than anything before.” — Greg Brockman [31:20]

• On Future of Coding:
  “Maybe the killer enterprise feature is refactors. It’s like rewriting your Coppola app or changing your Facebook.” — Greg Brockman [26:17]

• On Capabilities vs. Convenience:
  “If the capability is high enough, people will start doing a specific thing...then the convenience, there's so much pressure...to bring down the [barriers].” — Greg Brockman [17:00, 17:45]

---

Timestamps for Key Segments

• [00:00] — Why OpenAI did everything “backwards” and the genesis of the scaling law
• [03:14] — Dota 2 project: scaling compute and management lessons
• [08:11] — The Turing Test, economic value, and personalization
• [10:11] — Product/research interplay, API-first strategy, AI Dungeon
• [13:42] — AI’s early impact on medicine, life coaching, education
• [17:00] — Product bottlenecks: capability vs. convenience, operating system limits
• [18:29] — AI as mathematician: expectations for true scientific breakthroughs
• [20:28] — Energy and compute as the inevitable bottleneck
• [24:02] — “Vibe coding,” full-stack AI coworkers, and the centrality of codebase refactoring
• [27:41] — How OpenAI thinks about new products (Disney analogy)
• [29:00] — Brockman’s unique upbringing and early math experiences
• [30:53] — Predicting AGI: reflections and OpenAI’s metric for success

---

Conclusion

In this engaging and insight-rich conversation, Greg Brockman lays bare the counterintuitive, conviction-driven, and ever-evolving journey of OpenAI—illuminating why the biggest leaps in AI have come from both relentless scaling and openness to emergent, unpredictable value. The future of AI, according to Brockman, lies at the intersection of capability, convenience, and collaboration, with breakthrough applications often discovered in the most surprising corners of daily life and software development.

Transcript

[Sam Altman]

This is totally backwards from how you're supposed to do a startup, right? You're supposed to have a problem and we had no idea what the problem was. Is there a world where the AI becomes the manager and that, you know, gives you ideas and gives you some tasks to do? This was probably the hardest project I've ever done because it felt totally doomed. Right. It's like, I know, like every instinct, every builder instinct of mine actually feel doomed. Oh, it felt totally.

[Lex Fridman]

Greg in 2010 dropped out of MIT to become our first engineer and went on to become Stripe's CTO. In 2015, after he left, he co founded OpenAI. OpenAI is really cooking at the moment and Greg is one of the most productive people I know.

[Sam Altman]

Yours. Wow, you really have some old school.

[Lex Fridman]

Photos, some deep cut GDB trivia. So, yeah, we figure we've got to put people at ease, you know, make people feel at home.

[Sam Altman]

Very nice, thank you.

[Lex Fridman]

Okay, well, I just dive straight into all the questions I have, which is the last.

[Sam Altman]

All right.

[Lex Fridman]

If you were not working in AI, how could one have known that something was about to start working? People were telling you that AI was the future in the 1970s, in the 1980s and the 1990s. And then very quickly in the late 2010s, everything started happening.

[Sam Altman]

Well, I was someone who was not in the field. And so I remember very much what it was like 2013, 2014, it felt like every day on Hacker News there'd be a new deep learning for X article. And I remember being like, what is deep learning? And I knew like one person in the field and I asked them to introduce me to more people in the field. And I just kept getting introduced to a bunch of my smartest friends from college. Now, if you actually look at the work that was being done, your 2012, basically image recognition, for the first time, you could solve with the neural net much better than anything else. And it just blew all these traditional computer vision approaches out of the water. It's like this learned system that was able to outperform 40 years worth of let's write down all the rules and try to sort of handcraft the algorithm for the task. And it's very easy to then be like, okay, well this approach, sure, it works for computer vision, but it's never going to work for machine translation. In 2014, suddenly you're getting great results in machine transl. And I think that this pattern was applied in subfield after subfield.

[Lex Fridman]

One thing I've been wondering about is so many different things are finally working at the same time. And so we have LLMs, which are obviously amazing, but then we also separately have image models really working. And we also have text to speech and speech to text working way better than they were before. And so what's the common factor behind everything starting to work at the same time?

[Sam Altman]

Well, it's deep learning, right? I think deep learning is the core.

[Lex Fridman]

Why deep learning for a long time? Like, you know, why didn't deep learning work in the 1980s?

[Sam Altman]

Well, so, okay, so if you look at the number of orders of magnitude of compute that we've gone through from 1940 to today, I mean, it's just astounding.

[Lex Fridman]

You think they're all explained by compute scale ups applied to the right algorithms?

[Sam Altman]

Of course the type of algorithm changes. And some of those results aren't even deep learning. But I think that fundamentally it is about compute and you need an algorithm that is scalable, that can actually absorb that compute.

[Lex Fridman]

Was OpenAI the first company to take the scaling hypothesis really seriously?

[Sam Altman]

I think that claiming the first is always difficult, but I think that it is clear that we. I sort of succeeded much more wildly sooner than anyone else. And so I think that we had real conviction behind what we needed to do. Some people think that OpenAI set out to prove the scale hypothesis, whereas it was almost the other way around. The scale hypothesis is what we observed as the thing that was working for us and really saw it for the first time. Actually, during our Dota 2 project, we started out with 16 cores to train a little agent on Jakob and Shimon who are getting the project from an ML perspective on desktop. And then they scaled up to 32 cores. And it felt like every week I'd come back to the office and they'd scale up by another 2x and we had 2x performance. And just so clear, you just need to keep going. Like, where does this thing peter out? And it just never did.

[Lex Fridman]

Founders get too much credit because you have an initial product that's like a pretty reasonable idea. And then you listen to the customers and you follow what's working and you're saying that was kind of OpenAI with the scaling hypothesis where you started trying to make Dota AI's work and you noticed that adding more computers worked really well. And you said, where else we'll just throw more compute out of yield benefits.

[Sam Altman]

I think that's like to first order correct. And I think one thing that distinguishes OpenAI from the typical startup is we did everything in reverse, right? It's like you're supposed to Have a problem to solve. No one cares about the technology form.

[Lex Fridman]

The entity up front.

[Sam Altman]

Exactly. Yes. And for us, we really chase the technology without any idea of how it would be applied.

[Lex Fridman]

Yes.

[Sam Altman]

And a lot of pursuing a technology really is you have to let reality hit you cold hard in the face. There's just no other way to achieve results. You can't will it into existence. You can't convince people that this is the thing. It's like you have to actually make the system work. We have to just sort of figure out what is the right frontier, what are the problems, what are the things that are on the edge of working. And to really double down on those.

[Lex Fridman]

What else do you take away from the dodo work? How else does it. You could say you could have just started with LLMs and we could have skipped that period in the wilderness. But it sounds like it was somewhat formative for the OpenAI organization.

[Sam Altman]

Well, so I think Dota had many lessons, one of which actually was a management lesson for me. I remember when we started out the project, I tried to set a list of milestones, right? It's like, okay, this date we're going to beat this player. This date we're going to beat this player.

[Lex Fridman]

That didn't work.

[Sam Altman]

It did not work at all. I remember our first milestone came and. Exactly. And so you realize that you cannot control the outcome, right? You cannot set outcome based milestones. What you can do is you can control the inputs of. We're going to try these experiments by this date. We're going to implement this feature by this date. And that is what actually worked. And I remember it was one of those things that was just like a story that I could not have sort of written it any better if we'd intended to. Where we beat our in house like best player. And then we were playing as a semi pro and he was just trouncing us, trouncing us. And then suddenly we were starting to get pretty good. So we showed up at the international, this tournament blind first day. We had three players that we played against. We went 3, 030 and then two, one we're like, oh no, we lost. What happened? And it turned out that this pro that we're playing against, that he had used an item we'd never trained against. And we were like, oh no, we're totally going to be hosed. So what do we do? Well, we just need to change the training. And so people stayed up all night to get this done. They added this extra item in there. 4:00am they finally get the job running that Wednesday, we're supposed to play against the number two and the number one player in the world. And our semi pro plays against it. And he's like, this bot is totally broken. And we're like, oh no, we clearly had a bug. Something terrible has happened. And he was like, look, it's taking all this damage. It doesn't need to. I'm going to go kill it. He goes in to kill it, he loses. He was like, that was weird. He'd realized that what had happened was it had learned a baiting strategy. And then we realized, well, we have a superbot, but it's so bad at the beginning because it's trying to do the baiting. So what if we just stitched the two bots we have together and then that bot was just undefeatable and we played against this number one player and won. And to me, this is like the story of how deep learning works, right? Is it's like you kind of can't control where you're going to go. You can control everything that goes in. You can put these metrics and these measurements and you can have sort of the evaluations and that being able to gauge where you're at is almost as important as being able to make the forward progress. But if you get all those elements right, then you can do true magic.

[Lex Fridman]

Aren't you also describing something that worked really well for an organization where it was motivating to stay up all night? If the prize was impossibly far away, it wouldn't have been as motivating. But the fact that there was a near term reward function and you were able to show concrete progress.

[Sam Altman]

I think so, yeah. I think some of my favorite engineering stories have the same character. I remember you and I staying up all night to get our ISO 8583 integrated.

[Lex Fridman]

There's something about staying up all night for critical projects that actually have important history. Turns out, yeah, all startups. Glad to hear the tradition of the live wild opening eye. So Dota has fallen. Chess has fallen. Go has fallen. We've passed the Turing test, I think by anyone's measure, with people comment on how there was little fanfare when we did, but we seem to have pretty clearly done. So what's a good new Turing Test?

[Sam Altman]

Well, I'll tell you two things. One is that if you look at the strict version of the Turing Test, I would actually claim we haven't done it yet. So no one's really gone that extra mile to say, can we actually have an AI that is fully indistinguishable from a human. And it's not clear if it's even a good task. Right. But I think that the right question to your point is like, well, what is the milestone that we should be chasing in terms of capability? I remember talking to one of our board members in 2018 and he said, look, I get that we're all excited about near term AGI, but it just doesn't feel like is on track. And I asked, well, what do you mean? He said, in a world with near term AGI, you would expect massive economic value to be delivered by AI already and where is it? And in 2018, I think that was a very fair criticism. And clearly that's starting to change now.

[Lex Fridman]

It feels like one thing that may really change the AI market is personalization. Up to quite recently, when you asked ChatGPT a question, it was like walking into a shop off the street. They've never met you before, they know nothing about you, whatever. That's obviously not ideal for this close part of your digital life. I'm curious how you're thinking about personalization from a product point of view, because it feels to me like the most meaningful change since the chat interface two and a half years ago.

[Sam Altman]

Two and a half years ago. I mean, I think it's absolutely critical and I think it is very rightly considered to be kind of a next frontier. I am someone who always, when I just Google something, I go into incognito mode because I don't even want my computer to remember that history. And I always used to go for temporary chats on ChatGPT, but now my usage has totally reversed. I want ChatGPT to know, to remember everything. I want it to remember all of my interactions because it's useful.

[Lex Fridman]

Okay, so you guys figured out from a product point of view how to make the memory actually work better.

[Sam Altman]

And this is actually so a product point of view, but also really research point of view.

[Lex Fridman]

And I presume there's a flip flop between the product and research, where when you find something that's useful from a product point of view, then the product people say, and I am just a product person like you researchers go, actually make this good. And then that kind of kicks off more research.

[Sam Altman]

Well, we actually. So to some extent that's a failure mode in our. That I think that we really don't want to have that kind of silo and we really want to blur the lines and have people cross, collaborate. And so it's very different mindsets from how you would traditionally build a product. Versus how you do research. And part of what had happened actually was that we had GPT3. We knew we needed to build a product in order to be able to continue to raise funding. And we were like, well, what product do we build? And we wrote down a list of like 100 different products. Right. We could do a medical thing. And then you're like, okay, well now we have to sell to hospitals, we're going to have to hire doctors. And you just realize you give up on the G in AGI, Right. You're going to go for a specific thing. And so someone had the idea of saying, well, why don't we just make a API and let people figure it out. And again, this is totally backwards from how you're supposed to do a startup. Right. You're supposed to have a problem. And we had no idea what the problem was.

[Lex Fridman]

Yeah.

[Sam Altman]

And so we're going back into the problem. And so this actually felt like this was probably the hardest project that I've ever done because it felt totally doomed. Right. It's like, I know like every instinct, every builder instinct of mine actually feel doomed. Oh, it felt totally doomed.

[Lex Fridman]

It wasn't just like open ended or something?

[Sam Altman]

No, it felt doomed.

[Lex Fridman]

You were still doing it.

[Sam Altman]

I mean, it's like at some point if you have. There is definitely no other path. There is no other path. It was the only shot we had. I remember someone also saying like, I can't imagine anyone paying for samples from this model. And I was like, might be right.

[Lex Fridman]

I'm still trying to imagine it myself.

[Sam Altman]

Yes. And it was just not clear were we above threshold or below threshold. And we showed it to people and people were interested. But that's very different from people being like, I will build my company on top of this.

[Lex Fridman]

So what was the first use case to get any traction?

[Sam Altman]

So AI Dungeon. What was that again? There you go. So AI Dungeon was a text based adventure game.

[Lex Fridman]

Oh, sure, yeah.

[Sam Altman]

Yes.

[Lex Fridman]

Yeah. Okay. But that was like. That was real revenue or that was non zero revenue.

[Sam Altman]

Yes. And in fact I believe they were our first paying user.

[Lex Fridman]

And that get you confused where you're like, ah, clearly the future of OpenAI is gaming.

[Sam Altman]

I know, Right back to our roots.

[Lex Fridman]

Exactly.

[Sam Altman]

And it's interesting too because yeah, we had dreamed of all of these applications and medicine and all these things and you start with the gaming application, but we could see signs of life on so many other things. I think in many ways GPT3 was like the world's best demo machine. Right. When we released the API, people were Coming with all these cool things you could do, but making them reliable, reliable was so hard. And it really wasn't until the next generation of GPT4 until we started to figure out how to do post training well, that then you were actually able to build real businesses on top of these things.

[Lex Fridman]

Bill Gates was saying recently that GPT4 was the best demo he'd ever seen since Xerox PARC. You know this quote since, I guess.

[Sam Altman]

He said it to me the night that he saw it. Yes.

[Lex Fridman]

So that's high praise. I want to touch on the medicine thing because you've mentioned it. So like you said, you know, I think your family has kind of personal stories. You've talked about getting very valuable diagnostic help. We ourselves, actually, it's much more minor in our family, but we managed to fix a cat thanks to, you know, debugging it with an LLM. And I think this is an interesting example. Right, because so many people that I know have had some kind of experience like this. And maybe it's because you actually don't get that much time from a doctor. Are there other examples like this medicine application where you're seeing a lot of success, that many people have similar stories but we just hear less about?

[Sam Altman]

Yeah, I think it's a great question. And by the way, I think like medicine is an example of one where I kind of thought it was going to be one of the last domains that we successfully be able to add value in, but it turns out that the bar is so low and you just need to exceed WebMD. And so I think that we have seen sort of other areas that are like a real common theme. Like one that's very interesting right now is like the life coach, like life advice kind of application where you just.

[Lex Fridman]

Talked about that it's actually really taking off.

[Sam Altman]

Yeah, it really is. And so I think that there's things like education is another area that just like clearly is really having an impact. And there are studies coming out now that actually show that people are able to learn better through the use of these tools.

[Lex Fridman]

That's to be expected, right? Like it is the Bloom 2 Sigma effect in the product.

[Sam Altman]

Yes, yes. And that, for example, is like why Sal Khan, that's why he started Khan Academy, was to think about if you can give personalized tutoring to everyone. And we showed him GPT4, he's like, this is the thing we need to become a GPT4 app. And so I think that there are these really amazing applications that are affecting everyone's daily lives. Obviously programming is Another one I think that people are seeing all across the board in professional context. We're heading to a world where just like, if you want to do productive work, you don't have access to computer, you're going to be hampered. And so similarly, not having access to AI is heading in the same direction.

[Lex Fridman]

Okay, so speaking of not having access to AI, I will posit that these days it feels like AI product development is mostly OS limited. Is that how you feel? Like, are we stuck at the moment?

[Sam Altman]

I do feel a little of the stuckage. But not to worry, it is overcomeable. But, yeah, I think it is true. Look, like two years ago we released plugins in ChatGPT. Do you remember those? Yeah, yeah. And that was like trying to make it so anyone could write apps that then ChatGPT could access. And the models were just not that good. Right. We limited it to like three plugins at a time. You could have only so many functions and stuff, and it just wasn't that reliable. And now we're in a world where MCP basically is really taking off and is a way to hook up your AI to different tools and very much like kind of trying to take that same type of idea and really make it work. Now, the world that we're in is very similar, where there's certain interfaces we don't have, being able to access your phone and all those APIs. And there's a question of is the model above threshold to actually use them or not? And my observation has been that basically I think that there is maybe a lag of like six months of different interfaces that are hard to access, but once we have a model that's good enough, we will find a way, people will find a way. And so I think that we're in a world where I have every expectation that we will get the future that has been promised, but it's just going to take some work.

[Lex Fridman]

I feel like there are many moments where I'm using my phone and I want a single button where it just. It's like, you know, chatgpt, what do you think of this? Like, I need your comment, I need your fact check, I need your explanation. Something like that. And you take a screenshot and you, like go into ChatGPT and you click upload photo and, you know, it feels like very 1993 versus the button on my phone that just says, hey, ChatGPT, what do you think about this? And obviously you guys are not empowered to go build that. That's what I mean by it feels somehow like we're a little operating system.

[Sam Altman]

Well, I definitely get it, but I'll say, I think that there are two dimensions. And this is kind of how I've been thinking about things since we released the API back in 2020. There's capability and convenience. What you're referring to is the convenience, Right. It's like, pretty inconvenient to do the screenshot and paste it. But the thing is, if the capability is good enough, you are willing to accept any sort of inconvenience. Right. It's like if this, you know, by taking the screenshot and showing it to ChatGPT, it could give you amazing insight. It could tell you, like, how to, you know, build stripe in some way. And it takes you like a month to do it. You have to crawl to the top of the mountain. Like, you'll do it, right? Like, the convenience will not stop you. And so the point that I'm trying to make is that if the capability is high enough, people will start doing a specific thing.

[Lex Fridman]

They'll discover the use cases and the convenience will just catch up.

[Sam Altman]

And then in the convenience, there's so much pressure. There's pressure on the phone manufacturer, there's pressure on us, there's pressure on everyone in order to bring down the.

[Lex Fridman]

So I just need to be patient. It'll be great in three years time.

[Sam Altman]

Yes. And really lean in. Use the AI.

[Lex Fridman]

A criticism people like to levy of AI is, yeah, it's great and handy and all, but it hasn't come up with a single novel advance in mathematics or science. Well, you could have if you'd become a mathematician, but humanity has for keeping the scoreboard. What do you make of that criticism?

[Sam Altman]

Just wait.

[Lex Fridman]

Okay, so you think, take one of the Millennium Prizes. You think we plausibly will see that?

[Sam Altman]

I think for sure. I mean, there's no question like 2.

[Lex Fridman]

Years, 5 years, 10 years.

[Sam Altman]

I think that is the question. It's just timing, right? That is my. I mean, I would put two to five years as the right number. And I think ultimately this comes back to the question of benchmarks, right? Is that actually being able to solve a millennium problem is pretty high bar.

[Lex Fridman]

Yeah, pretty high bar.

[Sam Altman]

And once you can do that, there's so many other things that will definitely be possible. And I think that we're starting to see the leading edges of this. And to me, if we look at our definition of AGI, we recently started talking about this framework of thinking about levels of AGI, starting from chatbots to reasoners to agents to innovators to organizations, five levels and we're basically somewhere in level three right now. And level four, there's innovators. That's going to be different. I recently posted some pictures of our visit to Abilene, Texas, where we're building these big data centers together with our partner Oracle. And imagine taking that whole data center and just thinking hard about one problem, right? Imagine it just thinking about how to solve a millennium problem or how to cure a specific kind of cancer. Maybe it needs access to some apparatus, maybe it needs access to robotic wet labs, maybe it needs access to different tools in the world. But that level of computational power, coupled with the ability to experiment and learn from your ideas, that is going to be something the world has never seen.

[Lex Fridman]

Okay, so yet again, we haven't put a respectable amount of compute on these problems compared to what we will be doing.

[Sam Altman]

Yeah, we still have these tiny little computers.

[Lex Fridman]

So that actually gets to. In terms of these scaling laws, do they eventually run out because we just run out of compute, or do we eventually get to the point where we're inventing new kinds of nuclear energy? And so that is what unlocks the next level. A lot of energy that comes online now is for data centers, which was not true when you guys started training GPT2. And so isn't that just the upcoming bottleneck?

[Sam Altman]

I mean, it's as it should be. It really should be that it's energy manufactured into intelligence and that that's your only bottleneck.

[Lex Fridman]

But I'm saying that'll be like quite a plateau compared to the exponential growth we've seen over the past few years.

[Sam Altman]

Well, I think unless things really change.

[Lex Fridman]

In terms of permitting and plans for.

[Sam Altman]

Building and everything like that, this is, I think, the core. Right, Is that if you look at every trend in this field, there's these exponentials, these S curves that sum up to exponentials. Sure.

[Lex Fridman]

But these exponentials were mostly existing in like tech Silicon Valley space where it was pretty easy to have exponential growth. It's pretty hard in permitting and real estate and damming rivers and building nuclear power plants. It's harder to have exponential growth there.

[Sam Altman]

Let's see how fusion pans out.

[Lex Fridman]

Yeah, okay. But even fusion, most industry observers would say, is still kind of five years away. And so where it's the next five years of.

[Sam Altman]

So look, I think that it is very possible that we end up bottlenecked on energy. And that's actually one reason that we've been spending a lot of time really trying to advocate for the fact that we just need far more power. And I think that my observation of the market is that ultimately the capitalist markets do provide. I think there's this absolute tsunami of demand that is coming our way. But I feel some confidence that, again, when there's enough pressure, when there's enough clarity of this is the bottleneck. And it's not just for any company. Right. It's really for national competitiveness. And you look at other countries that are just building huge amounts of power far more than we are. I think that actually, for America to remain competitive, there is just no choice but to build.

[Lex Fridman]

We've got to figure out power.

[Sam Altman]

We do.

[Lex Fridman]

Speaking of bottlenecks, everyone was talking about the data wall in 2023. I think this is an interesting thing where no one is talking about the data wall anymore. And yet it doesn't feel like AI progress has slowed down. Is it just test time, compute? Is it like people were wrong about the data wall and where it presented a bottleneck? Is there actually still a data wall? But it's two years away?

[Sam Altman]

It's basically all of these things. Right? It truly is. Right. It's like you keep changing the paradigm. That is the real core of the Kurzweil view of the world is that fine, this one way of doing things taps out. And if you just look at that one way of doing things, you feel hopeless. You feel like this is it, but somehow you will find a new S curve. And I think that's what's happened. For example, synthetic data, for example, reinforcement learning. Right. If you think about the RL paradigm, fundamentally that's a data production mechanism. Right. And it's just that the AI happens to be training on its own data, and then you learn on it very rapidly, and then you learn on that. And each of these has taken us much further. And so I think there are lots of algorithmic ideas, lots of techniques, lots of ways of even using the existing data better. And so I think that fundamentally the S curves continue, and if you zoom out, it all looks smooth and uninterrupted.

[Lex Fridman]

So it's kind of like chip miniaturization, where each generation people are like, okay, well, that's the smallest you could possibly make a chip. That's it. We're done with miniaturization, and somehow we.

[Sam Altman]

Figure out about it. Yes. And now one difference with chips is I think at the end of the day, there is some clonc limit.

[Lex Fridman]

Right. But we've never been that close to our size.

[Sam Altman]

Yes, yes.

[Lex Fridman]

Where does AI coding go? And in particular, vibe coding is all the rage right now. It's Kind of the term of 2025, it's sort of working. It's very impressive. No one is really fully letting AI software engineers run end to end in production. So I'm just curious, what are your one to two year predictions on what happens with AI coding?

[Sam Altman]

Well, my general observation is that once something kind of works in this field, the next gen is going to be great. And so I think that's where we are right now for AI coding. And so I think what we're going to see is AI is taking more and more of the drudgery, more of this like pain, more of the kind of parts that are not very fun for humans. Now one thing that's very interesting is that I think that so far the vive coding has actually taken a lot of code that is actually quite fun and left behind the review and the deployment and these things that are not fun at all. And so I'm hopeful that we're actually going to be able to make a lot of progress on these other areas as well. But fundamentally we should really end up with a full AI coworker. And I think it really will be anything you want to create. You can be the manager and you can really have this team of software engineering agents. Now the thing that I think will be very interesting to see is is there a world where the AI becomes the manager and that gives you ideas and gives you some tasks to do. And that's something that again is just like totally backwards in terms of how we think about it. But are there ways in which you can actually have outcomes for companies and actually have people whose jobs become much more meaningful because they have an AI who really deeply understands them in the same way that your AI doctor really deeply understands all of your needs?

[Lex Fridman]

Isn't part of the common thread that we're talking about here? Often places where AI tools underperform, it's because they're trying to do something generally like voice recognition is not that good because it's trying to recognize all voices as opposed to trying to recognize my voice in particular. And similarly with AI coding, they work well in places where you need no context at all. And we're single shotting an app based on publicly available libraries. And in places where you have to understand a million line code base, you haven't fully figured out how to do a good job of that. Is that a fair parallel to draw between all these challenges?

[Sam Altman]

Well, I think there's two things in there. One is that I think this is already changing, right? So if you look at something like Codex, it's actually great at operating in a big code base. I ask it for where functionality is implemented and it's better than I am at finding it, which is kind of a wild fact. It's super cool to see it grappling around and just going and exploring. And actually this is one thing that we really shot for with Codex was to build a tool for software engineers who are not necessarily vibe coding. It's not about building a new app from scratch, which is a cool demo, but that's not actually how most software gets written. And actually I think maybe the killer enterprise features is refactors. It's like rewriting your Coppola app or changing your Facebook.

[Lex Fridman]

Did Hip Hop to do static php.

[Sam Altman]

Exactly. If you think about it, the amount of deep, sophisticated thought that is required to accomplish every factor is actually not that high. There's a lot of mechanical work. That's just the sheer volume of it. That's hard and that's an AI shaped problem for sure. So I think we're going to see a lot more productivity on all sorts of AI tasks as a result. Now we are in a world where you said a second thing, which is maybe you need to kind of narrow down more. I think that the way these models work is you actually do want one model that knows more and more things and you want it to have some personalization to you. But the fact that they kind of have this one base model that kind of knows everything is actually a very useful starting point. So I do think that you're going to see a world where we'll have more and more capable based models and figuring out how do you really connect it to all of your organization's code and context and history?

[Lex Fridman]

How does OpenAI decide what products to do? I'm just curious how you think about when to develop specific products or when you think, oh, you can trivially do that with ChatGPT and that's good enough.

[Sam Altman]

Yeah, this is a really tough question, right? It's something we really struggle with and I think that over time, I guess actually when we first launched ChatGPT, we were left with this. Well, we're an enterprise business and we're a consumer business. And that seems terrifying. Terrifying as a startup. And I remember talking to one of my board members who said, it just feels like what you have is an unfocused strategy at first because you're just doing all these different things. But if you think about it, maybe an analogy is to a company like Disney where you make one core asset like little Mermaid. Right. And then you productize it in all these different ways. Right? You think about Little Mermaid, the ride, the, you know, the lunchbox, the T shirt. And I think that we have some element of that. We have the core model, and then we have a question of, well, what are the applications that this can add a lot of value to quickly. Right. With, like, a small amount of additional work? And so I think the question of what areas to go into are how far does it take us off the general path, the return on how important is this domain area, especially for achieving the bigger goal, how much synergy is there across it with respect to other things we work on? So coding is one where there's very clear synergy, very clear roi. Because if we can speed ourselves up, that's something that accelerates everything.

[Lex Fridman]

How is being from North Dakota, how's it shaped you?

[Sam Altman]

Look, North Dakota was an amazing place to grow up.

[Lex Fridman]

Well, that actually. Come on.

[Sam Altman]

It.

[Lex Fridman]

Actually, I've been there, and it was great.

[Sam Altman]

Okay. It was incredibly safe. Our doors didn't even have working locks. Like it was that kind of place. I had a lot of freedom academically, so sixth grade, my dad taught me some algebra. Seventh grade was the first time they split you into advanced math. And so I was going to be taking pre algebra. So my mom took me to go see the teacher. The teacher. And we asked, can you skip? And the teacher looked at us very condescendingly and said, every parent believes that their child is special. I can guarantee your son will be plenty challenged in my class. And so after a month of me sitting in the back just playing games on my calculator, and she'd call on me randomly to try to trip me up. I'd just look at the board and be like, 2x. She said, okay, fair enough. Your son is nothing to learn in this class. And so they moved me into eighth grade algebra. But then eighth grade rolled around, and I had no more math left in my middle school.

[Lex Fridman]

You went to the college, right?

[Sam Altman]

Well, so I did in high school, start going to University of North Dakota and took a bunch of classes there. But also, I was connected to a lot of people who were the top math kids in the country through things like math camp and the math competitions.

[Lex Fridman]

So you're saying the social scene was not too distracting in North Dakota?

[Sam Altman]

Not too distracting, but it was. Yeah, it was definitely fun.

[Lex Fridman]

Last question. Do you remember we were going to camp by Sea in 2017, and I asked you how far AGI was away, and you said, two or three years. Did I say that you did. Yeah, yeah, yeah.

[Sam Altman]

I don't see the recording.

[Lex Fridman]

Well, I'm just trying to think, like, was I right? Were you right? How should we grade that? Because we didn't get AGI, but we didn't not get AGI either. And so I'm curious if you have any reflections from your own AGI prediction journey.

[Sam Altman]

I think that we are. I will say, I think that AI is surprising. I think that that is like the single most consistent theme. Is that the thing we were picturing? We got something different, but we got something better, more magical, something that is more helpful. And so I'm actually quite happy with that now. Predicting where you go, it's. Again, it's really hard to manage the outputs here. One goal of OpenAI that we have successfully achieved is every year to have at least one result that just feels like a step function better than anything before.

[Lex Fridman]

I think as long as you see us kind of, you just want to have one really awesome AI feeling thing each year.

[Sam Altman]

That kind of thing.

[Lex Fridman]

I like that. Yeah, that's a good way to tie it back, which is the way we grade that prediction is that you've stopped setting metrics based on outputs.

[Sam Altman]

Yeah, exactly. Exactly. Yes. But it does feel we're getting really close to something pretty magical.

[Lex Fridman]

I agree. Thank you.

[Sam Altman]

Thank you.