← The a16z Show
The a16z Show
The 2045 Superintelligence Timeline: Epoch AI’s Data-Driven Forecast
01:00:10
Epoch AI researchers reveal why Anthropic might beat everyone to the first gigawatt datacenter, why AI could solve the Riemann hypothesis in 5 years, and what 30% GDP growth actually looks like. They explain why "energy bottlenecks" are just companies complaining about paying 2x for power instead of getting it cheap, why 10% of current jobs will vanish this decade, and the most data-driven take on whether we're racing toward superintelligence or headed for history's biggest bubble.
Loading summary
Transcript95 lines
- [00:00]
David Owen
People are spending a lot on these models. They're presumably doing this because they're getting value from them. You can maybe argue like, oh, well, I don't think that value is real. I think people are just playing around, whatever. But, like, whatever they're paying for it. That's a pretty solid sign. We're almost giving you here the useful answer of like, I don't think it's a bubble because it's not burst yet. When it's burst yet and you'll know.
- [00:22]
Jaffa Edelman
It'S a bubble, people often make the case, oh, AI hasn't been profitable yet, and they're spending more to make it profitable. In reality, they'll have paid off the cost of all the development they've done in the past very soon. It's just that they're doing more development for the future.
- [00:36]
David Owen
Will they regret that spending? How much are they spending? You can look at Nvidia and how much they're selling each year, and you can see whether it keeps on growing and you can see whether stuff is kind of looking good to continue.
- [00:48]
Jaffa Edelman
Math team is unusually easy for AI. I'm going to be honest. People often make claims about it being like this, you know, intuitive, deep thing that it would mean that AI has achieved something, some huge level of intelligence for it to solve. I think in practice this is just like, you know, making a piece of art. It turns out to be farther down the capabilities tree than people might have guessed.
- [01:08]
David Owen
We sort of had this with chess decades ago, right? Like, computers solved chess very well. And everyone was thinking of this as the pinnacle of reasoning. And everyone, as a result, kind of concluded like, oh, well, of course computers can do chess.
- [01:22]
Jaffa Edelman
The interesting scenario to think about, you know, 20% chance, 30% chance something like this will happen in the next decade is like, you know, a 5% increase in unemployment over a very short period of time, like six months due to AI. The public's reaction to this will determine a lot. There will be very, very strong feelings about AI once this happens. I think there will be a bunch of very strong consensus on what to do on things that we don't normally think of as things that people are considering. I know when this happened with COVID there was a several trillion dollar stimulus package in a matter of weeks to days. It was breakneck speed. I don't know what that will look like for AI, but I think it's like everything else in AI, it's exponential, which means it will pass the point of people sort of care about it to people really care about it. Quite fast. I just expect wherever we end up there will be this certain thing which we would have considered unimaginable a year ago.
- [02:16]
Host (a16z Podcast Host)
Are we building towards the biggest economic boom in human history or the fastest collapse? Right now, AI labs are burning billions on compute. Anthropic just built a data center that uses as much power as Indiana State Capitol and Microsoft's planning, one that rivals New York City. The bet that AI will eliminate entire categories of work before the money runs out. David Owen and Jaffa Edelman from Epoch AI have done something unusual. They they've actually measured what's happening. They tracked down permits, analyzed satellite imagery and calculated exactly how fast these data centers are scaling. Their conclusion challenges both the skeptics and the true believers. They don't see a bubble. They see revenue doubling every year with inference already profitable. But they also don't see the software only singularity that some predict where AI recursively improves itself overnight. Instead they forecast something stranger. A world where AI solves the Riemann hypothesis before it can reliably fold your laundry. Where 10% of current jobs vanish but unemployment might barely budge. Where we hit artificial general intelligence not with a bang, but through a series of increasingly surreal milestones that keep moving the goalposts. Along with a 16z partner, Marco Mascoro, we cover their timeline predictions, what stops or doesn't stop the scaling and why the political response might happen faster than anyone expects.
- [03:39]
Marco Mascoro
Guys, there's a lot of conversation about the macro. Are we in a bubble? How should we even think about this question? We're going to get into forecasting later on, but why don't you just take a first stab at how you approach such a big general question.
- [03:49]
David Owen
Yeah, I mean for me at least the way that I thought about this a little bit is I look at kind of the big indicator being how much people are spending on stuff like COMPUTE and I guess maybe some sense of will they regret that spending that's relevant but how much are they spending thing like you can see, you can look at Nvidia and how much they're selling each year and you can see whether it keeps on growing and you can see whether stuff is kind of looking good to continue will they regret it side I mean that's just 2bc right? Like we'll actually have to wait and see. It does seem as if most COMPUTE gets spent on inference that companies don't so far regret using to offer their products. So I mean on that side I'm like thinking not to too bubbly yet but yeah, I low Confidence. And there's other stuff to think about.
- [04:40]
Jaffa Edelman
Yeah. Right now, the amount of money companies are actually earning in profit, not including the cost to develop the models initially, seems to be very positive, such that if they stop developing bigger and bigger models and just stick with the ones they've had, they'd have earned a profit pretty quickly at the current margins. And in this sense it doesn't seem bubbly. On the other hand, at any given time, they are investing in building even larger and larger models. And if that goes well, then they'll earn more money. And if that doesn't go well, then no matter how profitable they are right now, it'll be a small amount of money compared to how much they would have spent. So I think right now there are not financial signs that there's a bubble. A lot of people worrying about bubbles just aren't necessarily used to the level of spending and just like the level of success that sort of happened in, like, scaling. But if there is a bubble, it could happen very suddenly and be pretty bad.
- [05:29]
David Owen
So, yeah, I think we're almost giving you here the useful answer of like, I don't think it's a bubble because it's not burst yet. When it's burst yet, then you'll know it's a bubble.
- [05:38]
Jaffa Edelman
Yeah, yeah. I do think, like, you could imagine a world which has all the spending and the current level of success does not. Like, people often make the case, oh, AI hasn't been profitable yet and they're spending more to make it profitable, but right now it's not making anything. And in reality, they're making. They'll have paid off the cost of all of the development they've done in the past very soon. It's just that they're doing more development for the future. So I think there is this underlying financial success so far that I wouldn't expect to see. If there were at the very least, an obvious bubble.
- [06:10]
David Owen
Yeah, that does seem very relevant. People are spending a lot on these models. They're presumably like, you know, users to use them. They're presumably doing this because they're getting value from them. You can maybe argue like, oh, well, I don't think that value is real. I think people are just playing around, whatever. But like, whatever they're paying for, that's a pretty solid sign.
- [06:30]
Kirsten
I guess one quick question related to this is, like you talked in the report of the AI in 2030, basically, that you haven't seen signs of basically these models kind of plateauing or like the capabilities keep increasing and you have the benchmarks you have, the amount of data that is going, the amount of compute. Do you think phases or parts of the models are plateauing though? Like for instance pre training? Are we seeing some sort of plateauing in that or do you think people are still exploring some innovations in that stage? And Kirsten, what do you think about that?
- [06:59]
David Owen
Yeah, I think this gets a bit harder to look at. Like we get to an area where there isn't as much public data to say a lot. Right. It seems as if pre training is comparatively less of a focus than it was before. Partly because you have this exciting new direction of. Well, newish direction of post training where they've done so much about reasoning, whatever. But then I don't necessarily take that as evidence of like oh no. And that means pre training, you couldn't scale further, whatever. It seems as if there is meaning painfully more data out there. It seems as if plausibly like even a lot of this stuff is quite synergistic. You develop a better model, you like use post training stuff to make it better. You get a load of data of the model actually being used successfully or not. A lot of that can probably go into free training next time.
- [07:50]
Marco Mascoro
You aren't projecting a software only singularity where AI is able to automate AI research which is an automated feedback loop. Why not?
- [07:57]
David Owen
Yeah, I mean I guess I'm answering this and I'll yafe say more and it's like for me it's like that report, it's no one person's kind of oh, this is like the forecast, this is the prediction. Right. This report very specifically looks at what are the current trends. Are there reasons that they clearly couldn't continue or might not and if they do continue, where do they lead? I think whether you see this self improvement thing, that's very hard to do from a sort of trend extrapolation basis. Right? Like currently AI stuff does help AI R&D at least a little in terms of stuff like coding or selecting your datasets and creating those, whatever. But it's quite hard to actually measure and it's not really helping in some big way like this kind of self improving thing would suggest. There are reasons that you might think it could be very hard. People have discussed before how possibly if stuff just depend a lot on scaling up compute, then maybe automating a lot of the R and D isn't that helpful, helpful. I find that somewhat compelling but I think it's also just. It's pretty uncertain. It's hard to speculate about something that's quite out of regime like that.
- [09:12]
Jaffa Edelman
One thing that needs to happen in order for a software only singularity to occur is you need to be in this world where scaling up the amount of researcher R and D time basically allows you to improve AI enough that it makes up for the lack of, of being able to scale experimental compute or pre training. I think that something you would expect to see if this were the case is maybe not that much experimental compute being used in practice and instead all of the money is going towards researchers. Now there's a very good case that there's a very large amount of money going towards researchers. But as far as we can tell, experimental compute, which you seem to need to do research is receiving a similar amount of money and that in fact it's receiving many times more money than the final training runs that are actually of the models that are actually being released. I think this is in my mind is a strong update towards oh, you need to do very large scale experiments to do research and that we don't really have good evidence that researchers and just researchers would be able to speed things up without doing more experiments. However, there are pretty good arguments on either side of this. I tend to lean towards no, you actually need to do more experiments and that means you can't get this software only singularity. But I don't think the people who claim otherwise are like crazy. I think they're making some like they have very reasonable differences and we're both speculating on something where the data is currently pretty sparse actually related to that.
- [10:36]
Kirsten
What do you think on. So if you have some of the exploration that researchers are trying, I mean obviously people are exploring a lot with rl, trying to go beyond verifiable domains. And what do you think about the argument, for instance, that gradient descent is really good on learning in the current data set that you're giving. Right. And if you keep training this over and over, it's going to start forgetting things that it was trained before, right? Like catastrophic forgetting. And there's this argument, right, like, well, kids don't learn that way. Maybe there's some imitation learning that kids do. Maybe there is some sort of exploration that they do. And I wonder what you think about it. And it sounds right, like if kids really would just learn on imitation learning, I think parents would have a great time just raising kids. But it seems like the reason why they have such a hard time raising kids is because they explore all these different things. What do you think about it in terms of the algorithms and the things we need to keep improving these models over and over beyond the data and the compute.
- [11:33]
Jaffa Edelman
I am cautious about comparing how AI is learned to how humans learn. Not because I don't think they are comparable, but because I think we know a lot more about how AIs learn right now than we know about how humans learn. And people like making sort of assumptions about how human learning works and saying, oh, AI doesn't do it that way. And I don't know, maybe that's true. Maybe human kids learn via rl. I'm not very. I think that, yeah, I don't have strong opinions on whether or not you need to change to a method that's more like what we think kids do right now. I suspect people will find some method that works to use the compute available because they've been able to do this in the past.
- [12:13]
David Owen
Yeah, I'm also sort of reluctant, I guess, as well. It's one of those things where when we point to particular issues, like the example of catastrophic forgetting, it's sort of, well, okay. But as we've scaled up, we have managed to do quite well at like, having models that remember more and more things. This isn't to say that hence the problem is solved, hence we're done, hence no more algorithmic innovations necessary or anything like that. But I'm not exactly going to write it off.
- [12:47]
Jaffa Edelman
Yeah, I definitely don't think we've seen any slowdown yet in capabilities from any of these concerns people have. I think that people always have these sorts of concerns. I'm reluctant to believe any given one of them until this actually shows up in numbers I can see on a graph, which I just don't think has happened yet.
- [13:09]
Marco Mascoro
Dario Anthropic has said, he said in March 2025 that within six months AI will write 90% of code. And of course, that hasn't happened yet. He also said we have, you know, we could have AI systems equivalent of a country of geniuses in a data center as soon as 2026 or 2027. How do you evaluate why anthropic is so bullish or what is the crux of difference between what they believe and perhaps what you believe?
- [13:35]
David Owen
My model, at least, which I don't know if it's right, but what it is is that they think a bit more like the people who believe in you automate R and D. And that gives you very quick takeoff. So they see it as like, yep, we're working on these AIs that are great for kind of research, engineering, type coding, and at some point they're going to be useful and that's going to rapidly accelerate us to develop the next ones and then it's going to be quick progress.
- [14:04]
Jaffa Edelman
Yeah, I think that it's hard to tell the extent to which. I don't think we've gotten a lot of evidence that there's sort of views of this like software only takeoff are wrong insofar as like they were taking a little bit longer to get to like the minimum level of competence for AI to get you There definitely seems to be the case, but I don't know, it's hard to tell the extent to which we've actually had significant updates on this. I know Dario often qualifies what he says by saying as soon as or something like this. So this is like maybe more so the faster timelines he gives.
- [14:44]
David Owen
Although I'm not sure. Yeah, there has also been, I think sort of Talmud style commentary where people are carefully looking at his exact wording and then at wording of other people's discussion of how many lines of code that are generated by some teams at anthropic generated by cord code and whether this does or doesn't satisfy what you said. So it gets a bit tricky.
- [15:08]
Kirsten
I remember there was the paper from the uplift paper that was claiming that actually models would slow you down. But I think it mattered a lot what models they were using at the time because I think they were pretty outdated by the time the report came out. And I mean in my personal experience, you definitely become way faster and it just saw so much more for you. Like you're just having the whole context on your code base. That's such a huge advantage that I think for human just would be really hard to do.
- [15:35]
Jaffa Edelman
I mean far more than 90% of the code I write is written by AI these days. But I know I'm not like the average coder at all, but it's definitely, I don't think it's like a wild prediction at this point that 90% of code is going to be written by AI. I mean, for all I know, somewhere at OpenAI there's someone just. Or that, you know, with AlphaCode doing evolutionary algorithms on, having tons and tons of trials, trying to, you know, million shots. Some hard problem that it's just like it's really unclear how many lines of code are actually being written by AI right now. I don't think it's such a wild. It's by a lot of like people's intuitive sense in terms of like, oh, is 90% of the job of a programmer being done by AIs? Definitely not. But there's this more complicated sense of how much is being written by AI. Probably not 90%, but it's hard to tell.
- [16:32]
David Owen
Yeah. And I think that is a very meaningful distinction. If you were to measure how many lines of code are being written by tab completion, then it's probably quite high. But you don't necessarily expect that that's taking on that much of the programmer's really hard work. That uplift paper that you mentioned, I find it really interesting and really good. And it's also surprisingly recent in a way. Like, you know, you mentioned the models are outdated, but I mean, this was early 2025. So these were models that people actually did think were helping them. And in the paper, they even got them to say ahead of time, like, how much do you think this will speed you up? And they said, yeah, I think however much they then asked them afterwards, how much do you think this sped you up? And they're like, yeah, yeah, it sped me up. And I feel it does reveal actually, like, it might be hard for us to judge whether we were sped up or not.
- [17:24]
Jaffa Edelman
Yeah. One thing that might be happening here is that a lot of the code that's getting written by AI is code that wouldn't have been written otherwise. So it's not really speeding up things that would normally happen. But, you know, there's a lot of simple graphs or simulations I run that might have not gotten written otherwise. And so it's hard to tell exactly what's going on here in terms of the impacts. I think at the end of the day, the most reliable indicator here is going to be how much money these people are making from programmers and from, you know, subscriptions in general. And it's a lot of money. I think there's definitely indications that people are finding a use for them, and probably a decent amount of that use is for coding, but not exactly for the metric of doing 90% of an existing coder's job.
- [18:08]
Marco Mascoro
Yeah. Biology is this phrase that's been being used a lot, which is AI is.
- [18:14]
Host (a16z Podcast Host)
An end to end.
- [18:15]
Marco Mascoro
It's middle to middle, which is meant to imply that, you know, we're going to need a lot more human involvement than some people, you know, typically think. What is your mental model of what AI is going to do for. For labor markets, either on the sort of lower end and on the higher end in the next, you know, decade, let's say.
- [18:36]
Jaffa Edelman
Oh, in the next decade, I like, on the higher end, I'm definitely like, you know, probably I expect new jobs to be created. Everyone could still be influencers, but on the higher end, it's like there are not very good individual things that you can point to where it's very obvious that AI can't automate that job at this point. Now, you could argue, okay, but there's some unknowns, and I think it's like pretty reasonable. But those unknowns, we sometimes, you know, AI gets up against its limits and we figure out what they are, and then it learned, surpasses that. And I don't know, at the higher end, it definitely seems plausible that it could just automate all of the. Basically all of existing jobs, with the exceptions of ones that require manual labor, that people actually care about being done by a human. It just does not seem at all implausible to me that that can happen or that that could happen very fast, with the caveat there being like there's probably some regulatory pushback. If that happens on the lower end, I don't know, it could just, you know, could be a bubble and doesn't have any impact. The thing I talk about when I'm talking about the interesting scenario to think about, which I'm not, I don't know, 20% chance, 30% chance something like this will happen in the next decade is like a 5% increase in unemployment over a very short period of time, like 6 months due to AI being released is something that I think will have a very substantial impact on the world, both in terms of how people think about AI and sort of how much attention it gets. And seems plausible to me, but far from guaranteed.
- [20:20]
David Owen
Yeah, I think I strongly agree with being just highly uncertain. It seems very plausible to me that you end up more or less kind of. This generation actually is exactly where we run out of progress. It would be kind of crazy, but it could happen. And then it's like, oh, okay, everything is very much just generating more jobs for technical people to try to integrate it into doing kind of useful but janky things for all of the existing work people do. The stuff where it kind of becomes a crazy runaway thing that you can really automate large swathes of remote work with. I mean, my timelines are, I guess, probably a bit longer than Jaffe's, but yeah, I mean, it seems hard to rule out that something really big happens in a decade. A decade's quite a long time.
- [21:11]
Jaffa Edelman
I think I would be surprised if there were not 5% of jobs that exist now which AI has automated away over the course of the next decade. Honestly, I'd be surprised if it's not 10% of the jobs that exist now, I think how fast that happens and the extent to which those people find other jobs is something which I don't think I have seen compelling evidence for either way. And we're and probably depends on how fast various things go and exactly what jobs are automated. I think that 10% over the next 10% of current jobs seems like a pretty reasonable lower. It's not quite my lower bound, but you know, a pretty reasonable number over the next decade. But this might not show up in overall employment numbers.
- [21:55]
David Owen
Yeah, this is interesting. I mean definitely like the kind of. To the extent there is a mainstream economics view of this stuff, it would probably be that automation happens at the level of task rather than occupations and occupations can as a result go down quite a bit. But a lot of the time you're automating these similar tasks across lots of jobs. I think this is compatible with what you're saying. It's just that some jobs get really hit by it. I don't know. I find it quite hard to think about. I'm not sure what even the historic base rate for jobs ceasing to exist is. I know there are problems with this like the historic employment data series. There is actually quite a high, I believe, base rate of just the tasks in a job. Changing jobs themselves, changing jobs kind of going away, coming in. So yeah, even this 5% thing, I don't know if to think, yeah, that would be like a big effect or kind of. Yeah, that's actually roughly the size of effect you've already seen from something like software. I don't know.
- [22:59]
Jaffa Edelman
Yeah, probably 5% of jobs that existed before software no longer exist. It seems pretty reasonable, but I'm not confident to this. It's definitely something which like, I don't know, I expect especially if revenue trends continue, I expect to know a lot more about this in a couple, in a year or two, probably within the next year because it will just be the case that, okay, we will have AIs earning enough to substantial to be like a substantial part of the economy. If it's not showing up in unemployment, then we've learned something about what it's doing. We've learned that it's able to do this without showing up in unemployment numbers or maybe it will show up at unemployment numbers and we'll see exactly what there's been like some early work looking at like indicators of this. There's a lot of things that complicate looking into this because interest rates also have effects on like the sort of things you might care about or just like normal churn or also it's possible that tech companies, you know, maybe they'll lay off a bunch of programmers so that they have the capital to build data centers. And, and are those programmers being laid off because of AI? I don't know.
- [24:08]
Marco Mascoro
Maybe if you had a kid that was a freshman in college and they were asking, hey, you know, what should I major in if I want to have a great career, you know, what might you tell them? And if they asked you about, you know, computer science or math or you know, engineer.
- [24:21]
Jaffa Edelman
Yeah, exactly, yeah.
- [24:23]
Marco Mascoro
What would you say?
- [24:27]
Jaffa Edelman
I mean, I'd probably say not prompt engineer. I think in general people get better at using AI is very easy to use. Yeah, I think it's a good question. I think they should probably major in something where if they're majoring in programming, the thing that they should be or computer science, the thing that they should be looking for is not being a person who's going to the skills that are going to be useful are not going to be knowing a programming language. It's going to be more general purpose skills, ability to work with other people, communication skills, this sort of thing. I don't really know entirely if this points to a particular major. Most majors are probably not majors that are actually relevant for your job.
- [25:12]
David Owen
Yeah, I guess I'd sort of be like, well, there's not too much that you can do to plan around the super crazy futures. So I guess go for something that you're passionate about that's useful in the worlds but don't go crazy in that way. I actually think that, yeah, computer science, maths, if you're passionate about them, they're very good because you'll learn interesting things that are valuable in many worlds. But I don't know. I gave advice to a younger relative recently and they chose to study drama instead. So.
- [25:44]
Jaffa Edelman
I do think that, you know, one of the things that if you have a better time in college, that's like four years of your life, you've had a better time during. And at the end of the day, if it's a crapshoot, which of those things is actually going to give you a better time in the future? Planning for the present is a lot easier.
- [26:04]
Kirsten
I mean, it's definitely becoming really hard to know. Right. I remember the problem engineer was obviously a joke because everyone believed two years ago that that was some sort of viable thing. And obviously models are phenomenally better. I'd just being great prompters. So obviously that's Kind of like one thing that has been happening is it's really hard to predict what's happening as these models keep getting better. One question that I have related to this is obviously code is such a big market and it has had such a big impact. One that I'm very excited about. But it's still much earlier, I think, is computer use.
- [26:38]
David Owen
Right.
- [26:39]
Kirsten
It's basically automating all the digital tasks that you're doing in your computer. And there's very few benchmarks around this, like, whether it's Web arena or the OS world. You talk a little bit on your report about benchmarks. Curious on, like, what do you think is missing in that space? Like, why we haven't seen yet that moment where. The moment, for example, when Sonnet 3.5 came out, or cloud code or codecs where we saw significant improvement on coding in general. We haven't had that moment for computer years. What do you think is missing there?
- [27:10]
David Owen
Interesting. I mean, there have been improvements on computer use, for sure. I do have. I mean, this. Maybe I'm going out on a limb here slightly, but also I do think that there is a sense in which models are a little bit artificially hobbled by their vision capabilities. Like, it does seem as if a common pattern you see when you try to get models to do stuff with a GUI is they kind of get a bit confused about manipulating it and, you know, in a way where it's like, okay, this is interacting with your general propensity to get infused as you would in like, difficult, long coding problems. But it's kind of exacerbated because, like, you're not able to just easily look back on the thing and see kind of, ha, I was wrong. You instead go down like some awful dead end of just, I'm just going to click this again and again and again. So I think that's part of it. I think there is something here also probably about kind of long context coherence stuff. Like those tokens to represent the GUI are pretty big. And then you're filling up your context window as you go with, like, oh, yeah, well, I had all of this stuff that's happened before and you seem to just run into a kind of spiral of increasingly less sensible outputs. So I feel like these are two of the big things. But I don't know if that answers your question.
- [28:33]
Jaffa Edelman
I found computer use. I don't know. This was the first year I found computer use actually useful. We use ChatGPT agent in our data center research because a lot of what we have to do is find permits which are all going to be on janky county by county databases of error permits for, you know, the county that Abilene, Texas is in. And I don't know what databases exist for every county in the US chatgpt does normal chatgpt can't search them because it's these, you know, these actual user interfaces. You can't just search them with, you know, URLs because they definitely don't work that well. And it's able to navigate this such that I can just ask it to find me permits on a data center in a particular city and it will come back with air pollution permits and like tax abatement documents and all of this stuff that let me learn a huge amount. And this is just like because of the improvements we've seen in computer use over the past year or so. I'm excited to. Yeah, I think it's just going to get better from there, but I've definitely found it starting to get to the point where it's actually useful.
- [29:45]
Marco Mascoro
What's your mental model more broadly for what is going to happen to productivity or sort of economic economy statistics in general?
- [29:54]
Jaffa Edelman
Are you.
- [29:55]
Marco Mascoro
Some people say GDP growth would be, you know, 5%. I think it's a Tyler Cowan view. I think some people would say no, no, we should get up to 10% growth or maybe even higher if we truly have AGI in terms of how we understand it. What's your model of what happens to the productivity?
- [30:10]
David Owen
I think my kind of baseline guessing would be, you know, I forecast out kind of if revenue keeps growing the way it has in theory, for it to be worth spending that much on those chips. To do that inference, you should be getting something kind of similar to that value out of those chips by then. So then you could just draw from that kind of like, oh, okay, so extrapolating to 2030, you need. And I think for there it was in the report, I don't know, I calculated but I think it was on the order of like a percent kind of GDP increase that's in a few years. Right. That's not presuming AGI, that's presuming like if Nvidia stock revenues keep like growing as they sort of previously have and you assume that they make roughly as much compute from it as before and so on. If you actually get something, I mean, AGI is like, yeah, people use it to be emptying different things. I think if you actually get something that can do any tasks that humans can do remotely, then presumably you see A lot of growth. It feels sort of difficult to guess exactly what kind of a lag you're going to see. I think there's reasons to think, oh well, maybe people will be slow to adopt stuff. How do they learn to trust it, Whatever. There's other reasons to think, well, they're already using these technologies. A lot of it might actually be quicker than most growth. And indeed adoption's been quicker for LLMs than for many previous technologies. So yeah, I think it sort of gets hard at that point to model. At some point on our site we had some rough numbers where it was stuff like what if you, you know, doubled the virtual labor force, what have you, 10 times that, whatever. Then you see these like crazy GDP boosts. I don't know whether that's the most reasonable way to think about it. I think a lot of it comes down to whether you imagine that like, yeah, you really get something that can do everything versus you get something first but can do a meaningful fraction of remote tasks, but maybe can't do an entire bucket of them and then it bottlenecks you more. So I guess it's again this thing of like my best guess on current trends is this Fairly well defined few percent of GDP in 2030 thing, which is already pretty crazy by economic standards. But then once you go much further, it's like, God, my predictions are just going to be even crazier. I'm reluct to make them.
- [32:46]
Jaffa Edelman
I am going to be slightly less reluctant. And that's what we're here for. Assuming in the next 10 years we get AI that is capable of doing any remote job as well as any human. I think, you know, 30% GDP growth seems like a lower bound on something that's reasonable. Assuming you get this is a big assumption that a lot of people are going to that, you know, it's. There's a lot going on in that assumption. But assuming that happens, I think you either are going to get like 30% GDP growth or you know, negative 100% GDP growth because everyone's dead. It's just like, you know, it's just like at the end of the day it seems like you're going to have AI that can scale that. If you have AI that can scale there, you could probably have AI that scales even farther. And right now I think the like economic models I have seen of what happens if you get this sort of full replacement. You can automate a job or you know, either show this sort of extremely fast wild takeoff or with a couple of, or, you know, you have some people attempting to do this who then say, and then you like look down through paragraphs and it's like assuming current levels of assuming AI is as capable as GPT3. You know, I think the smaller numbers just like, you know, they're either nearer term predictions or predictions that aren't looking at like the full, the upper end of what sort of capabilities you might see in the next 10 years.
- [34:17]
David Owen
Yeah, I mean, it does seem hard to imagine a world where you have this supply of virtual labor that literally can do any stuff that humans can do and then it doesn't lead to crazy things. I definitely agree with that. I guess perhaps maybe some sort of a, I don't know, a heavy regulation situation, but there are, they're doing.
- [34:37]
Jaffa Edelman
Yeah. I think there exist worlds in which things don't go crazy after that. It does seem like those worlds are not in an indefinite stable state. But, you know, it's not impossible. But it does seem like the default there is you either go crazy up or you either go crazy down. And it's probably going to be one of those two. If you get to a world where it's like genuinely AI can do any job as well as any human. I think people, I don't know, it seems wild to me to claim that given that your default case should be not super ridiculous changes, it's just like that's a lot of things that your AI can do right there. And that's like, yeah, it just seems like it should have fundamentally changed the economy in one direction or another.
- [35:22]
David Owen
My intuition is a lot of the disagreement. I mean, probably some of it does come down to sort of cached beliefs people already have. But I do also think some of it is that when people talk about like, oh yeah, AGI, AI that can do a remote job, whatever, even though we feel like we're talking about the same thing, maybe sometimes we're not. I don't know. I've certainly had examples of conversations where it's like, yeah, AI can do any remote job. And then they discuss stuff that it can't do. And the stuff that it can't do, it's like, well, no, that's also a remote job. That's the kind of thing people currently do. So I think there is some of this.
- [35:57]
Kirsten
What do you think? Like, I mean, you talk about benchmarks on your report, but I wonder like 2027, 2028, what are going to be the right benchmarks to measuring the progress more than the economic growth more the capabilities on the model, like intelligence on the model like we had in 2012, Alexnet, obviously that, that got solved long ago, but that was probably not a measure of AGI by any means. Do you think the same would happen with the current benchmarks we have? So, Sweepbench, mmlu, let's say we maxed out on those benchmarks. What comes after that? How do we measure that? Is it sort of like GDP growth with these models? Is it sort of breakthroughs in science? How do you think is the right measure going forward?
- [36:41]
David Owen
Yeah, I mean, I think most of what we have is likely to be solved. And indeed the examples you gave are like pretty close already, like mmlua's basically solved. Swedebench is like possibly close. Depends a bit on how ambiguous some of the questions are. There's some details, but it's really getting there. I mean, I think some directions are obvious. You kind of do similar things, but harder and a bit better and trying to make them a bit more realistic. And people are doing this. There are harder software benchmarks that people have made more of an effort to try to curate and that cover larger tasks, for example. I think there's also perhaps some question of kind of budgets involved. I do think there's this kind of thing where obviously if you just burn money, it doesn't intrinsically make the benchmark better, but probably you are going to see something where you're just going to have to devote more resources on average to them. If you're trying to prove a higher level of capabilities to a higher standard of proof, probably it's going to involve more effort in developing them. I do also think, though, you're going to see examples of relatively small, kind of small numbers of things that are just very impressive. And these are also a valuable signal, like when you see arms being able to do things like, oh, yeah, I just refactored this entire code base and it was really useful, then this is going to be useful. And even if it's not yet formalized into a benchmark, if you've seen it for yourself, it's going to be kind of useful for you as evidence. And then people are probably going to make benchmarks that cover things like this to try to systematize them.
- [38:23]
Marco Mascoro
I want to go back to our question on timelines, and I want to ask you about a few different sort of milestones and get your perspective on timelines there. So first is what is your rough timeline for a major unsolved math problem being solved by AI?
- [38:40]
David Owen
I actually wondered. Yeah, because you had a few of these that you said for us to look at. When you say that it solves this, I mean, is this unassisted entirely or is it kind of a news report? Or someone tweets that, hey, like, I dumped this at GPT and it solved it. And what counts as major something that.
- [39:00]
Marco Mascoro
We would all agree, like a substantive, you know, version of it. Not. Not a, you know, just a anecdotal.
- [39:07]
David Owen
You know, person describing it. But does it have to solve it on its own?
- [39:13]
Marco Mascoro
Yeah, let's go with that.
- [39:14]
Jaffa Edelman
Sure.
- [39:14]
Marco Mascoro
Yes, Anastasia.
- [39:16]
David Owen
Oh, yeah, Because, I mean, there's already cases, it seems, of LMS being. Yeah. Like people are debating a little bit, but mathematicians who seem trustworthy are saying like, wow, I used this. It was really helpful during my proof.
- [39:29]
Jaffa Edelman
I would not be surprised if AI solves a major unsolved math problem like the Raymond Hypothesis or similar in the next five years. I'm not going to say that that's my median case necessarily, but I definitely wouldn't be that surprised right now. It doesn't look like math is that hard for AI. It's just like some things turn out to be hard and some things don't. And math is just like one of the domains where it's rl seems to work pretty well, and where it's most other domains, it's not at the point where it's useful to a full professor. To the same extent I think it is for math, or getting very close to for math. Yeah. And also it's very unclear to what extent certain capabilities that it has unusually well, might actually turn out to be very, very useful. Like, maybe it'll turn out that there's like four papers out there that it knows about that have obscure results in them that when combined, solve some big conjecture, which is the sort of thing that it, like, might be much more feasible to figure out with AI than for a human to figure out or something similar. There's a lot of uncertainty here, but it just like, does not currently seem like something that AI is actually going to struggle with. People often make claims about it being like this, this intuitive, deep thing that it would mean that AI has achieved some huge level of intelligence for it to solve. I think in practice, this is just like making a piece of art. It turns out AI could just do that before. It could do a lot of other. Before it can remember things for more than a couple of days or whatever. Yeah. It turns out to be farther down the capabilities tree than people might have guessed.
- [41:11]
David Owen
Yeah. I think I'm also bullish, though. I do think that, yeah, it's one of those things where it's tricky and you really probably do need to define it quite well to get a good forecast on it, to hope to get a good forecast on it. I don't know. We've had this experience that with benchmarking mathematics, we got mathematicians to come up with problems that I think aren't as difficult as the kind of problems you're talking about. But nevertheless, they're like, yeah, if AI could solve this, it'd be like a big deal for AI progress. It would mean something to me. And then AI has solved them and usually their response has been kind of like, oh, yeah, that updates me a bit. Although, man, when I look at it, I just realized like, yeah, you can kind of brute force this, you can kind of cheese this, you can get through. And it's a bit like, oh, okay. I mean, what if there's a problem that for humans we consider sort of, oh, this would be quite big. And then, yeah, AI solves it. Okay, Ah, well, it solved it, whatever. We sort of had this with chess decades, decades ago, right? Like computers solved chess very well and everyone was thinking of this as the pinnacle of reasoning. And then they did and everyone as a result kind of concluded like, oh, well, of course computers can do chess. So yeah, I don't know. I suspect that math is quite nice for AI to do. I'm reluctant to go out and assert like, oh yeah, definitely AI is going to like solve some of the Millennium Prize problems in the next few years, but it would not at all surprise me if it solves quite impressive seeming things in the next few years to.
- [42:47]
Marco Mascoro
Then what about a breakthrough in biology or medicine?
- [42:52]
Jaffa Edelman
And we've already seen some of that with the. What's it called? Alpha. AlphaFold it. Math seems unusually easy for AI, I'm going to be honest. So to the extent where I'm like, ah, is it going to do the same exact level of like, oh, it on its own did this huge thing that seems to be a much bigger stretch to me. It definitely seems plausible, but there's a lot of other concerns there where it needs to be able to actually do experiments and get data and interact with the real world for a lot of these in a way that does not need to happen at all for math in particular, for certain. Yeah, it's just they in fact seem farther off. What seems more plausible to me is that we see it become ubiquitous that some tools of using AI in some sort of aspect of biology or chemistry or something useful like that, that certain aspects of it are enhanced. It also is possible that AI will make incredible strides without humans. But it's harder.
- [44:03]
David Owen
Yeah, I think again it's a bit tricky for where you draw the line. I mean I think you're not counting tools like Alphafold because if you were then probably you'd argue for that. Right. The inventors co won the shared Nobel Prize. But yeah, I mean I guess there's kind of different directions in biology. You could have AI being able to predict quite specific specific things like that or you could have something that's more general purpose. This so called co scientist or whatever they want to call it approach where it's more about like oh, it was able to look through the literature and have good ideas and there's different extents of human involvement. There already seem to be some results where impressive stuff is happening. I've not vetted them enough to really have a sense of like would this already count as having satisfied. Yeah. The sort of level of impressiveness you're looking for. I sort of assume that finding things that end up being meaningful will happen pretty soon if it hasn't already happened. But then maybe there's a question of kind of okay, but is it doing as well as human researchers are actually like prioritizing the best new ones to work on? I think most of these co scientist results have probably had pretty involved humans prioritizing though again I've not looked enough to say.
- [45:26]
Marco Mascoro
Lastly, how about for real superintelligence for your definition of superintelligence?
- [45:35]
Jaffa Edelman
I think I am on the record as saying that the median timeline I discussed or the modal timeline. Sorry, I think smodal. Yeah. Which might be on the early side compared to where my median is. 2045 was where when I did the podcast with Jaime we discussed our forecasting breaking down and everything going bananas is the terminology I have used. And that looks like super intelligence. I, you know I think that it's like the case that if we get AI that can do every single job that a human can do as well as any human can do that job in the near future, then this is, you know, means that scaling just works to get things much, much better and probably means that you are not that many steps that you are just a bit more scaling away from getting AI that could do anything that humans. Sorry, two things vastly better than humans.
- [46:43]
David Owen
Yeah, it gets hard to predict and I think as well it gets to be one of these things where the predictions get a bit unmoored from the stuff that you can properly model My sort of guesses My judgmental forecasts, to use the fancy term for just kind of can do any remote work tasks, probably have a median of about like 20, 25 years. I kind of struggle to imagine a world where that happens and people are like deploying it and doing research, and yet they're not making further progress to being able to do stuff much better. So I guess I have to be like, not too much longer after that for some definition of superintelligence. But yeah, all very uncertain. And yeah, it seems to break down a bit.
- [47:36]
Kirsten
You talk a lot about the progress in data centers, benchmarks, biology, and there was one interesting part that I noticed just in the field that is robotics is making a lot of progress with, let's say, world models and like the physical space. A little bit curious on, like, what is your take here? What do you think? It's. It seems like a lot of the problems in robotics can be solved purely with imitation learning. You might not need like a lot of sort of like breakthroughs in math or whatever. You can just basically learn it from a lot of data. And I think in the last couple of years has been remarkable just in robotics and world models overall. Curious on your take a little bit on this and if you did some kind of research in this space.
- [48:13]
Jaffa Edelman
So we've looked into what sort of amount of compute is actually being used to do these training runs, and what we found is that like compute, the training runs that are being used for robotics are like 100 times smaller than the training runs that are being used than the training runs that are being used for frontier models. And so there's a lot of scaling you can do there. I don't think that until plausibly, until very, very recently, there have been serious attempts to gather data for robotics at a massive scale. It's just the case you can hire a bunch of people to move around in motion capture suits if you need to. And there have been a lot of attempts to do that, although I think this might be changing. I think of robotics as mostly a hardware problem. A hardware and economics problem of if it costs $100,000 to build a robot, then it's not necessarily better than a human who could work for $20,000 a year or a very cheap human in certain countries or something, sorry, a the like sort of minimum wage in some countries that you might be able to afford labor for. It's just not obvious to me that there is a software problem here. The hardware, it does seem like, unclear. It's very unclear to me how much of a hardware problem is left. In particular, there's certain tasks which robots might be able to do, but are they actually the tasks that you care about a robot being able to do? If you want your robot to be able to nimbly walk around while lifting up heavy things and moving fast and react, then that's hard. That's a hardware problem that I don't think we've seen solutions for yet.
- [49:54]
David Owen
Yeah, I think my impression roughly matches this. It's sort of, I don't know. People fairly often talk about this distinction between remote work and physical work, I think because there's this perception of robotics progress lagging behind a bit and there even is some intuition that maybe, maybe this physical manipulation stuff is actually just harder. But I wouldn't conclude that with much certainty. Like Jeff has said, it feels like you'd kind of also want to see, well, okay, what happens if it gets scaled up in a similar way to even get a sense of like, oh, okay, was it actually harder versus was it just deprioritized?
- [50:39]
Marco Mascoro
Is there, is there anything we didn't get to that you feel is important that we leave our audience with?
- [50:44]
Jaffa Edelman
We did discuss the data centers release. We just did. I'm not sure if there's a good way to leave the audience with that. Yeah, let's get into it.
- [50:50]
Marco Mascoro
Okay, so you guys just did a, you know, released dentist project. Why don't you talk a little bit about what you were trying to achieve there and what you hope people take from it.
- [50:57]
Jaffa Edelman
Yeah. So we took 13 of the largest data centers we can find. These includes some, a few from each of the major labs in the US and we found permits. We took satellite images, including new satellite images of all these data centers. We figured out how to determine how much compute is in them based off the cooling infrastructure that they're building, as well as when they're coming online in their future timelines. So we understand this real world data and it's all available online on our website for free. This like to give insight into this giant infrastructure buildup that's happening and the pace of it. There's some things about it that surprised me a lot. For instance, we learned that the most likely candidate to have the first gigawatt scale data center is anthropic, which would not have been my pick, but anthropic. Amazon's new Carlyle project Rainier development seems on track to come online in January, followed shortly thereafter by Colossus 2. We also learned a lot about what the largest concrete plans are rather than just like marketing plans. Some people will throw around numbers, but the one we found that's actually seriously underway and has permits and is setting up the electrical infrastructure for is one by Microsoft which is going to be used by OpenAI, at least in part in Mount Pleasant. They're calling it Microsoft Fairwater. And that one's going to be use a size, use not quite as much power as New York City, but I think more than half.
- [52:31]
Marco Mascoro
What's stopping us from significantly increasing the, the cluster size? Is it cost? Is it supply lead times? Are there any other engineering breakthroughs required power?
- [52:43]
Jaffa Edelman
I think that people are approximately wrong that there's something stopping us and we are scaling up as fast as there is money to scale up approximately. I suppose they could want there to be all of the clusters literally today, but they're scaling up really quite fast. You're seeing these, these data centers which are using, I think the one I mentioned for anthropic Amazon is using about as much power, nearly as much power as the state capital of Indiana, which is where it's located. And the timelines on some of these, like the Colossus 2, are two years or less, which is just an insane thing to build this thing that's using as much power as the city. I think that plausibly you don't want to buy chips now. You want to wait for there to be better chips. I think that people think of. There's a lot of noise about things being difficult and scaling up, and I think this is because people are having to spend a little bit more than they would ordinarily have to spend. You can't use the ordinary sort of power pipeline which is designed to deliver this affordable infrastructure at a slow pace. You have to buy things that you wouldn't ordinarily have to buy and spend more than you would ordinarily have to spend, but not buy enough to slow it down. All of these things pale in comparison to the cost of your GPUs. So my actual takeaway from a lot of this has been, oh, we're not having too much trouble scaling up, but just like these plans are going really quite fast and it's not obvious that people would actually have the finances and desire to do them faster.
- [54:23]
Marco Mascoro
When people are talking about energy as a major potential bottleneck or having to increase our capabilities significant, you're, you're not worried that that's going to be a sort of durable, sustainable bottleneck? That's not.
- [54:35]
Jaffa Edelman
I think people like complaining because they can't just use the traditional plug into the grid for cheap Affordable power four years down the line pipeline. At the end of the day, the day there are expensive technologies that exist. Right now you could pay for solar power plus batteries. This is fairly small lead time. It might cost twice as much as normal power, but that's still way less than your GPUs. So you're going to do it if you have to. And you see people doing these sort of emergency things that cost them a bit more, you know, starting up their data centers. A common thing we see is people starting their data centers before their data centers are connected to the grid. I think Abilene was an example. Xai Colossus 1 is a prominent example of just finding ways around this that are expensive. And you complain about it because you know it'd be nice if you could do the cheaper way. And no one's used to having to do it this expensive way. At the end of the day though, it's just like, does not there seem to be enough solutions, especially if you are as willing to pay as people are in AI, that I don't really expect it to be a significant bottleneck?
- [55:42]
Marco Mascoro
Maybe let's close with this. If these systems get as powerful as we're discussing, I'm curious how the sort of political system is going to respond. I'm curious if you're sympathetic to the Ashton Brenner view that there's some potential nationalization that occurs, but how do you expect governments to respond? It's kind of remarkable of how not in the political discourse it is, given how powerful it is already. I'm curious how you think about that.
- [56:13]
Jaffa Edelman
I expect so. The thing I calling back to what I mentioned earlier, this concept of the potential for 5% unemployment increase in six months, I think that the public's reaction to this will determine a lot. There will be very, very strong feelings about AI once this happens. I think there will be a bunch of very strong consensus on what to do on things that we don't normally think of as things that people are considering. I know when this happened with COVID there was a several trillion dollar stimulus package passed at like in a matter of weeks to days. It was breakneck speed. I don't know what that will look like for AI, but I think it's like everything else in AI, it's like, you know, exponential, which means it will pass the point of, you know, people sort of care about it to people really care about it quite fast. If things keep going, I just don't know where we're going to end up. I just expect, you know, Wherever we end up, there will be. It will look like, oh, everyone suddenly agrees that why that's to do this certain thing which we would have considered unimaginable a year ago. And I don't know what that will look like. It might look like nationalization. It might look like pausing. It might look like, I don't know, going faster, guaranteeing better unemployment benefits. Who knows? I just think there's going to be some sort of strong response of some sort, and it's going to happen very fast.
- [57:38]
David Owen
Yeah, I mean, you know, you make the point that governments are maybe less interested than you'd expect now, but I mean, the current impacts, I think, aren't really that large. I feel like the attention is getting larger, but it's not that AI as of right now is that powerful and yet governments are already talking about it a lot. Right. And you have people meeting with heads of state from various hardware manufacturers and AI companies and countries talking about their AI strategies, stuff like this. So I feel clearly country, national governments are going to be quite involved. It's just a question of how. And yeah, I also am a bit unclear on that.
- [58:19]
Jaffa Edelman
I think that right now we've seen this thing in revenue and finances where it's been doubling or tripling every year. And my default assumption is that attention that AI gets from policymakers and governments is going to follow a similar trend where it will double and triple every year. This means that in the future, if trends continue, there will be a huge amount of attention. And it means that right now there's a lot more attention than last year. But you don't suddenly skip from very little attention to all of the attention. Although you do move quite. We are moving, I think, quite fast.
- [58:53]
Marco Mascoro
I think we made enough predictions that we'll have to have you back next year and at the end of the year and check in and see where we're at and then make it for next year.
- [59:03]
Host (a16z Podcast Host)
David, thank you so much for coming on the podcast.
- [59:04]
Jaffa Edelman
Thank you, thank you, thank you.
- [59:06]
David Owen
Thanks so much for having us.
- [59:09]
Host (a16z Podcast Host)
Thanks for listening to this episode of the A16Z podcast. If you like this episode, be sure to like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts and Spotify. Follow us on X16Z and subscribe to our substack@a16z.substack.com thanks again for listening and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. Should not be taken as legal, business, tax or investment advice or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details details including a link to our investments, please see a16z.com disclosures.