B (4:57)

Even if AI progress totally stalls, you think that the models are really spiky and they don't have general intelligence. It's so economically valuable and sufficiently easy to collect data on all of these different jobs, these white collar job tasks such that to Sholto's point, we should expect to see them automated within the next five years.

A (5:17)

If AI progress totally stalls today, I think less than 25% of white collar employment goes away. Sure, many tasks will get automated. Claude for Opus can technically rewrite auto generated transcripts for me, but since it's not possible for me to have it improve over time and learn my preferences, I still hire a human for this. Even if we get more data without progress in continual learning, I think that we will be in a substantially similar position with all of white collar work. Yes, technically AIs might be able to perform a lot of subtasks somewhat satisfactorily, but their inability to build up context will make it impossible to have them operate as actual employees at your firm. Now while this makes me bearish on transformative AI in the next few years, it makes me especially bullish on AI over the next few decades. When we do solve continuous learning, we'll see a huge discontinuity in the value of these models, even if there isn't a software only singularity. With models rapidly building smarter and smarter successor systems, we might still see something that looks like a broadly deployed intelligence explosion. AIs will be getting broadly deployed through the economy, doing different jobs and learning while doing them in the way that humans can. But unlike humans, these models can amalgamate their learnings across all their copies. So one AI is basically learning how to do every single job in the world. An AI that is capable of online learning might functionally become a superintelligence quite rapidly without any further algorithmic progress. However, I'm not expecting to see some OpenAI livestream where they announce that continual learning has totally been solved because labs are incentivized to release any innovations quickly. We'll see a somewhat broken early version of continual learning or test time training, whatever you want to call it, before we see something which truly learns like a human. I expect to get lots of heads up before we see this big bottleneck totally solved Computer Use When I interviewed anthropic researchers Sholta Douglas and Trenton Bricken on my podcast, they said that they expect reliable computer use agents by the end of next year. Now we already have computer use agents right now, but they're pretty bad. They're imagining something quite different. Their forecast is that by the end of next year you should be able to tell an AI go do my taxes. It goes through your email, Amazon orders and zack messages, and it emails back and forth to everybody you need invoices from. It compiles all your receipts, it decides which things are business expenses, asks for your approval on the edge cases, and then submits form 1040 to the IRS. I'm skeptical. I'm not an AI researcher so far be it to contradict them on the technical details, but from what little I do know, here are three reasons I'd bet against this capability being unlocked within the next year. 1. As Horizon lengths increase, rollouts have to become longer. The AI needs to do two hours worth of agentic computer use tasks before we even see if it did it right. Not to mention that computer use requires processing images and videos which is already more compute intensive. Even if you don't factor in the longer rollouts, this seems like it should slow down progress. 2. We don't have a large pre training corpus of multimodal computer use data. I like this quote from Mechanize's post on automating software engineering. For the past decade of scaling, we've been spoiled by the enormous amount of Internet data that was freely available to us. This was enough to crack natural language processing, but not for getting models to become reliable, competent agents. Imagine trying to train GPT4 on all the text to data available in 1980. The data would be nowhere near enough even if you had the necessary compute. End quote. Again, I'm not at the lab, so maybe text only training already gives you a great prior on how different UIs work and what the relationships are between different components. Maybe RL fine tuning is so sample efficient that you don't need that much data, but I haven't seen any public evidence which makes me think that these models have suddenly become less data hungry, especially in domains where they're substantially less practiced. Alternatively, maybe these models are such good front end coders that they can generate millions of toe UIs for themselves to practice on. For my reaction to this, see the bullet point below. 3. Even algorithmic innovations, which seem quite simple in retrospect, seem to have taken a long time to iron out. The RL procedure, which DeepSeq explained in their R1 paper, seems simple at a high level, and yet it took two years from the launch of GPT4 to the launch of Zero1. Now of course I know that it's hilariously arrogant to say that R1 or 01 were easy. I'm sure a ton of engineering, debugging and pruning of alternative ideas was required to arrive at the solution. But that's precisely my point, seeing how long it took to implement the idea. Hey, let's train our model to solve verifiable math and coding problems, but makes me think that we're underestimating the difficulty of solving a much gnarlier problem of computer use where you're operating on a totally different modality with much less data reasoning okay, enough cold water. I'm not going to be like one of these spoiled children on Hacker News who could be handed a golden egg laying goose and still spend all their time complaining about how loud its quacks are. Have you read the Reasoning traces of O3 or Gemini 2.5? It's actually reasoning. It's bridging down the problem. It's thinking about what the user wants. It's reacting to its own internal monologue and correcting itself when it notices that it's pursuing an unproductive direct how are we just like oh yeah, of course machines are going to go think a bunch, come up with a bunch of ideas and come back with a smart answer. That's what machines do. Part of the reason some people are too pessimistic is that they haven't played around with the smartest models operating in the domains that they're most competent in giving Claude code a vague spec and then sitting around for 10 minutes until it zero shots. A working application is a wild experience. How did it do that? You could talk about circuits and training distributions and RL and whatever, but the most proximal, concise and accurate explanation is simply that it's powered by baby artificial intelligence. At this point, part of you has to be thinking it's actually working we're making machines that are intelligent. Okay, so what are my predictions? My probability distribution is super wide and I want to emphasize that I do believe in probability distributions, which means that work to prepare for a misaligned 2028 ASI still makes a ton of sense. I think that's a totally plausible outcome. But here are the timelines at which I'd make a 5050 bet an AI that can do taxes end to end for my small business as well as a competent general manager could in a week, including chasing down all the receipts on different websites and finding all the missing pieces and emailing back and forth with anyone we need to hassle for invoices, filling out the form and sending it to the IRS. 2028 I think we're in the GPT2 era for computer use, but we have no pre training corpus and the models are optimizing for a much sparser reward over a much longer time horizon using action primitives that they're unfamiliar with. That being said, the base model is decently smart and might have a good prior over computer use tasks. Plus there's a lot more compute and AI researchers in the world, so it might even out preparing taxes for small business fuels like for computer use. What GPT4 was for language and it took four years to get from GPT2 to GPT4. Just to clarify, I'm not saying that we won't have really cool computer use demos in 2026 and 2027. GPT3 was super cool, but not that practically useful. I'm saying that these models won't be capable of end to end handling a week long and quite involved project which involves computer use. Okay, and the other prediction is an AI that learns on the job as easily, organically, seamlessly and quickly as a human for any white collar work. For example, if I hire an AI video editor after six months it has as much actionable deep understanding of my preferences, our channel and what works for the audience. As a human would this I would say 2032. Now, while I don't see an obvious way to slot in continuous online learning into current models, seven years is a really long time. GPT1 had just come out this time seven years ago. It doesn't seem implausible to me that over the next seven years we'll find some way for these models to learn on the job. Okay, at this point you might be reacting. Look, you made this huge fuss about how continual learning is such a big handicap, but then your timeline is that we're seven years away from what at a minimum is a broadly deployed intelligence explosion. And yeah, you're right, I'm forecasting a pretty wild world within a relatively short amount of time. AGI timelines are very log normal. It's either this decade or bust. Not really bust, more like lower marginal probability per year, but that's less catchy. AI progress over the last decade has been driven by scaling training compute for frontier systems over 4x a year. This cannot continue beyond this decade. Whether you look at chips, power, even the raw fraction of GDP that's used on training after 2030, AI progress has mostly come from algorithmic progress, but even there the low hanging fruits will be plucked at least under the deep learning paradigm. So the yearly probability of AGI craters after 2030. This means that if we end up on the longer side of my 5050 bets, we might well be looking at a relatively normal world up to the2030s or even the2040s. But in all the other worlds, even if we stay sober about the current limitations of AI, we have to expect some truly crazy outcomes. Many of you might not be aware, but I also have a blog and I wanted to bring content from there to all of you who are mainly podcast subscribers. If you want to read future blog posts, you should sign up for my newsletter@thewarcash.com Otherwise, thanks for tuning in and I'll see you on the next episode.