Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil

Summary

---

Training Data – Sequoia Capital

Episode: "Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil"

Release Date: July 29, 2026
Guests: Jerry Tworek (Co-founder, Core Automation, ex-OpenAI)
Rohan Anil (Co-founder, Core Automation, ex-Gemini, Google Brain, Anthropic)
Hosts: Sonya Huang, Pat Grady (Sequoia Capital)

---

Overview

In this episode, Sonya Huang and Pat Grady of Sequoia Capital sit down with Jerry Tworek and Rohan Anil, co-founders of Core Automation—one of SF’s most buzzed-about AI “NEO labs.” The main theme: Are we approaching the limits of current AI architectures, and how can automation and new approaches propel the field toward continuous learning and true AGI (Artificial General Intelligence)? Jerry and Rohan share candid reflections on their time at OpenAI, Google Brain, and Anthropic, delve into the architectural limitations of Transformers, and explain Core Automation’s ambitious vision to both automate AI research itself and invent fundamentally new learning architectures.

---

Key Discussion Points & Insights

1. Learning From Experience: Beyond Reinforcement Learning

• [00:00] Jerry contrasts the distinct ways humans learn physical skills (like football, akin to RL) versus abstract ones (like mathematics)—both experiential, but algorithmically different.
• He posits that while AI has made huge strides in learning from experience, reinforcement learning (RL) is not the final word; new, richer methods will emerge.

Jerry [00:00]: “Reinforcement learning is not the end of learning from experience. There will be better approaches that researchers will be coming up with in the coming years on how to use that data.”

---

2. The Transformer’s Legacy—and Its Limits

• [01:57] Jerry provides context for his recent “spicy take” about the need to move beyond the Transformer architecture.
• Rather than disparage Transformers, he emphasizes the need to deeply appreciate both their strengths and weaknesses before seeking replacements.

Jerry [01:57]: "Appreciating Transformer means understanding what it does well... But if you want to work on replacement, we need to see where we are, what problems we have solved, to start seeing what the next stage is."

• He notes that most recent energy has gone into making Transformers more efficient or cheaper—not more powerful or expressive.
• The true bottleneck to progress, Jerry says, is now the architecture itself, not just compute or scale.

---

3. Real-World Deployment vs. Lab Training—and the Architecture Bottleneck

• [05:05] The hosts ask why Jerry believes architecture is the central issue.
• The main tension: Models are trained in the lab on structured “benchmark” datasets but deployed in the messy, unpredictable real world.
• Attempts to scale up RL didn’t deliver AGI as expected; real-world tasks remained unsolved.

Jerry [05:28]: "We need to have models that learn at test time ... on their real world task ... there are two types of learning we could be doing at test time: in-context learning (limited in data, scalability) and fine-tuning (issues with catastrophic forgetting, data inefficiency)."

• Jerry argues a new, meta-learning algorithm—embedded architecturally—will be needed for true long-horizon continual learning.

---

4. Scale, Economics, and the Transformer "Miracle"

• [09:50] The discussion explores how Transformers became dominant.
• Scaling LSTMs was technically possible but more expensive and less fruitful—Transformers "won" because they were more economically viable to scale.

Jerry [09:50]: "The majestic thing about Transformer...is that transformers are economically valuable—training them [is] lower than the revenue that they generate, which is magic of machine learning."

• OpenAI, he says, outpaced the field by betting on scalable (rather than just efficient) algorithms—a contrarian move at the time.

---

5. Why Start a Research Company Now?

• [12:59] The hosts question why Jerry and Rohan left big labs to do foundational research in a startup.
• Jerry argues that big labs are locked in a competitive cycle, making them risk-averse and unlikely to pursue radical new paths.

Jerry [12:59]: "If the Transformer is profitable... it's very hard to put a lot of attention and energy to work on something that will maybe redefine the field in a year or two."

• Smaller labs mostly copy the big labs. Core Automation’s niche is to take the architectural “moonshot” big labs won’t.

---

6. Rohan’s Perspective: The Computational Limits of Transformers

• [14:30] Rohan reflects on being at Google Brain when Transformers emerged and explains the industry’s slow embrace.
• He describes Transformers as efficient at computation allocation, but "shallow"—rarely more than ~100 layers deep.
• Chain-of-thought and RL-based reasoning is essentially stacking inference-time computation, but extremely inefficient in both tokens and compute.

Rohan [19:23]: "Most transformers that we train are quite shallow ... Deep depth is like ... it's called deep learning because you wanted deeper representations ... No one has actually shown us learning extremely deep representations."

---

7. Forecasting the End of Transformers

• [20:41] Jerry and Rohan discuss how far Transformers can take us.
• Transformers are superb at distilling static world knowledge—but brittle if real-world distributions shift or if they can’t readily adapt at test time.

Jerry [20:41]: "If OpenAI and Anthropic stopped training new models... the model is getting less and less useful. Anything valuable has to be present in training, and when it's not, [Transformers] suffer."

---

8. Learning From Experience: Beyond RL

• [23:32] The "Age of Experience": RL is important, but just one method.
• Jerry describes how human learning involves multiple, distinct algorithms—RL for some tasks, deep thinking (like math) for others.

Jerry [23:32]: "There is one learning algorithm in a brain. I think there are multiple actually and they work together."

---

9. Optimizing for Efficiency: Merging Pre-Training and RL

• [26:55] Rohan details his history working on large-scale optimizers and their unexpected importance.
• A more end-to-end approach—considering both architecture and optimizer simultaneously—is essential for orders-of-magnitude efficiency gains.

Rohan [26:55]: "Optimization and architecture go together ... your optimization algorithm defines what architectures you will discover."

• He comments on the inefficiency of current RL approaches (spending compute to get limited feedback) and sees room for massive improvement.

---

10. Will We Ever Match Biological Efficiency?

• [32:01] Rohan doubts current hardware and approaches can ever match the data/computational efficiency of biological learning.

Rohan [32:01]: "Neural networks are quite different. The hardware stays, the neural network stays, but it's learning very inefficiently and you need a lot more of them and a lot of parallelism to get small amounts of information through. So until I think we design hardware to be much more like how humans operate... it'll be much harder."

---

11. The Automation of Research Itself

• [34:58] Core Automation’s mission is not to remove humans from the loop, but to dramatically amplify each researcher’s capabilities—akin to “giving each researcher a car, not just their own feet.”
• By rebuilding the research stack natively to modern, agent-powered workflows, the team aspires to run many more experiments, much faster.

Jerry [34:58]: "Automation by Core Automation is about giving each human maximum agency ... The speed of iteration, the speed of research... is something very, very different."

---

12. AGI: The Definition and the Goalpost

• [37:55] Jerry argues AGI is best defined as "a model that can improve itself without humans in the loop," which remains distant.
• No current LLMs can truly remove humans from the learning loop; the LLM-human hybrid is "really successful right now, but LLMs without humans, not so much. Not at all."

---

13. The Core Automation Master Plan: From Kernels to Architectures

• [39:45] Rohan and Jerry detail their immediate roadmap:
  • Building and rapidly iterating on new architectures
  • Focused effort on kernel generation, as a bottleneck for experimenting at scale
  • Hosting kernel optimization competitions with human/agent teamwork (e.g., the “QR kernel” challenge)
  • The goal: accelerate the cycle from raw idea to highly optimized, hardware-efficient prototype—using AI agents to assist with coding at the deepest levels.

Rohan [42:12]: "Kernels ended up being a bottleneck...most of the operations were novel that we haven't gotten a lot of people to look at it...There are only like two humans at Google who could write it...it would be ideal if models had those abilities."

• The ultimate goal: Construct an architecture and learning loop fundamentally better at continual, deployment-phase learning than Transformers.

---

14. Success Metrics for New Architectures

• [46:20] Jerry says the visual “bend in the plot”—a performance jump on specific metrics—signals success, but researchers instinctively know when they're on the right track.
• The real benchmark: A system that meaningfully adapts in deployment, improving at the tasks of core automation scientists.

Jerry [46:20]: "All the best plots in my life, I had them in a dream before they were real...We want to be evaluating our systems at doing the work of Core Automation scientists each day."

• Rohan jokes: the evidence? If the team can go on vacation, and the lab keeps improving itself in their absence, “extend the vacation…until we are on permanent vacation.”

---

Notable Quotes & Moments

• Jerry on architectural innovation:
  "I'm asking myself a lot, what is next in machine learning? At this moment, the bottleneck to better models and smarter systems is the architecture itself." [01:57]

• Rohan on computation and kernels:
  "You need your architecture to run efficiently on hardware. A theoretically optimal architecture is not useful to anyone." [14:30]

• Rohan on stepping beyond current RL:
  "RL is spending a lot of compute, not as efficiently...if we could spend it because you don’t get much feedback...it seems quite inefficient and an easy target to get orders of magnitude on top of." [30:52]

• Jerry on AGI:
  "For me, AGI is a model that can improve itself without human in the loop in any way. That's, I think, the moment where we can meaningfully talk about AGI." [37:55]

• Rohan on autonomy:
  "We go on a vacation as a team and see if the lab produces something better for the week...extend the vacation until we are on permanent vacation." [47:57]

---

Important Timestamps Overview

• 00:00–01:00 – Jerry: Human learning vs. RL
• 01:57 – Transformer weaknesses; architectural bottleneck
• 05:05 – Lab vs. real-world gap; need for meta-learning
• 12:59 – Why big labs don’t try radical projects; startup opportunity
• 14:30 – Rohan's experience with Transformer’s rise
• 19:23 – Depth limitations in Transformers; inefficiency of chain-of-thought RL
• 20:41/23:32 – Limits of Transformer; adaptability gap
• 26:55 – Efficiency, architecture/optimizer co-design
• 32:01 – Biological learning efficiency vs. neural nets
• 34:58 – Automation and research lab design
• 37:55 – AGI as self-improving systems
• 39:45/46:10 – Core Automation’s approach: kernels, architecture experimentation
• 47:57 – The “vacation” metric for automated research

---

Tone and Style

• Open, geeky, reflective, and slightly irreverent—both guests and hosts speak candidly about the field’s dogmas, missed predictions, and the challenge of making foundational progress.

---

Summary Takeaways

Core Automation’s founders believe the AI architecture revolution is overdue. As industry establishments double down on Transformer-based recipes, Jerry and Rohan aim to automate the research process itself—using both human ingenuity and agentic “research assistants”—to rapidly discover new models that can learn continually in messy, real-world settings. In their vision, real AGI is the moment when “the lab could go on vacation, and the research keeps improving on its own.” Until then, the field needs more than scale: it needs new architectures, new optimizers, new experiments—a new kind of lab.

---

Transcript

[A]

If I play football, for example, it looks very, very close to reinforcement learning. I kick a ball a lot of times and every time I adjust it a little bit and I see if it roughly matches what I wanted and there's some self reinforcement happening. When I learn mathematics, it's very different type of thing. It's like reading about hard concepts and thinking about them very deeply inside my head until things click and until I have them connected. And both of those in some way are learning from experience. They are just very different. We probably are spending the most compute ever on learning from experience. But reinforcement learning is not the end of learning from experience. And there will be better approaches that researchers will be coming up in the coming years on how to, how to use that data.

[B]

Jerry, Rohan, thank you so much for joining us today. The two of you are the founders of Core Automation, one of the hottest NEO labs in San Francisco right now. And before starting Core Automation, you led some of the most important research projects of the AI era. Jerry, you were VP at OpenAI, where you worked, amongst other things, on running the Strawberry and reasonings teams. And Rohan, you were one of two of the four pre training leads at Gemini, and before that led a lot of the fundamental AI research at Google Brain and were the fix it guy across Google and then at Anthropic. And so between the two of you, you've seen more than your fair share of what the world looks like in terms of doing frontier research. And so I'm very, very excited to dig in. Let's start with you, Jerry. You tweeted a very spicy take recently. The first step to replacing Transformers is appreciating deeply how far they were able to carry us. Is that a eulogy for the Transformer? What does that mean?

[A]

Thank you very much for inviting us here, Sonja. I feel like a lot of my interviews these days is explaining my tweets and what did I, what did I mean? But appreciating Transformer means like understanding what it does well, not solving the problems that it is solving well. You have to focus on its weaknesses. You have to understand good parts and bad parts, and it's very easy. And a lot of the work what people are doing in architectures is trying to make Transformers cheaper and trying to make Transformers more efficient. I very rarely see people thinking about how do we make Transformers more powerful? Trying to do more expressive. But, but like seeing someone weak parts and seeing someone's strong parts are almost, almost, almost the same thing. It's just, just understanding the shape of Transformer a little Bit more. Well, I think right now we are in this stage. We got really, really good at training really, really big models, and we mastered two algorithms, we mastered pre training at a large scale, and we mastered reinforcement learning at a large scale. And I'm asking myself a lot, what is next in machine learning? And I think at this moment, what the bottleneck is to better models and to smarter systems is the architecture itself. It is this moment to revisit the train we've been riding for the last six years of trying to add more and more parameters to essentially two of the same operations, which is MOE and attention. And when I'm thinking about it, like, where we are today and what we are doing, I am thinking a lot about what Codex and what cloud code are doing for us. And I'm really, really appreciative of those systems and of the coding and of the workflow automation and of the systems of the products that we have today that we essentially have built over those six years of scaling. And I think this is the first step of thinking, if we want to work on replacement, we need to see where we are, what problems we have solved, to start seeing what the next stage is, what problems we haven't solved yet, what kind of are we missing? And this is kind of. Whenever I use codecs and I am successful at a task, I also start thinking, why didn't I try to push, like, think harder whenever I come to work? There are a lot of things I do with Codex, but I still come to work, I still ask it to do certain things for me. And I'm always asking myself, why am I even needed there? Why is Core Automation is name and its concept is we want to be automating tasks, and why are those things not yet automated? Why is not Codex doing everything for me? And this is the question of, like, the research, where we want to go.

[C]

And.

[A]

And with that research, I'm trying to think, what kind of models, what kind of systems do we need? What kind of qualities do we need that we don't have today? And that's what I'm thinking a lot these days.

[C]

And you have this starting premise of the architecture is the issue, which I think is a contrarian point of view. So what led you to that point of view? What did you see that made you think the architecture was the issue?

[A]

It's fundamentally what is the issue? It comes back from the previous implication. What I think is the issue is that the models are being trained in the lab and are being deployed in the real world. That is the Fundamental tension that is there.

[B]

And

[A]

a bit of my disappointment comes from my personal story. Whenever we were starting the research and progress on scaling up reinforcement learning at OpenAI, I basically believe that scaling up reinforcement learning is a necessary stepping stone on a path to, to AGI. Since I started working at OpenAI and I was always reinforcement learning maximalist, I was always believed this is what we need to focus on, this is what we need to do. I've seen LLMs being scaled up to higher and higher levels through GPT3 to GPT4 and we're still doing very little. RL and I had this internal belief that the moment we start scaling up, we'll solve everything, we'll be able to solve all the problems. And we eventually started scaling up. Or I was just in there, I was in the center of it. I was thinking, here we are. If you ask Jerry, in 2024, when do we get AGI? I would say 2025 will be that year. This is where we solve everything. And I saw us training model after model. This model was getting better and better. All the benchmark scores were going up. And did we also solve all the real world tasks at that moment? Unfortunately, unfortunately not. We still have work. And I realized there was this bit of distinction as all the benchmarks that we are evaluating our models, they were essentially the same thing as we were training the models on. Like all the evals and training tasks are the same sides of the coin, but the real world distribution and real world task is much messier, much murkier, much more, much more different. Our training data didn't really replicate the real world use cases and despite us basically maximizing all the tasks. If you see ask anyone training models, hey, what is one of your main issues? I don't have hard enough tasks. I don't, I don't have what to train our model on yet. We are still not covering the entirety of the, of the real world distribution. From that my conclusion is we need to have models that learn at test time. We need to have models that learn with users on their data, on their real world task, on the real world distribution. And there when you are asked why don't we have that today? Why are transformers not learning anywhere? And there are essentially two types of learning that we could be doing at test time. We could be doing in context learning essentially of transformers, which is, it doesn't have fundamental problems of catastrophic forgetting. It doesn't have that issue. It is pretty data efficient. So that is great, but it's not very scalable. We only can have so much of it. This limit limited and has some more. Even more of mechanical limitations of what actually are you doing when you build context. But maybe we can come back to it later. But we have in context learning, which is very limited and very small amount of data. Whenever I'm using codecs, roughly around 20 minutes of usage, I need to compact it and move it afterwards, which is not that much. Not that much data. All we can learn is for 20 minutes. It's not that much. And the second thing is fine tuning. We could try to continuously fine tune our models, but then those have the issues of catastrophic forgetting. We have issues of very low data efficiency. And neither of those are very solvable. Neither of those are very easy to find ways. People have been trying. If those were easy to solve, some already solved it. So my personal belief is we need to find an algorithm that we can meta learn. We can express on the architectural layer that can represent how does learning look like? How does learning look like that can work on much, much longer horizons.

[B]

Do you expect the architecture will look transformer like? Because my understanding from the chief seats is that OpenAI had been trying to scale up reinforcement learning for a long time. And it wasn't until the transformer came about that it seemed like there was an even kind of scalable prior on the world upon which to even scale RL. And so how do you even go about trying to think about scaling up this new regime?

[A]

Yeah, that's a great question. I think those two things happened at the same time. But if anything that happened there was mostly about economics, because technically you can scale up lstms. Just no one really dared to go in that direction. And they did scale much more poorly. They're scaling. In a scaling loss paper, there is presented a comparison of LSTMs and transformers. And fundamentally the scaling of transformers was better. There is a world where we never invented Transformers and we would be scaling LSTMs and we will be having some models, but because they would be much more expensive to train and much less impressive as a product, we would have would have just a worse experience. And maybe no one would be able to convince people to spend as many dollars training those gigantic LSTMs because we wouldn't get a market return. The majestic thing about Transformer, which goes back to like, why do we have to appreciate transformers so deeply? Is that transformers are economically valuable training them. The cost of training them is lower than the revenue that they generate, which is magic of machine learning. And it's not guaranteed by itself. But for STMs it probably wouldn't be that way, which made it happen. But you can in many ways, you can scale most of the architectures. I think a lot of reasons why people didn't scale things before was because researchers before OpenAI had a lot of reluctance to scaling. It was often seen as unscientific. And research in algorithm was providing how do we become more and more efficient? How do we, for the same compute budget, get better and better results? And it was a bit of a Contrarian bet by OpenAI at that moment to try to say, hey, we don't care about better and better algorithms, we care about more and more scalable algorithms and how do we pour more and more compute and get better results? Which OpenAI was criticized repeatedly by many people in the community for a long time. But thanks to that, we have the models that we have today. And I think there are tons of architectures that can be scaled up. And I am part of the Core Automation's mission. And our belief is that a lot of architectural research happened at too small scale for too long time. A lot of people are trying to say, hey, let's try to first try our architecture on a small data set in a small compute regime and then see where it scales only after you prove itself. But for example, when you do work on reinforcement learning, you know that to get to any interesting results, you need certain level of compute to even see the capabilities in the model. Reinforcement learning needs a baseline of ability to only start working. So where I am coming from, probably there are many architectures that need a baseline of compute to even start doing anything, anything interesting, anything useful.

[C]

Can I ask you then maybe a touchy question?

[A]

Please do.

[C]

If you need a baseline of compute that sounds like a job that would be well served inside of a big research lab, why start a company to go do this?

[A]

It's a great question and I think in many ways it's likely a timing thing. Timing issue. Market is right now in a very specific place where the biggest and the most successful labs, by coincidence or by fate, are probably in the most competitive market fight ever right now. Which makes them not very keen on trying different paths, trying alternatives. If Transformer is profitable and if you can spend more efforts and more resources scaling Transformer to win in the next quarter, it's very hard to put at least a lot of attention and a lot of energy to work on something that will, that will maybe better or maybe, or maybe we'll redefine the field in a year or two. So I think the biggest labs and I Talked to basically all of them don't have that much interest in trying the alternatives to transformer. And the labs that are not the biggest are doing whatever they can to do what the most successful labs are doing. And everyone is trying to train the same coding agent. If you look at last week's releases, everyone is trying to release a coding agent right now. And I think we need different paths and different approaches here. So that's what the niche in the ecosystem we are trying to fill.

[B]

And Rohan, you were at Brain when the transformer was invented. Do you agree with Jerry's eulogy for the transformer?

[D]

Yes, in some sense. First, when the transformers, Ashish Noam and others came up with it, I had like work. I worked on my work on online distillation around the same time. We presented it at the same internal research conference. It wasn't a big deal. Internally there was only a few people who actually got it. A lot of people were like, oh, it's like, yeah, it's another work. And people were finding ways to. And it was also very focused on, at least the original work was very focused on a real problem, which is translation. So they solved like they beat LSTM on translation and it took OpenAI. I mean internally at Google there was definitely like Noam and a few others were definitely interested in scaling language models. I think it is until GPT2 and GPT3 that we saw the benefit of Transformers working quite well. At least the way I think about architecture is how do we spend computation? And Transformer is one way, very efficient way to spend computation. But now that I look at the industry, a lot of our computation is inference time and spending it on tokens. Let me ask this question, like if I want to optimize for a better architecture, I want to look at both pre training and RL together and I would like to find architectures that spend computation much better than current chain of thought token generation. To give a much better overview, I think of pre training as built the transformer with certain context length and RL comes in and it's like, well that's not sufficient. I need more computation. Let me do it via adding one token at a time. This is quite inefficient from like inference perspective. You're doing one token at a time. So most of the solutions have been finding to do better ways of speculative decoding. So it's like a band aid to a problem that we've picked something that can only generate one token at a time. So autoregressive decoding. There Is problems with the transformer in terms of how do we spend the computation. For the longest time I think most of the world was training very large dense models and it took the industry roughly two to three years to get to refine the architecture to what we now take for granted was not obvious to a lot of people. Sparsity and mixtures of experts and getting good training efficiencies with them. So then you can ask what's wrong with the transformer? Well, if the computational depth is poor, how do we increase computational depth? And just posing that question opens up like 20 new directions on how we can modify the mechanism to incorporate it. So I see to do work like this it takes time and usually fundamental research in the past have taken five, six years to land into industry. And it's largely from organizational knowing that it is important. This is the bet. Just like Jerry had the inner belief that RL is needed. Absolutely did not have that belief at Google. I was a pre training maximalist pre trainer biggest I think you guys are a good fit. So that inner belief and second is you need your architecture to run efficiently on hardware. A theoretically optimal architecture is not useful to anyone. It is something when it comes into practice. So you need the research inception to getting it productionized and getting kernels and everything written the end to end loop. And there's only few places right now which have integrated teams doing that. And I think we have built a team in a way that puts the experts together, not in different silos that we are accelerating on having everybody look at the problem holistically from end to end. So that's where I'm quite bullish. That's why I'm here. The current mechanisms are quite poor and if you leave it to the world I'm afraid that it'll take us a much longer time horizon before we replace the transformer. And I think a lot of folks are already complaining a lot on token costs. And that seems like as someone we're

[B]

not complaining

[D]

in terms of like. Yeah, exactly. I come from the Google mindset where we had to serve billions of people. So finding more efficient architectures that fit have a deadline on latency in the number of tokens that you can serve. So when I look at that the amount of the world that can use frontier tech is very little and someone or some group has to accelerate and make this better. And we are taking that shot at doing that. The current technology just scaled up is still only relevant to a subset of humans. And this is the bet we're making.

[C]

So one of the things I heard you say was the problem with transformers is the computational depth is poor. If that's the crux of the issue, tell us what does that mean? Why is that the case? How do you fix it?

[D]

I can give you one insight. Most transformers that we train are quite shallow. It's at most like 100 layers. Deep depth is like. It's called deep learning because you wanted deeper representations. There has been experiments on going into depth, but no one has actually shown us learning extremely deep representations. Chain of thought reasoning and rl to do chain of thought by model itself is one way to increase computational depth because every token you add, you add one more pathway. So then you can get out of this bottleneck that the pre trained architecture has set you up on. You can only do number of layers times sequence length. Now you can increase the sequence length and you get much stronger results. You can do inference time scaling. Now the issue with inference time scaling is that models now have to produce more tokens to get better results. And that's very one token at a time. And from this you can see like you can directly address many of these things. And this is like a subset of work that we are looking at, right, Making this much more efficient.

[C]

What's your forecast for the transformer based architecture? If it's not the end state, how far can it get us? When do we start to see it topping out?

[A]

I think it all comes back to what we are training transformers for and what we can do with them. We're doing pre training, which is very good at distilling all the knowledge from the Internet into transformers. And then we can rl them through, which is we basically can bake all the workflows that we want into a transformer. So what transformer is capped out is we have all the knowledge of humanity in the model together, of the relationships and how do they work together, how they can be combined. And basically any task that we have training data for, we can put into that model. And this can be gigantic model trained with a lot of compute on all the data in the world. And then if we ever stop training that model, what would happen? The question worth asking often and thinking about Transformer, what would happen if OpenAI and Anthropic stopped training new models and we got the transformer we have today and say this is it, this is the best model we have. Months pass your spouse and the model is getting less and less useful. Maybe the lab really recorded every human on earth, what they were doing and what their tasks were and their environments. And put them in the model, put them in a learning environment. But then what happens if anything of that changes? If there are new events in the world, if those new events have new relationships between them, if there are new types of tasks, if there are new code bases, new tools to use, Transformers are getting a lot of their usefulness and value through the things that are valuable have to be present in training. And when they are not, they suffer. There's some ability to adapt, but it's not very big and not very flexible. So in my mind, this is kind of the level where the transformers top, which in many ways what I think is a tool to use for us. If we kind of know if there's a human who knows the limitations of a transformer, they can schedule that model. They can write a prompt of what is the task that you want. And by doing the training we are doing, you can get very successful at that. And any task the model fails, you get added through training data and the model can succeed. But that loop has to go through the lab training the model for you. And if the model that fundamentally needs to be trained in the lab, how much do you think of it that this is the goal or you would want to be able to update the model somehow, not having to go back there.

[B]

Have you read the Rich Sutton and David Silver have this paper, the Age of Experience. Have you read it? I'm curious how much you agree or if you have any different opinions. Where do your opinions diverge?

[A]

Reinforcement learning is not a particularly new approach, particularly particularly new thing to do. So in some way. Age of experience, I think, always has been there and people have been criticizing a bit pre training because pre training very clearly is the other way of looking at the models, which is like we have static data. That data is mostly generated by others. Although I have this personal view that pre training today is largely distilling other models, older models, into the new model, because most of the tokens in the Internet are coming from AI. But there's clearly pre training, which is behavioral cloning, which is mimicry, which is compression of Internet data. But reinforcement learning is not something that people haven't been thinking and people haven't been doing. Reinforcement learning was used to solve backgammon back in the day. Then it used to solve Go StarCraft Dota to solving programming right now. And every time it comes down to model writing its own experience and learning from that experience. But this is very clear and what I think is interesting and what I think is still perplexing to people that reinforcement learning Is not really the only way to learn from experience. And there will be more and there will be a little bit more of I think you can call it algorithmic but essentially innovation of how we learn from experience. Just because reinforcement learning is only one way to do it. It's a mathematical formulation and especially right now how we are using it. It really likes those parallel rollouts for variance reduction. Comparing how the model does in parallel versions of the world, which is not how we do, not how we learn from experience. We learn from our experience much more efficiently and much more. We use those in many ways at some moment I've been trying to explain to people what brain does, how we learn. So there's one learning algorithm in a brain. I think there are multiple actually and they work together. But if I play football for example, it looks very closely to reinforcement learning. I kick a ball a lot of times and every time I adjust it a little bit and I see if it roughly matches what I wanted and there's some self reinforcement happening. When I learn mathematics, it's very different type of thinking. It's like reading about hard concepts and thinking about them very deeply inside my head until things click and until I have them connected. And both of those in some way are learning from experience. They are just very different. So summarizing my thinking of the learning from experience is that we've been doing it for a while. We probably are spending the most compute than ever on learning from experience. But reinforcement learning is not the end of learning from experience. And there will be better approaches that researchers will be coming up in the coming years on how to use that data in a richer nature settings.

[B]

Interesting. Rohan, I'm curious since a lot of your work has been around optimization and efficiency, how do we get to a orders of magnitude more efficient? I guess more compute efficient and more data efficient learning algorithm.

[D]

I'd start with measurement. I think pre training as we define it right now is about compression. We look at the placidity and then measure how do we decrease the perplexity. And then we find that scaling and increasing parameter count and putting more compute is the way. And every time we increase compute in log scale we get this epsilon more improvement in these metrics. I think this is fine for building the prior but I think this is the wrong way to look at the problem. We should be looking at the end to end. What are we training these models for? Look at the outcome. Like for example, I trained this model and gave it to Jerry. Jerry will do RL and destroy all the perplexity metrics that I have created. Right. So then it's sort of like it was the best way we had so far to attempt to solve the problem. And I think the labs and everyone else have done a great job in producing intelligence that's super valuable and makes my work so much fun. But it was the bootstrap process. To get there, we have to combine pre training and RL together and that's like where one order of magnitude improvement would come from. And that's like a training procedure. You can say it's a learning algorithm. In terms of optimization, my story is I started optimization at Google for logistic regression back in 2016, GoT Nerd sniped by it, worked on some solvers for what we used to call Sibyl, which was the large scale linear solver that was used at Google before neural network took off and then replaced this. Then I asked myself, what do I want to work on with neural network? And it was quite clear I want to understand the training algorithm and make it better. And then someone, Vineet Gupta just showed up one day at my desk. It's like, I heard you're really good at writing optimization methods. We have this idea that we worked hard on a whiteboard and what turned out to be the Shampoo algorithm, can you help us implement scale, make sure it works at large scale for neural network training? So when I was working on this, I thanked my manager, Yong Hui Wu, who supported it throughout that till my end of my tenure, 2024. But largely the community and most of the people were not as excited by this idea. And for me, this was the most exciting thing because I was like, I'm putting in computation and making training better. This is the thing I have to figure out. I will spend as much time I would take to do it. And then people were making this assumption, oh, what's the upper bound? You could still use Adam, that's fine. Why are we. You could spend all the time on everything else, not optimization. But in some sense, optimization is like you have a model, you're optimizing it, you want to optimize it better. Now, connected to some of the stuff that we talked about architecture, what has happened is that a lot of the work that we've done in architecture is to make these networks train. And in some sense, it's like two sides of this coin and optimization and architecture go together. You could have a stronger optimizer, train a much more harder to optimize model and get better performance, or you can use a weaker optimizer on Easier to optimize models and get decent performance. So these trade offs that appear all over and for me, I spend a lot of time working on it. I think we used it for Gemini 1.5 flush and then the community started getting more interested in it. There was the soap paper published. We have an entire literature of shampoo, soap, all like bath time, the things that you would use. And then it was quite clear like. So that was like maybe a 2x improvement over what was happening. But even then, if you look at shampoo, it's quite weak in what it's doing. It's not using all the information that's available to you when you train. And as you use more and more information as part of training, you can get better improvement. And in some sense, like your optimization algorithm defines what architectures you will discover. Like I have colleagues, it's not very popular in the literature. It's only like maybe four people in the world care about it. Kind of ideas that are extremely interesting, like residual connections, have been extremely useful for training neural networks. There are folks who've now gotten rid of them and learned deeper representations, but they needed a better optimization method. So for me, optimization methods and the question you asked, which is like, how do we get there? It's combined with architecture and thinking about the problem end to end is where a lot of the computational efficiencies end. And I also see RL as spending a lot of compute, not as efficiently. And so if we could spend it because you don't get much feedback and you're spending a lot more compute because you have to decode all this long chain of thought to get this one bit of information into the network seems quite inefficient and an easy target to get orders of magnitude on top of. I can go on talking about optimization altogether.

[B]

I love it.

[A]

Rohan, do you think we'll ever approach

[B]

or surpass biological learning efficiency?

[D]

I do not think so because I think we would need to change. Maybe that was a strong statement. At least with the hardware we have, it seems pretty unlikely. It it our biological like we have something, as Jeff Hinden says, model computation. So we built our own circuit as we grow up and we built our own learning algorithm with the hardware and then we die and then we're gone. Neural networks are quite different. The hardware stays, the neural network stays, but it's learning very inefficiently and you need a lot more of them and a lot of parallelism to get small amounts of information through. So until I think we design hardware to be Much more like how humans operate, maybe more analog. Figure out how to deal with analog circuits, figure out how to do it error correction, Figure out how to get information through. It'll be much harder. I think we are safe.

[B]

Safe. That's an interesting way to put it. The idea that pre training and RL should be optimized end to end seems like such a clear, maybe obvious statement. Do you think the labs realize this and is it just hard for them to get rid of org chart and process to be able to make that come together? Or what stops the labs from being able to unify the two?

[D]

I don't think it's as obvious because it's a completely again different optimization problem. You have a prior, you're doing rollouts, you have higher variance. And then pre training is much large batch like more parallelism or compute for the unit of time that you can spend. So it is not an obvious thing for folks to combine these two training procedures until you think a bit more like why is it that the naive combination doesn't work? So that's one. The second one is if I poll some of the best researchers in these labs, they would say, oh, this makes sense, we should probably explore it. But it would be probably not in the top bucket because they have to train a model for the next cycle. Before, as Jerry said, there are companies now competing for release cycles because tokens are not sticky. So it's much harder to do even long term research of six months in many of these labs in the environment they are in.

[B]

So it seems like one of the core premises for core automation is you're starting at the lab at a time when I know Sam's been talking about the AI scientists, I think Dario's been talking about the AI scientists. It seems like your job as researchers has actually fundamentally changed and you get to start the company native to that era. And as a result you're maybe able to run a lot more experiments than otherwise might be possible. How automatable do you think the research job is and how are you guys approaching building your lab to be as I believe you're one of your missions is to be the most autonomous lab

[A]

there is, most automated lab, AI lab in the world. And to start with, I think the automation, the version of automation by core automation is about giving each human maximum level of agency in some way. We're not trying to really get humans out of the loop, which is like one version to automate, but it is about give humans ability to do the most with their amount of time, time whenever you are walking, you can get some distance. Whenever you get a bike, you can go walk a larger distance. Whenever you are a car, you can go much, much, much, much larger. Whenever humans started farming, they had to farm by hand and work on a small plot of land. When you have a machine, you work on a much, much larger plot of land. Personally, I'm both really great fan of the current coding agents and very happy in some way. It is what I've been working for many years, both doing coding research and working on various versions of AI scientists inside of OpenAI. And in the end I realized starting a company to realize that vision is one of the best ways to realize it because the way you can do research today is very, very different because a single researcher can do much more. In the end, the speed of iteration, the speed of research, the speed of how quickly you can move through ideas and how quickly you can can get data on your ideas is something, something very, very different. And you can try to move the old structures around it and the team's workflows, how data is gathered, or you can try to build, like you said, you can try to build natively for it, for processes that maximally empower each researcher and allow them to just, to just iterate on their idea much quicker. We are here and we are trying to rebuild the full deep learning stack and try to think how we can do almost each operation differently. What are various options and if we can execute at least even one of those experiments a day, that's already a pretty good iteration speed versus anything that was done before. And there isn't really any fundamental laws of physics reason why not. And maybe one day we get to 10 of those a day, maybe one day we get to 100 those a day. And fundamentally for that search process optimization process, we should be able to just find things that work in a better deep learning setting. And what we are trying to do is within almost all of us. We are a team that is very agent built and automation build. We're trying to do an experiment, how far we can push those things and how much an organization tries to do as much as we can with a small team, how far we can get with that?

[C]

When will we know that we've reached

[A]

AGI at some moment? I used to say it's very much in everyone's heart, whatever they consider AGI. OpenAI says the system that can outperform all humans in economically valuable work. But it goes to my previous statement, what if OpenAI stops training models? Would that still keep working? Would that still keep the automation level the same? The same? Or would that two duck drift? For me, AGI is a model that can improve itself without human in the loop in any way. That's I think, the moment where we can meaningfully talk about AGI because that is in some way it is a sub definition of the previous one. Because improving AI models is actually a job that humans can do. It is economically valuable work and it definitely is the case. But removing humans from loops with models has been actually notoriously difficult. Difficult. So far we haven't come anywhere close to it. It's very, very hard for me to find any task where LLMs were able to get humans out in the loop. The human LLM hybrid is really, really successful right now, but LLMs without humans, not so much. Not at all. And what I have seen there in 2024 and in early 2025 is that the current path doesn't get us there. And I think we need some pretty serious research on this company or

[C]

unlock

[A]

how do we make our models learn and adapt at test time on a deeper level than we've been so far.

[B]

I feel like we've alluded to this throughout the conversation, but it's kind of been one of these five blindfolded people trying to find the elephant things. What is the grand master plan for Core Auto that you're willing to share?

[D]

We can share our six month roadmap in some sense building architectures as I said would. It's not just how good the architecture is, does it run well and can we get users, including ourselves as part of the lab to use it? Right. So then that's directly like we can do many things now but the thing that is going to be difficult and the thing we want to automate away is kernel generation. So we have a set of hardware GPUs Blackwells that we have to train and run inference on. We will build the best model that we can to to basically reduce the time from having a very cool idea that can make these bottlenecks go away in the architecture to having them run at the highest TFLOPS on GPUs and in some sense like current coding agents plus humans can go a long way. But an example of this is our QR kernel competition that we hosted with GPU mode. It's for running this very old linear algebra operation qr. It's used for optimization like shampoo line of work uses it many other places uses it and you want to run this efficiently on a B200 node. If you use co solver for the shapes that we care about, you get some efficiency. And then a human plus some search loop can get you something like 7x. But it requires the high state human. Like there's maybe three people in the world and spend about $100,000 on these coding agents over a span of four weeks to get to a solution that's 60x faster. So these models today are nowhere close to getting that 60x faster kernel. And there is a real bottleneck now. That was a single problem. It has like perhaps three different operators work on this panel, do this matrix, multiply, I fold it back in and do this repeatedly. That's what the secure factorization of a matrix would look like. And if you give this problem to Entropic Models, OpenAI Models, Gemini, it just wouldn't solve it. Our models are not even close to solving this problem. So for us, it's like something that we've talked about, something that we are getting close to is sort of like getting to that point because that's our inner loop to having more efficient architectures.

[B]

Why kernels? Is it just because maximize intelligence per flop of compute you need to generate

[D]

kernels in some sense. I've had three projects. Two of them have kind of landed in the industry. So first is secondary methods. Kernels was the bottleneck because you have to run it well. If you are at a place like Google, you cannot spend 10x amount of compute and get a 2x win. So I could only spend maybe a budget of 20% and get the 2x win. Everyone's there, happy, be great. So I think that's the market, right? You spend less than you get. So kernels ended up being a bottleneck there because most of the operations were novel that we haven't gotten a lot of people to look at it. There's only like two humans at Google who could write it, Rasmus and Peter Hawkins, because it was deep XLA llo code that you have to write to make this work. And then that took them two years to do. The other idea I had with one of my coworkers at that time was like replacing some of the parameters in a transformer with extra memory. And we called it Ngrammar. Ngram memory. We worked on it in 2020. We had versions of it internally deployed. Not the big version, but the smaller version. But there I needed something that can accelerate sparse gathers and scatters as part of training. It required a hardware change and a hardware making use of the hardware. It never arrived. I had conferences set up with the TPU team, us And a bunch of others. We were talking about it during COVID like oh, we're going to have it happen. And it never arrived. I also was using TPU Xenanthropic while I was leaving. Just barely started the surface of being able to do it. But at the same time, six months before that Deepseek Road, their N gram, which is an improved version of adding more memory, showed scaling loss that yeah, you don't need moes. You could actually replaced it with these N gram embeddings. For me that was like. It was like a five year thing and I was very happy for that.

[B]

So much of the state space to explore isn't even possible if you're not

[D]

writing kernels kernels and you need to be assisted in writing kernels or solve that kernel to have the highest performance. And the roof line is pretty high. So it's like the qr. If I use coosolver's qr I get some performance. If we use the competition winners QR, you get 60x faster. And that is a completely different playing field now it opens up an entire new set of algorithms you can apply in terms of training transformers, training optimizers. QR is so fundamental in analyzing the eigendecomp decomposition and many other things. So it is a thing that I think also if you think about it, only few people have the skill set too and they're very much not at the same place. It's like one person here, one person there. And it would be ideal if models had those abilities.

[A]

Maybe I'll summarize a little bit and talk from the high level of what we want. Core automation is a lot created to build models that continuously learn from deployment. We believe, as I mentioned, that transformers are incapable of continual learning. There's no way how to put continual learning on transformers. So we know we have to find a different architecture in a way. Our quest is to find that new architecture, find that transformer replacement. And we want to build the most automated lab to do it. We want to be able to build experiments at scale the quickest we can iterate on them, try a lot of new architectural ideas, have strong priors of what we want to do. To search the space of architectures efficiently to find to that go to that place fastest than anyone else. That's what we want to do. And all the work we are doing on kernels on large scale training on trying new architectural ideas is exploring that space.

[C]

So you're going to and experiment your way into finding a superior architecture. How will you know when you've found it, what are you looking for to

[A]

say aha, this is the one that's a great question. There are always two angles in my mind. In my experience, every successful research had a plot that shows something that other plots don't show. There's one line that is a little bit Bendig in a different way way and you're saying this is what you want. But at least my experience with research always has been that plot is already quite late in a journey where most of the time you already know what you want and already know what you are up to. I am a bit joking, but it's actually true that all the best plots in my life, I had them in a dream before actually they were real. I kind of knew what I was looking for. Just the question is when it actually clicks, if you know what I mean. Because most of the time you kind of know what you are looking for but you are not finding it. You try one thing and it doesn't work. Try second thing, oh, it doesn't work. But eventually all the right pieces fall into it. And most of the deep learning systems are very intricate. So usually you have to get five things right in a row for the thing to start working and then eventually you get the plot that looks like you want and then you know. So I think what we are looking for is systems that learn and test time and if we see meaningful long term adaptability of our systems and we're joking, but it's a real. We want to be evaluating our systems of our everyday work. They get better at doing the work of core automation scientists each day.

[D]

Yeah, we go on a vacation as a team and see if the lab produces something that better for the week. Give the then what do you do

[C]

when you get back?

[D]

We'll see you for extend the vacation

[A]

two times and four times until we are on permanent vacation.

[B]

That is a beautiful note to end on. Rohan. Jerry, thank you so much for joining us. You've both worked on really, really transformative work for where we are today and I'm so excited to see you starting a lab on this edge, this new journey and very excited to see what you're able to come up with. Thanks for joining.

[A]

It's been great to be here and chat with you. Thank you.

[D]

Sam.