← The MAD Podcast with Matt Turck
Loading summary
Transcript92 lines
- [00:00]
Brian Catantaro
If you accept as the truth that we're going to be running at the limit, then what that means is that the way to get more intelligence is to be more efficient. We can't get more intelligence by applying more force if we're already at the limit. We have to be more thoughtful about how we use what we have. We build tools, we build external organs that help us solve problems. You know, we, we have an external stomach, we call it a kitchen. Now we're creating an external brain. What is the implications of an external brain? Pretty profound. Nobody actually really knows.
- [00:32]
Matt Turk
Hi, I'm Matt Turk. Welcome back to the Matt Podcast. Open source AI is having yet another moment with powerful new models arriving almost weekly. And my guest today is one of the very best people to unpack it all. Brian Catantaro leads Nemotron, Nvidia's family of open foundation models. Now, not everyone realizes Nvidia has a massive effort to build frontier AI models, but it employs hundreds of AI researchers. And Nimotron 3 Ultra immediately became one US open weights model when it was released just a couple of weeks ago. We begin this conversation with the state of open source AI and the race between the US and China. And then we go deep inside Nemotron 4bit training hybrid member transform architecture, mixture of experts, multi token prediction and multi teacher distillation, all in plain language. And finally we get a rare look at how a modern AI research organization actually runs. How you get many brilliant minds to build one model instead of 100 papers. Please enjoy this awesome conversation with Brian Catantaro. All right, Brian, excited to do this. It seems that open source is having a banner year. So you guys at Nvidia just released Nemotron 3 Ultra, which is an important moment and the best open source open weight model in the us. That was just a few days ago. And then Even more recently, GLM 5.2 came out and that was another moment. So it seems that things are accelerating in open source AI. It feels like a great place to start. What's your assessment about where we are and how wide the gap between closed source and open source currently is?
- [02:10]
Brian Catantaro
Well, it's really exciting to see all of the energy going into open technologies for AI, because we know that open technologies make it possible for people to innovate. You know, the Internet is such a great example of that. We actually did have closed Internets. I don't know if you remember things like America Online and Prodigy back in the day and they were great. And open Internet has also been amazing. Right? Like so many different companies have been able to figure out how to transform their work thanks to an open technology. The application of the Internet to retail is very different from the application of the Internet to healthcare or manufacturing, but all of them have been totally transformed by the Internet. AI, I believe, is also a very transformational technology and also a technology that needs to be applied in very diverse ways. And because of that, I believe that open technologies for AI are really fundamental. And it's very exciting to see continued investment and development of open technologies for AI from so many different organizations around the world. And I hope that that continues.
- [03:24]
Matt Turk
And what's your sense for how far behind open source is compared to closed sources has been? The big trend of the last few years has been this sort of narrowing gap. Do you think that open source is almost there or the bar keeps getting raised by the closed source models?
- [03:44]
Brian Catantaro
Well, I feel like this question,
- [03:47]
Matt Turk
it's
- [03:47]
Brian Catantaro
maybe a tempting question because it's fun to set up kind of competition, but I actually feel like the whole AI community is moving very fast. And, and if you look, for example at the progress in AI, whether it's closed or open, just over the past three months, it's been incredible. And so if you're in a field that's moving really, really fast, I think that's more important than any particular gaps that might exist between different models. Because the most important thing is how is AI developing as a field?
- [04:18]
Matt Turk
What do you think the drivers are to continued progress in open source AI? Is that the communities and big companies like Nvidia being behind it? Is that the global competition with China, what propels open source AI forward?
- [04:34]
Brian Catantaro
You know, I think there's a number of things that, that are pushing open technologies for AI forward. One is just the demand. You know, there's so many organizations that want to customize AI and want to integrate it deeply into their work in a way that really requires open technologies for AI. And so, so I think the demand is certainly there. I think also it's just the best way to develop technology, and we've seen this, you know, for, for many decades, that technologies developed in the open move quicker because we can all learn from each other. And in an era where we're undergoing the most exciting thing to happen in technology in our lifetimes, with the development and the deployment of AI, what else do computer scientists want to work on other than making AI awesome? And if working together as a community is the best way to do that, then that's also a driver that pushes the community towards openly developing technology.
- [05:30]
Matt Turk
To ask maybe a slightly Cynical question. There is at least a part of the community that's wondering whether open source as an ecosystem, not Nvidia, but in general has been progressing in part based on the ability to distill closed source models. And in a world where we seeing the anthropics and fable fives of the world starting to discourage distillation, do you think there is a chance that open source AI progress may slow down in that context or as a result?
- [06:10]
Brian Catantaro
You know, in my mind there's no question that when the technology community decides to make huge investments in the most transformational technology of our time, that there's going to be rapid progress and also that that technology is not going to be controlled by a small group of people because that's just not the way that the industry works. You know, we, we do our best work, we have the most impact with our work when we're able to each think about it in our own way and apply it in our own way. So, you know, I love the closed AI APIs, whether from anthropic or other people. I think they're amazing, you know, really, really impressed with the work that those labs are doing. But they're not the only labs in the world. There's lots of labs around the world and lots of people have a good idea. It's not the case that there's only a few labs that have the monopoly on all good ideas. That's just not true. That's not how humanity operates. There's a, there's a lot of bright people on this planet. And you know, the community of course cares deeply about this technology. It's obviously so transformational, has such profound impacts on so many things that, that of course many people want to be involved in that. And so I think over time we're going to see that community oriented approaches to developing and deploying AI are going to continue to strengthen and be widely adopted because that's really the history of how we built things as a, as a human species.
- [07:43]
Matt Turk
Do you think that is globally true as well? So, you know, in particular with respect to China, this perception that yes, a lot of people have great ideas around the world, however, a lot of progress from Chinese models were directly inspired or perhaps generated through distillation from the closed source models. Is that just kind of like press rage, bait or from the perspective of a leading AI researcher, you're very impressed by the novel ideas that come out of China as well.
- [08:22]
Brian Catantaro
You know, perhaps unusually, I actually did work at a Chinese company for about two and a half years. I worked At Baidu, I worked in the Silicon Valley AI lab along with Andrew Ng and as well as Dario Amadei. And we all worked for a Chinese company and saw how smart, hardworking, creative, inventive our colleagues were at the rest of Baidu. And, you know, that experience has, has stuck with me. I think it's absolutely false to say that, you know, the achievements of some other country are all being created by sort of, you know, copycat mentality. It's just not, it's just not true. Now, do we all learn from each other in the technology community? Of course, you know, of course, of course we learn from each other. But, you know, I would say, you know, it's been a really good thing for the world that the Chinese AI community has been so open with what they've been building. I think it's enabled a tremendous number of companies to build things that they couldn't have done without that community. And I think it's also spurred technological progress throughout the AI ecosystem. So, you know, I'm really grateful for the contributions that our colleagues in China have made over the years. And you know, I, I would love to encourage a spirit of openness amongst AI labs around the world outside of China as well. You know, I was really excited when OpenAI released the GPT OSS models a while back. And then of course, Google's been doing great work with Gemma. Absolutely thrilling to see that. And you know, we're pushing Nemotron along here at Nvidia as well. So I think there's, there's a chance for the rest of the world to catch up to China in the sense that we can understand the benefits of working together as a community to build technologies for AI in a way that I think China has frankly been leading.
- [10:31]
Matt Turk
Great. What is the case for a customer to be using open source models these days? What is your fundamental advantage?
- [10:43]
Brian Catantaro
Every company is built around a secret. This is a secret that has to do with not just their intellectual property, but also their platform, which has to do with how do they interact with problems and customers, how do they think about solutions to what their customers need? And it is always the case that the value of AI is greater when it can be more tightly connected with those secrets. Because AI depends on data critically. So the more valuable the data that goes in, the more valuable the solution becomes. Now, every company, when it's thinking about how to deploy AI, has to think through what are the implications for the core secrets of our company. And there's a lot of circumstances where due to trade secrets or Trying to think through the business model or even regulatory requirements that there's data that you really have to treat very carefully by law. And it is much better to do that when you are able to think that through and implement it yourself. Thinking about the integration of AI, the way that AI interacts with customers, the guardrails that are put in place, you know, every company has a specific understanding of its customers and therefore what. What the customer needs. And the amazing thing about open technologies for AI is that they allow customers customization. Right. So companies can think this through. They can build things that. That really matter for them. And, you know, I started out this conversation talking about the Internet and about how the Internet, the deployment of the Internet has been done in very different ways for very different industries. And there's a lot of desire to do that as we see AI change the way that we work and play throughout the entire economy. This is really spurring a lot of demand for open technologies for AI.
- [12:39]
Matt Turk
Great. I'd love to go into a bit of a deep dive into Nemetron, but before we do that, maybe a few minutes on your story, your background, what was your past to where you are today, including the Baidu detour.
- [12:58]
Brian Catantaro
So I started work at Nvidia in 2008. At the time, I was a graduate student trying to figure out parallel computing for artificial intelligence. And I thought Nvidia had a chance of changing the way computers work AI,
- [13:13]
Matt Turk
which was a lonely. Presumably a lonely quest. Right. In 2008.
- [13:17]
Brian Catantaro
Oh, it was. It was very chaotic back then. People thought I was crazy. And, you know, I remember going to ICML in 2008. I published my first paper training models on the GPU. And people asked me why I was there. People said, this is not a good paper for icml. We just do fancy math here. And I was like, well, but I think computing actually matters. Matters a lot for AI. If we could train bigger models that had more capacity to learn, we could probably solve more problems. And they kind of nodded their heads in the doors like, well, we're not really sure why you're here.
- [13:48]
Matt Turk
Isn't a GPU a thing for gaming as well, presumably?
- [13:52]
Brian Catantaro
Right, yeah, there's also that. Right. Which we continue to run into that idea. Actually, a GPU is whatever Nvidia says it is. We make them. So a GPU is a thing that we make in order to accelerate the world's most important computations, which in 1995 was graphics. And for a long time now, it's been AI. So, anyway, I started at Nvidia I was in the research group doing strange things about trying to make compilers, libraries for AI on the gpu. That led to the creation of first Copperhead, which was a Python embedded language that compiled to the GPU, which I think foreshadowed a lot of things in TensorFlow and PyTorch. And then that led to the creation of cudnn, which was Nvidia's first product for deep learning on the gpu. And I really enjoyed working on that. But I was always wanting to see more firsthand about the applications of AI. And at Nvidia, I was mostly working on libraries and compilers for AI. So I thought, well, you know, when Andrew Ng asked me to go build the Silicon Valley AI lab with him at Baidu, I thought, oh, this is a great opportunity, because even back then, Baidu was very advanced in its application of AI to its core business. And so that was a fantastic opportunity for me. The Baidu Silicon Valley AI Lab was an amazing place full of brilliant people that were working really hard.
- [15:33]
Matt Turk
What was it like working with a young Dario? Was there any signs that he could become who he has become?
- [15:42]
Brian Catantaro
Dario was brilliant from the beginning. I remember I interviewed him, I was on the panel, and at the time, he had been working in bioinformatics, so he hadn't been working on deep learning or the things that we call AI these days. Um, but it was very clear that he learned extremely quickly and also that he thought extremely deeply. Um, I think, you know, the thing I admire most about Dario is the strength of his conviction. Um, you know, I've been working in this field for a long time, and I've believed also that AI is going to transform the world. But I don't think that I believed in it as completely as Dario did. And perhaps that was because, you know, my academic training during my PhD was full of a lot of caution. I don't know if you remember, but AI was old and bad in 2005. It will never work that people did with computers. They started doing it in 1945, right? And so there had been so many grandiose promises that failed to deliver over the years. And so I came to AI with a lot of caution. In fact, back then, we used to call it machine learning, which was basically a dodge. Like, we just didn't want people to know that we were working on AI, because then they would be like, oh, we've heard about that ever works, right? So I came to AI with a little bit of this, like, you know, academic caution, like, oh, we should you know, we should hedge a little bit, like, I don't know if now's the time, like. And Dario, you know, his strength of conviction and his understanding of the moment of how the technology was developing, this time it was actually going to work, and then the implications of that on, you know, how the technology should be developed, what kind of institutions to build. I think he's done a spectacular job. And so, yeah, working with him, it was always, always a fun experience.
- [17:42]
Matt Turk
So then you went back to Nvidia and walk us through the journey.
- [17:46]
Brian Catantaro
Yeah. So 10 years ago, actually, in 2016, Jensen called me up and said, hey, would you like to come back and build an applied research lab? And I thought that would be a fantastic opportunity. You know, I've always loved Nvidia. I've loved the way the company works, the convictions the company holds. You know, Nvidia is a very unique company. It follows through over long time periods, you know, and I've seen that with cuda. I've seen it with our deep learning technologies, I've seen it with our ray tracing graphics technologies, our AI for graphics, you know, over and over again. Nvidia is not afraid to put in five or 10 years worth of research in order to change the world, you know, and working at a company that has that strength of conviction and the ability to follow through is kind of an ideal thing for me. I just really, I just really love the support that the company gives, gives its researchers to invent the future. And so I thought I'd come back. The first project that, that I worked on actually became dlss, which some of your audience may know about. But DLSS is our real time AI for graphics and it makes a small GPU run like a big GPU. It's about 10 times more efficient because rather than computing the color of every pixel for every frame, we use AI to infer the color. And, you know, these days, 23 out of every 24 pixels is being generated by our AI model. When you're using DLSS to play games. And gamers love it. It's become the standard way of playing games because it's just so much more responsive and it's more beautiful. Our AI, we train it offline on huge data sets and it's able to render graphics in real time more beautifully than traditional methods do. We recently actually announced DLSS 5, which is a fully generative version of DLSS. And I am so excited about it. It represents culmination of 10 years worth of research on how to make real time graphics much more beautiful. And so that's part of the journey here for me was real time AI for graphics. But then at the same time we also started a language modeling project. And this was back in 2017, before Transformers were big and before language modeling started taking over the world. But I just had this intuition maybe built on, you know, some of the things that I had seen while working at Baidu. I just had this intuition that, you know, working with text and understanding text was going to lead to better reasoning, which was going to lead to better application of AI in all sorts of domains. And so we started this project called Megatron. Megatron stands for the biggest, baddest Transformer. That's why we named it that. And it was really a systems project to show the world how to train the largest Transformer models on Nvidia's hardware. Back at the time, some of your audience may or may not remember this, but there were being claims made that the only way to train big Transformer models was on the tpu because after all, the Transformer had been invented at Google. And so, you know, we looked, we looked at, you know, we loved the Transformer paper. We thought, wow, this has amazing potential. We tried it out on our own language modeling tasks and it worked so much better than the RNNs that we had been using before. And also we saw immediately that there was an enormous systems opportunity to co optimize the gpu, the networking, all of the compilers and software that would enable people to scale Transformer based language models really dramatically. And we thought this is something that could really have an impact. So we started the Megatron project, which then led to, I think, basically helping the whole industry figure out how to train extremely large LLMs. And also led to the foundations of today's Nemotron project where Nvidia trains its own LLMs for its own purposes. So that's kind of the history.
- [21:55]
Matt Turk
Great journey. Okay, so let's go into all things numotron. And before we get into the specifics, that's the obvious question that I'm sure you've been asked many times, which is why does Nvidia care in the first place to be building model and investing very significant efforts into creating its own family of frontier models?
- [22:19]
Brian Catantaro
You know, nemotron has two jobs. The first job is to help us understand how to build the systems of the future. Nvidia is an accelerated computing company and that means thinking through the world's most important computational challenges from first principles and designing systems, which includes a lot of software in order to make it Possible for people to invent and deploy things that never could have been done with standard computing. But in order to do that, Nvidia has to deeply understand everything about how AI works. That's how we co design all of the systems and software for our main product line. So the first job of Nemotron is to make sure that Nvidia continues to exist so that we can continue delivering meaningful acceleration in an era where Moore's Law has died. And the acceleration that we get these days comes through specialization. But again, specialization comes through understanding. So that's Nematron's first job is to help Nvidia understand how to build its core products. Nvidia's second, or Nematron second job is to support the ecosystem. One of the most valuable things that Nvidia has built over the years is all of the people around the world who build and deploy amazing AI using Nvidia's technologies. And we think that it's necessary for open technology for AI to continue to exist from Nvidia to help support that. Nemotron's not trying to be the only open technology for AI. We love all technology for AI for the very straightforward reason that whenever AI is further developed and further deployed, it's an opportunity for our business. So this is, you know, we're very explicitly trying to develop our ecosystem because that's good business for us. But we're not trying to be the only provider of technologies for this ecosystem. We love seeing other companies contribute as well. The most important thing for Nematron's second job is just making sure that it continues to be possible for companies of all shapes and sizes to build and deploy their own AI.
- [24:28]
Matt Turk
By the way, Moore's Law is dead. Is that official?
- [24:32]
Brian Catantaro
It's been dead for years.
- [24:33]
Matt Turk
It's been dead for years. Why is that?
- [24:35]
Brian Catantaro
Well, you just look at the progress in semiconductor manufacturing. You know, the original statement of Moore's Law was economic, right? It was about we can afford to put twice as many transistors on the same chip in every, whatever, 24 months, whatever the time period is. And these days that is absolutely not the case. And it hasn't been for probably five or ten years. Right now we are still scaling our systems right through a number of ways. One is just applying a lot more silicon to it, right? We are also getting. Transistors are continuing to get smaller and more efficient, although at a slower pace, but they're also getting quite a bit more expensive at the same time. So the, you know, in an era where, where Moore's Law was alive. The best way to make the system of the future was to take the system of the present and then just shrink it and, and maybe double it at the same time. Right, but in an era where we've been living for a while now, where you don't get economic benefits from taking your existing design and shrinking it, you really have to be more clever about how you use every part of the system. That that's an era where accelerated computing is much more valuable than ever. Because the work of thinking through the problem from first principles and co designing absolutely everything from transistors to algorithms and applications in order to reduce waste and deliver meaningful acceleration, that's more valuable than ever.
- [26:08]
Matt Turk
Fantastic. To play back what you were saying a minute earlier, it makes good business sense for Nvidia to be in the model business because one, it helps design better chips and two, whatever is good for AI is ultimately good for Nvidia, which makes a lot of sense. That Nemotron effort is reasonably recent. Right. It started in 2023, I believe. Maybe walk us quickly through the key releases. I believe in 2023 there was Nematron 3.8B as a key release. Or am I missing a step?
- [26:41]
Brian Catantaro
Yes, yes, yes. So you know, you know, the, the numbering is somewhat lost to time. It. I almost feel like we're in the Lord of the Rings and it's like, you know, there's like some ancient like relics that we're digging up out of an old mine. You know, this is a long time ago. You know, the original what, what we originally called Nemotron1 was actually a project that we did with Microsoft. We jointly trained a 530 billion parameter model, I believe that was released in 2021. And so this is GPT3 era and that's what at the time we called it Megatron, Turing nlg. Turing was what Microsoft was calling their language model efforts at the time, but that in retrospect we called Nemotron 1. Then along the way we built a few more. We got up to Nemotron 3 and then llama came along and we were really excited about that. We were very happy that META was supporting the open AI technology space. And so we started taking our language model technology and adding it to LLAMA models, which then resulted in llama nemotron1 and that was the first reasoning model built on llama. We were really proud of that. That was 2025, might have been 24 I believe. I can't remember. Somewhere around there. And then, yes, and then we, we continued to, to develop that and you know, last we, so we, the numbers kind of started over again. We released a Nemotron 2, I believe it was last year. And then we quickly followed that up with Nematron 3 because we, we needed to put MOE support in. Nematron 2 didn't have Moe support and that made it kind of uncompetitive against other models like GPT. OSS20B was just like so fast because of MOE. And so we were like, okay, we've got to, we've got to put, put the MOE in. So that became Nematron 3. Now we're in a slightly difficult state because, you know, we're working on Nematron 4, right. But we already released a Nematron 4, which was in 2024. We released a 340B model called Nematron 4. And so I'm not exactly sure how we're going to solve this marketing problem. I didn't create this marketing problem. So I'll, I'll do my best to, to make it clear that Nematron 4 of, of whatever the next, whenever we release that is different from the, the 2024 NE. But in any case, we've been working on this for a long time. I think more important to us than any particular generation is just the sustained commitment that Nvidia has to developing these models. We've been doing it for a while. I think our models have gotten dramatically more useful in the past year, which is a reflection of two things primarily. One is that the whole company has come together. So there are many different teams around Nvidia that now understand how important this is to Nvidia's future. And so there's dramatically more people and better ideas that are going into Nematron. And then number two, along with that, we've been able to scale the compute resources that go into it. Obviously it's very important to have good computing infrastructure to build AI. We've recently increased our investment substantially because we believe that this is really, really key to our company company's future.
- [30:22]
Matt Turk
Fascinating.
- [30:23]
Brian Catantaro
But just to continue the thought, I think it's really important that everybody knows that we've been doing this for a long time. We are increasing our investments substantially and Nvidia is a company that follows through. You know, we followed through over 10 plus years with CUDA and we're doing that with Nemotron now.
- [30:37]
Matt Turk
That's very helpful because I think the broader world is just starting to catch up to the fact that there is a very substantial open source frontier AI research effort that's been happening. So it's really interesting to hear that there's been this progression and now this family of models that we're going to talk about in a second. Another important moment seems to be the creation just in March, three months ago, of the Nemotron coalition. Do you want to explain briefly what that is?
- [31:11]
Brian Catantaro
So nemotron exists to help support the ecosystem. And we were thinking, well, this is a different kind of AI project than other projects around the industry, right? Because we're not actually trying to dominate in any way, we're just trying to support. We don't, we're not trying to control the way that AI is being integrated into all these companies. We're just trying to make sure there's good AI. But we thought, well, maybe if we worked with people while we develop it, then it's going to be more useful for them, it'll be easier to integrate and because we will consider what they need from the beginning. And you know, Nematron has always been collaborative. I was telling you that, you know, long, long time ago, our first big model that we trained, we, we did with Microsoft, right? It was a joint effort where Nvidia and Microsoft researchers worked side by side to build that. And that, that ended up, I think, helping both Nvidia and Microsoft. I think we both learned a lot from that experience. And so because Nemotron is not trying to compete with other companies, but rather support, because we're going to be putting it out there openly anyway, why not collaborate before the thing is built? Rather than Nemotron being a project that Nvidia does all on its own and then posts on the Internet and says, hey, why don't you try this? We think it might be good. Why don't we make sure that it's good for the partners that are interested in by working with them before Nematron is even created and incorporating any sort of feedback, evaluations, environments, benchmarks or any other kinds of technology that other people want to bring. It turns out that the entire ecosystem, there's a lot of companies that really want open models to succeed. And so they have a self interest, they have their own vested self interest and making sure that open technologies are excellent. And so why not work with them and let them contribute however they'd like to making Nemotron better. So that's the idea of the Nematron coalition. It is not an exclusive coalition. We're not trying to be the only model out there. All the companies that we work with are free to continue doing the work. However makes sense to them. And yet these companies want to work with us because they want to make sure that open technologies for AI keep developing quickly and that they have a chance to influence how that happens.
- [33:38]
Matt Turk
Great. What's the current state of the new Motron family? You got Nano, you got super, you got Ultra. What do those models do and what are the use cases for them?
- [33:50]
Brian Catantaro
So Nano is a 30 billion total, 3 billion active parameter model supers 120 and 12 and Ultra is 550 and 55. They're designed really to fit it's kind of small, medium and large deployment scenarios. Nano can be really capable for things that don't require nearly as much knowledge or reasoning. But obviously for the most capable model you go for Ultra. Super in a lot of ways is our most popular model because it represents kind of a great balance between cost and intelligence. So we kind of like having this small, medium and large approach to building a family just because our customers seem to respond to that pretty well. But you know, the most important thing from Nvidia's point of view that people are doing with LLMs is agents, right? Is building agentic workflows. Having an agent working on your behalf solving problems for you night and day is such an exciting way of approaching the problems that we have to solve. And it's our dream to make Nemotron amazing for that purpose.
- [35:08]
Matt Turk
That's our goal, to double click on this. At a high level, Nemotron family is focused on agentic reasoning with a particular focus on making it efficient. Is that the right headline?
- [35:20]
Brian Catantaro
That's right, yeah. Nemotron has always been speed first approach to building models because Nvidia is an accelerated computing company. As I was saying, we're trying to think through what is the problem here computationally from first principles. And you know, Nemotron 3 family has a lot of things in it that are we're really proud of. For example, Nemotron, Ultra and Super were Pre trained using four bit arithmetic. We pre trained those in MVFP4 which is a not trivial thing to do to invent the algorithm so that your model can converge to an excellent result using such coarse arithmetic required a lot of invention. Really proud of that.
- [36:03]
Matt Turk
Do you want to explain maybe for people what 4 bit is versus 16 bit for example?
- [36:08]
Brian Catantaro
You know, actually there was a fantastic post I saw on Hacker News yesterday where somebody let you upload a picture and then it would basically posterize it. Basically reduce the colors to fit different number formats, including NVFP4 and MXFP8 and some of the other formats that are out there. And so you could kind of swipe around and look what it does to the colors of the picture. And you know, it's really quite dramatic. Four bits is not a lot of bits, right? That's only 16 values. Now of course, these are all what are called block scaled formats. So groups of numbers also come with an 8 bit scaling factor. And the specifics of this can get rather complicated. So maybe they're not quite as important. But the reason why we want to do this is because first of all we have dramatically higher throughput for these formats in our GPUs, specifically on Blackwell Ultra. And secondly, we know that it's going to save an enormous amount of energy. One way to think about the computational problem of AI is that we are going to be running at the limit, whatever the limit is. It could be an economic limit, like we only have so many billion dollars to buy servers with. It could be a power limit. We only have so many gigawatts that we can afford to train a model with whatever the limit is. We're going to be running at that limit. Every organization is, because why? Because the value of intelligence is so high, you know that, that people are going to, they're going to invest because they know that they're going to get return. The value of intelligence is enormous. So if you, if you accept as the truth that we're going to be running at the limit, then what that means is that the way to get more intelligence is to be more efficient. We can't get more intelligence by applying more force if we're already at the limit. We have to be more thoughtful about how we use what we have. And you know, four bit number formats are dramatically cheaper to move around. They take up less space in memory, they take up less picojoules when you move them from the memory or even on the chip, around the chip, much less energy when you compute on them. So that's really driving the investment in four bit formats. And I think these days four bit formats for deployment are very well established. It's pretty straightforward these days to make a good quantized four bit checkpoint that you can deploy and that gets you a lot of inference cost and speed advantages. But using 4 bit formats for pre training, that's quite a bit more challenging because you have this numerical solver that's you know, optimizing the weights and you know, it can be quite sensitive. So if you, if you don't treat the numbers right, your model can diverge. And instead of actually getting a model done through pre Training, you end up with, you know, basically just that run diverged, which is, you know, always, always scary. So it took a lot of invention for us to be able to pre train Nematron in four bit. We're really proud of that.
- [39:26]
Matt Turk
Okay, great. All right, so as we get into slightly more technical things, the architecture of Nemotron is hybrid. Is that right? So it's a combination of transformer and Mamba state space, which is a slightly more exotic form of architecture. Walk us through that. Yeah.
- [39:49]
Brian Catantaro
You know, we published a paper in 2024 that showed that you actually get a smarter model by combining state space models with transformers. And we actually did a sweep of how much of the model should be full attention and how much of it should be a state space model in order to get the lowest perplexity, basically the best language model that you could get. And we found that you actually want it to be mostly a state space model with a little bit of attention. And kind of the intuition behind that is that the state space models seem to be better at kind of this intuitive, kind of impressionistic understanding of a sequence because they're kind of summarizing the entire sequence into a constant space. That's how they work. So instead of having the ability to look at the entire sequence randomly, they summarize everything at every step into a constant cache or little scratch pad that they're, they're working on. And that constraint seems to actually make them smarter at some tasks that involve like global understanding. On the other hand, the advantage of full attention is that it can pick out very specific bits of information and look at those. Exactly. It doesn't lose anything. There's no lossy compression going on. You can actually see the whole thing. And so we found that, you know, using both of these together was actually better than using either one on their own. And that is independent of the speed benefit. That is just the model is smarter. And you know, since we published that, I think a lot of other labs have also found this to be true. You know, a lot of models these days are being built with hybrid SSM approaches. For example, QN has done that. KIMI is using what they call Kimilinear attention these days. So it's become, I think, quite widely adopted to use some sort of state space model in conjunction with full attention for, for the, the base architecture. Now it also has some speed benefits because the amount of memory that you need to hold that state space cache is actually constant with respect to your sequence length, which then means that generally you can fit much higher batches on the GPU when you're training and doing inference because the memory requirement is lower and it keeps the GPU fuller and busier and therefore provides some pretty important efficiency benefits as well.
- [42:32]
Matt Turk
So the models are also based on an MOE mixture of expert architecture. Walk us through that and maybe remind people what MOE is in the first place.
- [42:43]
Brian Catantaro
So mixture of experts is a form of sparsity. The idea is, wow, you want to train a model on the entire Internet. You want it to remember absolutely everything about the history of everything. But when you're answering a particular question, does it seem reasonable that it needs to actually think about the entire universe in order to answer that question? Actually, no. It seems like it's quite sparse. Right. It seems like we're using a language model to explore a very tiny space of ideas in order to answer a question or solve a problem. We want the model to be able to draw from the entire universe. We want to train it so that it understands everything that it possibly can, but when it's actually running, it doesn't really need to see all of that information. There's been a variety of approaches to sparsity that try to take advantage of this property, but mixture of experts has been the most successful. And the way that it works is that the neural network has what's called a router that is learned that is going to decide to send activations to a subset of the experts. For every token that's flowing through every layer of the model, it's going to be making choices about which fraction of the model is going to actually get to interact with this token as we try to understand it, build up representations of the problem, and then generate the next token that we're going to output.
- [43:59]
Matt Turk
So it's a little bit like if I have a company with 550 employees, but 55 of them are in engineering, I want the 55 employees who are specialists to come to my meeting about engineering and not the rest of the company.
- [44:16]
Brian Catantaro
That's right, yeah. Or you can think about it as a library. Like if you go into a library to do research, you don't read all of the books in the library. Like, your first job is to figure out which books do you need to look at in order to find the answer to your question. And so, so that's kind of the the idea behind moes. Now, moes have fascinating implications for the systems that we build. So with Blackwell, for example, Nvidia went all in on MOEs. That's why we built NVL72, which allows up to 72 of our GPUs to read and write each other's memory at very high speeds, very low latency. Now why is that important? It's because as you put a token through the stack of layers, at every layer you have a router that's routing that token somewhere else. Why don't you partition your experts so that the experts are not sitting every expert on every gpu, but you have a subset of the experts assigned to each GPU and then you're routing the tokens between the GPUs very dynamically as you push the token through the network. Now this is impossible to predict in advance where the tokens need to go, because it's very specific to that particular token for that particular model. And so that's why we built NVL 72. And that's why Blackwell is so amazing for inference for today's AI models, is because we thought deeply about mixture of experts when we were building it. This is speaking to Nematron's first job. If we hadn't been working on understanding AI, we wouldn't have been able to build Blackwell properly. And that has translated directly into increased deployment of Blackwell, which we're very excited about.
- [45:55]
Matt Turk
Is what you just described called latent moe, or is that a different concept?
- [46:00]
Brian Catantaro
Latent MOE is a specific innovation that we have in Nemotron 3 family and what it does is actually reduces the amount of communication that has to be sent through NVLink during MOE computations by basically down projecting it. So, you know, every token produces a vector and the idea is like we're going to take that vector and learn a way to compress it and then send that compressed thing through the network and then we're going to uncompress it at the other end. And as a result we save on network bandwidth and we also get four times the number of experts for the same inference cost. So you could think about it as like, you know, our library of books got four times bigger and we get to read four times more books at the same inference cost because of this particular innovation.
- [46:48]
Matt Turk
Is MOE in general becoming the default architecture for Frontier AI?
- [46:54]
Brian Catantaro
Yeah, I believe MOEs have been the default in Frontier AI for a long time. They're just a really good combination of inference cost and intelligence.
- [47:04]
Matt Turk
Great, great.
- [47:05]
Brian Catantaro
But they have drawbacks as well. They take a lot more memory. If you have a very small amount of memory, a dense model is going to be smarter. And they also, they tend to work best either if you're running at batch size one. So you're running basically a single job or you're running a huge data center with like infinite queries coming in in the middle. They can be a little bit tricky.
- [47:26]
Matt Turk
Another important characteristic of Numertron 3 Ultra is a 1 million token context, the long context window. How important is that in the overall mix and what does it enable the model to do?
- [47:41]
Brian Catantaro
The longer the context length, the more challenging problems we can solve. With a language model that allows us to do things like append all sorts of information to a query, which could be a code base, it could be instructions. In the long term, I'm hoping that I have my own personal LLM that's able to read all of my emails and help me answer questions about that. The more information that we can attach to a particular query, the more useful the model can be. Now it can get more and more expensive to reason over large amounts of input data. And so that's one of the reasons why there's usually a limit on how big the context length can be. But with Nemotron 3 we tried to push it as far as we could go. We think a million tokens is a lot of tokens and you can do a lot of things with that.
- [48:36]
Matt Turk
Presumably is particularly helpful in sort of multi step agentic workflows. And there's this whole separate discussion around context compaction to make sure that the model doesn't get lost in too many tokens. So how do you all think about this?
- [48:52]
Brian Catantaro
100%. I mean compaction, that's a thing. If you're using an agentic workflow you deal with all the time. And compaction tends to work pretty well because language models are pretty good at identifying the most relevant things and summarizing. And you're basically trying to summarize your context when you compact it. So compaction is not a bad approach. I think having models that can just natively reason about larger amounts of data is just inherently more useful. So of course we want to push the boundary on that as well.
- [49:26]
Matt Turk
Great. Can you talk about the multi token prediction, which is also very interesting if
- [49:32]
Brian Catantaro
you're running at a low batch size, which is when you are trying to get the most interactivity if you're in a data center. So you want the model to respond as quickly as possible and it's okay for it to be more expensive, your cost per token might be higher, but you want the result as quickly as possible. Or if you're running locally, so you might be running at batch size one just because you're the only person using it, it Turns out that the GPU has extra execution capabilities that are just lying there unused. The bulk of the work when you're running in these scenarios is actually fetching the weights from memory and then you push the token past those weights and then you fetch more weights from memory. But it turns out if you push two tokens or even five tokens through those same weights, it would cost basically the same amount of time. Because the expensive thing is not doing the math to push the token through the weights. The expensive thing is just reading all of those weights from memory, all those parameters they have to come in. And so the idea with multi token prediction is to take advantage of this by having the model predict multiple tokens at once. Let's say that the model predicts five tokens. We know the first token is correct. The next four tokens may or may not be correct. So then what we do is on the next pass, we take those four tokens and we stick them into the model and then run it through. And at the end we check, you know, the model then predicts another set of tokens, right? Then we check, were the extra tokens we predict last time correct? If so, then we just accept them and then we get like a forex speed up. And if they were incorrect, then we only accept the ones that were correct
- [51:16]
Matt Turk
and then
- [51:18]
Brian Catantaro
proceed from there. So the benefit of this is it doesn't degrade accuracy at all because you're using the model to double check, right? So all this speculation is going to get checked during the next token that you run through the model. So it doesn't degrade your accuracy at all to turn on multi token prediction, but it can give you a speed up and it's probabilistic depending on the acceptance rate of your predictor, you know, so if your predictor is more accurate, the acceptance rate goes higher, you get a higher speed up. So with, you know, with our recent Nemotron models, you know, we're pretty proud of our acceptance rates, but we're always trying to make them better, you know, always trying to improve that acceptance rate. This is a really good example of accelerated computing. You know, with multi token prediction, the speed that you get is a function of the accuracy of your model. The more accurate your model is, the faster the inference is, the cheaper the inference is, the more accurate it is. That's not usually how it works, but in this case that's how it works. And what that implies is that, you know, if we're trying as Nvidia, as a company, to provide meaningful acceleration to the world's most important computational workloads. This has to be an important part of how we think about it. You know, if there's a 3x cost reduction or speed improvement for inference, which is the most important computational workload of 2026, if that's on the table and it depends on the accuracy of the multi token prediction network, then that's something that Nvidia needs to understand very deeply because it's going to affect our business directly.
- [52:47]
Matt Turk
Fascinating. To continue on the tour, multi teacher distillation. We talked about distillation a little bit upfront. What does that mean in the context of Nemotron 3?
- [52:59]
Brian Catantaro
So with Nemotron 3 Ultra, we did post training using something called multi domain on policy distillation. And what that entails is that we have many different aspects of the model we want to improve. For example, science understanding is different from math theorem proving, which is different from coding, which is different from agent harness interactions. Right. With Nematron 3, I think we had about 10 or 15 of these teachers. And so the idea is that you take these teacher models and you push them as far as you can go on some specific domain. So you just don't worry about making good at everything, just make it really, really smart at this one domain. Then you have a collection of these models and you want to create one model that learns to be good at everything. And we do that using a specific reinforcement learning technique that a lot of labs these days use called mopd. And the good thing about this is that because the teachers are supervising, they can give really dense rewards to the student model. Basically every token is getting supervised and so the student can learn really quickly and then become almost as good as all of the teachers at all of the things. So one benefit of this is that it really helps the team work together better. You know, if, if you don't have a technique like this and you have, let's say 500 people working to try to make a model better, and one team's like, well, I'm trying to make it better at this thing. And then another team's like, I'm trying to make it better at that thing, there can be a tug of war where it's like, well, who wins? You know, and, and if you have to make a choice like, oh, I'm gonna make them, I'm gonna choose to prioritize this one over that one, then you make the other team feel like their work doesn't matter. You know, it's just really hard. This is one of the challenges of building AI in 2026 is that you have to figure out how to get the people to work together, even though you're only building one thing at the end of the day. And so this particular technology has been really instrumental in helping more people work together to make Nemotron stronger.
- [55:05]
Matt Turk
Fascinating. So it's just as much a technology question as a human organization question.
- [55:10]
Brian Catantaro
Exactly.
- [55:11]
Matt Turk
Okay, fantastic. Let's put a pin in this and get back to this in a second because it's a fascinating topic. In terms of the post training that you just alluded to, one of the exciting things that you all did in the context of Nematron is also to publish the data, the training data. Does that include per industry data for specific reinforcement learning tasks? Yes, that's the beauty of a conversation like this today, where you guys can actually talk about those things. So where does one get the data from? Or post training, reinforcement learning focused efforts? Obviously, one of the key questions in the world today is that LLMs or AI systems have become great at coding and great at math. The next big question is, can they become great at law and consulting and then all sorts of different domains? And part of the black box of closed models is how people go about doing all of this. Where did they get the data from? To the extent that you can talk about all of this, I'd be very curious about how you guys have gone about it.
- [56:25]
Brian Catantaro
It's not an easy question to answer because it is quite complex, but I would say we rely on a number of things. One is that we do purchase data from companies that are building data sets that you can purchase. And to the extent that we have the rights to redistribute or to open up that data, we do as part of our Nematron data effort. With Nematron, we are trying to be maximally open with the data that we release because our goal is to support the ecosystem. Our goal is not to be the only model out there. And we love it when we hear of other models around the industry that are using our data sets to make their AI stronger, because that means we're succeeding at our job to keep the ecosystem thriving and growing. Now, we also are big believers in synthetic data generation. We use an enormous amount of compute running language models on our own systems to create synthetic data that then helps our models be better at solving problems in specific domains and we release a lot of that data as well. Now, it's of course not very straightforward to do this. Like AI is always garbage in, garbage out. So you have to work really hard to make sure that any synthetic data that you create is actually adding value. That's actually helping the model generalize and solve problems more intelligently. But those are the primary ways that we go about building our datasets.
- [58:02]
Matt Turk
Since we're talking about post training and RL in different domains. Just curious to get your thoughts on where we go from here in terms of generalization. So just to build on what I was saying a second ago, the industry seems to be marching from coding and math, which are domains with verifiable rewards, to different industries. Do you think that this is where things are going and that AI industry as a whole is going to be able to cover those next few domains as efficiently as coding or math?
- [58:37]
Brian Catantaro
Coding is really special because it's a very intellectual exercise that created a lot of economic value, which then meant that we had an enormous amount of tokens that we could learn from, as well as tooling that allows us to verify whether, you know, our. Our models are actually solving problems. So coding is always going to have a special place in our heart and something that I think AI is going to continue to get much better at because we have this special relationship with it with regards to other domains. I think what I'm excited about has to do with significantly more diverse environments for AI to learn in during reinforcement learning. I believe that, you know, reinforcement learning is such a general form of teaching an AI how to solve problems. We're just getting started at figuring out how to apply that. And I think as our environments get more sophisticated, the AI then learns more understanding of the problems that it's trying to solve, as well as the implications of the actions that it can take. Then it becomes much better at actually solving those problems. When I look at the environments that we're using today, they're still fairly simple, all things considered. And I think that's going to become significantly more complex and diverse over the next few years.
- [60:16]
Matt Turk
All right, so you mentioned making 500 people work together, and I said that we would get back to it because it's so interesting. So just taking a step back, like, tell us about the research organization at Nvidia. Like, how is it structured? How does it all work?
- [60:33]
Brian Catantaro
Well, Nvidia is not structured according to an org chart. We have one, but it's not actually the best way of understanding how we work. My team, for example, is not part of the official Nvidia research team. My team is actually part of the organization that builds the GPU. And my team is not the only team building Nemotron. There's probably 10 teams around the company that have significant involvement in building Nemotron in different parts of the company and in enterprise software in our AI software division. The part of Nvidia that actually designs the GPU also significantly is involved in building Nematronics. So there's so many different teams that have to work together. We always like to say that the mission is the boss rather than the organization. But what that implies is that people have to figure out how to work together, which is challenging in the sense that humans are naturally tribal creatures. And it's not natural for us to be friendly with people we don't know very well or trust coworkers that we don't have success working with in the past. And actually the name Nemo Tron reflects that. We had the Nemo team, which was building software for AI, and the Megatron team, which was building primarily focused on systems research for building large language models. And we decided to work together and then start calling our projects Nemotron, reflecting sort of the. The collaboration between these teams. Since then, Numicron has dramatically expanded. There's so many more teams that are part of the effort. And it's really important that we have structured it in this open way inside of Nvidia. We are inviting volunteers from around the company to come help build Nvidia's AI. We think it's very important to the future of the company. And as that vision continues to develop, more and more people want to join. That's fantastic. We're really excited about that. And it means that we then have to figure out how to organize the work so that everybody has a chance to contribute and feel heard and feel like their ideas are fairly evaluated on the path towards impact. We have a formal process for doing that. We have an internal website where people share ideas, and then those ideas are assigned to one of 25 different leads that are, you know, over various parts of building Nemotron, they interact with those ideas. Some of those ideas get further developed, some of those ideas get deferred until, you know, the next, the next time we go around building a new model. But we're trying to build Nemotron in an open and inclusive way so that, you know, we can really come together as a company to build it. I think, you know, organizations that figure out how to collaborate to build AI succeed, organizations that struggle with control over who owns the AI tend to waste a lot of effort. And so Nvidia's success and Nematron success, I think is directly proportional to our ability to collaborate. Something that I care Deeply about.
- [64:03]
Matt Turk
Fantastic. But you mentioned earlier that despite the fact that you work at the number one undisputed leader in GPUs, you all, as a research organization, don't have all the GPUs that you would want in the world. So how does the allocation of GPUs and compute happen? Is that based on how promising an idea is or early success? Do you give GPUs withdraw GPUs based on success?
- [64:33]
Brian Catantaro
Well, it's a really complicated question and it's obviously a difficult problem for everyone in the industry to figure out how to allocate their compute inside Nematron. So we have a budget for Nemotron and inside Nematron we allocate compute based on what we think the needs of the project are. We have a hierarchy. So we have a set of programs, and inside of each program we have a set of projects and each of them put forward their requests. And then we have a two week cycle where we review requests and we review the budget and then we make decisions in kind of a hierarchical way and then compute gets decided that way. Now, having said that, this is something that I think we can still do better at. It's hard when we're making decisions about compute allocation because every researcher is convinced that their idea could change the world if it just got a thousand times more GPUs attached to it. Right? And they might be right, it might actually be true. And yet we're running at the limit. We don't have 1000x more GPUs for every idea that we have. We have to operate within the limits that we have. And so it is a challenging process. We try to incorporate as many people's perspectives into that as possible so that it's as much as possible a shared sense of understanding, maybe not agreement. So there may be times when one project feels like it really deserved more GPUs because the impact of that would have been so high, but it didn't get it. We hope in that circumstance that they have an understanding of why some other project did get more GPUs and why that was considered more of a priority during this particular allocation round for the company, so that people can at least understand, you know, that there's a reason for the allocations that we have. Having said that, you know, this process is always improving. There's always more work to be done to make this more transparent and more fair. And, and then of course, my, my number one is just to get more GPUs so that, you know, we can Also fund more things, because I would like to do that too.
- [66:52]
Matt Turk
How do you balance useful research with great exploratory research?
- [66:59]
Brian Catantaro
My belief is that research needs to be bootstrapped. Research is a chicken and egg problem. So it is always the case that every researcher believes, if I just had a lot more resources, my idea would change the world. Actually, it's important that researchers feel that way because if you didn't feel that way, you wouldn't have the conviction that's required to go do something crazy and new. Right? So you have to believe. And so of course you start with that belief, but then how do you translate that belief into something that other people can understand? Right. That other people are willing to invest in? This is what I call the chicken and egg problem. Right? Because like, once your research ideas is obviously good and impactful, it's easy to get resources. But how do you get it to be obviously good and impactful without those resources? Right. So the way you solve chicken and egg problems is by bootstrapping. This is an iterative problem solving approach where you do something small, you get some sort of signal about this is a good idea and you tell people about that and then you ask for just a little bit more. And if people saw like, oh yeah, that, you know, that experiment turned out pretty well, that's pretty intriguing. We should probably do a little bit more there, then you're on track. Right. And that over time, you know, iterate a lot, iterate quickly, iterate. Many times you can bootstrap to finding significant resources for your idea and also usually attracting more people to come along with it on the way because they have a chance to see that this idea is going to change the world and then they want to be part of it.
- [68:33]
Matt Turk
Is that how the moonshots at Nvidia got started as well over the years, whether that's in AI or otherwise? So it was bottoms up, somebody coming up with a good idea versus Jensen saying this is what we need to do.
- [68:49]
Brian Catantaro
Well, you know, Jensen has lots of good ideas too. And so the company is very responsive to his ideas. And that's, that's important as well. But Jensen very explicitly says all the time, this is a company of volunteers. You know, each of us is here because we choose to, we could, we could be doing something else our lives, but we choose to be here. And so, you know, we, we tend to make decisions, especially for early stage research. It tends to be very bottoms up because, you know, it's sort of an invitation like bring, bring your best ideas. Let's let's figure out, you know, what are all of our best ideas and then we'll take a step from there. Now, do we sometimes have top down ideas that are important for the company strategy? Of course, you know, of course NVFP4 pre training is one of those. So we decided as leadership of the company, we're going to really invest in FP4 hardware. Now it's time to go invent some optimization algorithms that succeed in using it. So we told the team, we didn't say to the team, you have to work on NVFP4 pre training. What we said is there's an opportunity, we're making a big investment and if we can figure this out, it will be significant for our company. And then we let the people who are interested in that work on that and as a result we succeeded. You know, so, so it is a balance of like bottoms up and tops down, but it always has this bootstrapping feeling even, even with something like NVFP4 where there's a significant like strategic top down component, the actual technical solution, which is very intricate and complex and has a lot of moving parts that came from the researchers themselves. And you know, that's my belief is that research always comes from the researchers themselves. You can't tell research exactly how to go solve a problem because then it wouldn't be research, it would be engineering. But in a world of AI where the most important problems we have to solve all have this research component, there needs to be freedom for researchers to innovate if we're going to make progress.
- [70:54]
Matt Turk
Listening to everything you're saying, I'm struck by how entrepreneurial the culture at Nvidia still seems to be. So, like it's a very large company. So I'm sure there's all sorts of politics and you mentioned the tribal instincts. I'm sure all of this is happening, but especially given how long the company's been around, the phenomenal success, the fact that people have been making a lot of money internally, it still seems to be very entrepreneurial, bottoms up driven, maybe meritocratic. Is that the right takeaway?
- [71:27]
Brian Catantaro
Yeah, I mean, one thing that's very unusual about Nvidia is the tenure of its leadership. Jensen Huang has been running the company for 33 years, but he's not alone. There are a lot of other very senior leaders in the company who have been there for three decades or longer, including my boss. And these people remember what it feels like to work at a very small Nvidia and they know what it feels like to work At a very large Nvidia, they have a shared sense of ownership for the company. You know, Nvidia is a place we often say no one fails alone. And the point of that, that's just a statement of fact, right? You work at a company, it's one company. You all succeed together, you all fail together. You work in accelerated computing. Accelerated computing is the composition of thousands of technologies. If any of them fail to deliver acceleration, the value is destroyed. It doesn't matter whether the chip is great, if the compiler sucks. At the end of the day, the thing that you're selling is time and capability to researchers that are trying to build the future of AI. And if they don't get that, it doesn't matter whether it was the, you know, the transistor or the math unit or the compiler or the library or the networking or anything else along the way that. That failed to live up to its expectations. The whole thing in composition fails. The whole value is destroyed. And so we have a deep understanding of that culturally at Nvidia, and it is something that motivates the way that we work together.
- [72:59]
Matt Turk
Maybe to close the conversation, I'd love to zoom out, get your take from the perspective of somebody who's as deep into all of this as it gets about where things may be going. So, like, who knows, in a few years, but I don't know, in the next year or two, maybe there's some visibility. I read somewhere that you're not necessarily a big singularity kind of person. Is that fair?
- [73:26]
Brian Catantaro
True.
- [73:27]
Matt Turk
And why is that?
- [73:30]
Brian Catantaro
Well, I think that intelligence is just so incredibly multifaceted. I always think about this question, like, if a company were to be looking for its next CEO, would it find the next CEO by looking for somebody who won the International Math Olympiad? Probably not, right? Even though it's incredible for people. Like, I could never even compete in any way at the International Math Olympiad. And those people are amazing, right? They have just incredible brilliance. That's not the right kind of brilliance to run a company. If we look, for example, at other aspects of our culture that are really important, for example, musicians, what kind of intelligence does it take to become a hit musician? Don't assume that it's all luck. It's not. These people are working hard and they're very smart in ways that I might not understand with my PhD, right? I might not have that kind of intelligence. And so when I think about intelligence, I think it's just so multifaceted and so contextual. You know, it really depends on the situation. It's not just about raw intelligence. Raw intelligence is kind of like the horsepower of an engine, but an engine running without wheels doesn't go anywhere, right? So, so intelligence, the impact of intelligence has a lot to do with the context that the intelligence has put in the harness, the platform. And so when I think about that, I think the Singularity is. Although it's an attractive idea, I think that it's really a wrong headed idea because it doesn't really take into account these other factors. So I believe that artificial intelligence is going to continue to develop at a rapid pace. It's going to unlock significant capabilities for people in every aspect of, of our world economy, people doing every kind of work. I'm very excited about the opportunities that it's, that it's going to bring. I am also a little bit concerned with how we're going to manage the transition. So I do think that transitions are hard for humans in general. Like we're, we're conservative generally and you know, there is going to be a lot of change. This is a profound change in the way that we think and the way that, that we work, the way that we learn. Ultimately, I have faith in our ability as humans to figure it out. You know, we've done it in the past. This is how, this is who we are. We, we build tools, we build external organs that help us solve problems. You know, we, we have an external stomach, we call it a kitchen. It creates enormous value for us. We can eat things that we couldn't eat without a kitchen. Right now we're creating an external brain. The implications of the external stomach were pretty profound for us as a species. They led to agriculture, which led to organized societies, the way our cities are built. So we think about what is the implications of an external brain. Pretty profound. Nobody actually really knows. But what I do believe in is the power of humanity to solve problems and to learn and to incorporate new technologies in ways that benefit us. I also believe that the problems we face as a planet all require more intelligence. Every single one of them, whether that's inequality or climate change or any of the other structural problems that I think are very worrisome that we face. The solutions to those are going to require invention and intelligence. And what that means for me is that the only kinds of tools that we can really create moving forward are going to be AI because the problems that we face are all about intelligence. And regardless of the technological approach to solving those problems, the solutions will always be called AI. And so that makes me hopeful for the future, but also somewhat respectful of the challenge that it is going to bring to us as we try to figure out how to live in a new way with this new external brain. But I believe in our ability to learn and to change. And I think ultimately this is going to make our lives better.
- [77:50]
Matt Turk
Do you guys feel the AI backlash that seems to be forming internally? Is that something that you all perceive, think about, and if so, do you think it's a communication problem that our industry may have? In particular, given what you just said about all the obvious potential of AI?
- [78:09]
Brian Catantaro
You know, I'm always worried about the way that the public thinks about technology and interacts with it. It matters a lot. And it is definitely the case that societies that want technological advancement have more technological advancement than societies that don't want change. So I think it is actually important to think about it. One thing that's interesting about AI is that I believe it tends to be much more accepted when it is part of everyday life. And at that point, people stop thinking about it as AI. It's just, oh, this is the tool that I use. Like, do you care whether it's AI that's helping you route your car? When you ask the map application to help you drive somewhere? I mean, it is. There is actually sophisticated AI that's going into that, but you're not really thinking about that. Right. You're just using a tool. And so I feel like people's acceptance of AI comes with experience. Right. The more experience we have working with it, the more we learn how to work with it productively, I think the more comfortable we become with it.
- [79:19]
Matt Turk
Great, Brian. So it's been a fascinating conversation. Maybe as a very last question, to make sure we cover it, I want to make sure that we talk about safety. What is the state of safety currently, and where does open source and closed source sort of fit in the safety conversation today?
- [79:40]
Brian Catantaro
Safety is on everybody's minds right now, watching the fable release. And the way that the government interacted with that, I think is a consequence of concerns about safety, about these models. They get stronger and stronger, and then they could be misused. And there's different approaches to thinking about safety and trying to define safety. I have maybe a slightly unorthodox opinion about this, which is that I think open technologies are generally safer because there's more sunlight. You know, when more people are thinking about the safety of a technology and evaluating it and then contributing to making it safer, I think that's inherently safer than having a small group of people being in charge of safety for everyone else. I also think with artificial intelligence, because it is really about ideas. It's really about exploring ideas in different ways that diversity is more safe than monoculture. And what that means is that there's going to be different beliefs. Like, diversity isn't just about, like, the easy stuff. Diversity is about the hard stuff. Like, when people have deeply felt disagreements, they really, really, totally disagree with each other. Making it possible for people to explore their ideas in a diverse way, I think is more safe than trying to create a walled garden where certain ideas are considered safe and certain ideas are considered unsafe. And this is controversial in today's AI environment, which I think is interesting, because we've had hundreds of years of tradition that speak directly to this. In the United States, for example, we have laws about freedom of conscious conscience and freedom of speech. And, you know, it's not because we didn't consider for thousands of years, would it have been safer if we didn't have those? Right. We tried that. We tried actually having a monoculture about, like, these ideas are safe to talk about, these ideas are safe to believe. And we found that to be much less safe than a pluralism where we officially don't take a position about what ideas are safe. We actually found that is much safer as a society to support diversity than it is to try to keep everybody safe, top down. And so I believe that open technologies for AI are inherently the safest way of building AI.
- [82:26]
Matt Turk
All right. Love it. Controversial take to close the conversation. Brian, it's been fabulous. Thank you so much. Really appreciate your spending time with us today.
- [82:35]
Brian Catantaro
Thanks for inviting me.
- [82:38]
Matt Turk
Hi, it's Matt Turk again. Thanks for listening to this episode of the MAD podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing, if you haven't already, or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.