Meta’s Chief AI Scientist Yann LeCun: The Path Toward Human-Level Intelligence in AI [Ep. 473]

Summary

Podcast Summary: Meta’s Chief AI Scientist Yann LeCun: The Path Toward Human-Level Intelligence in AI [Ep. 473]

Podcast Information:

• Title: Into the Impossible With Brian Keating
• Host/Author: Big Bang Productions Inc.
• Description: A podcast exploring our scientific and human understanding of the world, featuring conversations with visionaries from arts, sciences, humanities, and technology. Hosted by Dr. Brian Keating, Chancellor’s Distinguished Professor of Physics at UC San Diego.
• Episode: Meta’s Chief AI Scientist Yann LeCun: The Path Toward Human-Level Intelligence in AI
• Release Date: December 29, 2024

---

1. Introduction to Yann LeCun and AI Frontiers

Timestamp [02:15]:
Yann LeCun opens the conversation with a quote from 2001: A Space Odyssey, stating, “Any sufficiently advanced technology is indistinguishable from magic,” setting the tone for a deep dive into advanced artificial intelligence.

Brian Keating:
Introduces Yann LeCun as Meta’s Chief AI Scientist, highlighting his pioneering work in AI architectures like JEPA (Joint Embedding Predictive Architecture), which aims to build explicit mental models of the world and reduce output randomness. Keating emphasizes the potential transformation of fields such as physics, education, and healthcare through these advancements.

---

2. The Limitations of Current AI Systems

Timestamp [05:49]:
LeCun addresses a provocative statement he made in the Wall Street Journal: “AI is barely as smart as a cat.” He elaborates that current Large Language Models (LLMs) like GPT-4 manipulate language impressively but lack true understanding of the physical world.

LeCun:
“LLMs can pass the bar exam, but we still don't have domestic robots that can do what any 10-year-old can do in one shot.”

He contrasts the intuitive problem-solving abilities of cats with the current capabilities of AI, emphasizing that while AI can handle symbolic representations, it fails to grasp the complexities of the real world, such as planning and reasoning based on physical interactions.

---

3. JEPA: Advancing Beyond Autoregressive Models

Timestamp [13:43]:
LeCun introduces JEPA, an architecture designed to overcome the limitations of self-supervised learning in AI. Unlike traditional LLMs that predict sequences of discrete tokens, JEPA focuses on understanding and predicting continuous, high-dimensional data such as images and videos.

LeCun:
“JEPA stands for Joint Embedding Predictive Architecture. It trains systems to find good representations of data by eliminating unpredictable elements and focusing on what’s useful for prediction.”

This approach aligns more closely with how humans develop mental models, allowing AI to better understand and interact with the physical world by abstracting relevant variables and ignoring irrelevant details.

---

4. AI and the Future of Scientific Discovery

Timestamp [08:02]:
Keating poses a critical question about whether the current focus on GPU and LLM approaches is stifling innovation in other scientific areas.

LeCun:
“LLMs are a hammer, and now everything looks like an L. That’s a mistake. We need to go beyond autoregressive architectures towards systems that can understand the real world and acquire common sense.”

He underscores the necessity for AI architectures that mimic human-like understanding and planning, which are essential for breakthroughs in complex scientific fields such as physics.

---

5. Dark Matter of AI: Self-Supervised Learning

Timestamp [36:03]:
LeCun elaborates on his analogy comparing self-supervised learning to dark matter in AI, highlighting its fundamental yet underexplored role.

LeCun:
“I use an analogy where self-supervised learning is the bulk, the dark matter of AI. It represents the majority of what we learn without explicit supervision, much like dark matter constitutes most of the universe’s mass without being directly observable.”

He stresses that while supervised and reinforcement learning methods are well-understood and applied, self-supervised learning remains the elusive component necessary for achieving human-level AI.

---

6. The Path to AGI and Safe AI Development

Timestamp [44:21]:
When asked about the timeline for achieving Artificial General Intelligence (AGI), LeCun eschews the term AGI in favor of “human-level AI” or “Advanced Machine Intelligence (AMI).” He estimates that human-level intelligence in machines could emerge within five to six years, contingent on the success of architectures like JEPA and advancements in computational power.

LeCun:
“Building safe AI systems is akin to engineering reliable turbojets. We won't have a magic bullet, but through objective-driven AI and robust guardrails, we can ensure these systems amplify human intelligence without posing existential risks.”

He emphasizes the importance of aligning AI objectives with human values and implementing multiple layers of constraints to prevent unintended harmful behaviors, drawing parallels to how human laws function as societal guardrails.

---

7. AI’s Impact on Education and the Role of Professors

Timestamp [73:42]:
Keating inquires about the implications of AI on the profession of teaching and academia.

LeCun:
“Human interaction in education—such as mentorship and ethical guidance—remains irreplaceable. AI will augment this relationship by providing advanced tools and personalized learning experiences, but the fundamental role of professors as mentors and researchers will persist.”

He envisions a future where AI assists both educators and students by enhancing the learning process through intelligent systems, thereby transforming but not eliminating the professor’s role.

---

8. Personal Reflections and Evolving Perspectives

Timestamp [76:51]:
LeCun shares his personal journey and evolution in the field of AI, notably his change in stance regarding unsupervised learning.

LeCun:
“In the late '80s, I was skeptical about unsupervised learning. However, influenced by Geoff Hinton, I recognized its potential and fully embraced it by the early 2000s. This shift was pivotal in my advocacy for self-supervised learning as a cornerstone of future AI advancements.”

His openness to changing his views based on new evidence underscores the dynamic and self-correcting nature of scientific inquiry.

---

9. Optimistic Outlook on AI’s Transformative Potential

Timestamp [62:50]:
LeCun conveys an optimistic perspective on AI, comparing its potential impact to the invention of the printing press.

LeCun:
“Intelligence is one of the most desirable commodities missing in society. AI that amplifies human intelligence could be as transformative as the printing press, fostering the dissemination of knowledge and driving societal progress.”

He believes that, much like the printing press enabled the Enlightenment, AI will empower humanity to achieve unprecedented advancements, provided its development is guided responsibly.

---

10. Addressing Concerns About AI Safety and Control

Timestamp [52:06]:
Keating raises concerns about creating AGI systems that might lose control, prompting LeCun to discuss the inherent differences between human desires and AI objectives.

LeCun:
“The notion that intelligent systems inherently desire to dominate is false. Safe AI development hinges on constructing objective-driven systems with aligned goals and robust guardrails, ensuring they operate within defined ethical and practical boundaries.”

He dismisses fears of malevolent AI by highlighting that, unlike humans, AI systems do not possess intrinsic desires unless explicitly programmed, and with proper design, they can be aligned to serve humanity’s best interests.

---

Conclusion

Throughout the episode, Dr. Brian Keating and Yann LeCun engage in a thought-provoking dialogue on the current state and future trajectory of artificial intelligence. LeCun’s insights into the limitations of existing AI models, the promise of architectures like JEPA, and the essential role of self-supervised learning underscore a vision of AI that complements and augments human intelligence. His optimistic outlook is tempered with a pragmatic approach to AI safety, emphasizing the importance of aligned objectives and robust constraints to harness AI’s transformative potential responsibly.

Notable Quotes:

• Yann LeCun [02:07]: “Any sufficiently advanced technology is indistinguishable from magic.”
• LeCun [05:49]: “AI is barely as smart as a cat.”
• LeCun [36:03]: “Self-supervised learning is the dark matter of AI.”
• LeCun [62:50]: “AI will amplify human intelligence as the printing press amplified knowledge dissemination.”
• LeCun [52:10]: “The notion that intelligent systems inherently desire to dominate is false.”

---

This episode provides a comprehensive exploration of the evolving landscape of artificial intelligence, blending technical discussions with philosophical reflections on intelligence and the future of human-AI collaboration. Listeners gain valuable perspectives on how AI can be developed thoughtfully to serve as a powerful tool for societal advancement.

Transcript

[Brian Keating]

Right now, the Home Depot has spring deals under $20. So what are you working on? If you're planning on cooking out this season, head to the Home Depot so you can fire up the grill with deals on charcoal. Right now, get two 16 pound bags of Kingsford Charcoal for only $17.88. Was $19.98. Don't miss spring deals under $20 now through May 7th at the home Depot. Subject to availability valid on select items only.

[Yann Lecun]

This episode is brought to you by Chevy Silverado. When it's time for you to ditch.

[Brian Keating]

The blacktop and head off road, do it in a truck that says no to nothing.

[Yann Lecun]

The Chevy Silverado Trail Boss get the rugged capability of its Z71 suspension and.

[Brian Keating]

2 inch factory lift, plus impressive torque and towing capacity thanks to an available.

[Yann Lecun]

Duramax 3 liter turbo diesel engine. Where other trucks call it quits, you'll.

[Brian Keating]

Just be getting started. Visit chevy.com to learn more. Welcome back to into the Impossible. Today we're going to dive deep into the frontier of artificial intelligence with a pioneer, Yann Lecun. Jan only answers to one man at Meta. That's right, the Zuck. Today we'll find out what makes Zuck tick and along the way we'll explore Jan's controversial claims. Jan is a visionary and the motivating force behind new architectures like jepa, a self supervised AI approach that builds explicit mental models of the world, reduces output randomness, and opens new frontiers for understanding, predicting and solving complex challenges in physics, education and healthcare. It may just transform the way we learn and teach. So join us for a mind expanding conversation on advanced machine intelligence and the nature of intelligence itself. We'll push the boundaries and explore the evolving role of educators in an AI driven future. And we'll even explore the financial incentives for AI that drive a lot of the profit margins at places like Meta. Now let's jump into this conversation with Yan, the man behind the Metaverse.

[Yann Lecun]

Any sufficiently advanced technology is indistinguishable from magic. Open the pod bay doors.

[Brian Keating]

Hal hey Meta. Who is Yann Lecun? Yann Lecun welcome to the into the Impossible podcast.

[Yann Lecun]

Pleasure to be here.

[Brian Keating]

These are my favorite piece of technology. In fact, it's my second version of these and I've actually used them with a real CIA spy to kind of diagnose how these are the ultimate spy tools. So you are to be commended on these devices. I actually had an Apple Vision Pro, which is kind of a trick to buying a public university professor's salary, but I returned it. So I returned it within the Apple return window.

[Yann Lecun]

I'm on my third pair of the. The Ray Bans. I had the first version and then the second version, which I killed because I was selling and flipped, and they went underwater. It was a kind of early model. So I sent it back to our colleagues, and they're trying to figure out what went wrong, and they sent me another pair.

[Brian Keating]

Yeah, I really feel like they are the future. I mean, anyone who's had an Apple vision or even the meta quest, to be honest with you, I enjoy it. If I'm playing a game or something, my kids steal it from me. But they're heavy, and humans don't like to have their peripheral vision kind of concealed because a predator will come from behind us. Right. So in my case, I love these to kind of augment reality. The quality is great. This is not a meta commercial. But the point is, I think Apple could have done a lot better, and I don't see that surviving. I'm very interested to see the Generation three when those come out, and maybe I'll cajole you into giving me an early trial. But today, we're here to talk about physics. But we have to start off, because I am the associate director of the Arthur C. Clarke center for Human Imagination at UC San Diego. And you may notice in my background, I have a quote, and I'd like you to maybe tell my listeners what that quote means to you. Open the pod bay doors.

[Yann Lecun]

Wow, That's a quote. A famous quote from 2001 A Space Odyssey. And I must say that this movie had a big influence on me because I saw it when I was nine years old when it came out, and I was very, very impressed by that movie because it was talking about all the stuff I was fascinated by, you know, the universe and space travel and how intelligence emerged. I was nine years old. And intelligent computers and things like that.

[Brian Keating]

Most people don't realize that's where the word podcast comes from, that we're on. There's an engineer at your rival. One of your rival companies, Vinny Chieko at Apple, who was inspired by HAL 9000, just like you, and he saw the prototype of this white, gleaming, little, tiny device called later, and he said, we got to call it the ipod, and then the rest is history. And that's where podcasts come from. So we always open the audio episode of the show with actually Dave talking to Hal to open the pod bay doors, which he refuses to do, which some of your fellow researchers, including recent Nobel laureate Greg Hinton, you know, kind of might be terrified by. But I want to start first with a quote that you were quoted in the Wall Street Journal in October around the time of the Nobel Prize, and you said something very interesting to me or very provocative, as is your want. You said, AI is barely as smart as a cat. And I to myself, jan, you haven't met my cats. You know, my cats knock over glasses of water just to spill them on my laptop. And they play with a dead mouse just, just for fun. And I can't imagine an AI doing that because, you know, the only way to stop it would be to have a laser pointer in every room in the metaverse. So what did you mean by that? And why was that meant to comfort me?

[Yann Lecun]

What I meant by that is that the, the best of our LLMs, you know, can manipulate language in pretty amazing ways, but they basically have no understanding of the physical world because they're purely trained on text. So what? The image of the world that they get is through human representation of it, which is extremely symbolic, first of all, but approximate, simplified, discrete. The real world is much more complex than that. And our AI systems are completely unable to handle the real world, which is why we have LLMs that can pass the bar exam, but we still don't have domestic robots that can do what any 10 year old can do in one shot, can learn in one shot without even spending any significant brain power. So now what I say about a 10 year old is actually true of a cat. If you see cats trying to jump on a bunch of furnitures to reach a particular object of interest, they, they sit down and they kind of move their head and they plan their trajectory and then they go bounce, bounce, bounce, bounce, bounce, bounce. And you like, how did they do that? Right? So they can plan, they can reason, they understand the physical world. They have an extremely good model of themselves, of their own dynamics and intuitive physics about many things. And those are things that we are incapable of reproducing with computers today.

[Brian Keating]

It always seems remarkable to me, both the optimism and the pessimism about these objects. And I wonder, there's something called the Sudates trap or something like that where a rival power comes to power, it's weaker, and then the dominant power spends all of its attention focusing on that. It's sort of related to sunk cost fallacy. And I want to get your impression about this. Are we sort of dooming ourselves? At least in the physical sciences, this GPU and LLM approach is just sucking all the oxygen up in this space. I mean, as far as consumers are concerned. And then that drives Nvidia to be, you know, worth $3 trillion. Are we now, you know, fully basically committed to the, you know, GPU plus LLM model, and will that not stifle actual innovation in physics, for example?

[Yann Lecun]

Well, yes or no? So, yes, if we keep being obsessed by LLM and LLM are sucking everything else out of the air, and currently it looks a little bit like this. LLMs are, you know, kind of a hammer, and now everything looks like an L. And that's a mistake. Yeah, I mean, that's the point I've been making, that LLM are not the be all, end all of AI. They're surprisingly powerful given their conceptual simplicity, really. But there's a lot of things they cannot do, and one of them is representing and understanding the physical world. And certainly planning actions in the physical world is not something that LLMs can do. So I think that's the big challenge over the next few years in AI. It's going beyond autoregressive LLN type architecture towards architectures that can perhaps understand the real world, acquire some level of common sense. And the way our intelligence operates, at least for complex tasks that require or deliberate conscious reasoning, is that we have some mental model of how the world works. And we can imagine what the result of our actions is going to be, the effect of our actions. And that is what allows us to plan, because we can imagine what the result of a sequence of action is going to be. We can optimize that sequence of actions so as to achieve a particular goal. That's exactly what goes on when the cat is standing and looking up and trying to figure out what trajectory to follow. That's planning. And they have a mental model themselves. They have a mental model of the material they're going to jump over. We do this absolutely all the time, very often without even realizing. And that should be of interest to physicists. And I hope we can talk about this next.

[Brian Keating]

Yeah, no, I like to bring up this guy here, Albert Einstein. So in 1907, he had this Gedankin experiment where he envisioned a freely falling observer and an elevator. God forbid the cable snaps and the elevator falls, that such a person would experience no gravitational force field. And he called that the happiest thought of his life. And I want to ask you, Jan, in what case, or is it possible that A, a computer could have a happy thought, let alone the happiest thought, and B, without embodiment, without some visceral sensation, to know that pit in our stomach that we all know when we go on a roller coaster or when the elevator moves weirdly. How can an AI, a computer system, ever have the capability to construct new physical laws like Einstein did?

[Yann Lecun]

The short answer is that today, no, like the AI systems today cannot have this kind of intuition. Even though, I mean the, the AI systems that are the most appropriate, that are applied to scientific discoveries today are specialized models, right? So you want to predict the structure of a protein or predict the interaction between two molecules or the property of a material. You develop somewhat specialized models for this. And you can't use LLMs really for this kind of stuff. They're just going to regurgitate whatever they've been trained on, but they're not going to be able to come up in new things. And those models, of course, are powerful in the sense that they all predict chemical reactions that nobody tried before and properties of material that nobody ever built and things like this. So they are a little more outside of the beaten path. They can go a little bit outside the beaten path more than LLMs, which basically are ways to index existing knowledge. But they're not, they're not going to have this kind of insight that Einstein was famous for, not yet. But the hope is that at some point they will. My big question, scientific question and interest is how to do that is what kind of process, through what kind of process do we humans, but also animals, build models of the real world? And one big thing there is the figuring out the appropriate representation and relevant variables of a system or something that you're interested in modeling and what's the right level of abstraction of that representation. So for example, you and I know that if you want that we can collect an infinite amount of data on, let's say Jupyter, and there's like enormous amounts of data that we know about Jupiter, right, in terms of weather, density, composition, temperature, all the everything. But now who would have thought that to predict the trajectory of Jupiter for the next few centuries, you only need to know six numbers, three positions and three velocities and you're done. You don't even need to know density, composition, rotation, anything like that. It's just six numbers, right? So the, the most difficult step to being able to make predictions is finding the appropriate representations of the reality and eliminating all the stuff that's irrelevant so that you can make those predictions. I've been obsessed for the last several years with an architecture that I think is capable of this, that we call jepa, which I may explain if you want.

[Brian Keating]

Yeah, yeah, I'd like you to. And I just want to thank you for bringing up Another reference to 2001 A Space Odyssey. The planet Jupiter, of course. Course is where they find the mysterious monolith that then allows them to transport across probably a wormhole or something. Yeah, talk about jepa. What is that? I'm not familiar with that.

[Yann Lecun]

There is one characteristic of LLM or one trick that LLMs use, which I've been advocating for a very long time, called self supervised learning. So what is self supervised learning? It's basically, you take an input, it could be a sequence, or it could be anything, it could be an image, you corrupt it in some way, and then you train the system to recover the full input from the corrupted one. In the context of language and LLMs in particular. So there are several types of natural language understanding systems. But an earlier one, before the current crop of LLMs, is one in which you take a piece of text, you corrupt it by removing some other words, replacing them by blank markers or substituting some other words, and then you train a gigantic neural net to predict the words that are missing or are wrong. Right in the process. After doing so, the system learns a good internal representation of the text that can be used for all kinds of potential applications. Right. As input to, say, a translation system or sentiment analysis or.

[Brian Keating]

I never thought I'd say this. I gave up my morning coffee. Not just temporarily, but because I found something that makes me feel so much better. It's called hi Nandaka. And this is not just another coffee alternative. It's the upgrade your body needs. I used to depend on the morning coffee as soon as I woke up. But after the buzz wore off, I felt jittery, anxious, and drained by the afternoon. That's when I started using Nandaka and everything changed for me. Instead of a quick caffeine hit, I get steady, sustained energy that lasts all day long. No crashes, no afternoon slumps. It even staves off my hunger. So I don't need that muffin that I used to crave in the morning. Just gives you clear, focused, calm energy. And my digestion from this is much better than with coffee. I don't get bloating or stomach pains. My mood is more stable, and most importantly, my metabolism seems to be working better too. Now I'm a doctor. I'm not that kind of a doctor, but I know that I'm not craving sugar or snacks anymore. And I've even dropped a few pounds, and not just a few pounds, from my chin to my stomach. Unlike other mushroom coffees that I've tried those that basically use mycelium, which is basically grain powder. Nandaka uses 100% fruiting body mushrooms, the part containing the powerful compounds that support my energy, focus and longevity. It's crafted with ceremonial grade cacao. So delicious fermented probiotic teas and adaptogenic herbs to fuel my body and mind. And it will do the same for you. Every ingredient, a source for maximum purity and potency, sustains my calm energy with slow release caffeine that provides clear focus without burnout. The functional mushrooms like reishi and cordyceps promote clarity and cognitive performance, really boost my mood and focus. And last but not least, because it has no fillers, no preservatives and no junk, I find it really improves and supports my digestion. The polyphenols from Pooh, air tea and cacao help with my gut health and reducing all my sugar cravings. Right now peak is offering 20 off for life and a free starter kit with your purchase. That's a rechargeable frother. It's so cool and fun to use a glass beaker so you can feel like a real scientist like me. And together you'll make the perfect cup every time. Just go to peaklife.com Impossible. That's P I Q U E L I F E.com Impossible. Trust me, your energy, your gut and your future self will thank you.

[Yann Lecun]

Summarization Whatever you want. Okay, now LLMs are a special case of this, where you build the architecture of the system in such a way that to predict a word in the input, the system can only look at the word to the left of it. Okay? So it can only look at the previous words to predict a particular word. So now you don't need to do the corruption process anymore because the architecture basically intrinsically corrupts the system by preventing the system from looking at all the data. It can only look at what's to the left of a particular word to predict that word, right? So you put an input and then train the system to just reproduce its input on this output. Okay? So that's self supervised because there is no task that you ask the system to accomplish. There is no differentiation between input and output. Everything is an output and an input, right? So that's our supervised learning. Now that works amazingly well for language, and it works really well for DNA sequences and all kinds of stuff, but it only works for essentially sequences of discrete things like language. So language is there's only a finite number of words in the dictionary. You can never predict which word will follow a sequence of words, but you can predict a vector of scores or probability distribution over all possible words in a dictionary. And that's easy to do, right? It's just a big vector of numbers between 0 and 1 that's up to 1. What do you do about natural data? Okay, Data that comes to you from a sensor, say a camera. So now that your data is video, or let's say it's just an image. So what you could try is try the same thing, right? Take an image, corrupt it by masking pieces of it, and then train some neural net to reconstruct a full image. That's called a masked autoencoder. Nae, it doesn't work very well. And in fact there is various ways to train systems to reconstruct from partial views, right? They're called autoencoders, but there are various ways to train them. The math technique is just one. And none of that really works really well. A lot of those techniques, by the way, are inspired by statistical physics. So one particular method to do this is called variational autoencoder. And the variational comes from variational free energy. So it's the same math okay, as statistical physics.

[Brian Keating]

Does it fail though, Jan? Does it fail because of some missing boundary conditions or initial conditions like the three body problem? I had an undergraduate this past year try to answer the following question. If you had the orbit of Mercury for the last 10,000 years, which we can, you know, we actually know its orbit from JPL and NASA, right? So we know it's orbit over, you know, something like 200 of its years because its year is much quicker than ours. And I said, given that data, could you predict that there's something missing from Newtonian mechanics? In other words, that you have to then augment it with a new variational method, which is the variation of the Einstein Lagrangian, and it couldn't get it. It could. There was nothing it would do. It knew that it could predict that there's some anomalous procession of Mercury, but it couldn't get the equations. And we had to basically force it by feeding it the analog of Einstein's equation. We want this answer to the. If we had LLMs in 1899, could we have predicted Einstein's theory of general relativity? And the answer, at least with type models that we're using machine learning, we couldn't do it. Is that what's missing from. Is that why it fails in sort of sense, it's missing some core insight that Einstein's genius had to come up with or something else?

[Yann Lecun]

No, it's something else.

[Brian Keating]

Okay, what is that?

[Yann Lecun]

It's much more pedestrian, unfortunately. It's the fact that to predict a continuous, high dimensional, continuous signal like an image or a video, it's very difficult to represent a probability distribution over all possible images. Right? You can, when you're predicting a word, you don't know exactly which word comes, comes after a sequence, but you can approximately.

[Brian Keating]

Sure, yeah. It's not going to be gibberish if it's actual language.

[Yann Lecun]

Right, right. And you know, if you have a verb, there's probably a complement that comes after things like that, right? So you can't do this with video. So if you show a video, if you train a system to predict what happens in the video, right? You show it a segment of video, then you stop the video, you ask it predict what happens next. And then of course, you show it what happens next and then you train it to actually predict this. It doesn't work very well. I've been working on this for the better part of the last 15 years and it really doesn't work.

[Brian Keating]

I trust you.

[Yann Lecun]

And the reason it doesn't work is that if you train the system to make one single prediction, the best thing you can do is predict the average of all the plausible futures that may happen. And that's basically a blurry image. Because even if we take videos, like our video right now of us speaking, I could be saying a word or another, I could be moving my head one side or the other, I could be moving my hands one way or another. And so if the system has to make one prediction and we train it to minimize the prediction error, it's just going to predict the average of all the things that could happen. You're going to see blurry versions of my hands, blurry versions of my face, very blurry versions of my mouth. And that's not a good prediction. And so that just doesn't work. Basically, self supervised learning by reconstruction or prediction does not work for natural signals. Okay, so now I'm coming to this idea of jepa. Okay, so JEPA stands for Joint Embedding Predictive Architecture. So what's an embedding? An embedding is a representation for a signal, right? You take an image and you don't care about the precise value of all the pixels. What you care about is some representation which is going to be a list of numbers, a vector that represents the content of the image, but does not represent all the details about it. Okay, that's an embedding. And joint embedding is that if you take an image and you take a corrupted version of that image, or let's say a slightly transformed version of the image, different viewpoint, for example, the content of the image doesn't change, and so the embedding should be the same. So a joint embedding architecture is trained by, is basically a big neural net. And you train it in such a way that when you show it two versions of the same image of the same thing, you produce the same embedding. You force it to produce the same embedding, okay, the same output, essentially. And then the P. The predictive is, let's say a version of the image is a frame in a video and the corrupted version is the frame before. So now what you need to do is predict the next frame from the previous frame or predict the next few frames. So we produce few frames, and that's called a jepa. So joint embedding predictive architecture, right? You have two embeddings, one that takes the future of the video, one that takes the past, and then you have a predictor that tries to predict the representation of the future of the video from the representation of the past of that video. When you use this type of architecture to train a system to learn representations of images, it works really well. There's a number of different techniques that my colleagues and I and many other people have come up with over the last few years to do this, and it works really well. So we can learn good representations of images. We're starting to get good representations of video, but it's very recent. But then what you can imagine is now that you have this principle that I was talking about for Jupyter, where you have data about Jupiter or Mercury, and then you ask the system, find a good representation of all the data you have, eliminating all the stuff you can't predict so that you can make predictions in representation space. So eliminate all the stuff you cannot predict, the weather on Jupiter, all kinds of details that you really would not be able to predict, and eliminate all that and just find a representation such that you can make predictions at a certain horizon within that space. And in my opinion, that's really the essence of kind of understanding the world that you do when you do physics, right? You're trying to find a model of a phenomenon, eliminating all the stuff that is irrelevant, and then finding a good set of relevant variables that allows you to make predictions. That's really what science is all about.

[Brian Keating]

Is there an analog in that that is subjective? Like, you speak about temperature of a model and things like that. Do you have to still specify parameters like that.

[Yann Lecun]

Not particularly in this context, because when you have this kind of architecture, or at least the simple one I described, you eliminate the stochasticity, to use a penetic term, in the prediction. You're basically. I mean, when the system is trained to do this, it trains itself simultaneously to find a good representation of the input that preserves as much information as the input as possible, but at the same time it's still predictable. So if you have phenomena in the input that is unpredictable, like chaotic behavior, random things that you just can't predict, individual motion of particles and stuff like that from thermal fluctuation, it's not going to keep this kind of information. It's going to eliminate this and just keep whatever relevant part of the input are useful for prediction. So let's take a chamber full of gas and we can measure all kinds of stuff about it, including the position of all the particles, which is an enormous amount of information which you can't predict because you have to know how every particle interacts with the walls and the heat baths and everything. So you can't really make any prediction of that. Then probably the system spontaneously will say, well, I'm not going to be able to predict the trajectory of each particle, but I can measure pressure, volume, maybe number of particles and temperature. And look, when I compress, the heat goes up. So PV equals NRT or something. That's really the essence, I think, of where perhaps machine learning connects with, with science or physics in particular. Another aspect of this which I think is fascinating, which really we haven't explored nearly enough yet, because that's kind of a bit of a new concept is the idea of the level of abstraction of a representation. So people do this in physics, in science, right? In principle, we could explain everything that is occurring between us right now in terms of, I don't know, quantum field theory, right?

[Brian Keating]

Yes, I suppose, although there be the subjective nature of human consciousness might play out.

[Yann Lecun]

That's just particles interacting, right? So in principle they could all be reduced to quantum field theory, but of course it's completely impractical because the amount of information you would have to manipulate is just ridiculously large. Right. We use different levels of abstraction to represent phenomena that again eliminate the details. Right. Quantum field theory. And then on top of this we have particle, elementary particle theories, atomic theory, molecules, materials, and then you go up the hierarchy, you can go pretty high up, and then have biology, objects interacting with each other, and then psychology at the top or something like that. So finding good abstract representation levels of Representations allow you to kind of understand what goes on, but eliminating all the irrelevant detail is really kind of at the root of intelligence. Really.

[Brian Keating]

I think I had this debate with someone I'm guessing is not a great friend of yours or champion of a Peter Thiel back in May, and it was. Whether or not we can extrapolate from LLMs, there is something emergent about them, but it's not clear that what they are currently lacking is what will get them to a level of artificial Einstein. I call ae, you know, artificial Einstein and that's, you know, the training data. So you talked about this in conversation with our mutual friend Lex Friedman last year, early this year. You know, basically that LLMs have 20 trillion tokens, but a 4 year old has order of magnitude more. And maybe they get more, maybe they do get that 10x. But I always say is what's missing is what's preventing us from coming up with the analog of general relativity or a theory of everything. For example, is it that AI currently doesn't know the plot of Gladiator 2? I don't think so. In other words, there's so much, there's infinite amount of training data that could be provided in tokenized form, but there's something very different. You know, Einstein didn't need the Fast and the Furious one to come up with general relativity, the principle of Lorentz invariance, so. Or Poincare. What is the most likely route to new physics in your opinion? Is it the type of, you know, JEPA and more, you know, kind of visual data and modeling based on observed phenomena, predictive bridges between the two? Or is it something like symbolic regression, which is totally different and does require, in my opinion at least. I'm an experimental cosmologist, so I'm not a theoretical physicist. But. But the point being, does it need some supervision that only humans can provide? So what do you think is the best route to focus it on a theory of Everything, which is currently the holy grail in my field. What would be the most likely route to that, in your opinion, in terms of tools and techniques that a young person might want to pursue?

[Yann Lecun]

Okay, so I think there are a number of techniques that people have been working on for a while that will probably have a lot of utility in the short term. So things like symbolic regression. My screamer has been kind of working on this kind of stuff and he was connected with NYU and the Flight Iron Institute and everything. And I've connected with him as well. So yeah, it's really interesting stuff, this work on this kind of stuff going back decades, but it didn't work very well at the time because computers were not that powerful and everything. So there's been a lot of progress. I don't think this is going to produce systems that have the kind of insight that we're talking about, the kind of insight that Einstein have had or a lot of other physicists like Feynman or things like that, and certainly not produce the theory of everything just by itself. I think what's missing is much more fundamental. And so it's the ability to construct mental models of the world that the system would be able to manipulate.

[Brian Keating]

In.

[Yann Lecun]

Its mind and use corner cases, extreme cases. I mean, this is the kind of gedonging experiment that you were talking about before that. Einstein was famous for having a mental model of something, making a hypothesis and then trying to push that model to kind of an extreme case and see what's happening there. Or like the gedunking experiment that people use very often to explain time contraction due to relative speed, right? So if you observe someone on the train shining a light that bounces up and down through two mirrors, that person, the light goes at the speed of light and it just travels up and down a particular just the height of the wagon. But to someone observing this from the outside, the light is bouncing up and down in diagonal. So it's actually traveling a longer time, but still at the speed of light. So it must be that time is contracting, right?

[Brian Keating]

Today's episode is sponsored by Acorns Early. April is Financial Literacy Month and it's the perfect time to start teaching your kids the value of money. And Acorns early makes it easy with their Smart Money app and their debit card for kids, Acorns Early's Chores tracker teaches kids that hard work pays off. You just set up the tasks in the app, set a payment amount, and tick the chore off when it's done. Sort of a project management structure for your home. A home operating system. I love it. Savings goals help kids turn savings into a habit. They can even choose their own target dates and turn on the optional autosave feature to help them get to their goals. Kids get a sense of independence wherever they spend and what they earn with their very own debit card. Plus, kids can build up their money knowledge and their confidence with bite sized financial lessons. The Acorns early app has tons of fun interactive courses that help my kids learn all about things revolving around money. And money, as we know, makes the whole universe go around. No matter what your kid's age is, they're going to have to learn about money. And so I found Acorns early incredibly important for positively affecting both the relationship that my kids have with money and my relationship with them. It kind of adds an intermediary, a third party that really they trust sometimes more than me because of the Acorns early professional level of detail. So you ready to help your kids grow money skills that will last a lifetime? Just head to acorns early impossible. That's a C O R N S E-A-R-L-Y.com impossible. Or download the Acorns early app to get started. Sign up now and your first month is on us. Acorns early card is issued by Community Federal Savings bank member FDIC pursuant to license by MasterCard International. Terms and conditions apply. Monthly subscription fee starting from $5 per month unless canceled.

[Yann Lecun]

I mean, that's a very simple aposteography. It's kind of obvious, but you have to make the enormous assumption that the speed of light is the same for everyone. Here is the thing that you were talking about. Training data. Do you need to know about. Do you need to have watched Gladiator to be able to come up with relativity? Of course you have it. The interesting thing is that historians of science seem to say that Einstein was not aware of the experiments at University of Chicago, the Mickson Morley experiments that showed that the.

[Brian Keating]

Sorry, I have to interrupt you. That was at Case Western, my alma mater. What was that?

[Yann Lecun]

Case?

[Brian Keating]

Sorry, I can't let that one go. Sorry, go ahead.

[Yann Lecun]

Indeed, indeed. They were trying to prove the existence of the ether and they couldn't prove it. And they saw their experiment was deficient. And I don't think Einstein was aware of it. At least historians are saying he was unaware of it. So even in the absence of experimental evidence for his hypothesis, he was kind of able to come up with this concept. I mean, that's pretty amazing.

[Brian Keating]

And of course he had great competition from none other than Henri Poincare. Sticking with your countrymen, I think, Jan, not knowing you for more than 40 minutes now, but I think you're a deeply closeted physicist. I mean, so am. But I want to bring you out of the closet. It's safe now. So you've said something incredible recently that I just can't resist because it is literally in my field. And that's that you compare self supervised learning to the dark matter of AI. And much, much like dark matter in physics, it's essential we know it exists or we think it exists, although there are some apostates I'll get into, if you're interested. So we know, I mean, neutrinos are a form of dark matter. We know it exists. We know they're not sufficient to account for the observed amount of missing matter. But what did you mean by that? What did you mean by dark matter being analogous to self supervised learning?

[Yann Lecun]

That remark actually is eight years old now. I made it many years ago at a keynote. And in the audience was my former colleague Kyle Kranemer, high energy physicist from NYU at the time, who's not Wisconsin. And he said, you should not have used dark matter as the analog. You should have used dark energy, because that's really where most of the mass is of the universe. So I was trying to explain the following analogy, that the bulk of what we learn, we don't learn by being told an answer, or we don't learn by trial and error. We just learn the structure of our sensory inputs through self supervised learning or something similar to this. We don't actually know what humans and animals use, but it certainly feels a lot more like self supervised learning than it feels like either supervised learning or reinforcement learning. Right. So supervised learning is situation where you have a clear input and a clear output, and you train the system to just map that input to that output. Right. Show a picture of an elephant, tell the system that's an elephant. If it says it's a cat, correct the parameters so that the output 10 comes closer to elephant. Right. That's supervised learning. And then reinforcement learning is you show it an elephant and you wait for the answer and you just tell it whether the answer is correct or not. You don't tell it the correct answer, you just tell it whether it's correct or not. Maybe with a score of some kind. Okay, and now the system has to search among all possible answers, which one is the correct one? If there is an infinite number of answers, it's super inefficient. So reinforcement learning is so inefficient that it cannot possibly explain the type of efficient learning we observe in humans and animals. Supervised learning cannot possibly explain either because most of what we learned we're not taught, we just seem to come up with it. Right? And certainly animals, there's a lot of animal species that become really smart without ever meeting their parents. A good example is octopus, but there are plenty of examples in birds and various other species. So they learn a lot about the world and they never meet their parents. So they're not being told anything, they're not being taught anything. And then there is this sort of amorphous thing that we now call self supervised learning. And that's really where the bulk of learning really takes place. And if anything, the success of LLM really is a sort of bright demonstration of the power of self supervised learning. So I use an analogy where I showed a picture of a chocolate cake and it said the bulk of the cake, the genoise of the cake if you want, is self supervised learning. The icing on the cake is supervised learning and the cherry on the cake is reinforcement learning. If you want to quantify the relative importance of the different modes of learning, that's the right analogy. And when I was saying this in 2016, the entire world was completely focused on reinforcement learning. Reinforcement learning was going to be the path towards human, human level AI. And I'd never believed in this. And so that was kind of controversial. It's not anymore. And so then I said, there is chocolate in this bulk of the genoirs of the thing. That's dark matter. Yeah, that's the dark matter of AI. That's the thing we have to figure out how to do. And it's kind of like we're in the same embarrassing situation as physicists where we know how to do reinforcement learning and supervised learning, but we don't really know how to do this self supervised learning thing that represents the bulk.

[Brian Keating]

Hey cosmic explorers, it's time for some astro trivia. Do you know the difference between a constellation and an asterism? There are only 88 official constellations and the last one was added way back in 1930. But I have over 900 ratings of into the Impossible. And while you can't make your own constellation, you can make an asterism of 5 stars. A collection burning bright enough to make Orion's belt jealous. So do that on Apple podcasts. Scroll down to ratings and review. Tap the five star button and leave your thoughts. Or on Spotify, follow our show, tap the star rating. Don't forget to listen to all episode if you want to leave an actual rating. And please don't forget to follow or subscribe to the show wherever you're listening to this the matter that you and I are made up of these chunks of rock, which I'll give give to you when we finally meet up someday. These are meteorites from the early universe, are from our early solar system. I give them away to anybody who has a edu email address at my website. The point is, this is very important. You know, people say, oh we, we're, we don't Even know what 90, you know what's 80% of the matter is in the universe. But you know, the 20% that we do know about is extremely important and without which we can't have this conversation. And last week I talked to relative colleague of yours, Stephen Wolfram, and staying on the topic of dark matter, he believes that that dark matter unconventional idea that he has is that the universe is a hypergraph, according to him, that evolves via pure computational rules and that time is generated by the sort of update rate of the hypergraph. And he suggests that as time and temperature are related through laws of thermodynamics via entropy, he's actually suggesting that dark matter is what he calls space time heat. Not asking you specifically to comment on that. I actually don't fully understand it. We debated it because the question I had is can it? Okay, so there's the, there's dark matter that we know exists. There's dark matter that and we, we don't know what it is. It could be, it could be some strange new particle like the axion. It could be, you know, some new force field we don't understand. But there is dark matter that we know about. Absolutely. Neutrinos, 100% WIMPs, weakly interacting massive particles. So can space time heat account for neutrinos which are about 1.9 Kelvin today in universe? And so we kind of fought that out. But generally speaking, what do you think about this hypergraph idea that the universe is pure computation? Does that hold any interest to you as a researcher?

[Yann Lecun]

I don't know about the hypergraph idea specifically, but I can tell you I've been fascinated by the connection between physics and computation for a very long time. When I started my career at Bell Labs in 1988, all of my colleagues were physicists. The lab was a physics lab. I was the only sort of non physicist. I don't want to say computer scientist because my undergraduate degree is actually in electrical engineering and I did a lot of physics, but all my colleagues were physicists. And I had a brilliant colleague called John Denker who was in the office next to mine. And both of us were very interested in sort of fundamental questions in physics and how they connect with computation. We attended a couple workshops at the Santa Fe Institute, one of them organized by Wojciech Zurak, who's also into those questions. I don't know if you know any of his work, but there were people like Seth Lloyd, who was just finishing his Ph.D. at the time. We're talking 1991 or something like that, 1992. And people like Marie Gellman and John Wheeler. And so John Wheeler gave a talk. His talk was it from Bit. Right. That's the title of a series of lectures that he gave. And he says it's all information at the bottom. We have to figure out how to express all the physics in terms of information processing, essentially. And so I found this concept fascinating. And I've been sort of ruminating on this idea for a very long time. Not in concrete enough terms to actually write a paper on it, but some interesting ideas around this. Certainly one connection on this that I had is this idea of revers computation, right. Which of course has become kind of a big thing because of quantum computing now. But was it that popular in the early 90s?

[Brian Keating]

Jan, we also, speaking of physicists, I took questions from the audience and I just received one question via text message from my good friend, and possibly your friend, Max Tegmark. Are you willing to answer Max's. He has two questions for you that he texted to me. First one is, when do you, Jan, expect AGI, defined as AI that can do almost all zoom jobs?

[Yann Lecun]

I resisted the use of the phrase AGI. And the reason is not that I don't believe in the concept that AI system will eventually become as intelligent as humans. I certainly have no doubt that at some point in the future we will have machines that are as intelligent as humans in all the domains where humans are intelligent. There's no question this will happen. Okay, no doubt it's a matter of time, but calling this AGI is complete nonsense because human intelligence is incredibly specialized. We have a hard time kind of accepting this concept that human intelligence is specialized, but it is very specialized. That's why I don't like the term. The term I've been using is either human level AI or ami. So that stands for advanced machine intelligence. This is kind of the term that we use internally at Meta, we pronounce it abi. Because friend. There's a lot of French people. Okay. Also means friend, right? In French. But that's the same concept, Right? So now, how long is it going to take? Strangely enough, I get asked that question by people like Mark Zuckerberg. And the reason is it's an important thing to know if you want to invest tens of billions in infrastructure to train big AI systems. If you want to be able to tell people, within a few years, you're going to be able to wear those smart glasses that you were showing us initially. And in those glasses there will be an intelligent assistant that you can be with you at all time. You can ask Any question, it's going to be smarter than you, possibly, and you shouldn't feel threatened by that. It would be like having a smart colleague that you can talk to and ask any question. So how long is it going to take? So I think to have possibly a system that at least to most people feels like it has several intelligence as humans, if all of the plans that all of the things that we are imagining will work, okay, so those JEPA architectures and some other ideas that we're playing with succeed, I don't see this happening in less than five or six years. But now is it going to happen in five or six years? And I think there's a distribution with a tail that's very long. And the history of AI is that people just keep underestimating how hard it is. I'm probably making the same mistake right now. When I say five, six years, this is if we don't run into a major obstacle that we didn't foresee. If all of the things that we're planning to try out actually work, if things kind of scale, if computers accelerate and all that stuff, there's a lot of things, a lot of planets that need to line up for this to happen. So that's the best case. It's not going to happen next year. You might have heard from some other folks.

[Brian Keating]

Sam Altman. Yeah, right.

[Yann Lecun]

Yeah, Sam Altman, Elon Musk, various people, or Dario, Emily. Yeah, it's going to happen within the next two years or something. No, what may happen in the next two years is that it's going to be more and more difficult to find cases where common people will be able to ask questions to the latest chatbot that the chatbot would not be able to answer. But again, where is my cat? Where is my domestic robot? Where is my level 5 self driving car? Where is the self driving car that can learn to drive in 20 hours of practice without killing itself?

[Brian Keating]

I wonder how much you just, you know, not as an expert in this field, but just someone who's fascinated by it and has benefited. My life has just benefited so much because now, you know, I've got a bunch of kids and I don't read them, you know, stories. I ask, you know, meta to read them stories. No, no, I don't do that. But I don't think there's anything wrong with it. Morally, I feel fine, because if you're reading somebody's book, it's basically the same thing. But I think we're kind of arguing about stuff that's maybe the most analogous Thing I can, I can point to is like the Drake equation, like the Drake equation parameterizes basically a statement about optimism for detecting aliens. And it's based on a whole bunch of parameters. And those parameters are always given to us without any uncertainty. And you as a scientist and I know the most important thing are the systematic and statistical errors are simple. Systematic errors are hard. That's where the physics is. That's where the intuition comes in. That's where the craftsmanship comes in. So, but in these questions, so you always get numbers like, oh, there's abundant billions of civilizations in the universe, or there's none, depending on what you choose for your error bar, and likely too for AGI. It's such a nebulous thing. So people define in all different ways. I agree with you. I don't think it's true. But I think the Keating test, if I could be so bold, would be something like, come up with a new law of physics. Come up with a solution that makes a prediction that can be testable and falsifiable, that we can then say, this is never. This is truly new. It's not reproducing, it's not predicting. It doesn't have temperature dependent. So what would you say if you could have the Lecun test instead of the. I think the Turing Test was great for, you know, 100 years ago, but the analogy with the Drake Equation, Drake equation is like, who is. Who's talking to us? And the Turing Test is like, who's listening to us? But I don't think that's sufficient either one of those. What would you Say is a LeCun test that you'd be comfortable with?

[Yann Lecun]

Here's the bad news. I don't think there is any single test that would work. That's probably right, because any area or sub problems that you can formulate, there is probably a sort of specific solution towards solving that problem with superhuman performance. And we see this with computers. That's a history of computer science, right? Computers can calculate faster than humans now. They can translate thousands of languages in any direction, can play chess. A $30 gadget can beat you at chess, right? It can beat vhs, certainly. You know, a lot of those tasks that we came up with, like games, we came up with them because they're hard for humans and it turned out to not be that hard for machines, right? So like, you know, every search algorithm, like, you know, shortest path in a graph, things like that, that your gps, your map software uses, your map application uses. Those are fairly simple algorithms and they have Superhuman performance. So any particular application, Ari you pick, there's going to be a specific, specialized solution for it. And so no single test is going to test for intelligence. And what we're observing now is that people are being impressed by the fact that LLMs can manipulate language. And it turns out manipulating language is simple. It's much simpler than we thought. In fact, it has to be simple because it only popped up in evolution in the last few hundred thousand years. And given the difference between the genomes of humans and chimpanzees or something like that, it only represents like a tiny portion of the genome, if anything. Actually maybe a tiny, tiny portion, maybe the equivalent of a couple megabytes of genomic information, which is really not that much. And in the brain, language is handled by two tiny areas right here and right here, the BRCA area and vernicke area. BRCA area for producing language, Vernicke area for more, for understanding. Understanding. We get fooled into thinking those things are intelligent and generally intelligent because they behave a little bit like humans, but really they're very shadow. We see this when we try to build systems that can accomplish very simple physical tasks. And it's just excruciatingly complicated. They really can't. I mean, I don't think we have a good solution, although there's progress being made in robotics and stuff like that because of machine learning. But we're still not nowhere near where we need to be.

[Brian Keating]

Max asked second question. You can probably guess this, Jan. What is your plan for preventing loss of control over even smarter AGI?

[Yann Lecun]

Okay, so Max and I disagree on this, right? And we've been on various panels discussing this issue. I also disagree with Jeff Finton on this. Right. We're good friends, but we disagree on that question. So first of all, there is the sort of implied idea that if a system is intelligent, it wants to somehow take over or dominate. And that's just completely false. Not only is it false, it's not even true within the human species. It is not the smartest among us who want to be the chief. We have examples on the political scene. That's right.

[Brian Keating]

We won't talk politics.

[Yann Lecun]

The idea somehow that that intelligence is necessarily associated with a desire to dominate. It's just false to have a desire to dominate, or to or not a desire, but just even dominate by accident. There has to be some hardwire drive into the entity for competition, for resources, for example, or for influencing other entities to be able to profit from it or whatever. And this is a characteristic, the desire to Dominate is a unique characteristic of social species within biology, right? So you have domination, submission behavior in baboons, you have them in chimpanzees, you have them in wolves and dogs, you have them in humans. Although in humans there is another way to acquire status in human society, which is prestige. And you and I are both academics, so we're not influencing others by force, right? We're doing it by prestige, or at least we hope we do. But then take our tongs. Ourangutongs are not social, they are solitary. They're territorial, as a matter of fact. And they don't have any desire to dominate anybody because they don't need to. So this idea that there is an intrinsic desire to dominate, that is, as.

[Brian Keating]

A contractor, I don't pay for materials I don't use, so why would I pay for stuff I don't need in my mobile plan? That's why the new my biz plan from Verizon business is so perfect. Now I can choose exactly what I want and I only pay for what I need right now with my biz plan. Get our best price as low as $25 a line. Visit verizon.com business to get started today. Price per month with five plus lines.

[Yann Lecun]

Includes auto pay and paper free billing.

[Brian Keating]

And special intro offers, discounts, taxes, fees, economic adjustment charge and terms apply. Offers end June 10, 2025.

[Yann Lecun]

Linked with intelligence is just false. It's just false. Okay, so now there is the question as to, you know, how do you make sure that the drives of an AI system are going to be. The objectives are going to be aligned with human values and those systems are not going to destroy us by deliberate actions, but also not by accident. If you project, if you extrapolate the capabilities of LLM, then you might be entitled to be scared because LLMs to some extent are intrinsically unsafe. Now, they're not very smart, so it doesn't matter. But they're not controllable in the sense that the way they produce an output is not by optimizing an objective, is not by trying to figure out the sequence of actions that will satisfy an objective. They just produce one token after the other, autoregressively without thinking very much.

[Brian Keating]

Right, but why do you say that's not safe? I mean, my toddler does the same thing, but has no power, right? There's no embodiment, there's no controllability network that could allow her to actually do something to me, even though she might be stochastically crazy. Right? I mean, by an Adult standard.

[Yann Lecun]

She could be like sitting on your lap in front of your computer and then sort of randomly slap on the keyboard and destroy your entire file system.

[Brian Keating]

Right. But not launch nuclear war, right? She's not going to, no.

[Yann Lecun]

Because she won't have that power. But people don't have that power either. An AI system will not have that power either. Unless we're very stupid in the way we build that. What your daughter has is a set of drives that were hardwired into her by evolution, some of which have to do with just exploring the world and things like that to learn how the world works, basically. But some of them are very specific. So there is a drive for kids around when you're old to want to stand up, because that's the way you learn to walk. Right? So they're happy when they stand up. That's hardwired. There are hardwired objectives. So I'm not talking about hardwired behavior. I'm talking about the fact that you touch the lips of a baby and the baby will start sucking. Right? That's just hardwired behavior directly. There's like a neural circuit that just does that. I'm talking about hardwired objectives. So things that you're driven to do, but nature doesn't tell you how to do that. Like Edie. Right. I mean, nature doesn't tell you how you find food. You have to hear that's out by yourself. I mean, with the help of your parents.

[Brian Keating]

But Max's fellow Swede, Nick Bostrom, was on the podcast and he's famous for this paperclip problem. But I pointed out to him, you know, you can't produce infinite paperclips because there's only so much iron inside of the core of the Earth or on any asteroid or on any solar system. So it's kind of preposterous. They first have to have the agency, the desire, the target teleology, the purpose built. And I don't want to say it's impossible, because that's one of my rules. I don't say anything's impossible. But it seems like Max is obsessed with perhaps proving something which can't be proven. He's asking you provably safe. But what if that's proving a negative?

[Yann Lecun]

I don't believe. I mean, also, Stuart Russell also wants to is looking for provably safe AI system. I think that's just as impossible as a provably safe turbojet. You can't prove that a turbojet will be safe, yet we can build incredibly reliable turbojets Right. That can fly you halfway around the world in complete safety with only a two engine airplane. Right. I mean, that's mind boggling in terms of technology. But we can build those things. Like AI is going to be the same. There's not going to be a magic bullet, there's not going to be a proof that we can build safe systems. But we're going to engineer safe systems. And the way we're going to engineer them, I think that's the way I think things are going to go, is that we're going to build systems that are objective driven. I call this objective driven AI. And it's the fact that the output that is produced by the AI system is not the result of just producing a token after the other is a result of optimizing an objective with respect to a set of actions you take. So you have some mental model in your head, in the mind of the system. The system has a mental model of the situation it wants to or the environment it wants to act into. It's imagining a sequence of actions it's going to accomplish. And through the sequence of action and its mental model, it can predict what the outcome is going to be. And now you can check whether this outcome satisfies a set of objectives. So one of them is, did I accomplish the task that I set out to accomplish? Okay, but making a thousand paperclips. But then there might be other objectives that are more like constraints, and they would be guardrails. So you pay a high price for killing someone or hurting someone, for example, or maybe for taking certain actions that will consume too much energy or whatever. So, so you can imagine having a series of those objectives, some of which are guardrail, some of which are task objectives. And then the way the system produces its output is that it, through optimization, it searches through the space of action sequences for one that minimizes all of those objectives and guardrails. Now that's objective driven. Those systems cannot be jailbroken unless you break them. But you can't jailbreak them like you can jailbreak an LLM by giving it a weird prompt which will kind of go outside of its conditioning if you want. Right. So the system cannot be jawbroken. The only outputs they can produce are outputs that satisfy the guardrails according to their internal mental model of the situation. And now the game to make a safe AI is going to be how accurate can those mental models of the situation can be? And what guardrails do you have to put in to make sure that those things are not going to Go haywire and transform the planet into paperclips. And that's really easy to do. And we know how to do this for humans. We've been doing this for humans for millennia. It's called making laws. A law is a guardrail objective that tell people, okay, maybe the act that you're planning to do here seems good for you, but if you do it, you're going to go to jail for five years. Okay? So that changes your cost function, right?

[Brian Keating]

And for AI, we'll blow a circuit in the GPU that will cause it pain.

[Yann Lecun]

You know, humans can choose to ignore that guardrail, but an AI system that is objectively driven will not be able to ignore it by construction because it will have to optimize it so it won't be able to escape those guardrails.

[Brian Keating]

I always call it like the AGI of the gaps. It's kind of like this God of the gaps theory. They act as if. And I love Max, and he's been very kind to me personally and I've had him on many times. But. But the point is we're not just going to one day have AGI, right? It's going to be an iterative process. We already see it, right? We see Waymo, right? So right now, I actually saw Waymo in LA last week, but it was being driven by. It was very disoriented. There was a guy in the front seat driving this Waymo. I was like, what's the point of that? But I feel like, you know, if we started to see that, you know, like, so call that Generation Zero or full self driving, you know, Tesla, I've been in them, they don't drive on the sidewalks. You know, that would be the shortest way to get somewhere. You know, the Waymo could just, you know, drive through the sidewalk to get past traffic. It was, you know, busy. But it doesn't do that. So we'll see evidence if there are guardrails, literal guardrails, in the case of cars, say, driving off the rails, we're going to see that in iteration 0.1. And then, you know, it's not just one day. We're going to have Hal, and even Hal, I would say, like the first operating system wasn't built with an operating system system. Like, there's always precursors, you know, the first, you know, computer wasn't designed on a computer. I mean, by definition. So the first truly AGI is not going to be made by an AGI. And so it's not possible to have this recursion. But I want to give you the last word there before I turn to the final questions. I want to ask you about being a professor in this new age. But do you have any other thoughts about AGI? I have millions of more questions, but we'll have to wait till another part. I know you're very busy. Any other thoughts about, you know, the dangers or. That's the most common question I think you get. And my audience asked.

[Yann Lecun]

I have a very optimistic view. Right. I mean, I think intelligence is one of the most desirable commodity that we're missing the most in society really, to make progress. So I think the effect of having machines that amplify our intelligence effectively might be as transformative as the effect of the invention of the printing press in the 15th century and the dissemination of books, which allowed dissemination of knowledge, which actually gave a good reason for people to learn to read, for which there was no reason before, and then led to the dissemination of the Enlightenment science, democracy, freeing people, freeing themselves from the dogma of religion and becoming more rational and things like that. So I think that had a profound effect, right? It had. Had temporarily really bad effects also. Like the printing press allowed the ideas to be disseminated that caused people to kill each other essentially for 200 years in Europe because of religious dogma or something, but eventually really had brought down the feudal systems and caused the American Revolution and the French Revolution and things like that. So I think AI will have similarly transformative effect of amplifying human intelligence, assisting humans in accomplishing tasks that otherwise would require other humans. I think there's a bright future for humanity if we can do this right. And I'm not particularly scared of the kind of scenario that Max is talking about.

[Brian Keating]

So you think that AI will literally be more transformative for humanity than the Facebook poke feature of the early 2000s. But speaking of Facebook, so you're an academic. How do you think of yourself? Are you a professor? Are you a scientist? Are you an employee at a, you know, top three, you know, firm on Earth? How do you identify yourself if you're woken up in the morning by super intelligent AI and they asked you it?

[Yann Lecun]

I'm a scientist. I'm also, I would also say I'm an educator, not just because I'm a professor in university, but because also I, I do things like this, you know, trying to talk to the, the wider public about certain aspects of science, but I'm really a scientist. So I spent about equal time during my career in industry and academia. I started my career in industry at Bell Labs, which then became AT&T Labs and then I worked briefly at the NEC Research Institute and then I was in my early 40s when I became a professor. I'd never been a professor before and then for 10 years I was just a professor. And then. And after that joined Facebook. At the time created Fair. I ran FAIR for four years, so I was a research manager, I guess, and this was part time. I was also kept being a professor at NYU, basically 2/3, 1/3 more or less 2/3 Facebook or meta now and 1/3 NYU. After four years of running Fair, building it up and running it, I stepped down from running it. So now I'm chief scientist. I'm chief AI scientist at Meta and I'm what's called an individual contributor, which means I'm not a manager. I don't have any people, anybody reporting to me. I don't run anything. I hate running things anyway. I'm a terrible administrator, okay? I disorganize. I don't like doing it. It's torture for me. Did it for four years, but it's really torture. So I'm much more interested in sort of intellectual impact. So the effect I'm having at Meta is that I'm plotting a path towards human level intelligence, because that's the scientific quest of my life, if you want. But also it's like there's a product desire to have intelligent assistants in your smart glasses that have human level intelligence. So that's one of those rare cases where the lofty cosmic goal that you might have lines up with what people who pay you are willing to pay for.

[Brian Keating]

Right, everybody. I know that if you're enjoying these types of conversations, you're going to love my Monday Magic mailing list, where I explore the secrets of the guests that come on the show and other exciting facets from around the world of STEM, science, technology, engineering and math. And best of of all, I enter each and every one of you into a competition to win one of these little babies right here. A meteorite. Yes, that's right. A fragment of the early solar system produced in a cataclysmic supernova event which ignited with as much intensity as I have for the members of my Monday Magic mailing list. I know you're going to love it. So go to Brian Keating.com yt to join the mailing list and enter into the competition to get one of these beauties each month. But if you have a. Edu email address, you're guaranteed to win one. If you live in the United States, go to brianketing.com edu to get your fragment not of the metaverse, but of the real early universe. Now back to the episode. I could see how the video kind of technology that we're talking about earlier might make for better filters on Instagram. And I use, I use some of it, actually. I have a hack for you, Jan. I don't know if you've discovered this, but I think you're on mute. Workday is starting to sound the same. I think you're on mute. Find something that sounds better for your career on LinkedIn. With LinkedIn job collections, you can browse curated collections by relevant industries and benefits like Flexpto or hybrid workplaces so you can find the right job for you. Get [e-mail] jobs finding where you fit. LinkedIn knows how. I'm hoping that you haven't. So I can do something to impress you. You ever go on an airplane and wi fi is like $19 and it's a two hour flight? Or you can do messaging for free. Have you ever been in that situation? Did you know that you can use WhatsApp AI meta AI, your creations. You can use that for free. That counts as messaging for free. You don't have to pay for any WI fi and you connect to the Internet. Internet.

[Yann Lecun]

That's true.

[Brian Keating]

You knew that.

[Yann Lecun]

Damn it.

[Brian Keating]

I wanna, I wanted to impress you. Come on.

[Yann Lecun]

That's very cool. I don't think a lot of people realize this. Yeah, yeah, you can access the Internet.

[Brian Keating]

19 bucks to, you know, use some, some 1 megabit per hour Internet.

[Yann Lecun]

Anyway, don't say too loud because the, the airline companies.

[Brian Keating]

I know, I know. Well, yeah, luckily our audience is not as big as, you know, Lex Friedman's yet, but we'll see. What is Mark's, you know, interest in this? I mean, is it going to make, you know, is it just to make better filter, I mean, AGI versus what Facebook's. Meta's main products are WhatsApp, besides the chat, you know, Llama and then Instagram, Facebook messenger and Store and all the things that are so cool about Facebook. Meta. What marks interest? I mean, I'm very interested in what drives him at a scientific level perhaps because, you know, he's not trained as a scientist, obviously, but he, he has this vision of the future and it can't just be about consumption, right? There must be some other reason. What is his mission? What is his massive purpose in life?

[Yann Lecun]

His massive purpose is to connect people with each other. That's the entire purpose of Meta. It's just connecting people and it's very focused on this. But connecting people also means connecting not just people with each other, but connecting people with knowledge or helping people in their daily lives. Right, so the future that is the vision of the future is. You've seen the 2013 movie her, right? The Spark John movie, right? So there's this guy running around with AI assistant that is with him at all times, right? Either in glasses or things you put in your ears, or little things with a camera that you wear. And he is with the assistant at all times. I mean, that's a vision of a future where every one of us would be kind of walking around with, with super intelligent assistants working for us. It's like a business leader, politician or an academic working around with a staff of people working for them who are smarter than them, which is inevitable. I don't know about you, but I tend to hire people rather than me.

[Brian Keating]

It's incredible.

[Yann Lecun]

That's a good thing. I mean, people should feel empowered by that future. So if you want that, if you have that vision of the future and Meta has that vision, is building the devices for that, right? Then you need AI with human level intelligence. Essentially. It's a product need. If we had it today, it would have a huge impact. Everybody in the world will use it. We've done some. When I say we, it's the company as a whole. I'm not connected with them, but people have taken a few of those Ray Ban glasses to rural areas in India, giving them to farmers. They absolutely love it.

[Brian Keating]

Look at this and tell me what you see, tell me what's going on and, and imagining what it could mean.

[Yann Lecun]

I mean, they look at their plants and they say, well, this one looks diseased. You know, what disease is that? Or what is this bug? You know, do I, do I need to do something against them or, or like, you know, should I harvest now? What's going to be the weather tomorrow? I mean, all kinds of questions. And they can use. And they can do this in their own local language.

[Brian Keating]

I could talk to them and, you know, some Hindi or something like that and they could.

[Yann Lecun]

Well, Hindi is a widely spread official language, but most people in India don't.

[Brian Keating]

No, no.

[Yann Lecun]

Right, yeah.

[Brian Keating]

They have 900 different dialects there, right?

[Yann Lecun]

So I heard 700, but it's close enough. And then, you know, between 20 and 30 official languages.

[Brian Keating]

So as we wrap up, I just have two more questions. So the first one relates to what we do and I identify myself also as a, well, you know, first of all, as a father, husband, et cetera. But then as a scientist, and then as an educator as well. And I always like to ask the question, you know, Galileo lived my hero. He was Galileo. And not just because he has AI in his name, but, you know, he was a professor, he was a scientist, but he also had to make money. And so he would make telescopes only if he could sell the instruction manual, which was like the Sidereus, Nuncius and the dialogue and so forth. But I call what you and I do the, you know, the second oldest profession because there have been professors since the year 1000 in Bologna, Italy, and then Oxford, obviously, and then Sorbonne. I don't have to tell you. How do you see AI impacting it? Why should someone. Listen, I'll say for me, why should they learn cosmology, general relativity from Brian Keating when they can learn from this guy, Albert Einstein? What is the threat and the opportunity for me and my colleagues at. As professors?

[Yann Lecun]

I mean, it's clear that the profession of knowledge transfer is going to be transformed pretty deeply. And we might need to find sort of new economic models for research, for science, for scholarship, which plays an important role in society, which is not necessarily linked with education. Now, I think there is still the. But the whole idea of the PhD and the advisor, which is a bit like the Jedi Master and the Padawan, right? I mean, that kind of relationship, I think would still exist because it's not just the knowledge, right? I don't know about you, but I probably learn as much from my student as they learn from me. It's just a different type of. Of learning that takes place or communication of information. But the interaction, I think, is important in terms of behavior, ethics, what is interesting, what to work on, what's the practice of science and research. In a way, in the future with AI, all of us will be bots of a team of virtual people. You can think of it this way. And that would include business leaders, professors, I mean, anybody, right? Not leaders, just everyone will be like a person today with a staff of smart people working for them. That will be true of students, too. So your interaction with the student will not be with just the student. It will be with the student, augmented by the AI systems that the student has access to.

[Brian Keating]

I find it wonderful. I'm not threatened by it. I've tried to do, you know, even to. Even. I build telescopes and, you know, look for the heat left over from the Big bang. I'm still fascinated by it, and I actually think it's people now. We're at the beginning, right? Said that we're at the worst that AI will ever be. Right now, it's only going to get better. And I can do so much so quickly. And there's certain things you have to be, you know, kind of supervised. But. But it's, it's, it's. The thing that means so much to me is how pleasurable it is. I mean, talking with the glasses and asking about the physical world and interacting with it, but also learning. It has already an IQ of 120 in every subject on Earth, almost especially with the new Llama models. I'm so excited about those. But the last question I want to ask you comes from the namesake of this podcast. It was Arthur C. Clarke's statement that the only way to know the limits of the possible is to go into the impossible possible and transcend them. But he said something else. He actually said a couple of very funny things. He said, for every expert, there's an equal and opposite expert. So I use that on my department chair every now and then when I want to get out of a teaching. But he said the following, Jan. He said, when a elderly. I'm not calling you elderly, but I'm saying when an older but distinguished scientist, certainly you are, says that something is possible, he is very likely to be right. But when he says something is impossible, he is very likely to be wrong. I want to ask you, Jan, what have you been wrong about? What have you changed your mind about, if anything?

[Yann Lecun]

Oh, I changed my mind about a number of things. One example was I, you know, in the sort of early, early days of neural nets, you know, when I started interacting with Geoff Hinton. I did my postdoc with geoff Hinton in 87, 88. And at the time, I had negative interest in things that we called unsupervised learning. At the time, I thought this was ill defined. I didn't see the point of it. And Jeff thought this was the most interesting thing. He'd been working on Boston machines for which he got the Nobel Prize. Nobody uses Boston machines anymore. But that's beside the point. It was a very influential view, and it was mostly for unsupervised learning. And he had this vision that the bulk of learning had to be unsupervised. He was right about this, and I basically rallied to his, to his side in the early 2000s. So it took a long time from the late 80s to the early 2000s before I kind of changed my mind about this and then became a true believer in that thing. And they started kind of advocating for it in more explicit terms in the 2010s, but I was clearly wrong about that.

[Brian Keating]

Yann Lecun. Hey, Meta. How do I say thank you very much and good weekend to Yann Lecun. Merci beaucoup and bon weekend to you. Jan, this has been fascinating.

[Yann Lecun]

There's one thing I need to tell you because you were talking about telescopes, so. I don't build telescopes, but I do astrophotography as an amateur. I don't do science, but I take pretty pictures.

[Brian Keating]

I'd love to see it. Maybe I follow you on Twitter. Instagram. So maybe you'll post some stuff there. That would be wonderful.

[Yann Lecun]

I've posted a few pictures in the past. Yeah.

[Brian Keating]

Yeah. That's great. Yeah. I mean, the original astrophotographer is Galileo, who sketched out the feelings that he had about the universe, not just what he saw. Yann Lecun. This has been fascinating. I can't wait to meet you, Maybe in person when I come to visit the Simons foundation, the Flatiron Institute. It'd be a pleasure to meet.