C (3:17)

Yeah. And so the piece opens. So first, everybody can now see, you know, this reference to the great Christopher Nolan movie Memento. So the piece opens. Sorry.

A (3:31)

With a twist on machines. With.

C (3:33)

With a twist on machines. So, so why, why, why did we open the piece with this metaphor? You know, what, what is the kind of like memento like experience of. Of, you know, AI models today? And kind of like, what. What is the frame that. That we used here?

A (3:53)

Yeah, absolutely. So I guess I'm not sure if everyone in the audience has seen mem. If you haven't, here is kind of like a quick blurb. But basically the main protagonist, Leonora Shelby, has a form of this amnesia where he cannot form new memories. So he goes about his life with kind of like this cut of the. After which point he has kind of these long term memories, but really cannot retain anything new that he experiences. And so what he does is he uses the sticky notes where he writes some of the notes to himself. He pulls out his Polaroid camera to capture the moments as he goes on about his life. And I mean, he even tattoos some of the memories that he wants to imprint in his memory, basically throughout kind of the movie. And so why does this matter? Right? Like, this is just kind of like an explanation, but why does this matter? And AI models, like, honestly, it kind of maps one to one to how AI models work today. So we have the training phase where we basically encompass all of the world knowledge. And that part is what we call pre training. And after the training phase, we basically have the cutoff date, after which point we deploy the models into the world and we call that inference. And so the question is, honestly, the model is basically frozen, but the new experiences, new knowledge still persists. So how do we go about scaffolding it today? We augment it with agent harnesses. We use retrieval mechanisms like rags. We also even, like, the example that I like to give is we have the system prompt that essentially serves as a tattoo. It's basically kind of like all of the scaffolding that we build around the models to really learn through feedback and through use. So this post is honestly kind of like an opportunity to dive deeper into the topic, really discuss why continual learning matters.

C (6:04)

Yeah, so we're living right now in this paradigm of in context learning. And basically the thrust of this piece is like, okay, we need to move into kind of like a continual learning paradigm, but you Call out like a couple of different companies, you call out cursor, you call out openclaw. These teams that are kind of doing income context learning things in that modality that seem to be pretty effective. So where do you feel like they still fall short and where do you feel like we still need improvements?

A (6:44)

Any honest argument about continual learning pretty much has to start with in context learning because it genuinely works. We see that with examples like Karpathi's outer research project. Kind of like the other examples we give in the article is open claw. Like the underlying model was available to anyone. But what's really made it a special magical moment is this kind of like orchestration of the context, right? Everyone has the access to the context. But here OpenCloud really utilizes your file system. It creates kind of memories, right? And it even has like a special bash access. So honestly, the skeptics would say why complicate things, right? These naive but janky interfaces really tend to work and they will continue to win because they're so fundamental. And so that's where we spend a lot of time thinking about it with Mat. And the example that I really like is given by Yu San, who is currently at Stanford. It's an example from mathematics. You probably have heard about Fermot's last theorem. That was pretty much an open problem for over 350 years. And the thing is, there were a lot of mathematicians that tried to tackle it. All of the literature, researchers, like research papers were available to anyone. But what Andrew Wiles did, he basically went into near isolation for seven years and had to invent new techniques to bridge basically two fields of branches of mathematics, elliptic curves and modular forms. And please don't ask me about that because that would be above my pay grade.

C (8:32)

But we're not going to get into cryptography here, don't worry.

A (8:36)

But basically the learning here, it's like a true genuine discovery that you really could not have even learned from all of the information and like whatever pre training data that humans kind of had before. Yeah, yeah, I guess. Just also to ground it a little bit more, I want to give a few concrete, practical examples. And the first one is essentially in adversarial security. Like imagine there is a new jailbreak attack. You have your model deployed in the wild and it's being used. Imagine you try to update your system prompt to say like, don't do this, like it's not going to work, right? Because all of the parameters in the model have learned to be helpful to the users. So you really have to encompass that kind of knowledge in the weights where the attackers don't have access to. Right. The attackers have access to your context just like any other user. And so you have to use something else like the weights to really tackle it. And the second example is like, the second example is imagine your favorite JavaScript library, let's say React. You learn through all of your pre training data that there is a function called X. But at some point a new version of React comes out and turns out that it's a breaking change and all of a sudden X function doesn't exist. It's now a Y function. No matter how much you say it in the context, you cannot just override what's the most intuitive throughout all of the model parameter parameters to basically say X. And so those are kind of like the examples. And so the question is not whether in context learning works, the question is whether that's kind of the ceiling.

C (10:26)

Interesting. So you do something really, really helpful in this piece, which is you lay out where exactly learning happens or you call it actually compaction in the piece and maybe you can explain what compaction is. But I think maybe a good thing to do now would be to kind of like talk through this chart and then also talk through just, you know, who are some of the teams that are, that are really building in the space and what their different approaches are across the entire, you know, like non parametric to parametric spectrum.

A (11:01)

Totally. Yeah. Yeah. I guess like the question is whether it's not that we have the memory features or not, it's more about where does the compaction happen 2.0. So we make this very high level framework in terms of just the three buckets of the context, the modules and also the weights. And the distinction that like the one call out that I think is important is all of these are learning mechanisms. And even in context learning is still a form of continual learning. It's just not where we focus the most energy in the article. But context is essentially what we call non parametric learning where we don't actually update the weights. And that's what people are most familiar today with. So that's exactly what you think about RAC companies like Pinecone, companies that build agent harnesses, memory scaffolding like Leta MEM0. And there's definitely a lot of, a lot of activity here. The main defining factor in how things are currently thought about in this specific field is the limitation. And the limitation here is like you have a limited amount of context length and so how do you utilize it in the most Efficient way is kind of the main question here. And then we have kind of like the middle ground of you want some access to weights, you want to update it, but is there kind of a middle ground where you can learn and update your weights in some adaptable, smaller manner where you don't update the entire model? And so this is kind of the middle ground. And there is a great Stanford paper on this called Cartridges that kind of explains how you can update KB caches. And again, the energy where we spend in the article for the most part is on the weights and kind of like the parametric learning. That field is honestly still kind of in early states. So we have a bunch of companies that are doing kind of the RL data and systems, and some that are basically saying that transformer architectures are the bottlenecks and we need the novel architectures. And so we kind of list a few. But it's just worth calling out that in all of the labs that we talk to, even the labs don't just tackle one approach. They actually have multiple teams that tackle continual learning through the different kind of paradigms.

C (13:29)

So I think given that we have about five minutes left here, I think it might be good to kind of chat through some of the conclusions that you come to at the end of this piece. So you sort of suggest that we might even need to redefine what a model even is. So would love to kind of hear in your own words, what does that mean and how should we think about that?

A (13:56)

Totally. I mean, the models are currently frozen after the cutoff date, like what we mentioned before. But I think I come back to just what Ilya talked about just recently, and what he said was basically like, with AGI, we almost overshot the target. Yeah, we humans are not AGI, but we still learn on the job. We learn from experience. And that's what makes kind of humans kind of unique. And so that's kind of like the ultimate test. Like, how do we define that we got to continue learning is like, well, is there a system that is able to learn on the job and get better through use, just like humans? I think that would be kind of the question. And by the way, there are researchers from Berkeley and some of the other labs that are actually working on benchmarks that will hopefully help us define what is continual learning in a better form.

C (14:46)

And you, you also, I mean, you kind of get at the. You, you just got at this already. But you talk about, you know, models getting to this place of having, you know, glimmers of experience and, like, Is there some kind of concrete milestone for you that would tell you, like, you know, this. We, we. We have, you know, like, seen something that is just so far beyond what current AI models are capable of. Maybe it's, you know, a novel discovery or maybe it's just, you know, the ability to work for longer and longer and longer amounts of time without, you know, messing up in some way. Like, so. So would be curious to kind of hear. Hear what? What your kind of heuristic. Is there

A (15:31)

something that I briefly mentioned just now is honestly kind of on the job learning, learning from experience when you are deployed, whether you can get better. And the test that some people use currently is pretty simple. You basically have you train a model that is learned on XYZ data, and once you deploy, you just want to check whether it learns something out of distribution, something that it hasn't seen before. And we are starting to. You see some examples like the test time training done by Yu San with the Discover paper that kind of makes some of the novel inventions. It really changes kind of like the shape of the models to adapt and learn on the job to tackle specific problems.

C (16:17)

This is the paper.

A (16:18)

Yeah. Yes, exactly. Learn to discover it this time.

C (16:23)

Yes. All right. Homework. Homework for everyone who's. Who's watching at home is. Is to read this paper, you know, so do you have. Do you have any, you know, closing words, Malika? Do you, you know, like, should. Who should be reaching out to you right now to talk more about this? What kinds of people do you want to hear from?

A (16:46)

Anyone who is interested in continuing learning, especially founders. We would love to just chat with you. Otherwise we will be stuck in Memento

C (16:56)

in our own perpetual present. Yeah, exactly. We do not want to spend the next couple years of our lives writing on sticky notes, tattooing ourselves, you know, making videotapes, all of the crazy things that happen in Memento. Well, thank you so much, Malika, for hopping on the show. This has been a ton of fun. I hope we got to write a ton more pieces and, you know, like, you got to come on all the time. This was a ton of fun.

A (17:29)

Looking forward. Thank you. Thank you.

B (17:33)

Thanks for listening to this episode of the A16Z podcast. If you like this episode, be sure to, like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes, go to YouTube, Apple, Apple Podcasts and Spotify. Follow us on X16Z and subscribe to our substack@A16Z.substack.com thanks again for listening and I'll see you in the next episode. This information is for educational purposes only and is not a recommendation to buy, hold, or sell any investment or financial product. This podcast has been produced by a third party and may include paid promotional advertisements, other company references, and individuals unaffiliated with A16Z. Such advertisements, companies and individuals are not endorsed by AH Capital Management, LLC, A16Z or any of its affiliates. Information is from sources deemed reliable on the date of publication, but A16Z does not guarantee its accuracy.