B (3:38)

Well, I'm certainly not worried about research hitting a wall. Like, there's so many questions that we need to work on right now. And I'd separate it into two interesting angles, right. One is what are the right problems to be solving right now? What are the things that the models, the current generation of models we have can't do? Then there's a question of how do we go about it? What's the hypothesis that may give us the clue to how to solve some of these problems? In terms of what problems to solve, I think an important one is what do we do about memory? Machines have the ability to remember tremendous amounts of information. You're just stuck in there. The hard part is knowing when to pull on what piece of information to make a prediction, to generate information to reason. And so having this ability to be a lot more selective about all the information you've seen in context is super important. And already transformers were an important piece of that Attention is all you need. Well, it turns out it's not all you need. You need a little bit more than that. You need the ability to reason about information at different timescales, at different granularity, and so on and so forth. There's definitely a good piece of work to be done there, which really involves. And now we talk about the how, the choice of architecture, the choice of learning mechanisms, the type of data sets, the type of use cases that we need to look into. Another big research theme is on building world models. We hear a lot about world models, which are essentially the ability to take in all this information and predict the effect of actions. So when we talk about causality, how are actions transforming the world? This is what a world model should be able to do. Role models are absolutely essential when you want to build agents because these agents are going to take actions which is going to change the world. You want to be able to predict these effects. So whether you're building robots, and then we talk about physical world models, but also the agents getting deployed on the web, you need to build digital world models so that these agents, whether they're making financial decisions, communicating on your behalf, organizing meetings, that they have the ability to predict the consequence of their actions. So that's a big theme and there's a lot of different hypotheses about how to go about building these world models. And the third theme I'll highlight, and there's many more, but at least pick out the top three choice is about how do we build in reason efficiently. And right now a lot of the reasoning methods are still quite thorough based on sort of forward search methods and learning the right reward function. But I do think there's the transformer moment for reasoning and choosing action and being able to plan at different levels of granularity. We're still far away from doing that. And so there's all sorts of ways that it's being baked in LLM as a judge and things like that. Where AI systems give feedback to AI systems, order to train them is still very early days.

A (6:50)

Okay, I want to dig into a lot of what you just said. Let's start at the beginning. Let's start with memory. Is memory and continual learning two sides of the same coin? I mean, there's this idea that the models can search the web and they can find something in a session, but as soon as you close that session, they forget it. And I guess the reason why I'm going there is because a way that some people have suggested solving both of these is just making the context window massive and then just becoming efficient in the way that you navigate that.

B (7:19)

Yeah.

A (7:20)

What do you think about that hypothesis?

B (7:23)

The two concepts are related, but they're not exactly the same. And so memory is really about how do you address sort of what information to pull in in the context of the task you're trying to solve. Continual learning makes the assumption that the context keep on changing, therefore what you've learned keeps on changing. So there's a notion of non stationarity and that is really key to continual learning. I confess I have a little bit of trouble with continual learning as a concept because I feel the community has never been able to nail how do we articulate the problem in a way that we all agree on it. And so everyone who does work on continual learning takes a different flavor of it, which makes it, at least in my eyes, and I haven't worked a lot in this area, but makes it a little bit hard to know whether we're making progress or not. On memory. It's a little bit more standardized. The tension really is about. It's a question of efficiency and relevance. So the way to measure whether you're doing that is a little bit better standardized and you don't want to be just sort of remembering everything. And so it's a little bit better standardized how we articulate the tasks.

A (8:33)

Okay, let's go. We're going to touch on both of those now and then we'll keep going down the list with continual learning. Maybe I'm so far removed from it, I'm not struggling with it. So I'll give you my caveman thought about what this is and you can help us break it down a little bit. I mean, the problem has been articulated that the models, they don't change as they go about all these. I mean, think about how powerful it would be if, let's say the GPT model, which is speaking with 800 million or maybe more by the time this comes out. Million people a week, could inter. I mean, might be scary actually, but could internalize those conversations and learn from the discussions that it's having. That would almost. You know, I agree with you that the wall. We're not at the wall, but the question is, is there going to be enough data to keep making these machines smarter? And, you know, as they have these conversations, that opens up that ability to continue to grow and learn. But the model stays static despite all the fact, all these conversations that it's having with people. Isn't that the problem?

B (9:34)

I mean, don't get me wrong, right? Like, I absolutely believe we need to address the fact that these models need to keep on evolving. I have no doubt about that. I just mean right now, the progress in the research community that's working on continual learning isn't necessarily connecting to the work that's going on, on scaling now. The models that are released, right. You know, they keep on evolving. I would say, you know, the generative models we have today, whether it's ChatGPT, whether it's Gemini, whether it's our, the command models that the Cohere team is building, these models keep on improving. It's just we don't necessarily let them improve online, but we ship at different times, like a release of a model which has a particular characteristic. The advantage of doing that, frankly, is you can really test the model before you put it out there. You can put it through its paces in terms of performance, in terms of safety and so on. And I would be a little bit reluctant to just let the model keep running on its own because the learning can go very, very fast and you can switch out of a mode that seems completely reasonable very quickly, which, you know, we have seen a few times in the past.

A (10:40)

Yeah. I think we might be thinking about one of the same instances when Microsoft had this bot called Tay.

B (10:45)

Yes.

A (10:46)

I'll tell you a story. I actually broke the news of Tay, that Microsoft had this great bot, spoke with the people, wrote the first story about it when I was at buzzfeed, which is exciting. I pinned it to my Twitter profile. I went to sleep on the west coast and I woke up with all these messages being like, hey, that chatbot that you wrote about, the fun teen chatbot is actually espousing Nazi ideology. You might want to unpin that tweet. And it was because it kept learning. So, okay, maybe continue learning, you know, if it's done, because it has to also be done with some sort of fine tuning where you want to make sure that behavior, maybe it's preemptive fine tuning.

B (11:22)

Well, let's not release continual learning until we've achieve continual testing.

A (11:26)

That sounds like a very reasonable plan. All right, Memory. What makes it so difficult? I'll tell you one story. My Friday co host and I, with my Friday co host, Ranjan Roy and I, we both went into Gemini on Google and on Gmail and we asked, can you find the first email that I ever sent with my wife?

B (11:51)

Okay.

A (11:52)

Couldn't do it.

B (11:52)

Yeah.

A (11:53)

Is that because there's. Is it just because there's so many emails in there that actually, like, applying AI to. To try to figure out, like, what conversations have been had. Is that difficult or is it kind of a product problem from Google? Like, where. Why is memory so difficult? And how. How are we going to end up? Like, how is the research community going to tackle this?

B (12:16)

I mean, it's a little bit difficult to diagnose just from your description. I feel like I'm a little bit like a surgeon who's on the phone hearing the description of the patient.

A (12:25)

Have you asked Chachi P. About your symptoms?

B (12:28)

So I won't necessarily venture a precise diagnosis for your case, but nonetheless, I don't Think it's that difficult to figure out? I mean, I'd have to know what information is the bot pulling from, just in terms of visibility and privacy. Did you give it access to all of the information it needed to answer that? And that's the first one. And we do a lot go back to what we're building at Cohere, actually. We do a lot of deployments on site, so sometimes it's just a question, like we didn't activate the access to the right information to do it. So you need to figure out whether that access to the right information is there. And there's all sorts of reason that you may not want to give the bots access to all of your information all the time. So that's one practical consideration. The other one is retrieving the right information. And so did the query match how the information was encoded? Because in most of these, you may not want to just leave the information in raw form. It gets very expensive. I mean, you're one person, but at the scale that some of these companies are operating, you have to compress it, which we often called embedding. So you create embeddings of this representation, and so it may not have embedded the information properly. And then there's like, retrieving that information. And maybe it retrieved like 10,000 different items and didn't rank this one close to the top. And so it didn't generate the right response. But it could be that it knows of it, it just didn't show up at the top. So there's like a few different reasons, which makes it hard. One of them is the access to the information. What is encoding that information then? It's like, retrieve the information at the right moment.

A (14:05)

But when this stuff works, it's pretty magical. I was just in Claude, actually, and I noticed that Claude's memory capabilities have really improved. I was speaking, so I love to upload the transcripts of my interviews and, like, just, you know, get. Get a grading out, like, give me a rating on a variety of metrics. You decide. I tell the bot. You decide.

B (14:25)

Do you agree with the ratings that the bot is giving you?

A (14:29)

Definitely.

B (14:29)

Okay.

A (14:30)

Usually.

B (14:30)

Well, you've trained it well.

A (14:31)

I did. So some are good, some are bad. I actually had Gemini do a bunch of ratings, and it was like five, five on all categories. And I was like, that is wrong. And then I went to ChatGPT and Claude, and they were actually much more reasonable about it. But one of the interesting things that Claude did when I asked it this week, it started comparing it to the other interviews I had done.

B (14:53)

Okay.

A (14:53)

And it said, you know, you actually hit better points on this one. And this is why this one didn't resonate, in my opinion.

B (15:02)

Did you benchmark it with a sample of your audience?

A (15:06)

That's probably the next. And it'll probably. Because I'll take data out of the podcast analytics and drop it in these bots, it's going to be able to cross reference. So when it works, it's magical. And you've identified this as one of the areas where AI research really needs to concentrate. And this is the cutting edge. How good can this get? And what do you think? Do you think that it's at a moment of real progress or is it sort of party tricks to be able to get Claude to be able to do the things that I talked about.

B (15:38)

The question on rating specifically, like analyzing the information and sort of distilling some.

A (15:43)

Feedback more about the memory, the fact that it can call back memory in particular?

B (15:49)

No, I mean, we're making good progress on that. You know, extending the context length is kind of the easiest way to go about it, but there's quite a bit of progress that is being made on this.

A (15:59)

Okay, let's talk about reasoning. You mentioned reasoning as a cutting edge moment. The problem is efficiency. Is that really the issue here? I mean, so reasoning is the model basically goes step by step. It tries to answer, checks the answer, tries a different answer, then eventually decides, okay, this is probably what they want, and then it spits something out.

B (16:24)

Yes. I mean, that roughly happens this way. I think the challenge is really being able to plan at different levels of sort of temporal granularity. Right. So in terms of how you execute actions, let's say, you know, you're planning a trip. Right. You're not going to start by thinking of like, what are the shoes that I put on to go on my trip? Right. You're going to start by talking, thinking like, roughly what season, roughly what part of the world do I want to go visit? You start from the top level and then you take it down a notch, which is like, okay, you've identified a rough time, a rough place. Let's get more precise on the time and the place and maybe the activity and maybe who you want to go with. And then you take it down another notch, and that's when you start booking your reservations and so on. But sometimes you'll hit a blocker on the reservation and you can't get the flights or the hotel you want, and then you'll pop back up and say, like, do I change my dates, do I change my place, Do I change who I go with? I'm not going to bring the kids because then we can have more options so we can pop back up in terms of level of resolution. That's the part that the reasoning models don't do. They do really well at one level of granularity. So you've got a robot, you give it all these motions for the hands, the body motions. It can plan essentially to control the motors at that level of granularity. But the going back and forth between different levels of sort of resolution of action, it's really hard. So on the technical terms, we call it hierarchical planning. That's really hard to do that. Decomposition and keeping the information relevant as you go back and forth.

A (18:07)

Is that just a limitation of the large language model? Because the fact that an LLM can even do this in the first place, like, again, like it started with predict the next.

B (18:16)

They do it at the word level.

A (18:17)

Right, right.

B (18:18)

And out of the word level you do get the higher. The higher level. It is really impressive. I think that's the part that probably shocked a lot of people. They expected at, you know, back in 2023 or so, they expected that as you're generating tokens, you're not going to be able to generate sort of big ideas or a bigger plan. And yet it's pretty remarkable that it does it, which is why you get sort of, you know, different opinions in terms of some people's thinking, like, hey, like, it's already impressive. Like, like, let's just keep on pushing that way of doing things and we will unblock this. And other people being a lot more skeptical that you'll achieve it.

A (18:55)

Explain that a little more. So as it's typing, as it's. I mean, I think Andre Krapathy basically explained that the transformer is a computer and every time you generate a new token, you're going through a piece of computing. So the more you type, the bigger the computer is that you use?

B (19:10)

Yes. I mean, the more information goes in and the bigger your representation is.

A (19:14)

Okay, but are you saying that as this happens, the computer is effectively already thinking ahead? I'll give one example. Claude, just to go back to some anthropic research, they published this amazing research where they asked Claude to write a poem. And as it's writing the first line, it's already activating features in the model that's thinking. What rhymes with that? Yeah, which is amazing because again, it's technology that predicts the next word. But as it's predicting the next token, it's already thinking the next sentence, which to me is just mind boggling.

B (19:49)

Yeah. And so, I mean, this is why to some degree the emphasis on code and the ability to build representations of code and generate code is so interesting. Because when you look at code and for people who program before, the code has that structure, that hierarchical structure it's encoded in. Anyone who looks at a bunch of code, even if it's not necessarily a language you understand, you understand the notion of functions and variables and libraries and so on those different levels of granularity of the project, it's encoded in there. There's some hope that by training enough on code, the machine essentially infers these kinds of structural c. Fascinating.

A (20:36)

So, I mean, like you talked about the fact that this technology, and this is sort of the thing that sort of makes my head explode a little bit, the fact that this technology is able to do these things that you wouldn't think, given the architecture it is supposed to do. Same with. If you think about video models and image models, and by the way, one of your former colleagues, Yann Lecun, would always talk about how to generate, and I know he has some criticisms of video models. But to be able to generate AI video, you really have to be able to predict and plan what's going to happen in the physical world.

B (21:08)

Absolutely.

A (21:09)

And there's some embedded intelligence that even leading researchers, I don't think fully get that. When you, for instance, ask a model just to use Jan's favorite example to drop a pencil, there are so many permutations of where they can go. And now the models without, like, I mean, without having lessons of physics understand that it drops and maybe hits the table and might bounce up.

B (21:32)

Yep. Because it's seen enough data from objects that are dropped that have these kinds of behavior, but try to predict what's the behavior of a similar object dropped on, you know, on a different planet, and probably the prediction is wrong because all of the data was taken with our gravity constant.

A (21:50)

Yes. I mean, I will say as I'm talking about this, I did just see a video generated where a man's fingers came out of a styrofoam cup as he was holding it. So there's room to come, there's a.

B (22:01)

Lot of room for improvement.

A (22:03)

Now there is some talk Demis Hasabis was on recently talking about how Google's video models in some way have capability like these world model capabilities. They do understand the physics. And you brought up world models as another area where this technology really has the potential to grow. It's the cutting edge, still kind of undefined. I will say. And you know, going back to the caveman here, I'm a little bit confused about why. For instance, like one of the examples that you brought up earlier was that if you want a model to be able to like go out and like complete financial transactions and understand the implications of financial transactions, it has to know how the world works. But can't you just teach that in text? Can't you teach it like, if you use my credit card and you know, buy anything online, I will go bankrupt, like in, in text or even number logic, and therefore don't do it. Like, why does, and I think world models is like you, these models need to understand gravity. Why does a model need to understand gravity? To learn these basic rules of sort of the way that the world works.

B (23:07)

Well, and this is why earlier I sort of distinguished between like physical world models and digital world models, right? It's, it's possible that you can actually build really effective agents, web based agents that don't understand the concept of gravity. And it's possible you can build physical world models for robots that don't need to understand the functioning banking system. And so you can define the world as being like a contained environment. But if you want to deploy the agent on that environment, then it does need to understand the roles of that, that environment quite well. The challenge is getting enough coverage of data for all the possible futures, right. And all the different ways that the world could evolve subject to various events happening. So a lot of the cases today where it's actually most beneficial is where there's like a place for the human at the table. And I'll give you an example, right? People talk a lot about using chatbots for customer service, right? Like chatbots should be, you should just like plug them in, they will answer, answer all your questions, they'll be available 24, 7 and so on in reality. And there will be of course, many chatbots deployed for these kinds of cases. But one of the use cases we've seen that works really well is actually to have the bot pull together all the relevant information. You do customer service, you pull together all the relevant information from many different sources. As opposed to following a script being just chatbot, pull together all that information about the documents, the documentation that accompanies the system, the case on the client, the different problem, that description that you have, you pull all together that, and then you pose a diagnostic, and then you pose a few suggested actions, and then you keep a human in the loop to validate the plan and to carry out the action. And so that means that the human can, and these are more complicated cases than just your cell phone plan or something like that. But nonetheless, in those cases, what would have taken a long time, maybe half an hour to pull together all that information, distill it for a human. Now you can reduce that down to a 20 second analyze, verify and carry out the action. So if you have that ability to combine the human and the AI agent, actually you get often some much more powerful results. And it means if your world model isn't complete, humans in the loop, they figure out the pieces that's missing, they give that extra information, and then you bring that information back to train your agent. Then you get continual learning. There you go, it's there, we're getting there.

A (25:48)

Do you buy that the models need to understand gravity for AGI to be reached? I mean, there are basically like a couple schools of thought that you could basically train AGI on bits and, you know, letters and stuff like that, images. And then there are others that believe, you know, you really need, you really need these models to understand like, you know, not just the rules of poker, but like what happens when a person puts their hand on a poker table. What do you think?

B (26:17)

Yeah, I mean, I tend to actually place my bet not on the fact that we're going to reach like a single super intelligent agent, but on the fact that we are much more likely to live in a future where there's going to be many agents for many things. And so some agents will absolutely need to understand gravity. If we're going to have physical robots that are moving around in the world that are going to be hitting objects, that are going to be picking up objects and so on, they will need to understand that other agents that are dealing, for example, with our digital life may not need to understand that. And we also need to have a protocol for these agents to interact with each other and to talk to each other. I actually think that's a much more likely scenario. Rather than have the uber agent that needs to understand everything and have a fully encapsulated world model.

A (27:13)

There's a popular thing that AI lab leaders have been saying recently. They've been talking about how there's a capability overhang, how the AI technology can do a lot more than it's being used for. Do you believe that?

B (27:27)

Absolutely, yeah.

A (27:29)

Say, say more about it. Talk about what do you think is not being done that could be done?

B (27:33)

I see it every day. I mean it. I'll I'll open up a little window. Like one of the reasons I was super excited about joining GO here is because it's one of the few places that you know that we have a team that does research. So I get to see, see day to day what's happening in research. We have a team that does modeling, so I get to see the models that we're building, look at the evaluations, a full spread of evaluations. And we have a product, that product is an agentic platform that is going to real clients. So you get to see the whole thing and I see something that our models can do and I see some things that we've built into the products and then we go and there's a lot of customers that are not using the full functionality for all sorts of reasons. So I think between what we have in terms of capacity versus what's being deployed right now, there's a big gap between that. Sometimes the reasons are capacity questions. We talk a lot about superintelligence, big models. In reality, paying customers want a good trade off in terms of performance for efficiency. So we'll train bigger models, but we'll deploy smaller models because it gives us that trade off that like good enough intelligence to get the job done. And I'm like, well, we could give you so much more. No, it's good enough. So. And it's a perfectly, you know, rational position for them to be taking. So some of that is for efficiency reasons. Some of that gap is also because you're going into organizations which have systems and processes in place. And sometimes there's like a mismatch between what those processes are set up to do today versus what would be, I think, a more welcoming environment for an AI agent. So there's these kinds of things and then the other one is often I think there's a lot of intelligence that is not encoded. So the agents go, they plug into a bunch of internal system, they leverage all the business intelligence with privacy, security consideration, they leverage all that information. But sometimes there's big pockets of information that we're not leveraging right now. And if we did, if we connected into that, then we would be able to do a lot more. That impedance mismatch in terms of the information sharing from the organization or from the individual to the AI is another case where leaves a lot of machine intelligence on the table.

A (29:58)

So we're going to talk about enterprise in a moment, but let me ask you one question about how this applies to consumer. Sure. Obviously we talked about a lot of technology and the vision Is there within the big tech companies to have a, like a universal assistant, something like an Apple Intelligence or an Alexa? Plus, you know, both of them have rolled out in their own way, but both of them. And I guess, you know, Meta has their own product, Google has their own product. None of these are lighting the world on fire. Do you think that is. Is this another example of a capability overhang or is it that the technology is just not there yet?

B (30:34)

I think both are true. I think people are expecting basically been promised superintelligence. They are expecting magic out of these AI systems. It is not magic. And so I would say there's a big gap between expectation of what they can do today. Then there's also a mismatch between what people try to do versus what might be the strength of, of these agents. I compared a little bit. You know, you're working in a team, you get a new teammate in like day one, you may not know exactly what this person is capable of, not capable of. And it takes some time working together. And sometimes that person gets a lot better when you give them a lot more information. And sometimes you discover they have a new skill that they didn't have. But at the end of the day, you know, often that person isn't able to do everything everywhere all at once. And so I think there's both. Both these things are true at the same time.

A (31:29)

Yeah. There's also, I mean, a lot of corporate politics. I just wrote this, of course, I wrote this story recently in Big Technology talking about how there's like these two basic. And actually you're in a great position to talk about this or give us the real story here. From my vantage point, there's basically two trajectories that a lot of companies are on. The companies themselves have. I'm not talking about your customers, but if you take about like companies overall, many of them have struggled to put this technology into place, but individuals are starting to see the benefit. So you actually have like these companies with these pilots that are not getting into production, but then you might have somebody, you know, lower down, using Claude code who's like actually getting it done.

B (32:12)

Yeah.

A (32:13)

So what do you, what do you think about that and what do you think it means if we end up seeing that divergence?

B (32:18)

I think that is absolutely true. We see this all the time, even within our own companies. Yes. People's ability to leverage the technology varies a lot. I mean, the reality is we are moving towards a world where there's going to be more and more of that technology. And so the people who have the ability to understand and leverage the technology are going to have an edge.

A (32:39)

Okay, I agree. All right, last question before we take a break and go on to some more of the practical applications, some of the more coherent stuff. I still can't wrap my head around the fact that the AI labs are so close together in terms of the technology they produce. One builds some innovation, the next has the innovation. One seems like it leaps ahead, the next seems like it leaps ahead. Can you envision a scenario where one of the labs just kind of hits on something and, and can actually open up a lead against the others? Or is it just gonna be neck and neck forever?

B (33:20)

I think it's really hard to keep ideas in a box, especially because in many ways these ideas, they reside in people's heads. And I mean, you've seen as much as me the movement of people between these companies. Like they're always ping ponging back and forth. They carry the ideas with them, even if the code stays on one side. Once you've seen some insights, you can't unsee it. And so they may need to reimplement, they may need to articulate in different ways, they may give it a different name, but ideas just circulate. You can't keep ideas in a box. And that's why, honestly, for many years I've been so much an advocate for open science. I just don't believe that you can keep these ideas boxed in unless you're willing to keep people boxed in, which we are not willing to do. And so I don't think we have a way to close the ideas. We should embrace the fact that when you let the ideas circulate, all of us progress faster.

A (34:22)

Then the question is, let's say all these labs do reach superintelligence. That's been asked. Well, you can't hoard it, so where's the economic value in developing it?

B (34:33)

Yeah, we're still very, very early days in the technology, and we're even earlier days in terms of what are going to be the dominant economic mod. What is going to be the right business strategy in the age of AI? I think we need to give ourselves the time to experiment. Now we have 30 years or so perspective on the Internet and the economic impact of that, and it's going to take a number of years before we figure that out. But often those who develop the technology are not necessarily the same as those who scale the technology versus those who actually commercialize it versus those who actually control it and regulate it. So there's a pretty complex ecosystem that is all gonna arise out of that.

A (35:17)

Okay, well, at the other side of this break, we're gonna talk about some real economic impact of this technology already talk a little bit about what Cohere is up to and then we'll cover a lot more. So we'll be back right after this. Starting something new isn't just hard, it's terrifying. So much work goes into this thing that you're not entirely sure will work out, and it can be hard to make that leap of faith. When I started this podcast, I wasn't sure if anyone would listen. Now I know it was the right choice. It also helps when you have a partner like Shopify on your side to help. Shopify is the commerce platform behind millions of businesses around the world and 10% of all e commerce in the US from household names like Allbirds and Cotopaxi to brands just getting started. With hundreds of ready to use templates, Shopify helps you build a beautiful online store that matches your brand style. You can also get the word out like you have a marketing team behind you. Easily create email and social media campaigns wherever your customers are scrolling or strolling. It's time to turn those what ifs into with Shopify Today. Sign up for your $1 per month trial@shopify.com big tech go to shopify.com bigtech that's shopify.com bigtech you want to eat better, but you have zero time and zero energy to make it happen. Factor doesn't ask you to mu meal prep or follow recipes. It just removes the entire problem. Two minutes. Real food done. Remember that time where you wanted to cook healthy but just ran out of time to do it? You're not failing at healthy eating, you're failing at having an extra three hours. Factor is already made by chefs, designed by dietitians and delivered to your door. You heat it for two minutes and eat inside. There are lean proteins, colorful vegetables, whole food ingredients, healthy fats, the stuff you'd make if you had the time. There's also a new Muscle Pro collection for strength and recovery. You always get to eat fresh. It's ready in two minutes. No prep, no cleanup, no mental load. Head to factor meals.com bigtech50off and use code bigtech50OFF to get 50% off your first Factor box plus free breakfast for one year. Offer only valid for new Factor customers with code and qualifying Auto renewing subscription perks Purchase Make Healthier eating easy with Factor and we're back here on Big Technology Podcast with Joel Pinault, the chief AI officer at Cohere. And of course, this is part of our Davos series that we're hosting at the Qualcomm House here in Davos and running over the weeks following. So, Joel, it's great to have you. Let me give you what I've gathered as the use cases in business for AI, and you tell me if I'm missing any, and then maybe what you think is the most valuable. All right, I wrote four down. One is external chatbots, the customer engagement type of chatbots, the type like Brett Taylor talked about at Sierra. The other is internal knowledge. So let's say a company has knowledge within the company and it's all fragmented, and maybe there's a bot that you can start to query internal knowledge. Third is papering over systems that don't work. I don't think that needs much more explanation. It's like the story of success.

B (38:26)

Skeptical about that, but still.

A (38:28)

And then the fourth is automation.

B (38:30)

Yeah.

A (38:31)

Am I missing any big categories as far as AI in business? And where do you think the real value or the biggest category is right now?

B (38:38)

I think there's, like, different ways to slice it. I think that's a perfectly reasonable way to slice it. I think another way that I've seen it sliced is between predictive AI, generative AI versus agentic AI, which is like a whole other level of opportunity. And then the other way I've seen it sliced is more by application domains. Right. Like whether it's what AI is going to do in health care, what AI is going to do for scientific discovery, what it's going to do in banking, what it's doing, for example, public sector, and so on. So that's the other way that people have looked at the different classes of opportunity.

A (39:18)

And so what do you think the biggest is?

B (39:24)

There is so much potential, I hesitate to pick one. I will say, you know, quite frankly, where Cohere has placed its chips. And the core hypothesis is on the case of enterprise AI that needs really high privacy and security guarantees. I think there's a big cluster of applications, which falls a little bit in the second category that you outlined, where you have a lot of internal business intelligence information, perhaps fragmented, you want to be able to leverage all of that information to empower your employees. And so in that case, especially when that information is something that you don't want to pop up on the web through an API, there's an opportunity to build agentic systems that work in house with the local data, that inform the employees, and are essentially like close partners to the Employees.

A (40:17)

Can you give me like a use case or a case study?

B (40:21)

Yeah, I mean we do a lot of work for example in financial services because as you can imagine a lot of that data is quite sensitive in terms of information. Very concrete use cases we're seeing is for financial analysis. So you know, we have people whose job it is to advise various clients and they need to pull on diverse set of data like what's the, you know, what's the information that's relevant to this particular customer, what's the information that's relevant in terms of like the current landscape, the possibilities and so on and kind of pull all of that information to make up like a personal plan. A financial plan for a client is the kind of application that this technology can make much easier and you can essentially then query your plan, decide do I have enough information, do I need to gather more sources of information? And you can combine the internal with the external information but the output of that stays private, it stays secure, it stays in the hands of just the people who need to see that information.

A (41:18)

I'm glad you brought that up because I was asked recently by someone in the financial service industry what's going to happen to entry level employees who were doing a lot of that, you know, collating and pulling in the external information. Yeah, and I didn't have a great answer. I, you know, because you know, you pay, you pay entry level employees less than your standard employees and you anticipate there's going to be some learning on the job and some productive things. And now the question is what are these people going to do if they can do it for them?

B (41:49)

If these entry level employees are able to use AI properly, they're skipping ahead to the level where they can actually be fully functioning analysts and they can essentially do 10x the job with the tools. And so their growth in terms of their ability to deliver value to the employer has just been magnified by giving them the AI tools.

A (42:08)

So is then the threat really to the middle, the people who are mid career who are going to get, I mean it's like it's the old story of the social media intern who comes in and all of a sudden is managing like PR or marketing for a company, is it the Gen Z kid who like knows how to prompt and can use cohere and all of a sudden the person who's been doing things for 15 years in a certain way has to look over their shoulder.

B (42:35)

I do think that whenever you introduce a completely disruptive technology that is a lot of what you see, you See, the younger generation for whom that technology is native and is very intuitive and they really learn how to use it very quickly. And that just makes them so much more effective and productive. And folks who are not able to engage with a technology as quickly are finding themselves as a disadvantage.

A (43:04)

I just remember being early in my career and maybe this is why I didn't last very long in a company and had to go start my own. But, you know, having the energy and wanting to do things and you know, if I would have had something that could like build a prototype and I could bring that to the meeting and show it, as opposed to like, can I have a couple hours of the developer's time to work on this side project, that would change things.

B (43:28)

Absolutely. And to be honest, that capability is afforded to anyone in the company. It's not just the more junior staffers that have access to it. It's also the people who are in leadership position which instead of writing out a memo, suddenly can go out and produce a full fledged prototype. They don't need 10 people, 10 staffers to help them produce their prototype. They have an idea, they can quickly prototype it and they send that to the team to get moving with a project. So I think that kind of capability is going to open up new ways to set up projects across the organization.

A (44:07)

This cloud code thing has been interesting to watch. It went overnight from something that will auto complete developers code to we'll go out on the Internet and do things and build things to accomplish specific tasks. So is this idea of AI systems going out and doing things like on one hand, you know, I hear, I see the story of like, and I've said this on the show a couple of times, but like, you know, the former Amazon CEO of Worldwide consumers going out and vibe coding a CRM over the weekend, you know, that's, that's cool. But I'm also just like, you know, how real is that? So I'm curious. Oh, okay, you're giving me a look like, yes, it is real.

B (44:48)

Well, I think that goes back to my idea, right? Like, those who are able to prototype in this way, it doesn't mean that whatever you vibe coded into a weekend suddenly turns into $100 million business. Right? But it's a way to communicate with your teams, your intention. So as long as you have good ideas, you're able to share these ideas in a way that's much more real and to start prototyping much faster. Now, there's other ways to communicate your ideas. There's other ways to direct your Teams, but that suddenly opens up so, so much more.

A (45:18)

It is interesting how AI is. AI is many things, but it's a communication technology.

B (45:23)

It's becoming that. Right. And this is the kind of thing that the new coding agents are opening up.

A (45:32)

Does Cohere have like a version of this that it's working on?

B (45:36)

I would say like yes, we're working on the same kind of capabilities. We're building core generic models. I would say that that's. It's a bit of a different experience right now that we're offering in terms of the north platform. But there is a lot of that sort of collaborative work. There's a lot of this like going out and essentially deploying agents, leveraging external, external information. So there's some elements that are similar, but we're less focused specifically on coding use cases for now.

A (46:06)

Cohere obviously has raised a lot of money, more than a billion dollars. But this is not. I'll just like draw it out. OpenAI sneezes that over a weekend. You have a world now where AI is being developed by a handful of very big companies. Your former employer Meta is a big player. Amazon, Google, of course Microsoft and then OpenAI and Anthropic. With these, they raise the entire like years worth of VC money in a round. Now what do you think about the risk of the fact that so much of this is being concentrated in so few hands?

B (46:50)

Honestly, I do think it's beneficial to the ecosystem to have multiple groups who are able to develop models and to deploy them. Just to give you a concrete example, Cohere was very early on working on multilingual models. So the ability to understand information, digest information across multiple languages, 20, 30 and so on languages. We have a line of AYA models that is really well respected, open sourced and so on. It's just not on the radar of some of these companies that are very focused on English centric information. Completely fine. Different space for different companies. When we get into markets in Asia, when we get into markets in Europe, suddenly it matters to have a model that is actually state of the art across languages or across the local language. And so that opens up completely new market. Right now the opportunities are so broad that actually there's space for up and coming players to really keep on growing, to have a very healthy revenue, to bring in talent, to actually build new things that are different from some of these other companies are building. So I tend to think it's super healthy to have more rather than fewer companies that are building AI. And I think we're seeing the fact that going back to my idea of many different AIs who do many different things, even at the company level, this is what's happening. There's a number of players who are building different things and learning from each other.

A (48:26)

But the fact that big tech has so much of it, not a worry.

B (48:30)

It doesn't worry me.

A (48:31)

Okay.

B (48:31)

And I mean, you know, we could have a much longer discussion about it, but it doesn't cause me to lose any sleep over the fact that, like, what we're building at Cohere has like an amazing path to be successful.

A (48:46)

Okay. By the way, I mentioned anthropic and OpenAI, which have Microsoft and Amazon and Google have massive stakes and there are many more. Yes, somebody who does. Wario. Dario Amodei from Anthropic. Well, maybe not the fact that he got all those billions from Google and Amazon, but he does have some things to say about the big tech companies. Here's a thing that he said recently. Some of these companies are essentially led by people who have a scientific background. That's my background. Stemmas, Hassavis background from Google, DeepMind. Some of them are led by the generation of entrepreneurs that did social media. There's a long tradition of scientists thinking about the effects of, of the technology they built and not ducking responsibility. I think the motivation of entrepreneurs, particularly the generation of the social media entrepreneurs, are very different. The way they interacted, you could say manipulated consumers is very different. So basically, I don't think he wants.

B (49:43)

Them running strong opinions from Dario, which.

A (49:46)

Is, I guess, not something out of character for Dario Amonte. Do you think that's a legitimate concern? Because it's so interesting. You're a research scientist who also worked at a social media company. So if anyone knows the answer to this, it would be you.

B (50:01)

I mean, I think what's really important, like, no one is going to be good at everything, right? The question is like, how do you get others in the room to advise you on how to build something great? And I spent some time at Meta, I would say there was a very strong channel from researchers to the leadership team and the opinions were brought into the room. I think I've seen that certainly at Cohere, where the research team, the modeling team, the product team, there's a room where all these points of views can come together. I go back to this thought. You can't expect one person to have all that information. And as long as they're building up the teams that are diverse, that are listening to these diverse voices, like they will build Better products at the end of the day.

A (50:53)

Okay. On that note, as ads have started to enter the picture for generative AI, there's a wonder among outsiders like me about whether these companies will do things like engagement max and try to optimize for time spent so they can get those numbers up. I don't want to ask you whether you think that's going to happen or not, but I want to ask you as a researcher whether that's even economically feasible. Are the models now efficient enough where like a vis, let's say you were to show an ad a visit to just like to serve that visit with an LLM could be a profitable thing. Or is it still so expensive to serve these use cases that this even notion of engagement maxing doesn't make sense? Because economically it's not valid?

B (51:46)

I mean, in general, right. Like through trial and error we, we find economic models that are viable. Right? Like that's still how it is. So it depends a lot on the pricing model and so on and so forth. And so it's expensive ads to buy. But you know, it depends, you know, it depends on how the model is set up. So I don't know that this is the way that that gets rolled out initially. We'll have to see what, what's the progression of that. I do think, you know, we have the ability to tailor content based on the information we have that is there. That is a lever that's going to continue to be used from an economic point of view.

A (52:18)

AI sovereignty before we go. Countries are starting and institutions like banks are starting to build their own models or they're not relying on off the shelf stuff. So talk a little bit because this is something Cohere is working on. It's something I don't know a lot about. The fact that there is this push or at least it's something that is being discussed. So what is AI sovereignty and how is it playing out?

B (52:43)

Yeah, sovereignty has been used in a few different ways. In some cases it means the ability to have your own model. So in the case of financial services and banks, that is definitely something that they spend a lot of time investing in, thinking about, looking for solutions. They see the opportunity. They were, I think, early adopters even of, you know, previous generation AI technologies, predictive models, for example, statistical models and so on. And so they see this as the natural evolution. So they're pretty advanced in terms of their sophistication and their readiness for AI. And often they have the means to invest in it. And so we're definitely seeing A lot of interest there. Often, though, you know, I think the talent gap makes it a little bit harder for them. So sometimes they've tried to build their own models and so on, and they come to us and they're looking for solutions that are a little bit more mature, out of the box and so on. And so we have really solid partnerships going on there. The other way to think about sovereignty that we're hearing a lot is that companies want a robust plan for AI, and so they want options. They may be using one model, but they actually want to have another model to be able to compare to benchmark. If one model access gets cut off or too expensive, they have another one. And so there's an aspect of sovereignty that's really about building a robust strategy. It's not about just using your own or using one thing, but it's about having control over the access to the technology.

A (54:18)

Yeah. As you speak about it, to me, it's just amazing how fast this has moved. And going back to our first meeting, 2022, the fact that we're. It's 2026, so it's been three years and change, but it's. It's just a world of a difference year to year to year.

B (54:33)

Yeah.

A (54:34)

So, last question to you. Can the pace keep up?

B (54:38)

It is still moving very fast on so many fronts. You know, just the size of the investments. I think on adoption, we are so early in the curve. And so that's. That's going to be the next challenge to see. How do we. How do we enable this technology to sort of disperse through. Through society, through the. Through the business world, in people's lives, and how do we do that successfully? But, yeah, I think the pace, especially when it comes to commercialization and adoption is really very, very early days. So got a long way to go.

A (55:14)

Seriously. Well, Joel, we've spoken a handful of times. I always appreciate how you're able to take these big things that a lot of us are wondering about and grounded in the research and the practical side of things. So you're always welcome on the show, and thank you for coming on.

B (55:27)

Thank you. Always a pleasure.

A (55:28)

All right, everybody, thank you so much for watching and listening, and thank you to Qualcomm for having us here at the space at Davos. And we'll see you next time on Big Technology Podcast. All right.

B (55:40)

Thank you.

A (55:40)

That was great. Thank you so much. Thank you. Thanks, everyone.

B (55:49)

Foreign.

A (55:58)

Did you know your credit card points and miles can lose value to inflation? Credit card companies often reduce the redemption value of your points. And miles. Now imagine a credit card with rewards that can grow in value. With the Gemini credit card, you can earn Bitcoin or one of over 50 other cryptos instantly with no annual fee. Every swipe at the store or gas pump earns you instant rewards deposited straight to your account. Visit gemini.com card today. Check out the link in the description for more information on rates and fees. Again, if you're looking to invest in Bitcoin but don't know where to start, the Gemini credit card makes it easy. Issued by Webbing, this is not investment advice and trading. Crypto involves risk. Check Gemini's website for more details on rates and fees.

B (56:44)

Honestly, I put off building a website way longer than I should have. Then I tried Wix Harmony and it was way easier than I expected. I just described what I wanted and I had an incredible looking website. The best part? I could change anything myself or ask my AI agent for help. I had everything my business needed right there. So if you've been procrastinating, this is your sign. Start building a website for free@wix.com Harmony.