The Next Gen AI Models: Reliable, Consistent, Trustworthy — With Aidan Gomez

Summary

Big Technology Podcast: The Next Gen AI Models – Reliable, Consistent, Trustworthy with Aidan Gomez

Hosted by Alex Kantrowitz | Release Date: October 30, 2024

In this insightful episode of the Big Technology Podcast, host Alex Kantrowitz engages in a deep conversation with Aidan Gomez, CEO of Cohere and co-author of the seminal paper “Attention is All You Need.” This paper introduced the transformer architecture, a cornerstone of modern generative AI. Together, they explore the current landscape of AI, dispel prevalent myths, discuss the return on investment (ROI) in AI technologies, and forecast the future trajectory of artificial intelligence.

---

Investment and ROI in AI

Alex Kantrowitz kicks off the discussion by addressing the substantial investments flowing into AI, citing OpenAI’s recent $6.6 billion raise contrasted with reports of a $5 billion annual loss. He raises concerns about the sustainability of such investments given the escalating costs associated with data, compute, and energy required for training advanced models.

Aidan Gomez responds thoughtfully, emphasizing the long-term value of AI technologies. At 02:06, he states:

“I think those numbers are actually small relative to the long-term value that the technology will deliver. It is now time to prove that.”

Gomez highlights the transition from the proof-of-concept phase to actual production and widespread adoption, particularly in enterprise settings. He notes that Cohere is witnessing significant growth as businesses integrate AI into their operations, promising substantial ROI in the near future.

---

Dispelling Myths About AI

The conversation delves into the myth that next-generation AI models will be "godlike." Gomez dismisses the notion of Artificial General Intelligence (AGI) achieving superhuman capabilities imminently. At 04:25, he clarifies:

“I think the idea that we're building AGI or something, that's just going to solve all our problems for us. I think we need to set that aside.”

Rather than anticipating a sudden leap to AGI, Gomez envisions a steady improvement in AI reliability, trustworthiness, and competency. He anticipates continuous enhancements without expecting abrupt, transformative breakthroughs.

---

Scaling AI Models: Compute, Data, and Beyond

Alex probes into the scaling of AI models, referencing the exponential increase in GPU usage—from Meta’s Llama 3 requiring 16,000 GPUs to Elon Musk’s Project Memphis deploying 100,000 GPUs. He asks Gomez about the implications of such scaling.

Aidan Gomez acknowledges that more compute leads to better models but warns against excessive scaling without practical deployment benefits. At 08:32, he explains:

“If you're building a massive model, it's not actually useful for the world if it's too big to be consumed, if it's too expensive to actually deploy.”

Gomez advocates for balancing model size with usability, ensuring that advancements translate into tangible, accessible tools for users.

---

Artificial General Intelligence: Goals and Realities

The discussion shifts to AGI, with Alex referencing differing interpretations of the term. He distinguishes between AGI as comparable to human intelligence and superintelligence. Gomez aligns with a practical definition of AGI as intelligence matching human capabilities in various tasks. At 10:39, he shares:

“With that definition of AGI, I think it's both achievable and a fairly reasonable target. So we can measure how good humans are in any particular task and create technology that can perform that well.”

Gomez contends that AI will augment rather than replace human workers, enhancing productivity and creating new opportunities rather than causing mass unemployment.

---

Enhancing Work through AI Assistants

Exploring the role of AI as reliable assistants, Alex references OpenAI’s Zero1 reasoning model and its potential to evolve into more trustworthy tools. Gomez emphasizes the importance of models being able to reason and self-correct, enhancing trustworthiness. At 15:16, he states:

“It [reasoning] is a crucial piece in improving not only the accuracy or robustness or usefulness of the model, but also the trust in the model.”

He envisions AI evolving into collaborative partners that assist daily tasks, thereby transforming the nature of work.

---

Addressing AI Risks and Safety Concerns

Alex raises concerns about AI potentially causing harm through autonomous decision-making. Gomez reassures listeners by emphasizing human oversight and safeguards. At 16:35, he asserts:

“We have to plug them in and we have the opportunity to implement safeguards. So to make sure that before these models are put in any very high stakes situations, there's oversight that a human has to approve high stake actions.”

Gomez dismisses doomsday scenarios portrayed in media, advocating for thoughtful and controlled deployment of AI technologies.

---

Emergent Behaviors and AI Capabilities

The topic of emergent behaviors in AI models emerges, with Alex questioning whether Large Language Models (LLMs) can exhibit behaviors beyond their training data. Gomez clarifies that while AI can interpolate between known skills, it does not spontaneously develop unconstrained abilities. At 19:42, he remarks:

“I've never seen a model behave in a totally unexplainable way. They're really good interpolators.”

He dismisses the notion of an intelligence explosion, suggesting that AI’s self-improvement capabilities plateau rather than perpetuate indefinite advancement.

---

The Role of Synthetic Data in AI Development

Gomez discusses the limitations and applications of synthetic data in training AI models. At 22:08, he explains:

“Synthetic data probably doesn't get us out of that issue. Actually, I don't know if synthetic data outside of easily verifiable domains like math, it's hard to use synthetic data to drive outcomes.”

While synthetic data aids in areas like mathematics and coding, human expertise remains crucial for complex, nuanced fields.

---

AI in Enterprise: Real-World ROI Examples

Shifting focus to ROI, Gomez highlights Cohere’s integration with enterprise clients like Oracle. At 29:09, he shares:

“We're powering over 50 different applications within those software tools. And so it's actually starting to get into the hands of employees and drive efficiencies.”

Examples include automating job description creation, supply chain management, legal contract review, and healthcare data analysis. These implementations demonstrate significant time and cost savings, underpinning the economic value of AI in enterprise settings.

---

AI’s Impact on the Workforce

Addressing concerns about AI replacing jobs, Gomez emphasizes augmentation over replacement. At 39:56, he states:

“It's very assistive actually. So it's less about replacement, it's more about augmentation. What everyone's building are tools to augment their workforce to make them more productive.”

He envisions AI handling mundane tasks, freeing humans to engage in more fulfilling and intellectually stimulating work.

---

Partnerships and Integration with Cloud Providers

Discussing the role of cloud providers, Gomez notes Cohere’s collaboration with major platforms like Amazon and Azure, while also supporting on-premises deployments for regulated industries. At 40:32, he mentions:

“Cohere has had a long time focus on, on prem as well, because for a lot of regulated industries like finance and healthcare, a lot of that data doesn't actually go on the cloud.”

This dual approach ensures versatility and compliance across diverse enterprise needs.

---

Future Outlook: The Next 2-5 Years in AI

Looking ahead, Gomez predicts a continuous evolution of AI assistants becoming more capable and integrated into daily workflows. At 42:06, he forecasts:

“In the next two years, I think we're going to start to see really compelling assistance. It won't just be little convenience functions or small features. It'll look a lot like a partner that you do work with.”

Over the next five years, these assistants will evolve into trusted collaborators, deeply embedded within various systems and processes.

---

Personal Insights from Aidan Gomez

In a reflective segment, Gomez shares his astonishment at the rapid adoption and impact of the transformer architecture. At 27:26, he expresses:

“Even if I step away from being one of the authors of the paper, the impact and what the architecture has been able to do for the field has been a huge shock, a colossal shock.”

He credits his co-authors and acknowledges Google’s pivotal role in integrating transformer technology across its platforms.

---

Conclusion

The episode concludes with Gomez reiterating the immense potential of AI to transform industries by automating repetitive tasks and enhancing productivity. He underscores the importance of thoughtful implementation and the collaborative future of human-AI partnerships.

Alex Kantrowitz wraps up the conversation by highlighting the understated yet profound impact of AI in enterprise settings, challenging the perception that AI’s value lies solely in consumer-facing applications.

---

This episode of the Big Technology Podcast offers a comprehensive examination of the current state and future prospects of AI, grounded in the expertise of one of the field’s pioneering figures. Listeners gain a nuanced understanding of AI’s economic viability, practical applications, and the realistic scope of its capabilities, steering clear of both unfounded fears and exaggerated hype.

Transcript

[Alex]

An author of the original paper that launched the generative AI revolution joins us to sort out the technology's myths and facts and when we'll see a return on all that investment. All that and more is coming up right after this.

[Michael Kovnat]

Hey, I'm Michael Kovnat, host of the next Big Idea Daily. The show is a masterclass in better living from some of the smartest writers around. Every morning, Monday through Friday, we'll serve up a quick 10 minute lesson on how to strengthen your relationships, supercharge your creativity, boost your productivity, and more. Follow the Next Big Idea Daily wherever you get your podcasts.

[Tomer Cohen]

I'm Tomer Cohen, LinkedIn's chief product officer. In my new podcast, Building One, I interview some of the best product builders out there. People at the intersection of dreaming and building and learning. Together, you and I will learn from their experiences. If you're just as curious as I am, follow Building One wherever you listen and check out the conversation on LinkedIn.

[Alex]

Welcome to Big Technology Podcast, a show for cool headed, nuanced conversation of the tech world and beyond. We have a great show for you today because we're joined by Aiden Gomez, the CEO of Cohere, an AI platform for enterprise. And he's also the co author of the famous attention is all you need paper, which invented the transformer and started this whole AI thing. Aiden, great to see you. Welcome to the show.

[Aiden Gomez]

Thanks Alex. Thanks for having me.

[Alex]

Great to have you here. I want to begin with some myths and facts about AI. We have debates all the time on the show. Where's the technology going? Is it worth the investment? And no better person to ask than someone who was there at the beginning and now currently an entrepreneur in the space. So earlier this month, OpenAI raised 6.6 billion, but the reporting says they might be losing 5 billion per year. And you know, in some ways, okay, you need the investment to build the models. But in another way it's like, okay, you know, where does this end? Because the compute, the data, the energy to train these models keeps getting bigger. The requirements for that keep getting bigger and so does the money. And you know, does this ever become sustainable? So what do you think?

[Aiden Gomez]

I certainly understand the urge for people to see the numbers being spent on training and be concerned that it's not going to recoup in value. But I think that those numbers are actually small relative to the long term value that the technology will deliver. I think it is now time to prove that. So last year was very much the year of the proof of concept. People were getting familiar with the technology. It was their first time working with it. And so there were a lot of small tests and experiments. But this year is very much one of going to production and getting these models into the hands of people at scale. Of course, we're already seeing a high degree of ROI in the sense that there's now hundreds of millions of people who are using the technology. It's actually in their hands. It's part of their day to day. And so that certainly is roi. And with what Cohere focuses on the enterprise side, we're starting to see this technology get into the hands of employees and get into enterprises. It's a much slower process. It's a bigger lift. You have to integrate with existing systems within enterprises. You need to train employees on how to use this technology. But that's well underway now and we're seeing quite dramatic growth in adoption. So I think we will find ROI and it's coming soon.

[Alex]

And we'll talk more about the specifics of Cohere and ROI in the second half. But let's keep on this line because it goes to another one of these myths and facts, which is that the next set of models are going to be this godlike set of models. And you know, you talked about how there's going to be like a lot of cost at the beginning. Right. And that's necessary cost to train these massive models. And the sense that I've gotten, and from my reporting, one of the things I've heard is companies have been willing to make those investments because they think that this next 18, 12 or 18 months in model development is crucial and the capabilities will advance significantly as they put more compute data and energy into the process. So let's go to this myth, in fact, number two, which is, does the next set of models give us that, you know, God, like AI model? And so many people I don't know.

[Aiden Gomez]

About God, like, I don't think I'd ever use that term to describe what's coming. I. I think we're going to have some really powerful and useful tools emerge. I think that's what's coming. The idea that we're building AGI or something, that's just going to solve all our problems for us. I think we need to set that aside.

[Alex]

I don't actually think we'll get to that. But let me ask you more pointedly on this next generation of models. Okay, so you say we're going to see some new tools. What does the next generation. Because they are being trained on much more resources than, than they have been previously. So what tangible step forwards are you expecting to see from the next generation of models?

[Aiden Gomez]

I think the reliability and trust factor is huge. And also just the competency. Right. The accuracy with which it gives you answers. And so all of those are going to increase. I don't see a step change coming, but I see a steady continuous course towards very high accuracy, very high reliability AI.

[Alex]

There's been so much hype in the industry. This is one of the things that sort of comes along in this discussion, which is like everybody I speak to who's on the ground says, yeah, we're not going to see a step change with like, let's say GPT5, but exactly as you describe, more reliability, more consistency. Is there, I mean, is there worry that some of the air is going to come out of the, you know, this AI moment if, you know, because again, like I, when I say God, like I'm not. I don't believe that's going to happen. But I'm reflecting what a lot of the hype is starting to expect. And so if it's just steady, you know, steady improvements and reliability, which is actually like, you know, we both agree pretty big. But do you think that that sort of takes some of the steam out of this moment for AI because people will look at the step change as, as a failure, given where the hype is.

[Aiden Gomez]

Well, listen, I'm not one of the people who's saying we're going to be building godly. Yeah. I don't have much to say towards those claims. What I would say is that even if, just as a hypothetical, even if the technology froze and what we have today is all we get, there's so much good to be done, there is so much work to go to implement this technology across the economy, really boost productivity, drive better outcomes, build tools so the technology does not have to move in order for incredible value to be realized. We just need to go do it. And it takes a lot of effort and time and work to go realize that value.

[Alex]

Okay, and again, more of that is coming in the second half where we go a little bit more tangible. But let's stay with the theoretical or at least like the industry stuff. What do you make of the fact that the GPUs. So we talk about like the ingredients. Again, this is coming from your paper, right? The ingredients that are required for these models to get better, they need data, they need compute, need energy. And the compute right now is starting to go like through the roof in terms of the amount of compute that's being used to train models. So just for some context, so Meta's Llama 3 model, which was like state of the art, like 10 minutes ago, it used 16,000 GPUs to train that one. Now we're hearing that Elon Musk is building this super cluster, I think it's called Project Memphis, that has 100,000 GPUs, so multiples of what the cutting edge is being used to train on. So I'm curious if what you think that increase in GPUs are going to get us first and foremost. And then we'll talk about whether the right way to scale these models is with just throwing more compute, because I know you have a nuanced take on that. But first and foremost, if you go from 16,000 GPUs at the state of the art to 100,000, what do you think that delivers?

[Aiden Gomez]

It definitely delivers a bigger and better model. You have more compute. We know that scaling up improves things. There's questions around saturation and whether continuing to scale up is justified, whether there's going to be enough gains from that strategy to justify the increasing cost. My personal perspective is that building a massive model, it's not actually useful for the world if it's too big to be consumed, if it's too expensive to actually deploy. And so for cohere, we've been very focused on building the right size of model. But if your question is what is more compute unlock it will be a better model. Objectively, we know that scaling leads to more capability, a smarter model that's more reliable, and so that's the output.

[Alex]

And does that ever end? I mean, that's one of the big debates here is that basically you could add compute and data basically to infinity and it will continue to improve. Or is there a tipping, you know, sort of a saturation point?

[Aiden Gomez]

I don't think within any achievable scaling up for humanity that will reach that tipping point. It just saturates the gains become much, much smaller. And so you're much less willing to want to pay double the price for a minute difference. But it is pretty consistent that bigger is better and that just continues, but it tapers off over time.

[Alex]

I mean, OpenAI has talked about how like their goal is to build AGI. A lot of people in the industry talk about AGI's North Star. I know cohere is more like, let's make this practical for businesses, but I want to get your sense because that's not your Northstar. I think you can speak a little bit more about like more honestly about what it means and whether it's achievable. So do you think that let's just use this definition of AGI as intelligence that's as capable as humans in the tasks that humans do? Do you think that that is something that we should even be thinking about, or is it a marketing tool like, and is it achievable?

[Aiden Gomez]

I mean, with that definition of hei, I think it's both achievable and a fairly reasonable target. So we can measure how good humans are in any particular task. And then, yeah, I mean, it's a reasonable goal to want to create technology that can perform that well in that task. So I think based on that definition, I think when we start to think about, you know, you described it as like God like models, these are being described as, well beyond.

[Alex]

So, yeah, my definition is probably artificial general intelligence. And I think that this God like is the superintelligence thing that a lot of. And I think a lot of people will use AGI as a synonym for a superintelligence, which seems wrong to me. But there is this belief that once we hit AGI, we've already reached super intelligence, because if it can do everything that humans can do and doesn't get tired, doesn't need to sleep, doesn't need to get paid, necessarily, you're already at superintelligence, but. Sorry, go ahead.

[Aiden Gomez]

Yeah, but no, I think that definition of AGI is a reasonable one and I think it is exciting. I think that that's definitely the target. What we want to do is we want to create machines that have this unique property that humans have of intelligence, and we want to be able to deploy them in the places to take work off of the shoulders of people and put it onto these machines to make work better and easier. And in order for you to do that, in order for you to trust the machine enough to shift that work over, it better be as good as the human. Otherwise you're. You're paying some price, you're reducing inaccuracy. Things get worse, not better. And so that's a very reasonable objective.

[Alex]

And when do you think we might reach it?

[Aiden Gomez]

In many respects, we're already there in many fields. The models are still working, right? We're still like, yeah, but I don't think those two things are in conflict. I don't think that really.

[Alex]

Why not?

[Aiden Gomez]

Well, because I think that we will never see mass unemployment of humans. I think that this technology is going to unlock more opportunities. It will let us do more, as opposed to scaling Back what we do. Humanity is very supply side, constrained, not demand side. We want more, we want better, we want to be healthier, we want to do more, we want to have things be cheaper. And so we have all this demand and we're trying to keep up with our own society's demand. And this technology, its true promise is in bringing productivity and letting us do more. Now, you can zoom in and you can pick a specific field and you can say this field might be automated by AI. And I think that's true. And we should be thinking about retraining and shifting certain skill sets over to other new domains, like retraining people. But in general at the macro scale, I think this technology will create much more opportunity than it will take away.

[Alex]

I mean, if we have AI technology that can basically do work for us, whether it's knowledge work or whatever, right? I mean, we already have a lot of technology that can automate factory work. Why are we continuing to work?

[Aiden Gomez]

It brings purpose and meaning to a lot of lives and we enjoy it. I think that the right form of work is something that fulfills you and that is enjoyable, intellectually, interesting, compelling. And that's really what I want to spend my time doing. Is that as opposed to number crunching or. And maybe someone else enjoys number crunching, but for me, I'd rather outsource that to and far between my Excel spreadsheet. Right? So, yeah, I think that work in its best form is incredibly fulfilling. And that's something that humans will never give up. We'll always want to do that. But if we can hand off and if we can have an assistant that is on 247 and has access to all the information and tools that I have access to and I can ask it to do things, for me, that's a very compelling value proposition. It changes work in a way that is extremely positive, I think for almost everyone.

[Alex]

How far away do you think we are from having reliable assistance? Like, a lot of people looked at OpenAI's Zero1 reasoning model and they're like, oh, this is just kind of like a step toward assistive AI. What do you think?

[Aiden Gomez]

I think the notion of using reasoning or letting the model have an inner monologue to work through problems, think through them, make mistakes, but then realize that, catch mistakes and correct them, I think that's a crucial piece in improving not only the accuracy or robustness or usefulness of the model, but also the trust in the model because you're able to inspect how it arrived at its conclusions, how it decided to do what it did you actually trust it much more? It's explicitly written out. And so I think we've all known these sorts of tools would need to emerge. And yeah, I think it is a big step towards dramatically more reliable assistants, ones that you can trust and work with and give feedback to. I think it's really exciting.

[Alex]

Okay, and then where does that put you on the fear around AI? I mean, if AI can sort of go step by step, figure out these processes, realize where it went wrong, go back, take action. Right. I think that's sort of where people get weirded out is when these things start to take action on their own. What, what can they do that we're not prepared for? So what do you think about that? Are you worried AI might cause harm to people?

[Aiden Gomez]

I think it's really important to remember that we get to choose where we deploy models. It's not like they get to choose where they work or what they have access to. We have to plug them in and we have the opportunity to implement safeguards. So to make sure that before these models are put in any very high stakes situations, that there's oversight that a human has to approve, high stake actions. It's not carte blanche and the model is now smart and we just plug it into everything and say go at it. It's very much intentional and we're going to need to be thoughtful and careful about that. So I'm not scared of like a doomsday, like Terminator scenario. I think that media has certainly instilled that with lots of sci fi stories and it's a very compelling story, which is why, well before AI was remotely competent, we were coming up with stories about how this might happen. But I think the actual media.

[Alex]

Right. It's like also AI leaders are saying, how many people signed that statement that said we should be treating AI risk the same way we treat climate change and nuclear? Why do you think there's so many people in the industry that are stirring up the fear around this stuff?

[Aiden Gomez]

I think that's a great, that's a great question to ask them. I did not sign that letter. And so.

[Alex]

So it puzzles you as well?

[Aiden Gomez]

Yeah, yeah. I mean, I'm empathetic to the fears because, yeah, like this is a very salient story. That's why it's been so popular in sci fi and, and all this sort of stuff. So I'm actually understanding of why people are so attached to those stories. But as more and more evidence emerges that these models are much more controllable than we may have thought that they're a little bit less capable than we may have thought. It's harder and harder to make that narrative. And I think you see the discourse shifting now. I think the discourse has begun to shift away from doom and existential risk. And now it's much more about practical concerns, which I'm really happy to see. Stuff like, okay, this technology could be really useful for healthcare, but it could also cause harm if we don't do it the right way. And so specifically, how do we set up the safeguards to make sure that harm doesn't happen? Same thing with finance, right, and like distributing loans or something like that, or with people using them maliciously to pretend to be human and trick people. How do we prevent those things? That discourse is super productive. Like that's very effective. And so things are shifting in that direction now. And I'm excited to see that change.

[Alex]

Now, some of this fear comes from this line of people saying, oh, there are emergent behaviors in the models, right? That basically that they've found them able to sort of come up with things that are outside of their training set. And there have been some papers that say, okay, actually they don't really have any emergent properties or emergent behaviors. And as someone who wrote the paper that kicked this all off, what do you think about that? Can LLMs have any emergent behavior or discoveries that they weren't trained on?

[Aiden Gomez]

I think that they can. What's the right word? I think they can interpolate between skills. And so if they've seen how to do A and they've seen how to do B, they can get kind of the average of A and B, but they don't just go completely beyond anything that they've seen. I've never seen a model behave in a totally unexplainable way. They're really good interpolators. If you show them different domains, they can blend domains quite well. But yeah, I've heard the same thing about emergent behaviors. And I think the research is really inconclusive there. There's not a lot of compelling evidence that says we're going to have some total step change or capability take off. Even in the latest state of the art research, a lot of it's about synthetic data and models teaching themselves. And so self improvement is this notion of can a model actually teach itself without human intervention? This is now a huge part of model building. It's a big part of how we create data at cohere. And before this started to become mainstream and actually part of the production process of Creating these models, people were saying, self improvement, these things are just going to take off. They're going to become superhuman overnight and we won't be able to control it. Well, it turns out that doesn't actually happen.

[Alex]

Right. So this intelligence explosion or intelligence takeoff is not something that happens.

[Aiden Gomez]

No, it's not happening. It's not happening. It improves for it can self improve for a while and then it tapers off. And so, yeah, you get some good improvement out of it, which is why we use it. But then it plateaus. It doesn't just keep going forever. And so I think the evidence points firmly in the direction of a lot of those fears may have been misled.

[Alex]

Now talk a little bit about that. It's interesting you bring up the use of synthetic data and having the machine self improve, because another one of the big questions about whether this plateaus is, you know, does the world run out of data to train the AI? And I was watching one of your recent interviews where you talked about how, you know, back in the day you could run up to anybody and they can add knowledge to a model. But as the model got smarter, the models got smarter and smarter, it became less easy for people to add supplemental knowledge to them, which points to sort of running out of available data to make these AI models smarter. So how does AI generated data actually solve that problem? And where is synthetic data being used to make these models better?

[Aiden Gomez]

Yeah, so I think the example you gave is a good one. It's getting harder and harder to get the data that incrementally improves the model. And it's important to note that that's because the model is getting so much better. And so before we could just grab anyone off the street and they could teach the model something, and then that signal started to go away. And so we had to go to undergrad students in bio to teach the model about bio, and then we had to go to master's students and then PhDs. And we're kind of at that level where we're currently hiring PhDs to teach the model in their specific domain. But then after PhDs, where do you go? Right. I guess professors. What about after that? So I think the models are catching up with the state of knowledge across a bunch of different fields. I would say that synthetic data probably doesn't get us out of that issue. Actually, I don't know if synthetic data outside of easily verifiable domains like math, it's hard to use synthetic data to drive outcomes. So we'll be able to do it.

[Alex]

In so how is it being useful for you?

[Aiden Gomez]

Not. Not for making our models fantastic philosophers, or making them fantastic social scientists or something like that. For that, we rely on humans. What we do use synthetic data for is for crafting how the model responds to stuff. And in domains that are verifiable, like math, like coding in those places, it's actually quite effective. But that's still a huge domain of interest for people building and deploying these models. We want them to be good at math and computer science. And so more and more synthetic data is becoming a huge chunk of the data that we train on.

[Alex]

Okay, fascinating. One last part of this discussion is sort of what methods help get this AI to improve? And there's been a question of whether LLMs can take it, like, all the way, or whether you need to combine LLMs with different forms of training, whether that's reinforcement learning, although I guess that's part of it already. But the other side of it is, do you have to, like, build world models with robots going out in the real world and learning things, like, things like gravity and what happens when you bump into things which you just can't convey in text? So I'm curious if you think the current methods are able to get this field to the promised land or whether they need to be combined with others.

[Aiden Gomez]

There's definitely proof points out there which suggest large language models or the transformer architecture is capable of handling a bunch of different modalities. And so you can merge not just text, but video and audio as well into the model. So you can give them a much more balanced experience of the world. You can show them the world. You can show them videos that demonstrate physics. You can let them see, hear, or speak. And so as a platform, it does seem like this is a pretty good platform as far as they go. There's a more philosophical argument which is had among academics around. Is text enough or even is supervised learning enough? Is it enough for the model just to observe the world? Or does it need to take part in the world to really understand it? For instance, would you understand the world if you read all of the Internet and you watched every video on YouTube? Would you really understand it? Or do you need to actually be embodied, Be a little robot out there kicking a ball or running down the street? I actually take, I think the less popular view, which is the Internet is enough. And by observation, you can actually learn enough to be extremely, extremely compelling. I think that's if we're talking about AGI and doing things as well as humans do. I think that's Enough.

[Alex]

All right. I want to take a quick break, hear from our sponsor, come back, talk about roi, and then just talk a little bit about your journey. Aiden, from being somebody who wrote that paper to where we are today, I think it'd be interesting for listeners. So we'll be back right after this.

[Lars Schmidt]

From the minds of visionaries to the desks of disruptors, I'm Lars Schmidt, host of the Redefining Work podcast. Join me each week as we explore the new world of work through the lens of those shaping it. CEOs, HR leaders, investors, and more. Be a part of the conversation that changes everything. Subscribe to Redefining Work today.

[Alex]

And we're back here on big technology podcast with Aiden Gomez. He's the CEO of Cohere, also the author of the co author of the attention is all you need paper that kicked this entire generative AI moment off. Right. Invented the transformer. Before we get deeply into roi, Aiden, just a personal question for you. I mean, are you. What does it feel like having. Seeing. What does it feel like seeing your invention being taken in all these, like, wild directions and sort of being this key moment and a truly, like, step forward for the tech field?

[Aiden Gomez]

I mean, it's like, beyond my wildest dreams, I think. I don't take full credit for it at all. I assign the overwhelming majority of the credit to my co authors on the transformer paper. So it's hard for me to accept the reality of what the transformer has accomplished out in the world as my own. But it's so incredible. Like, even if I step away from being one of the authors of the paper, the impact and what the architecture has been able to do for the field has been a huge shock, a colossal shock. Just the technology we have today, I thought we'd be here maybe in like half a century, you know, not seven years. So it's really surreal and amazing.

[Alex]

Has Google effectively capitalized on it? Given that this came out of Google.

[Aiden Gomez]

I think Google has done super well. You know, they supported Google Brain in creating this technology, and it's been integrated all over Google.

[Alex]

All right, let's talk quickly about roi, or maybe let's go deep into roi. Well, we'll see how we end, how we end up here again. We talked about all this, all this money being spent on, you know, upfront costs, training the models. And you mentioned that even if the technology stopped today, there'd be so much work to do with it because there's a lot of benefit out there that isn't being realized yet. But Talk a little bit about the places that you're seeing already getting a return on the investment in terms of implementing generative AI technology, because I think in the common conversation, people don't even think those places exist, but it seems like you're seeing it on the ground.

[Aiden Gomez]

Yeah, I think today we're starting to see it integrated into production. In enterprise it's much slower than in consumer. There's much higher lift to actually get it integrated and there's a higher bar of trust necessary to drive adoption. Like I was mentioning earlier, last year was very much the year of the proof of concept. But this year we've started to see it go to prod. So there's some good examples of that with our partner Oracle, which they have this suite of applications which basically power enterprise hr, supply chain, all of these sorts of back office functions. And we're powering over 50 different applications within those software tools. And so it's actually starting to get into the hands of employees and drive efficiencies.

[Alex]

Wait, hold on, talk about what that looks like for an employee on the ground there. How does the software that they were working in change when you put generative AI in it? Through cohere?

[Aiden Gomez]

Yeah. So you're automating parts of the job, little tasks within the application. You can now just push a button and the model will do that. You might need to provide a high level. A good example might be in writing job descriptions. Right. So a manager, a hiring manager wants to hire for a specific job. What they want to do is just put in bullet points. I need someone who has this background, does this, et cetera, and then press Go and it will generate the full job description with everything that the company needs included there. And in a way it's actually presentable to the people applying.

[Alex]

And that's a known use case. Yeah. So let's hear something else.

[Aiden Gomez]

Yeah. Another good example might be in supply chain. When you're looking for an alternative supplier to one of your products, doing that search and retrieval and being able to iterate with a model and not just do single step search where you search over suppliers, but where you give feedback, you say actually no, that one that you just recommended doesn't work for this reason. And you're able to refine iteratively with this assistant or agent. And these models basically touch every vertical. And so there's no particular vertical specialization, it's totally horizontal. So we were working with a legal tech startup that helps with reviewing contracts and building an assistant for a lawyer to help them review contracts. More Quickly flag, you know, concerning terms, that type of thing. We're working with a healthcare startup that tries to use news and social media to track pandemics and are people getting sick in a particular area, reporting specific symptoms and so using models to screen for that. It really impacts every single vertical.

[Alex]

Can I take the devil's advocate position on this? Let me see if I can channel an AI critic and see what you think about this. Basically what they would say is job descriptions. Okay, it will save you a tiny bit of time if you have the AI, right, the job descriptions. If you're looking for a supplier, chances are if you work in vendor management, you're going to have a good familiarity with those suppliers anyway. If you're a lawyer, like, yeah, it might be a little bit of time, but you can comb through a contract and find out what's, you know, what might be concerning about it. This is your expertise. You're like almost like a narrow neural net trained for that one specific purpose. And now we're giving it over to AI. And they'd look at just the billions being invested in this technology and say, well, what am I really getting for that? If this is effectively doing some of these things that humans are quite good.

[Aiden Gomez]

At to begin with, I would counter and say that risk to supply chains is many trillions of dollars. I would say that lawyers are extremely, extremely expensive and you don't want them combing through your documents no matter how efficient you think they are. And same thing with doctors, we really want them spending time with patients, not combing through hundreds of notes and filling out forms afterwards. I would say those are. Maybe this stuff feels banal, maybe productivity feels boring compared to some of the hype of AI, but it is the value. This is what we're trying to build for. And so I would push back quite firmly on that.

[Alex]

React to this. I think this is a thread from Benedict Evans, tech analyst. There's an interesting difference between people outside tech sneering at generative AI as chatbots that get things wrong and make crappy, you know, quote unquote stolen images, and people inside tech who are mostly working on using it to automate a huge number of boring back office processes inside giant corporations for billions of dollars.

[Aiden Gomez]

I think that's a great observation. I think that there are some very superficial critiques of generative AI that have become very popular. I think the substance is in actually doing the work and getting this technology to be productive for humanity. And a lot of people are working on that right now. It's going to Take time, like I've said, but the opportunity is immense. It's the biggest in a generation.

[Alex]

Yeah, I think that's kind of the misconception and that's the interesting point about what this technology can do. So I was speaking with Flexport again, supply management, supply chain management. And I think about writing about how the fact that like supply chain is actually like ground zero for where this technology is being applied and useful. But they're basically, they're like getting faxed things, they're getting PDFs, you know, to try to log that and comb through that. You know, the volume is crazy. And they're using generative AI to read through the documents and give them actionable insights on it. And they're like, look, it's not going to be the most exciting use case, but this is saving us a tremendous amount of time.

[Aiden Gomez]

Yeah, I was about to say, I'm like, it's so boring, but it is so valuable. People don't understand the actual scale of impact of some of these crucial banal things. And if we can scale them up, make them more accurate, more reliable. Yeah, it really is world changing.

[Alex]

Isn't it? Kind of crazy that like the picture of AI again is this just like, you know, I guess Maybe it's because ChatGPT was the thing that started the hype cycle, but the picture, popular picture of AI is like this masterful again, like God, like technology that, you know, can do all these things and be this friend for you, like the character AI type startups and people talk about AI girlfriends, but then the value is really being realized in like the back office. I mean this is pretty crazy sort of divergence. There never. I don't think I've ever seen a technology type of divergence.

[Aiden Gomez]

Yeah, I mean, I think like the Internet is a good or like computing in general, like these general platforms for supporting new types of products and tools. Yeah, sometimes they have biases in certain ways, but it's all about diffusion, like diffusing into an economy, diffusing into our daily lives. And it takes time for that to happen. And we should remember that we're like 18 months into that journey. And so it's still, it's really so early. But yeah, I think the Internet has had huge impact both on the commercial side, on the enterprise side as well as with us as consumers and people and AI will be the same. There will be products that are pure play AI products targeting consumers that bring tons of joy and value to consumers. And then there will be platforms like Cohere that Enable huge value within the enterprise world.

[Alex]

Yeah, and again like talking a little bit about how impactful this is in enterprise, I think this is from Reuters. Accenture's generative AI business, which helps companies automate operations to save costs and boost productivity, recorded about a 50% jump in new bookings quarter over quarter. This has outpaced growth in Accenture's other core businesses as a go to consultant and outsourcing service provider for companies migrating their operations to the cloud. Analysts expect slow demand for such service as enterprise spending plateaus. So basically this is like finding ways to automate is like giving life to the consulting industry. What do you think about that?

[Aiden Gomez]

I think, you know, Accenture is a really good partner and there's just so much work to be done implementing this technology that that makes perfect sense. Like there's a huge technological shift happening and the technology has unlocked a whole new set of applications and so now we need to go out and do the work to realize it.

[Alex]

Yeah, and what type of partnerships are you having with Accenture? Is it like going into companies and again automating back office or like what is what's going on there?

[Aiden Gomez]

Yeah, so they're our solution integrator partner. And so yeah, it's about taking on projects inside of enterprises to help them accomplish something. Like maybe it's implementing for their finance team. There's some function that they're stuck on and that takes a huge amount of their time, but it's totally non strategic. They shouldn't be spending time on it. And so can we automate that or a big part of it using these models? It's about these strategic projects to try and unblock and automate parts of usually back office functions.

[Alex]

It's amazing how like I wrote about this a little bit in my book, but we're like living in the knowledge economy and even still like we've gone from industrial economy, which is like literally like pulling levers and pushing buttons to make stuff, to knowledge economy, which is all about knowledge. But even in the knowledge economy, so much of our time is like legitimately on like straight up, you know, repetitive kind of tasks that we wish we could automate to make room for us to do more knowledge stuff.

[Aiden Gomez]

I hope that that goes away to a large extent, but I don't think it will like I think there will always be on the margin these sorts of not good uses of our time that we spend time on and we'll continue to push that margin back and back and back and try to automate as much of that as we can, but it's a huge, huge project. What we're focused on is kind of building from the foundation. Start by automating the biggest of those, the ones that you're wasting the most time on, and then gradually get into more niche, targeted, specific automations or applications.

[Alex]

Is anybody using your technology to replace full time employees?

[Aiden Gomez]

I am not aware of that. I don't think I have any example of that happening. It's very assistive actually. So it's less about replacement, it's more about augmentation. Like at the moment, what everyone's building are tools to augment their workforce to make them more productive. I can't think of a single example of displacing people.

[Alex]

Okay, I know we're running out of time. One more thing I want to ask you about is sort of like the role of cloud providers versus the role of people buying direct and like how this is helping or what type of pressure this is putting on cloud. This is again we talked about this recently. So Anthropic, they just broke down, CNBC just broke down anthropic's revenue and third party APIs like Amazon and I think Microsoft Azure, if they're available there, let's say Amazon, 60 to 75% of their revenue. So how important are these cloud providers like Amazon, like Azure, in driving this forward?

[Aiden Gomez]

The cloud providers are great partners to cohere. That's where the majority of compute workloads are happening, but not all of the workloads. So cohere has had a long time focus on, on prem as well, because for a lot of regulated industries like finance and healthcare, a lot of that data doesn't actually go on the cloud. But certainly for many industries that are cloud first, that's the place that their AI workloads are going to happen. And so I think it makes sense for revenue to be coming from those sources. But for cohere we support both. And so it's perhaps a little bit more balanced.

[Alex]

And so your technology is basically going to work. Your company will basically work to integrate your technology into existing systems or you have your own software.

[Aiden Gomez]

So we build our own models from scratch and we build a platform that lets people plug in their data sources, the tools that their employees use into the models by a system called RAG retrieval, augmented generation. And that's something that we're specialized in. The guy who created RAG when he was at Meta is Patrick Lewis and he leads our RAG efforts. But it's basically the dominant architecture or system that enterprises are looking for right now. They want to customize these models with their proprietary data. And the best way to do that is with rag. So that's something that we provide out of the box in like a super simple plug and play way.

[Alex]

Let's end with this. Can you give us your prediction for what the AI field looks like in the next two years and five years?

[Aiden Gomez]

Yeah, in the next two years, I think we're going to start to see really compelling assistance. It won't just be little convenience functions or small features. It'll look a lot like a partner that you do work with, someone that you interact with every single day and you view as a collaborator over the next five years. I think it's not a major shift, but it's an increasing incompetency. The scope of those assistants will expand. They'll be trusted with doing much more, and they'll be integrated into many more systems, so they'll be dramatically more capable. So I view it as like a continuous change over time towards much more compelling independent agents that we can collaborate with.

[Alex]

Well, Aiden, thank you so much for coming on. Great to see you. And thank you so much for sharing everything about the industry in general and where, you know, companies are finding the roi. I do think that this idea that, listen, like, it may be quote, unquote boring, but hey, if it's saving billions of dollars, then don't tell me that that's a boring application of technology. That's kind of my main takeaway today, and I think it's pretty fascinating stuff that you're working on.

[Aiden Gomez]

Yeah, thanks for having me on. It was great seeing you, Alex.

[Alex]

You too. All right, everybody, thanks so much for listening. We'll be back on Friday, breaking down the news, and we'll see you next time on big technology Podcast.