← Software Engineering Daily
Software Engineering Daily
Crew AI with João Moura
00:44:51
Agentic AI is seen as a key frontier in artificial intelligence, enabling systems to autonomously act, adapt in real-time, and solve complex, multi-step problems based on objectives and context. Unlike traditional rule-based or generative AI,
Loading summary
Transcript55 lines
- [00:01]
Sean Falconer
Agent Take AI is seen as a key frontier in artificial intelligence, enabling systems to autonomously act, adapt in real time and solve complex multi step problems based on objectives and context. Unlike traditional rule based or generative AI, which are limited to predefined or reactive tasks, agentic AI processes vast information, models uncertainty and makes context sensitive decisions mimicking human like problem solving. CREWAI is a platform to build and deploy automated workflows using any LLM and cloud platform. The company has rapidly become one of the most prominent in the field of agentic AI. Joe Mora is the founder at CREWAI and he joins the show with Sean Falconer to talk about his company. This episode is hosted by Sean Falconer. Check the show notes for more information on Shawn's work and where to find him.
- [01:04]
Joe Mora
Joe, welcome to the show.
- [01:05]
Hey there. Thank you for having me, Sean. I'm very excited to be here. Talk about AI agents.
- [01:10]
Yes, yes. Yeah. Before we get there. So you're the CEO and founder of crewai. What's the story behind the company? Like, how did all this start?
- [01:18]
I gotta say, Sha, like people ask me this and my wife hates when I tell it just because it involves her. But I was working Clearbit for five years prior starting crewai and in Clearbit, I was leadering their enterprise product and also all their AI initiatives. I was there for like all the way to the acquisition by HubSpot. And when we got acquired by HubSpot, I was already playing with agents in my personal life for myself, basically building agents to help me kind of like do better at LinkedIn and do better at X. My wife was telling me, you're doing so many cool things at Clearbit, you should be posting about it. Right? But I'm very good at cheat posting on X, but I'm not that good on posting on LinkedIn. So I got a few agents to help me out and I got hooked. Like it was so easy. I was getting so many posts in a very consistent manner and I was like, you know what, I want to build more agents. So I looked around, I found nothing that could help me do it the way that I wanted to. So I decided to build my own. And that's kind of like how the open source started. And from that, to make it into a company was just a very organic process of like companies reaching out to me and saying like, hey, we're actually using CREA in production. Can you help us out with this? And I realized there's no way for me to provide that level of support, like by just making it an Open source project. Like, it needed to have, like, funding behind it to make sure that we could do it. And that's how things came to be. And it got very excited after that.
- [02:45]
That's really, I think the dream scenario for almost anybody who like, ends up starting a company is like, you know, build something that's useful for you. Starts as like the side project that's doing, you know, something that you're happy with. And then suddenly people are like knocking on your door and saying, hey, like, we want to pay you basically to use this project, or we're already using this project. I mean, it's awesome. I think it's interesting, you know, what you said around doing some of these agents to help you do some of your work, like posting on LinkedIn. And actually for myself, like, one of the first public AI based applications I built also was for helping me analyze podcasts that I do and then generate a first draft LinkedIn post. Because I don't always have time to, you know, put those things together, but I want to do the guest. Right. And help promote those things and so on. So that was like one of the first projects. Now I did not, at least as yet, turn that into a business. So it's amazing that you're able to, you know, go from those humble beginnings to actually creating something of real value.
- [03:40]
Yeah. I got to say for me back then was just like, I want to make sure that I was finding the time to post this. But like, as you know, if you want to write something that is really good, like, that has that, like, emotional connection that taps into your, like, experiences, it takes a while. Like you can take a few hours for it to think like something true. Right. So the thing was, this would help me get from a crazy kind of like one, two line idea to kind of like a two, three paragraph post that then I could basically, like improve in five minutes and it was ready to go. And it was funny because I preloaded those agents with some knowledge. So they knew about my resume, they knew about my, like, life experiences. I wrote things about this day that I fell for my bike or the day that I broke my jaw. And now that. And I got all those things in there, so it would draw all those interesting correlations that I think make this story very special. That then I could spend like a few extra minutes making sure that just hitting the points that I really wanted to. But that was great. Yeah. I still remember being kind of like on a meetup in the Bay Area. I think it was on shack 15 and people from Oracle reaching out to me and say, like, hey, so we're using Create in production. And that was a big aha moment for me. I was like, what? Oracle is using Crewai? So, yeah, from that point on, things starts to really, like, basically get a lot of traction. And honestly, I'm very grateful, if anything, as you said, I know how lucky I am from getting to work on something that I love so passionately. And also the fact that we have such a nice and incredible community behind us. So I take none of that for granted.
- [05:16]
Yeah, that's awesome. Yeah. And I feel like the company itself and the product has kind of jumped onto my radar maybe in the last six months. And I feel like I'm hearing more and more. Has, you know, the growth been crazy during that time? Has there been really like, are you seeing essentially this wave that's happening around agents and interesting crew?
- [05:32]
Yes, I gotta say, like, it's like record after record. Like, we keep topping records every week or so. So it was funny because on the Christmas week, we broke our record in how many crews we executed in one single week. And each crew here, for the people that don't know about this, is a group of AI agents. Right. So you could have 2, 3, 5, 7. I have seen crews with as many as 21 agents on it in this crew. The reason why these agents are grouped together is because they're trying to fix a specific problem or they're trying to automate a specific process. So that is why they're packed into a crew. So on that week of Christmas, we ran over 3.5 million crews. So those are kind of like tens of millions of agents in one single week. And that was all hackered high. But then fast forward a few days and I think a couple of days ago, in one single day, we ran 1.3 million crews. So, like, it's insane how much traction things are getting. And it's funny because there's the open source, there's an enterprise, there's socials, so there's all these different components that plays into the company. But I think open source and the enterprise adoption has been very impressive. And I think this year is going to be a little insane if things keep moving the way that they are.
- [06:55]
Yeah, that's amazing. So I think that's a good point to maybe just slow down a little bit on where we're going, just for people who are a little less familiar with this topic and talk a little bit about, you know, what is an agent and maybe get your perspective on this, because clearly Agents, they're having a huge moment right now. Even if you're not in AI, I'm sure you're hearing all about it and people are making grandiose predictions for 2025 about that. But there's a variety of definitions out there. Like you can get everything. You know, I've seen definitions of agent that basically suddenly everything is an agent. Like there's nothing that's not an agent and then there's really specific definitions like you know, agents have to be using tools. But in AI itself there's a long history of AI agents. It's not just about gen AI. Like there's, you know, even going back 1960s and 70s, you had rule based systems, it wasn't generative AI or LLMs, but you had this notion of an agent doing, you know, some work for you. So how is like, is the definition of a gen AI agent different from the historical definition or is this just, you know, a different algorithmic approach that's happening behind the scenes? Power the agentic capabilities. And how do you think about the definition of an agent?
- [08:03]
Yeah, no Sean, that's a great question. I think it's something else. I think it's something different. When you think about AI agents nowadays, I think it's mandatory for you to have the AI dictate the flow of the application. So if you have a traditional software and you basically have a class with functions and the functions are calling each other and you have kind of like an API call that calls AI in there, that for me is a workflow, that's a script, that is not an AI agent. Now if the AI, the LLM that you're powering, this is actually controlling the flow of the application and by that it has agency, then for me that is an AI agent on its current definition. So yes, I mean there's workflows, there's automations. Like honestly if you forget the name AI agents for a second, we're talking about AI powered automations. That's, that's the bulk of it. Right. But I think what throws like an extra spin on it and make it interesting for unlocking automations that were just not possible before is this ability to not only generate the content in real time, but also change the flow of the application in real time. So it's not always a predefined path of kind of like graphs and nodes where it's doing always the same thing, but you're extracting the full value of the LLMs by having like the LLMs itself control it.
- [09:26]
Right? Yeah, I Mean historically, when we think about engineering anything, it's essentially a person that is coming up with if this do this and this and this, and you're creating sort of this deterministic flow. But with the agent, you're sort of handing off that decision making process to the agent to figure out what is the plan of execution in order to accomplish a goal. I'm giving it a goal, just like you would give a person a goal and they go and they essentially figure that out.
- [09:53]
Yeah. And I would say that it's more of a spectrum more than anything else. What we're seeing out there in the real world for the thousands of use cases that we have been building so far, is that it's not like one or the other, it's a spectrum depending on how much of precision you need for your use case. So if your use case require less precision and you can rely more on these models, usually that means that you're generating more value just because you can have like less people overseeing this. These things can run over and over and it's on and then you can have that, like just go for it. But sometimes for certain aspects of the automation, you want to have more control, you want to have programmatic control of like what it's going to happen. So we allow you to do that as well on the feature that we call Create Flows, where you can basically say, all right, if this, then something else is going to happen. And that's basically a producer and kind of observer event listening approach. And that makes it extra flexible because you can decide when you want to have agency and when you want to have like more control. So I would say more of a spectrum than one versus the other. But definitely depending on the use cases and the precision that you're trying to get, you might optimize for certain aspects.
- [11:05]
Why do you need something like a framework like Crewai to build an agent versus doing something bare bones yourself?
- [11:12]
Yeah. I gotta say, I love this idea on engineering that you should try to do the least amount that you can get away with. Right. That's always a good way for you to get started with things. Like don't over engineer things. Always think from the ground up, like, what is the minimum that I can get away with it? And that's usually a good baseline. The problem is what we're seeing out there is these agents start very simple conceptually because they're like, all right, it's an LLM and a loop and I'm going to give it a prompt and it's going to do Something. And then you start to thinking about the tools. They're like, all right, so I need to give it a tools. That's not a big lift I can do to calling. I can do JSON parsing. All right, I got that done. Now you're going to run this at scale. And they're like, well, if I'm going to be using tools, I need to have a caching layer. If I have a caching layer, I need to have an expiration mechanism for this. And if I have two agents and they're working together, I want this caching layer to be shared. Well, if there are two agents, they probably need to share a memory as well. Well, if they're doing memory, maybe this memory should be a reg so it can select the. And then things start to get more and more complex. So I think where these frameworks come in handy and I think where create really shines is it abstracts away a lot of that and give you kind of like a DSL that allows for more simpler use cases. You can just use the dsl, but if you want to go super low level and change the prompts, changing the templates, changing out the inner workers, you can do that as well. But it checks all these boxes so that you have agent delegation, ability to communication, caching tools and memory, all that for granted. So you don't need to rebuild those same things over and over and over. And the other thing as well is as patterns are forming, like you might try to do something for your company, but the market is moving so fast that you're kind of like doing something that is suboptimal and a framework that's getting exposed to like a thousand companies are going to be able to learn from all those use cases. So those would be a few things that really can make a difference for some of these enterprises when they're using crewai.
- [13:17]
What about in terms of like the challenges around getting the abstraction right? Like, all this stuff is so new things are moving quickly so you could, you didn't have to make a decision. You're sort of, you know, having an opinion about how people should develop agents. And then you're encoding that essentially into this framework and you're saying this is what our opinion is about how to build these things. But how do you know that your opinion is the correct one?
- [13:40]
That's the thing, like a lot of like early days, you don't. Right. And that's the beauty of it, if you ask me. Early days, you don't. But then as people start to use it and you start to kind of like basically polish these rough edges and the ideas, you start to have a more clear take on what works and what don't. I think one thing that we did with Crew from the early days is a very opinionated way and I think given how fast the market is moving, that's what people need. People want to have an opinionated way of like how should I be building this? What is actually working out there? I don't want to try to think this from the scratch. So. So I think the fact that we had an opinionated take, if anything helped the framework to get as much adoption as it did. This idea that we from the get go are saying like, hey, this is what works. I remember looking back a lot of the inspirations technically from the DSL came from the Ruby community, specifically Ruby on Rails community. I spent quite a few years doing Rails in my career and I always loved how they prioritize developers experience and how that is a big piece of the language and the framework. And some people might say that sometimes it's too magical, especially Rails. And I agree with that. But I think there's something pretty about the ability to make something that almost read as plain English to the point that democratize accesses to that technology. Right. So the thing that we're trying to do is as long as we give you ways for you to go as low level as you want, is it okay to have something that is more high level and you can basically figure out pretty quickly?
- [15:12]
Yeah. I do think that especially with framework adoption, like it depends a little bit on what it is and what problem you're solving. But a lot of times DX is a big factor in terms of like something catching fire and people wanting to adopt it because it's like, especially when we talk about something like agents, which is probably not necessarily something that most people have spent decades like working in, they really just want to be able to like solve their problem and get going and get to that aha moment as quickly as possible. So if you can lower the barrier to entry and the amount of friction involved with getting them to that like hello world moment with an agent, then of course they're going to want to do more and invest time and get into more complicated use cases. But you need to sort of solve that problem to start with.
- [15:53]
Yeah. And I guess the other thing that adds to that is just like you got to use it, right? You can build something and not use it. So I think from the get go we have been using Kruse so much within our company ourselves that it just like it gives you clarity, right? Because you know what works, you know what doesn't. You see like when you're struggling, you see new engineering starting your team and non technical people join your team and trying to build crews and update crews. So it gives you the clarity that you get when like you're a major user of your own product. That I think it really, really helps. And I think honestly the community is kind of like it's a flywheel. Like the more people you get in the community, the more feedback you get, the more you see what works, the more you can like policy the edges. And I think the other thing that also plays a role in this is because I have experience with enterprise software and a lot of our team also does. I think that we understand that you can't just move fast and break things on enterprise, right? You can absolutely move fast but you got to not break that many things. So I think like being able to make things like backwards compatible in really fighting a lot like for not having breaking changes, I think those are things that are really also help drive adoption. Just because people know that the docs change, they know that the features are coming. But there were very few times where we actually shipped something that was like this is a breaking change. Like you're going to need to update your code to make this work.
- [17:23]
Okay.
- [17:26]
Sean Falconer
This episode of Software Engineering Daily is brought to you by Capital One. How does Capital One stack? It starts with applied research and leveraging data to build AI models. Their engineering teams use the power of the cloud and platform standardization and automation to embed AI solutions throughout the business. Real Time Data at scale enables these proprietary AI solutions to help Capital One improve the financial lives of its customers. That's technology at Capital One. Learn more about how Capital One's modern tech stack data ecosystem and application of AI ML are central to the business by visiting capital1.comtech can you break down.
- [18:07]
Joe Mora
Like what is the anatomy of an agent typically look like?
- [18:10]
Oh yeah. So usually again there's an LLM in the middle serves kind of like of the brain of the thing. There is a task that you're giving for this LLM. This LLM in Crewai it's impersonating like someone or a role. They're behaving as a certain like role and trying to try to accomplish this task and there's going to be an output. So that would be the core. Then you have tools and you can think about tools as integrations. Like it can be API calls, database connections, rag implementations, whatever it might be, SAP, Salesforce, SharePoint, you name it. Those are integrations that your agents are going to be able to call on their own as they basically try to accomplish that task. Then if you put those agents together into a crew, they now automatically have the ability to use certain special tools. Those tools are delegating works for one another, asking questions to one another. And then because they are using all those tools, they get that caching layer that we were mentioning earlier. So now if they use the same tool, passing the same arguments, they're going to hit this caching layer. So now because they are all in the same crew, you're going to have a few different types of memory. You're going to have like a shorter memory. And this applies to any frameworks out there. A lot of these concepts are cross frameworks by the way. You're going to have a shorter memory that basically allows your agents to remember everything that everyone is doing during an execution. You're going to have a long term memory that is basically going to be improving your agents across many executions. So as they learn about what they did wrong, what they did right, they basically kind of like improve over time. And then we also have an entity memory so that they remember specific definitions of things. So for example, if they learn what is a software engineer, they won't need to have to try to learn that again. They basically, they already know about it. So those would be kind of like the memory. The only other thing that I would say that adds to this is they're usually when you're bringing this into the real world applications, there's usually a trigger and a destination, right? Those things don't run on isolation. So there is usually a component of like, oh, this is going to be kicked off when a new entry in my HubSpot appears or a new Zendesk card is created and the destination is usually like something else are the same like, oh, it goes back into my Zendesk or it goes back into my Salesforce or it goes into my email. So I would say high level, those would be the major components that you see in there. But then again, things can get very complex as well if you just keep working on that.
- [20:49]
So you know, one of the things you talked about there was how an agent might have a role. And a lot of times you're configuring an agent role through things like the system prompt where you're saying, you know, you're an expert researcher, expert copywriter or whatever it is. So you know what's happening underneath the covers with the LLM, when you give direction like that, like is that directing the LLM to a certain set of parameters or adjusting different vectorized terms that influence each other to change how the LLM is going to like interpret and respond to things?
- [21:18]
Yes, 1,000%. I think a lot of people that are using these AI APIs right now, they have a high level understanding of how these models works behind the scenes. But honestly, if you look back to any AI model out there or most of the AI models out there before LLMs, even if you're thinking about prediction models or classification models, in the end of the day they're trying to all do the same thing. They have something that they want to predict and they have other, what people call features that they already know. Right. That's what people call in data science or ML. So you have the features that you know when you have the one thing that you're trying to predict. So a lame example would be what is the likelihood that's going to rain? You don't know if it's going to rain or not, but you know what is decision of the year, you know what is the temperature and you know your location. So giving those, if you have enough data sets from three, four years, you could pass this into a model that is going to try to create a mathematical function that giving the data that it knows, it can predict the data that is unknown and that's the likelihood of writing. So that is any AI model out there like it's what they're trying to do with LLMs, they're just not different. The only thing that changes is that in this case the data that you know the features are the text that you have written so far or the tokens that are outputted so far. And it's going to use those tokens to try to predict what should be the next token. So the more information that you put before the token that are trying to predict or the more qualified information, the better steering power you will have on the next token. So if you say it's 37 degrees Celsius on a winter, but if you change it to a summer that have a directly impact on the likelihood of. Right, right. So I would recommend people to try this out themselves. They can go into ChatGPT or Claude or whatever it be and ask for like give me stock analysis on Tesla and then open a new chat and try the same thing. But it starts with you're a FINHA approved investor and you're going to see they're going to have wildly different responses because they're steering the model in a different way. So role playing can definitely play play like a big role on how these models perform if you do it right.
- [23:37]
And in terms of the long term memory versus short term memory, how that works behind the scenes is essentially you're automatically populating the context, so you're doing some context data augmentation before whatever the new behavior is or the new event sequence that the agent's supposed to execute. But for the long term memory, you're just holding onto that is persistent, so it's always there within the context window. Is that the idea behind the long term memory?
- [24:01]
Memory a little bit. So the long term memory, it's actually a GRAQ database, it uses chroma behind the scenes, but you can customize if you want to. And the reason why we can do something like that is on Crewai, from the day zero, we made sure that when you're defining a task, and it was funny because when we created this back then, a lot of engineers didn't understand and kind of gave us trouble because of it. When you're creating a task for an agent, you have to not only describe the task, but you have to say what is the expected output. A lot of engineers got mad at us because of that. They're like, well, I'm already giving the task description, why do I need to say what is the expected output? What they didn't realize is because one, we're forcing you to do better prompting, even if you can't tell. So if you haven't been doing much prompting up to this point, this is kind of like a forcing function for you to think like you're supposed to think in writing the prompt. But then we can programmatically use an LLM as a judge to compare your actual output with what you said the expected output would be. And then we can extract learnings from the discrepancies in there, and then we can save that into a vector database and then your agents can query those learnings during execution to learn from while they have done wrong in the past so they can correct. So a lot of the long term memory is memory that doesn't apply for that specific run that you're doing of our agents with a specific data that you're putting in through that. But it's more about the way they're supposed to behave and things that they haven't complied with in the past that you expect them to, so they start complying with that over time. So the long term memory is injecting extra Context with learnings from discrepancies between what you expect them to give in the past and what they actually gave you. But everything happens automatically in terms of like learning.
- [25:56]
Are people actually training agents in the same way that you train? Like, you know, fine tune an LLM, or are you primarily relying on this prompt augmentation to provide the right context?
- [26:08]
Both. So we do have a training feature that is basically like similar to DSP pie in a way. It automatically tunes your prompt to be the optimum that it could be, so you don't have to worry about it. And it does in a way where it's conversational. So the agent does part of the work and they come back to you and say like, hey, this is how I'm doing it. Do you like it? And you can give it feedback. And then throughout that process of you conversating with the agent, it basically updates its prompt to kind of like make it better. So that feature is extremely useful and saves you a lot of kind of like prompting engineering. But we also have cases of people fine tuning models, especially small models. Small models, they suck a lot of times as agents they're amazing for a lot of different pieces. But as agents they have a very strong time to kind of like complying with specific formats that you expect and things like that. But if you fine tune them, they become like beasts. So we have seen people fine tune to have agents that basically output content in kind of like a voice, right? Like a company voice in the same way. So doesn't sound like AI. That's kind of like the most common use case that we see out there.
- [27:24]
What about in terms of an agent having access to files and tools like how do you control access? You know, data governance and controlling access is like a hard enough problem in any distributed system and you know, enterprise architecture and stuff like that, because you end up having to have essentially like different code rules at every piece of software independently. Like, how do you do that with agents?
- [27:47]
Yeah, so there's a couple answers on our open source. It's all programmatic, so you're going to have to figure out these things on your own. On our enterprise offering, you do have features around where you have an internal repository of tools and I think that is even available in our free tier. So you can like, if you're listening to this, you can try it out. You can go@create.com and sign up in the free tier. I think we already have the two repository. So as you build these tools, you can push them into this private repository and you can control access levels based on roles. So you can create a specific role and not only that role will apply for the agents, but also apply for the people. So you can have certain engineers in your team that is going to have access to tools that are going to give them access to SAP data or to Salesforce data or to Zendesk data. And you're going to have other engineers that won't have access to that, meaning that they won't be able to even see it. And they're going to be able to use those tools when building their agents and their agents won't be able to use it themselves. Now that I think is just a tip of the iceberg. I think when you're talking about the actual agents having the permissions and I think things got to start to get very interesting. I was talking with folks from Okta a few weeks ago. They're doing some amazing work, exploratory work on AI agents permissioning and authentication. Because you start to thinking about it and you start having all these questions. For example, does an agent's permissions change depending on the person that is triggering the agent? Like if I have extra permissions, I have level of access. Should the agent that I execute have the same level of access? And if so, what happens to the logs that are being generated? Like how I guarantee that this is not exposing PII and bursting of information. So we tackle a bunch of that on the enterprise. Like we have PII sanitizer and a bunch of other things. But I think like that's a big topic and that is just the tip of the iceberg. There's a lot more that we are going to start to encountering as we're rolling out those things like at large scale in the wild.
- [29:48]
Yeah, absolutely. I mean if you have some sort of like financial analyst agent that is available to your employees for some reason. Well, the person on customer support probably should have a different level of permissions into the financials of the business than perhaps the CEO or the cfo.
- [30:06]
Exactly, exactly. And we are seeing that firsthand in a lot of companies out there. I think again right now I think it's, it's early days for agents even though the team is so hot. It's a brand new industry, it's very early days. So a lot of the times all about like permissions of access is all about approving certain tools usage. But I think like as we get more into impersonating kind of like agents, I think like things are going to become more interesting.
- [30:32]
Yeah. You know, on the Given that we're sort of in the early days, like how do you think about like the maturity curve for companies being ready to do adopt agents? You know, it's hard enough to do any AI application at, well, at scale and then singular agents and we're talking about not even singular agents, but multi agent. Does multi agent end up compounding the problems that you run into with a singular agent?
- [30:55]
I gotta say it's insane to see how fast a few companies are moving, honestly. And I think that what is happening there, I think there is one, there's some FOMO that's kind of like driving the market, right? Like no one's going to be having their competitors kind of like eat their lunch because they're being more efficient. And I think the other thing is a lot of people like to compare kind of like AI agents with the Internet back in the day and the Internet red boom. And I think that's a fair comparison a lot of different ways. There's one thing that is a major difference though, and that I think it's driving a lot of the adoption and kind of like the eagerness to adopt agents. And that is if you were on the Internet on day zero, that would have zero impact on their bottom line. Like no one was online that wouldn't help your business in a way or shape or form. But with AI agents in what people are getting, like we're caught in the wind off is with these public companies, they have to file the reports. You start to see reports from companies like Walmart and you see how many millions of dollars they're saving because one quarter ago they implemented agent, they put AI in their case. I don't know if they're using AI agents or not yet, but you start to say, all right, so this have like actual bottom line impact. So I think that is driving a lot of the fast adoption that we are seeing in some customers. That said, a lot of times those teams are heavily technical still. So what we're seeing is even though you have companies like Microsoft pushing for very kind of like non technical folks to build agents, but I strongly believe will be the way that agents will go for the long term. Right now the teams that are being most successful in these companies are technical teams. You need to have like some term of sponsorship on the technical side to help you build this very custom use cases that apply to your company and that leads to more success for these companies that are being a little more eager. But for known technical teams to drive adoption like this, I think it's still very Very early for that to happen.
- [32:52]
I totally agree that. I think there is a really big difference between early days of the Internet and what we're seeing in AI right now in terms of the business risk. It's hard to think now because the Internet has become what it's become. But back in the 90s, it wasn't clear that people could actually make money doing business online because one, there was not that many people online, and then two, like, people just didn't know, like, if I take my, you know, storefront and put it online, does that suddenly lead to, like, money? But if you can have an agent that can reduce cart abandonment for when people are shopping by 5% or something like that, well, clearly that has a tremendous value for your business. I think there's inherently less business risk.
- [33:33]
If you can get these things working a thousand percent. And I think honestly, the combination of this idea of like, all right, this is able to actually drive impact from like a next quarter together with this FOMO of like, all right, my competitor might be doing something I think that is driving a lot of like this eagerness to adopt. But yes, those projects, honestly, as I said, depend a lot on the use case. A lot of companies that come to us, we usually start with lower precision use cases, things like backups, automation, sales, marketing, maybe some support. And then we start working our way up to more kind of like high precision use cases, like user facing, pricing, accounting, things like that. We do get some crazy wild cards sometimes. People that are coming in hot, like first time doing it, and they want to kind of like conquer the world with things. And we have some of those customers as well. But usually people start with low precision use cases and get comfortable with it, and then they start scaling from there.
- [34:34]
What's the most sophisticated use case in your opinion that you've seen?
- [34:39]
The most sophisticated use case that I have seen. Well, I got to say, there are things that I never expected to see people do. And that was a big Fortune 500 consulting firm working for another Fortune 500 kind of like media company. And they were using CREWAI to mimic video and audio editors. So as the media companies were streaming a live sports feed, they had agents that were using fine tune video and audio models to track the ball on the screen, automatically cut it, add sound over, and then push that as social media content. That was something that, I mean, I was not expecting to see anytime soon. Again, a very complex use case, but more kind of like an unusual side and another one that comes to mind that is a more complex one is Filling out IRS docs, a very complex use case. Honestly I thought that making my taxes suck. But now that I know what some of these banks have to go through, it's insane. Like they have sometimes too few forms that are 70 pages long of questions. And don't worry, they come with kind of like a manual, but the manual has 620 pages. So how do you read that manual when you fill that up? So using agents to do something like that is another use case that comes to mind. That was, was tricky at first but was very interesting once that we got it running.
- [36:20]
Yeah, I think like filling out RFPs is probably going to be a big. I'm sure companies are already doing that and then I know for sure that there's a bunch of companies that in sort of the bio informatic drug design space that are using. I don't know if they're, you know, technically agents, but at least AI to help fill out some of the forms. Because there's a lot of forms you have to fill out in order to go through the sort of legal channels to get a drug to a place where you can test it. And I'm sure there's human expertise checking those things over. But there's just a lot of work that you have to put into filling out these, these forms and it slows down and every second costs you money when it comes to like delivering a drug to market.
- [36:55]
Yeah. And the thing is like these forms, like it's not like you can have anyone fill them. Right. It needs to be someone that like is very much an expert on their feud to make sure that they're doing it right. So yes, those things can be very frustrating for sure. I think the other case that came to mind now is a lot of overlaps with factories. So people that are building. We work with a pharma company on use case like this. This one is not already in production just yet. But it's very interesting where agents are monitoring the sensors from the machines and again you could have just regular AI do this. But where the agent part comes in is if they pick up something that is odd about the data, they automatically cross that live with FDA databases and then they launch an internal investigation and kind of like what might be happening here. Does agents produce a full report that then goes into the people that will actually send a check the batch off that production. So yeah, very interesting use cases all around. I think it's going to be incredible. 2025 and 2026 definitely going to be like very exciting years for AI agents.
- [38:01]
When it comes to designing an agent, there's all these sort of different agent design patterns that exist, like reflection, pattern, react and so forth. How do people make decisions about what pattern makes sense for the problem that they're trying to solve?
- [38:15]
Yeah, I gotta say, I think if you're building something from scratch, you find people a little more curious to kind of experiment with all the different patterns and see what happens and how different architectures that you can go about things. What we're finding is that a lot of these use cases, customers want to focus on kind of like the end result. Like does this produce the end result that I need and is this cost efficient and time efficient? I think like time efficiency is not as big of a problem with agents as we have seen with other AI applications, just because people assume that those things are going to take a while to run anyway. But cost efficiency was a big problem. If you go back like a year ago now with like a race the bottom, with the LLM prices, not as much. Usually people start with react just because that's kind of like the go to and kind of like works the better. But one thing that we do internally is we map all the papers that are coming out and we implement a lot of features based on some of these papers. So a lot of kind of like the. We implemented a new feature recently around hallucination checking, using LLMs as judges as well. And that was entirely based on the paper for like a big university, but very cool paper as well. So honestly, I think as AI is getting more and more mainstream, you're getting more to the folks that are not necessarily reading the papers, they want to get value as fast as they can to actually drive kind of like value within their companies. But if you really want to see all the cool, interesting things, you got to be tuned into that.
- [39:41]
Yeah, so if I'm using crew, then I don't need to be thinking necessarily about this. This stuff's all abstracted away from me. And CREW is going to come, go and essentially execute whatever sort of agentic patterns make sense to solve the problem.
- [39:52]
Yeah, exactly. I mean again, you can change, you can go deeper level if you want to and can customize a lot of different things. Like you can customize the inner prompt, you can customize like if you want to have a hierarchical agents or not, you can customize the templates and everything. But yeah, like in crew, a lot of those decisions, if you just want to get going, they're ready made for you.
- [40:13]
What is the biggest challenge in getting enterprises to adopt agent Technology now, I.
- [40:18]
Gotta say I think it's very much early days as we're talking before. So it's interesting because education becomes intertwined with selling to some extent, especially when you're talking about enterprises as you ask. Automation is almost like a constitutive process. There's no process that is equal by between those bigger companies. So it's very kind of like a more hands on process on understanding their needs and making sure that they are getting the value that they want while helping them to get educated on evaluations and what it means evaluations and how they should be thinking about that. I know that you probably spend a lot of time on that yourself, just educating people on all of this and how they should be thinking about this. So I would say it's necessarily a challenge, but it's definitely different from what other sales motions that I have seen in the past where educational content is not as much intertwined with the selling process as you would. And then the other thing is just making sure that you're measuring results so you want to make sure that you're tracking roi. That's another thing that you want to make sure that customers understand from the get go and you're focusing on practical use cases. I hate when people come up to us saying like, hey, what should I be using this for? What are the use cases that I should be using it for? And we have helped some customers on that route and they became customers and that was great. But in the end of the day, the better ones are the ones that come like we have a clear need, we might not know how we're going to do it, but we want to help with this. So I would say if you're thinking about using AI agents, be another adopter and don't wait for other people's needs. Really think through how this could benefit you specifically and what are your pinpoints and then reach out to crew.
- [42:02]
What about the technical challenges? What are some of the things that people need to be aware of there?
- [42:06]
Well, data, right? Data is the big thing. At the end of the day, these LLMs are engines. They need the proper data for them to be working. So I think integrations is a big one. We have a bunch of integrations that we have built ourselves that kind of like usually help people get started, but there's always that internal system that no one touches for like seven years and you need to get data out of it and it's like a random API. So I would say the integrations, as one would expect, are still probably kind of like, the main hinge on getting a lot of those major use cases into production. Again, if it's something that is kind of like we already have seen things like SharePoint, SAP, Salesforce. Yes, you can get it done pretty quickly. Now, if it's an internal system or more specific CRM, then things get a little more complex. And that does slow down a lot of these implementations. Implementations. Because, again, people are eager to get to value and then now they need to deploy engineering resources to build integrations that. It feels kind of like moving backwards.
- [43:07]
So what's next for Crew?
- [43:08]
Well, I gotta say, I was talking with someone before the new year and people were asking about, like, what 2025 would look like. And I gotta say, we close more customers in the last six weeks of the year than in months before. And we close. Closed quite a few. So things are definitely taking a lot of hit. I mean, I signed a deal on December 31. December 31. Usually people are not even working and we're there signing deals with executives on kind of like major Fortune 500 companies. So I would say that for Crewai, the year is going to be either great or insane. What insane looks like in terms of not only customers and revenue, but actually in terms of growing the team, in terms of growing the company, in terms of maturity of the framework and everything that we're building. I think that's the things that are going to be very interesting and very curious to see how they're going to play out. And I think 2025 is where you're going to see other major players making a big move. Right. Like, Microsoft's definitely doing their thing. Salesforce is definitely doing their thing. ServiceNow is doing their thing. SAP is cooking some stuff. So we know there's going to be a lot of different players coming into this, so I think if anything, it's going to make everything extra interesting.
- [44:22]
Yes. Well, I'm excited for you. And Joe, thanks so much for being here.
- [44:25]
Thank you so much for having me. I really appreciate it. Thank you all for listening. And yeah, if you want to know any more about Crew or myself, feel free to reach out to me over x on LinkedIn. Thank you so much, Sean.
- [44:36]
Awesome. Cheers.
- [44:44]
Sam.