Loading summary
Transcript1 lines
- [00:01]
A
Welcome to the AI Chat podcast. Today on the podcast we're talking about one of my favorite AI companies, which is 11 labs, a new thing they just launched, which is essentially the ability that they are now offering for you to build conversational AI agents. So they're leveraging their voice models and they're actually letting you build agents right on their platform that people can talk to. These are conversational customer support. These are great for, you know, being a travel agent or helping you take an order at a restaurant. There's so many ways that these are going to be used. I'm super interested, going to be breaking down. I'm actually kind of walking through some of their documentation for their developers on how to set up an agent and I'm just going to essentially do that to explain to you how you build one of these things. I'll try to do it in the simplest way possible and some of the capabilities that it has. I think this is really interesting and this is kind of a bit of a deeper dive on how this actually works. But I think it's important because this is going to give you a really good vision of where this goes in the future, how this works and kind of everything you need to know when it comes to that. So overall, a very exciting topic. And before we get into that, I wanted to say if you're, if you want to get a daily newsletter sharing daily AI tips and different stories and tools that are breaking, go check out AI Box AI. At the bottom of the page there is a sign up where you put in your email and I'll send you every single day the top three AI news stories and just a little blurb. This isn't super long. I'm trying to make this short, useful, concise. A lot of people use it. We have thousands of people subscribed. Um, and so if you're interested, you can go get that newsletter and I'll, you know, we'll share daily kind of what's going on in the AI environment in a really hands on way. You will see, you can see tweets, what people are saying about the different stories. It's really interesting. I'm trying to make this thing useful for you. So if you're interested, go check it out. There's a link in the description to AI box AI. Okay, so what exactly is 11 Labs building here? How does this whole thing work? I think up until this point the 11 labs is, you know, known mostly just for giving a bunch of different AI voices. They do text to speech. Their head of growth, Sam Schuyler was talking to TechCrunch about this whole thing and said that a lot of their clients were already using the abilities of their new conversational AI agent to build some stuff. So in all of this they are able to use what language this agent is speaking, what the first message is, what the system prompt is to determine what the agent's Persona is. So you'd be like, hey, act like a travel agent, always say this, always do that. So you can kind of put all of that in and then you get to select what LLM you're using, Gemini GPT or claude, which is pretty awesome. And they've got a bunch of different like temperature responses and different settings you can change. So it's really kind of designed to be hands on. So I was over looking at, looking at 11 labs and how you actually set this up. They have a really interesting demo of kind of this whole tool in action where they have a travel agent that they've built and the travel agent essentially is talking to someone about a trip. They're asking a bunch of different questions. They're saying, oh, I don't want this, I want that. And what was really interesting to me is at the end of the conversation that I had, which it did great. If you know 11 labs or audio sounds pretty decent. It's one of the best, I think, AIs as far as voice goes out there. And of course OpenAI has a great voice, but it doesn't has nowhere near as many integrations and kind of stuff it can do. So at the end of the conversation it then has the history which I'll break down. It's pretty cool. So in their, in their kind of documentation they explain how to build one of these tools. They're building one for a fake store called Pierogi Palace. If you've ever had pierogis before, they're delicious. They're a Polish thing. It's got mashed potatoes and cheese in the middle of kind of like a dumpling maybe. I don't know, it's in a little like a Hot Pocket kind of thing. Anyways, they're delicious. Big fan. Anyway, so the thing that you're going to be able to do first with your assistant setup is that you go to their dashboard, you click on Create Assistant. You can either do it with a template or you can choose a blank template, which is kind of interesting, right? So they have one that is like a support agent. They have a template for that, which makes it really easy for you. They have one that's a Video game character. They have one that's a math tutor. And then you could just do a blank template. So, you know, depending on what you're interested in, you got a couple options to get this thing started. Then you're going to go and set your first message or your system prompt, which is essentially you could say something like, welcome to Pierogi Palace. I'm here to help you place your order. What can I get started for you today? So that's like the greedy message that it always says. Then you do what is called the system prompt. And the system prompt is essentially you telling it exactly how to act. You tell it what kinds of things to say. In their example that they gave, they said you're a friendly virtual assistant for Pierogi Palace, a Polish restaurant specializing in pierogies. It's located in the Zakopane Mountains in Poland. So you really get specific. And I mean, like, I can imagine a lot of restaurants will use this exact tool. They said your role is to help customers place order over voice conversations. You have comprehensive knowledge of menu items and their prices. So what's interesting is you can essentially use this. You could build kind of your. Your agent or whatever, your conversational agent, and you can tie this to your phone line. So when people call your restaurant, it could be greeted by this. Um, you could even have two. Right. One that's like screens, calls like, hey, what can I help you with today? And then depending on what you want, it'll send you over to like, oh, they're trying to make an order. Okay, send you over here. Oh, they're trying to make a reservation. Maybe I can take it or send it to another one that's set up for reservations that's tied into our calendar. So there's a lot of cool things you can do with this. Yeah, really excited. In their specific system prompt, they actually listed out what the menu items were. So, like, potatoes and cheese, Pierogi, Three Polish zloty per dozen. They have, like, prices and items, which I think is really cool. You just copy and paste a whole menu on here. And it's only pulling from this. It's not just pulling from, like, hey, I want to order this random beef stroganoff. And it's like, okay, sounds great, right? It just looks at what's on this, and it only lets them do what's on it, which is great. So it then also in their. In the prompt that they showed, like, their example, they said, these are your tasks. Greet the customer. And it kind of says how to do that take the order. It says, listen carefully to their selection, blah, blah. Confirm the order, calculate the total price, collect delivery information, estimate delivery time, provide order summary, close the conversation. So that's an exact flow of how this conversation works. And then it says guidelines. Use a friendly and professional tone throughout the conversation. Be patient and attentive to the customer's needs. If you're not sure, ask the customer to repeat, do not collect any payment information. Just tell them the payment will be handled upon delivery. Avoid discussing topics unrelated to taking and managing the order. Right. They're doing that because. So it could be like. And what's your opinion on the current political state of. You could just get these things off the rails. So they really have to put a message in there to, to stick to the script, which is kind of funny. Okay, once you have that in place, once you have both your greeting message, so the first message and then also the system prompt in place, you then go and you can actually configure your voice settings. So you can choose from over 3,000 different voices that they currently have in 11 labs. And yeah, you could listen to them, you could test them. There's a bunch, they have a whole marketplace where people upload their own voices. So there's a ton that's in there that's really cool. And I think they compensate people. So it's a, it's an interesting thing. So then after you do that, you go and actually test your assistant. So you, they have a little example button where you press order and then you have a whole conversation with that assistant. After that happens, you're going to need to configure how the data collection is handled. So you can configure how you collect and analyze all of the conversations to like essentially you can go and look through what was said, the transcript, but it also makes like a little AI summary, which is kind of cool of the conversation. They have an analysis section for the setting for the assistant settings where you can essentially define custom criteria if you're trying to like evaluate specific things in a conversation. And yeah, there's a lot of cool tools they have there. One of them is a goal prompt criteria. So this passes a conversation transcript to an LLM to verify if the specific goals was met. Right. So you could say the goal is to give them information about the menu or the goal is to have them buy something and then you can run it kind of through an LLM that's like based off of this conversation. Was this a success, failure or unknown? So they have a bunch of really cool tools like that. You can set up how you want to collect a lot of the data, including, you know, the customer's name and all that kind of stuff. And then you can go and view the entire history of the conversation, so you can see kind of a summary of it. And then you can see everything that was said in the conversation, which is really cool. There's a lot of really cool stuff that they've kind of built into this. So after that, you're ready to go and your tool is ready to take orders. Overall, I think this is a fascinating time. ElevenLabs is really pushing forward as one of the key players. They're competing obviously against OpenAI's whisper assembly AI, Deepgram, Speechmatic, Gladia, a lot of others, and they're also trying to raise right now a valuation of $3 billion. So they're actively trying to raise money. I personally think that 11 Labs is one of the best AI companies out there, one of the best audio AI companies out there. So I'm a huge fan of 11 Labs. Been using them since the beginning and have seen that they are only getting better and better. $3 billion valuation, honestly, to me, sounds perfectly reasonable. They compete directly with OpenAI. Well, OpenAI has demoed some cool voice tools and stuff. 11 labs just beats them to the punch as far as getting it out the door. So I don't know what eleven Labs is cooking up right now to compete with that, but a lot of those tools aren't, you know, all of those tools aren't even out from OpenAI. So I think there's a lot, a lot going on there that they have a lot of opportunity. In any case, if you enjoyed the podcast today, make sure to leave a review wherever you get your podcast. Make sure to go and sign up for the newsletter if you're interested in getting Daily EAI News Intro inbox in. Hopefully a really useful way to you. Thanks so much for tuning into the podcast today. Really appreciate it. Hope that you all have a fantastic rest of your day and I will catch you next time.