Loading summary
Transcript89 lines
- [00:01]
Michael Stelzner
Hey, before we start today's show, if you want to accelerate your AI learning, I have a solution for you. Become a member of our AI Business Society. You'll join me as we go deep with live AI training each and every month. Imagine crafting more persuasive content, creating stunning images and automating those time consuming tasks. It's all possible when you join the AI Business Society. Go to socialmediaexaminer.com AI and join today.
- [00:41]
Yash Gad
Welcome to the AI Explored podcast, helping you put AI to work.
- [00:48]
Michael Stelzner
And now, here's your host, Michael Stelzner. Hello, hello, hello. Thank you so much for joining me for the AI Explored podcast brought to you by Social Media Examiner. I'm your host, Michael Stelzner, and this is the podcast for marketers, creators and business owners who want to know how to use AI. Today I'm going to be joined by Yush Gad, and we're going to explore custom AI models. If you're concerned about security and intellectual property and you really want to take advantage of AI and you want to understand how you could potentially tap into some of the stuff that's going on in the open source world so that you can create something extremely sophisticated and also extremely safe and secure, I think you're going to find today's interview very fascinating. Even if you're not into that, I encourage you to listen to today's interview because it's going to give you insights into some of the challenges that we face when we're using some of these large models that are up in the cloud. By the way, if you're new to this podcast, be sure to follow us on whatever app you're listening to us on so you don't miss any of our future content. Let's transition over to this week's interview with Yash Gad, helping you simplify your AI journey. Here is this week's expert guide. Today, I'm very excited to be joined by Yash Gad. If you don't know who Yash is, he's an AI strategist and developer. He is the founder and CEO of Ringer Sciences, an agency that helps marketers build and maintain custom AI models. Yash, welcome to the show. How you doing today?
- [02:25]
Yash Gad
Doing great. Thanks for having me, Michael.
- [02:27]
Michael Stelzner
I'm excited you're here. Today, Yash and I are going to explore how to create custom AI models that allow you to scale innovation. Now, before we go there, I want to hear a little bit of your backstory. How did you get into AI? Tell us the story. Start wherever you want to start?
- [02:42]
Yash Gad
Yeah. It's really strange. You know, I'm not a marketer or communications person by trade. I'm actually a biophysicist kind of working in computational neurosciences. So that's what my Ph.D. was, was in. So I was doing a lot of developing neural network models and very traditional AI models that we were using to kind of understand how the brain controls eye movement. And so I got the opportunity to kind of really take this in a different direction when I joined with a marketing and PR agency and to really see like, what these kind of neural networks could do for like things like natural language processing or understanding behaviors or understanding network effects and seeing how we could use AI tools to really accelerate our understanding of those things. So that was kind of how I started seeing where AI could really go in terms of broader applications.
- [03:29]
Michael Stelzner
When was that?
- [03:30]
Yash Gad
That was back in 2013, when I made that jump into, into kind of agency world and really saw kind of working side by side with analysts, like the kinds of questions they were asking and really thinking through how models that I had worked with in my research could potentially be applied to the kinds of things that they did.
- [03:48]
Michael Stelzner
Okay, so that's really intriguing because you're very early obviously, right? I mean, in the AI space, did they even call it that or did they just call it neural networks back then? Or what were they calling it?
- [03:58]
Yash Gad
Networks? It was machine learning. It was a lot of different, you know, NLP tools. There was a lot of this, this kind of thinking. But if you really pull away the, the terminology, those things are basically the same building blocks that went into ChatGPT and, and other things.
- [04:13]
Michael Stelzner
Okay, so somewhere along the way, like tell us a little bit about the journey. You started your own business somewhere along the way. Right. And tell us a little bit more.
- [04:21]
Yash Gad
When I was with this previous agency, I loved working with analysts, really loved kind of building tools for them to really answer very specific research questions or to do a very particular kinds of analysis that was either just automation or automation plus some AI and really kind of thinking through, like, what that could unlock for them in terms of new offerings, new capabilities, new understanding. And so when I left that agency, it really was with a vision of rethinking how this could work for not just the industry that I'd been working in, you know, for most of my career, healthcare, but just in general the kinds of problems that we were seeing. Could we rethink that from first principles? And if I wasn't at this agency and I just wanted to do it on my own Terms, how would I do it differently? So that was really the founding principle for Ringer and that was in 2018. I really wanted to tackle all these problems fresh with all the most contemporary AI tools, the most contemporary data science tools, and really see what we could start building out. Probably different industries.
- [05:18]
Michael Stelzner
Okay, so is there some sort of AI thing? Does Ringer mean something maybe in the neuroscience world that the rest of us do not understand?
- [05:26]
Yash Gad
No. It really came as kind of a tongue in cheek thing where a lot of people that were enlisting our help at the, in the early days of Ringer really were interested in bringing us into be that expert for their data science team or for their, you know, their company in general. It's. We kind of use. The catch was you're bringing in a Ringer, you know, you're bringing in that, that kind of expert, that secret weapon. Maybe sometimes the teams don't want it in there, but they're appreciative once that person is in there.
- [05:54]
Michael Stelzner
Okay, so once ChatGPT rolled out, which is almost exactly two years from the day we're recording this, how did that change demand for your services? I'm just curious quite a bit.
- [06:04]
Yash Gad
It was very interesting because, you know, like I mentioned, we had been doing these things quietly and had been part of our DNA for, for years. We had just been, you know, taking it for granted almost that we were using these AI tool and then all of a sudden everybody was using these chat tools. Everybody was an expert now in AI and people would ask us these questions, you know, like different engagements. We were there for doing analytics or for some data science work. And they were like, hey, have you all used this chat GPT? Have you all like, what do you think about this? And we're like, of course we've used it and you know, here's some applications. And as we were talking through it with them, we realized that not only did we need to kind of start helping steer folks and strategizing how you actually think about this new tool, but they had no idea how to actually bring this into their organization or do this for real. Like it was this toy that they were playing with, but to actually bring it into their business, have it, have real function. Most of these teams had no idea where to start. And this is coming from really tech savvy companies that had really great IT teams, development teams that followed through some use cases that had no idea how to get started. You know, these tools were out there, but just there was such a barrier to entry, so they were just asking, asking questions and we just happened to be in the right place.
- [07:16]
Michael Stelzner
So what are you doing today for most of the kinds of clients you're working for?
- [07:19]
Yash Gad
So we very quickly developed a niche around what we're calling kind of custom language models, which is really starting to move away from, you know, let me just throw all my data into a ChatGPT or some other large model like this and see what happens. But being very precise about what we're doing, you know, a lot of these clients are sitting on proprietary data. They don't want to just be going out into, into the wild. And ChatGPT or that's very sensitive documents or they have a very domain specific problem they want to solve some really niche problem in healthcare or some really specific problem in financial services that they want to answer. And they need specific data, they need a specific understanding. These generalized models just can't do that. And so we kind of have been tasked with coming in, helping to stand up a very secure data structure, a very secure model that's not going to be sending this data out into the wild and then having this all baked within their IT infrastructure. Very secured model that's walled up but also customized to the point that it's very, very, very good at these handful of tasks that they want to do.
- [08:23]
Michael Stelzner
Excellent. And folks, we're going to be talking about something today that isn't just applicable to enterprise, right? I mean, like anyone who's listening, whether it be small business or big business, can benefit from what we're talking about today, even though a lot of Yush's are obviously more sophisticated businesses. So there's plenty of people listening right now and we're going to get into what exactly this custom model is. But let's start with the upside. And you've kind of hinted at a little bit, but why should businesses focus on custom AI models or custom language models as you refer to it? What's the benefit?
- [08:55]
Yash Gad
You know, when you start off with the security question that that's a good place to start because I think a lot of businesses, especially small ones, are sitting on a lot of proprietary domain knowledge and it could be completely detrimental or destroy their business if that, that information got gets out there. If it's part of like, you know, what ChatGPT is being trained on for the next model and everybody has access to that when they're poking at this model. You might lose completely what make, what differentiates your business. That's one part of it, but really the, the other parts of it are, you know, around, you know, how this model can be kind of really dialed into very specific tasks. Forgetting the privacy part of this, making it really specialized in one specific thing that you really understand. Well, that's a key distinction, you know, so that's these custom models are really good at doing those handful of tasks that are important to your business. And then the last part of it is especially for like applications that will be customer facing, which you know, we did for one of our clients. Prompt analytics is a big repository of data that people really haven't tapped into or called out as being like the next frontier for analytics. You know, really understanding how people are prompting these chatbots, how they're conversing with them and what data they're getting from them is going to be huge in terms of understanding people's behaviors, customers behaviors, and really could be that next bit of IP that companies sit on.
- [10:15]
Michael Stelzner
Fascinating. Okay, so that last part kind of intrigued me a little bit. So we'll get to that. So let's first define what the heck is a custom AI model. I think everyone who's listening knows what ChatGPT is. It seems to not be that. So why don't you define it in a way that we can wrap our heads around?
- [10:32]
Yash Gad
Sure, yes. When we kind of set aside like the framework for a custom language model, it's a couple of different components. So it's thinking about this now in terms of your data and now a secure store for that. Whether it's on premise like a physical computer in your, a warehouse somewhere or in your cloud environment, your sensitive documents stored and structured in a very specific way where we've kind of tagged that data for information that will be important for a very particular use case that you've laid out. Then it's the actual model itself. So this is, you know, all this is like part of that same custom language model data, the model and then kind of the front end of it. The model itself can be either a localized model, like a llama 3 or something else where it is sitting on a machine. It's not ever touching the Internet or drawing information from, from elsewhere. It's kind of very specialized to just look at that data that you've parked in a very secure location and then either fine tuned learning from that or actually referencing that in a rag approach. And so then you can use a localized model, you could use APIs like ChatGPT, but you can now control the kind of data that gets passed back and forth to those kinds of models. So you've created a very intentional connection to these models. Whether it's A localized model, whether it's these publicly available models and how your data corresponds to that. And then the last part of it is like creating your own front end onto this. So making sure that this interface is very specific to your use case. Do you need certain buttons, do you need certain reference material available in this front end to make sure that this flow does exactly what you need it to do? Sometimes you don't even need a chat interface. Sometimes you may not want that as part of what this, this model is being trained to do. And so really making sure that is aligned with exactly your use case, and then that you're tracking any of the interactions with that model, your own prompt analytics and storing bad data securely as well, because that's ip. That's IP that you've generated on your secure model on your specific use case. And so the custom language model framework is really all of these pieces working in harmony, controlled by that company.
- [12:36]
Michael Stelzner
Okay, so what I heard you say is, number one is a set of data that's secure, whether it be on your computer or up in the cloud. Number is picking the model, and we're going to talk about that a little bit later. Number three is having some sort of an interface where your customers or employees are using it. Right. And then number four is some sort of an analytics layer where you know that people are using it. Is, are those the components? Did I hear that right?
- [12:58]
Yash Gad
It's exactly it.
- [12:59]
Michael Stelzner
Okay, cool. So what I'd love to explore now is some creative applications of things that either you've done with your clients or hypotheticals so people can wrap their brain around what is possible with a custom model.
- [13:14]
Yash Gad
Yeah, there's some really interesting examples that we have of kind of ones that have worked out remarkably well. Our very first custom language model actually happened almost a year, a year and a half ago, where we really kind of set out the framework for what this could look like. We were having these, you know, as I mentioned, like these AI discussions with different partners as we were being brought in for different kinds of work. And they had a branding exercise that they wanted to do. They had a product launch they were getting ready to do. They had all of these assets, past press releases, past branding documents. They had competitors that have released similar products. And they were like, can we use ChatGPT or some AI model to kind of just synthesize all of this data and do something with it for branding? To us, it was an intriguing ask because we're like, yes, but there's things that would have to happen to make this successful. And so we took it upon ourselves to really create the framework for what became the custom language models. We took all this data, we, you know, we scraped the Internet for a lot of these different sources. We tagged it with very specific information that was going to be used for what audience was this about? What topics or kind of focus for the branding, you know, did this particular piece of content can relate to? And really just starting to organize this information for an eventual model. Then we had, you know, a model that we, that we kind of baked kind of a localized model. I think we were using a Llama model for this the first time around.
- [14:38]
Michael Stelzner
Just explain what Llama is for people that don't know what it is.
- [14:40]
Yash Gad
This is one of the kind of open source models that are, that are available so, you know, kind of different than a chatgpt. This is a model that you can fully download onto a machine that has a GPU and actually run it yourself. Kind of completely walled off from anything else.
- [14:54]
Michael Stelzner
And it's designed by Meta by Mark Zuckerberg. Right?
- [14:56]
Yash Gad
Mark Zuckerberg. Correct. There's a lot of these models that are starting to be released. There's actually a great leader leaderboard on Hugging Face that actually has a listing of both like public or kind of private models as well as open source models of how they're licensed. And there's new ones that are coming out every day that have, I would say even better performance than Llama. And it's great to kind of see the progress. They're all trained on different kinds of data, but the, the open source models, what's great about them is that once you download them, you can kind of then really understand how I want to modify this, how I want to interact with this in a completely safe environment. You don't worry about that, you know, your information going elsewhere or training somebody else's model. It's really just this now becomes your baby to play with.
- [15:37]
Michael Stelzner
Okay, so go back to the story. So so far, here's what we know. You've collected all this data from your client, you have gathered some information on the Internet and you've downloaded a Llama model. And continue please.
- [15:50]
Yash Gad
Yeah, and so we, we then decided to set this up in a, you know, a rag approach where we're not actually fine tuning the model, we're not actually modifying this base model, but we're instead having it interact with this data and pull it in as different queries are being given to the model. So it's, it says, okay, I need to reference documents about patient centric content that was about research. Okay, let me find which of the content was really about that. And now let me bring that into the context for what the model is going to respond with.
- [16:19]
Michael Stelzner
Okay, real quick, my audience is not going to know what RAG is, so why don't you just explain what that.
- [16:23]
Yash Gad
Retrieval, augmented generation. So it's basically exactly what it sounds like. I'm retrieving data from another source and now I'm augmenting the thing that I'm going to generate, the actual thing that the model is going to spit out with that information.
- [16:35]
Michael Stelzner
Okay. I've heard a lot of people throw the name RAG out there a lot. Is RAG something that all the models, even ChatGPT and Claude use, correct?
- [16:43]
Yash Gad
Yeah. Whenever you upload information to like a ChatGPT or to a Claude, they give you an option of just, here's some data that I want you to use for that Next thing that I want you to generate. Here, here's an Excel file, here's a PDF. It's basically doing that. It's kind of translating it into kind of a transient data source that it can now bring into the next thing that it's going to generate and saying, okay, here's all the data, reference this, and now here's the prompt the user gave me, combine it with this knowledge and now generate me the answer to the user's question.
- [17:15]
Michael Stelzner
I feel like this is really important concept for people to wrap their brains around. Right. Because if you're new to AI, you probably think of ChatGPT as this thing that's got this massive knowledge base out in the cloud. And a lot of times we've been trained to narrow it and act as if it's an expert, but what it doesn't have is your data. And if it sounds like what I'm hearing, you say, if you don't tell it to just exclusively reference your data, it's going to go ahead and reference other data as well, right?
- [17:41]
Yash Gad
Absolutely. And this is why you see a lot of, like, hallucinations. Like it's just making things up or it's going out and it's trying to scour the web and trying to fill in its gaps. It's grabbing random things it might be finding in the search results. If you haven't been intentional in feeding it a very specific amount of data, it will just go off the rails. And if you are actually saying, only reference these documents that I've given you, you're more likely to get answers that are actually very relevant to what the user was asking for.
- [18:07]
Michael Stelzner
Okay. Since we're down this rabbit hole, when you're dealing with this rag thing, do you have to call it out by name or do you just exclusively tell it? Hey, only pull your data from these sources. Do not make anything else up. Is it something along those lines?
- [18:20]
Yash Gad
It's exactly that. A lot of the current models, like Even like a ChatGPT or Cloud or some of those, you really just include that in your prompt itself. That, as you're putting it in, is like, hey, I just gave you some documents. Reference those. If nothing else, don't make anything else up. You know, you can actually specifically tell it. Don't lie. Don't, you know, don't, don't extrapolate. Just reference explicitly what's there, and that'll give me the answer to my question.
- [18:42]
Michael Stelzner
Okay, so back to the example. We've got this company that you're working with and they're doing a launch. We've gathered all this data, we've installed a local model, in this case, Llama, and continue, please.
- [18:55]
Yash Gad
So then we set up, you know, a light chatbot, and then we just started prompting it. We started prompting you with the kinds of questions that a branding team might have generate me mission vision statements based on, you know, a patient centric focus with research elements or a provider or HCP focus with these kinds of elements. And we just generated draft and draft and draft of a lot of these kinds of things. And it was very quick. I mean, it took a bit to get the prompting just right. But once we got it right, it was just cranking these things out really quickly. And so end to end between gathering the data, setting up the framework for the model, prompting this, generating a deck that could then go in front of the C suite of this company. It was two weeks end to end, and this is a branding exercise that probably would have taken, you know, a larger agency months to do and, you know, tons of, you know, wrangling data and discussions and iterations. And so we very quickly got to something that was very usable. We had a workshop with the C suite and presented this. And really their changes were, okay, I like these drafts that you have, but I might take this sentence from draft one and this sentence from draft two and merge those together. But those were the extent of the changes, largely. They loved what came out of the models because of how we had backed them up with this data.
- [20:07]
Michael Stelzner
I love it. Okay, so one of the questions that might be going through some of the more sophisticated audience's mind is, can't I do this already with Claude what's your reaction to that? I mean, is it a safe place to start? Store all your company data?
- [20:20]
Yash Gad
I would say no. I would say that there were very specific things in the data sources that we use so that it wasn't just public information, it wasn't just stuff that you would scrape off the Internet. There were very specific things that had not gone public yet about the product that was about to be released, things that they would not want going elsewhere. And this is, I think the main thing that we hear from IT teams across a lot of these enterprise clients is we don't want these tools in house because we're afraid that people are going to just throw sensitive material out into the wild and not realize it is stuff that will be used to train the next iteration of a model. They're very paranoid about that. And so, and rightfully so, that is information that you need context around and don't want to just be misused in different ways. And so there's the same thing for this client. I think they really wanted to make sure that this information was being very tightly controlled and only being used for this exercise and not the rest.
- [21:13]
Michael Stelzner
Many of the top experts you've heard on this show will be speaking at Social Media Marketing World 2025 and with your AI ticket you can attend at a very economical price. You'll discover practical AI workflows and advanced AI automations that will increase your productivity and save you time. Imagine getting live and in person training from Matt Wolf, Chris Penn, Jeff J. Hunter, Rachel Woods, Molly Mahoney, Brian Piper, Jeff C and many others. Isn't it time to enhance your career by fully embracing AI? Grab your tickets now at social mediaexaminer.com Aicon we also talked about social media reports as an example that you were going to share, if you could share that one too.
- [22:00]
Yash Gad
Yeah, this was a more recent ask that we got. So anybody that's worked in an enterprise setting with a lot of different agency partners and as a marketer, you know that the one thing the agency partners love doing is sending social media reports or earned media reports or you know, media reports over to the client. You know, these great decks that show all of these different metrics about how well a campaign did, how well their marketers did, how well an event, you know, went for in terms of, at a conference and just they have these, they cherry pick these different metrics, they have different ways of describing it that make them look good. They throw all these over to the client, our client who was sitting with all of these, they said Well, I don't know how to make sense of any of this. So can you build me an AI model that will actually make sense of all these different reports that I have, but in the way that I think about them? So we built an AI framework that basically took his understanding of how all of these different metrics fit together. Like, you know, these 10 agencies might call this one metric, 10 different things. But what do I want to call it and how should I think about all these different versions of it? Does, you know, is 1 10x the other one because of some scaling factor they have? And I want to normalize all this. Okay, that should be baked into this process as well. You know, are there different channels like, you know, there might be used different metrics for LinkedIn versus Twitter versus something else? Okay, how do I normalize all of this? Is it for different disease areas that I may want to think of these things differently? Okay, now how do I take all that into account? So baking a model with all of this knowledge. So again, using a rag approach, can we use all this information within a model to now make it able to just take in these PowerPoint presentations that all these agencies would throw over the fence, ingest them, and just spit out an Excel sheet that is now normalized all this data and organize all this data. So no chatting, no nothing like that, but just an interface where they could, you could upload all this data and get back fully clean, normalized data. And so that's what we ended up prototyping for this client. Worked out remarkably well. You know, there were hundreds of these presentations that they had gotten with, would have taken forever to go through, and probably still would have been a very manual exercise to kind of piece it all together. And we were able to just crank through this in one go and gave him a repeatable process. So now any new reports that come in, he's able to just throw them into this tool and get out this great normalized data.
- [24:14]
Michael Stelzner
Love it. Okay, let's say we want to start with a custom model. Where do we begin?
- [24:18]
Yash Gad
So the very first step and the thing that we always push people to is what is the use case? What is the real use case? What are you trying to solve? And it can be something small, it can be something really, you know, large, but getting to that very specific thing. So like, you know, for the social media report example, you know, it was, can I get to normalize data from all of these reports? That was the base ask from the brain exercise one. It was, can I get to mission and vision Statements that are patient centric and HCP centric from all this data that we had. Great, that's a great starting point. From there then we can start unpacking. Okay. In order for a human to answer that question, what data would that person have needed? What knowledge would they have needed about their space? What documents would they have needed to reference in order to now make that use case doable by a human? Once we got those two pieces, then we can start building out the framework for a custom language model. It starts with the data, it starts with being building that secure data set. But then it starts to become. Okay, now what model should I go with? That is an important part of this. So there's a lot of considerations. You know, we, we talked about some of the open models that are there. There's a whole family of these that are all been trained on different sets of data.
- [25:32]
Michael Stelzner
Real quick, before we get into the models, let me just ask a couple clarifying questions and we'll get to the models for sure. So number one is what's the thing you want to solve for? It sounds like in one case it was like a one and done thing with your first example. Right. And in another case it' an ongoing thing. Right. And I would think that in an ideal world businesses would want to develop something that would have ongoing benefits for the company. But I can also see scenarios where there's this monumental task like digging through mountains of data to try to find something. And in that case it might be a worthy exercise to essentially save them time if they're up against the wall. Right. Or give them maybe a second opinion or save them having to go out and hire a massively expensive branding agen. See, right. But are you finding most people their one thing is an ongoing thing or is it typically a one time thing?
- [26:26]
Yash Gad
It's almost always an ongoing thing. I think because they, to your exact point that they, they want to see that long term value, that return on investment of like I'm building this AI thing, sure you can, you can weigh that against just a one time contractor doing something. But there's a lot of value to be had if this is a repeatable thing. And even just searching for a bunch of documents or sifting through hundreds of thousands of old documents that I might have, that's probably something people will want to keep doing, you know, to keep having this chatbot live, to keep feeding it with more information and be this knowledge repository. So we're always steering folks toward making these things repeatable or expandable by. Okay, well what if you used it for this one use case, for understanding this, this bit of old documents, you have great. Now what if you start adding new documents, does that unlock new use cases for you? Can you start seeing ongoing value for this thing and helping them to realize that it's not just this one pet project they did, but it can become the starting point for building out a suite of things within their organization.
- [27:22]
Michael Stelzner
Now onto the documents and the knowledge there's going to be some people listening that are all bought in on AI, but they got to persuade somebody inside the company that's, that's been told AI is, is going to steal my data and stick it out there in the world. Can you give them a couple of tips or talking points on how they can have their cake and eat it too when it comes to this? Objection.
- [27:39]
Yash Gad
We've had so many of these AI discussions. A recent client, actually for the first time that we'd ever seen this in kind of our history, you know, had an AI due diligence form that they actually had us jump through. And actually we're asking the hard questions of how is the model that you're using for us going to be trained? Is our data going to be used for training somebody else's model? Is our data going to be going elsewhere? And I was like, wow, this is a really sophisticated IT team that was starting to ask these questions. So these are the things that we've started responding with. You know, when you start thinking of walled off models, the first thing that you're doing is ensuring that that data is never going outside of your own IT infrastructure. So the IT will always have their arms around where that data is going. You're locking down the model so you're ensuring that the, again, the models themselves are never going to be contacting the Internet. They're not going to be going out outside without it's express permission. Everything is very tightly controlled. There's no kind of points where, where somebody else could get in on your data, get in on your model. Everything is tightly controlled by the IT team if they are in fact really, really worried about even cloud security. And we have a client that we just started working with it, they have had security breaches with their cloud environments, have had data breaches. So they actually want a fully on premise solution and that's also an option. So if an IT team really wants to be locked down, these are solutions that can be implemented in an organization where you have a machine locally hosted and has all the data and has all the models right there on that one. Machine inside your office and it doesn't go anywhere else and can only be accessed internally on your local network.
- [29:10]
Michael Stelzner
Okay, thank you for answering those questions because I know those are important things. Let's talk about models. You know and I know and you already mentioned this hugging face thing, but it seems like every day there's a new update to these models. So when you select a model, are you stuck with the model forever? I mean, help me understand how important this decision is.
- [29:30]
Yash Gad
That's a great, great question. So I would say that if you're building these kind of solutions the right way, you shouldn't be ever stuck into a single model. These models are constantly being trained on different data. They're all going to be good at different things. None of them are going to be good at all things. You know, we have, we've seen some models that are really, really great at like, healthcare examples. They've been trained on tons of healthcare data, claims data, a lot of background information. They're really good answering medical questions, whereas a different AI model may be really bad at answering those same kind of medical questions. But then one might be really good at answering questions about content or social media because it's been trained on tons of social media data. And so it's really, you know, the first part of selecting a model is really understanding what has it been trained on and what is it going to be good at?
- [30:13]
Michael Stelzner
How do you even know that? Where do you find that information?
- [30:16]
Yash Gad
Most of these models have good documentation. The ones that want to explain how they got to their model have been really good about, you know, telling you what kind of information this has been trained on. How far back, you know, was this the last ten years? This is the last three years of data. Where does it stop? When has been taken out? What kind of assumptions have they made about the data? You know, the good models have actually given you a lot of, you know, exposition about this. The bad ones, you know, it's a black box. And I'm very wary always of black boxes because you don't know why it's going to be, you know, react weirdly sometimes to things you're prompting.
- [30:48]
Michael Stelzner
So which ones are the black boxes? Are we specifically referring to Claude and ChatGPT?
- [30:53]
Yash Gad
Yeah, and mostly those, I think in ChatGPT more so I think they're intentionally keeping a lot of it vague because it's just a giant amount of information. I think it would. To document everything they, they've been using for training would be a monumental task. But I also think that there's probably A reluctance to show all of the.
- [31:10]
Michael Stelzner
Different details about that because of copyright infringement. I'm sure of it. Right, okay, so what are the models that, I mean, as of we're recording this today, what are the models? If someone wanted to kind of start with a checklist of models they might want to analyze that they could lock. What are the models they might want to be looking at?
- [31:26]
Yash Gad
I think the two that I would start steering people toward, Mistral and Llama 3 are probably the first two that I would kind of go to. You know, there's, there's a great leaderboard on Hugging Face which is again just a great repository of tools, open source models, leaderboards, examples, you know, recipe books of people having implemented these things before of just code that, you know, people have used in the past. And they have a great leaderboard that basically is just open LLMs and it's currently showing, you know, accuracy on different levels of tasks, relevancy of responses, things like that, and just user reviews on how well these models performed on a variety of tasks. It's a constantly evolving space and they talk about the size of the models, how easy they are to deploy. And this is a constantly changing thing for us. You know, one of the big things is if I'm suggesting to a client, this is the open model that I want you to use. Exactly the question you asked, in six months from now, a new model comes out, how easy is it going to be for me to swap in that new thing? And so we want to build our solutions in a way that the models themselves become hot swappable. So if the next, you know, Llama Llama version comes out, I can swap it in for the Llama three or I can swap it out for the Mistral that I was using before without any other disruption to the entire other system that I've created. I can just swap out this model. One of our early enterprise clients actually had the great idea of wrapping a lot of these models, even the public ones, with their own API wrapper so that we were basically accessing this wrapper and not any of the models directly. So that way it's fully under it's control. They knew which models they wanted this.
- [33:04]
Michael Stelzner
So it sits in the middle. Explain what that means a little bit.
- [33:07]
Yash Gad
Yeah. So basically imagine like if I'm normally interacting with these public models through an API like the ChatGPT or Claude or you know, even a locally hosted Llama somewhere, I would be hitting an API or be making a call to them, I would send some information to them. The model would process it and then send it back via that API. Now imagine that this company has instead decided to wrap that in their own infrastructure, their own code that basically becomes a pass through for that information, but it can flag, you know, hey, I just saw that there was some sensitive data that came through that I'm going to block that call from happening or I don't want you sending this much information. Let me cut that, that information off. So you can put in some of these guardrails there and you can make it so that it's a single entrance point to any of these models. And if it decides they want to add in 10 more models, they can do that within the structure of that API and we just have to connect to it one way. And so this becomes a very flexible system now where somebody else can manage the different models that are available to this custom language model. But now you've built out this infrastructure that is very scalable with changing models over time.
- [34:12]
Michael Stelzner
Okay, this is really fascinating because I've had some people on the show that talk about make.com integrations and it kind of feels a little bit like that, but obviously more sophisticated because you're using API calls and all that fun stuff. But basically what I'm hearing you say is you've got this, for lack of better words, technology that sits between the user and the model and it's got some decision making structures that are going on and maybe it's smart enough to know that if it's this kind of a task, go over to this model and if it's this kind of a task, go over to this model. Is that what I'm hearing you say?
- [34:45]
Yash Gad
That's exactly right, yeah. You can make a lot of those kind of decision points so you can bake that into kind of some of the infrastructure. And again, this gives it a lot more control over bandwidth, over model selection, over, you know, just how users are kind of doing this. And it gives them an in between place to track user usage prompts, things like that, you know, on their own. So it just, it gives you again, a lot more control. That's really what this framework is about, is really giving people more direct control over all these different pieces instead of I'm sending things out to a black box.
- [35:14]
Michael Stelzner
Okay, I want to ask you a couple questions about APIs and then I want to get into Mistral and Llama 3. So some of this stuff is selfish. Like we've got somebody in house that understands how to do some of this stuff and maybe some people that are listening have Techies in house and maybe they're not as sophisticated as the people you work for. But my understanding is when you make an API call to like Claude and or to chat GPT that that data is protected. I want to know how confident you are that that's true.
- [35:40]
Yash Gad
I mean, protected is a very strong word.
- [35:43]
Michael Stelzner
I mean secure maybe, you know, secure.
- [35:45]
Yash Gad
From other people seeing it. But I mean, cloud is definitely seeing what you're prompting it with. OpenAI is definitely seeing what you're prompting it with. If, for example, I sent through that API an entire, you know, confidential brief from my company that's out there, that's going to be used in that next iteration for Chad GPT. And so what you send out through those API calls, if it's not being monitored, if you haven't put your own structure in place, is just going out there. Now there are systems that, you know, have people started to build, like, you know, like I was mentioning some of these IT teams where it actually will look at that prompt and say, like, you know what? Actually I, I can already flag financial information in here. I can flag different pieces of information here that absolutely should not go out.
- [36:25]
Michael Stelzner
Like customer identifiable information or something like that.
- [36:28]
Yash Gad
Correct, correct. But these systems inherently will not be flagging that for you. You know, and if you look at the terms of service for a lot of these things, even some of these great tools that are out there, like, you know, Jasper AI is I think, a great one that a lot of people have started using that, you know, really gives you some flexibility in developing your own models. But if you look at the terms of service, they, they do flag that the user is kind of on the hook for understanding what they're putting in there, what they're uploading into these systems. So this kind of takes that out of, you know, the user wanting to understand that or having some understanding of that. If you've kind of built this securely from the get go.
- [37:01]
Michael Stelzner
Okay, cool. So we have a lot of people right now that are trusting that the API is not going to train the model on the data, but nobody really knows. It's the honest truth. And you got to play it safe, especially if you're a larger entity. Mistral, my understanding is that's a French company. Is that right? And then Llama is kind of explain a little bit about what those two models are and what your experience is with them and what kind of their benefits are of using them.
- [37:26]
Yash Gad
Yeah, so both of them are kind of open models that have kind of been developed by these different groups and They've been trained on a wide variety of data. Both of them not as wide as ChatGPT. I think like, you know, they've been kind of more restrictive. The timeframes, the main restriction, the main differences that we've seen between the two models are the size of deployment as well as how responsive they are and the flexibility they have with a lot of different prompts and a lot of different language. So Mistral, what we've seen between, if I'm doing side by side comparison of Mistral vs Llama 3, Mistral largely has performed better in most of these cases. It's faster, you know, the smaller models work better than smaller models. For Llama 3, there's usually different tiers for a lot of these open models, like a small set of weights or medium set of weights in a really, really big model. And this basically tells you how big of a footprint this is going to have on your computer. Like if I download this thing, one thing might be, I'm just making numbers up here. We're like one gigabyte versus 100 gigabytes versus a thousand gigabytes, you know, and there may be just different orders of magnitude there, but then also the hardware that's required to run it. How fast inferences is going to be like if I'm prompting this model, how fast am I going to get, get information out of it? All of these things get impacted by the performance of the model, how the model is structured, how much it's optimized for users implementing it themselves. Mistral I've seen is a little bit better on some of these things than the kind of comparable thing in llama 3. It also seems to do better with some of the open ended kind of messy language that you might see if you're, you know, you have like a social media example or you're trying to kind of synthesize, kind of mess your data. It seems to do better with that than Llama 3 does. Llama 3 seems to be a good generalist. I think, you know, in, you know, it's, it's good, decent performance in all things. And I think it was probably the first really good open model that was out there. A lot of the different Internet forms and things, you know, really are kind of baked around that. In fact, one of the best resources that I know, I think I've used still, you know, is the Llama Lounge, which is a subreddit on that really has a very active community around, you know, just helping people navigate these, these open models. It started off around Llama and the deployment there, but now has expanded to all the open models that are out there.
- [39:45]
Michael Stelzner
I mean, I know Zuckerberg has kind of unlimited money and he's determined to kind of win because he believes Open will win. And it's very counterintuitive to the way he's built all of his companies, but people have this natural distrust for him. Do you feel like the developer community is embracing llama3 or are they more embracing Mistral? I don't even know who's behind Mistral.
- [40:04]
Yash Gad
I do think that they are largely embracing Llama 3 for general purpose. I do think that, you know, it's what I liken it to Is this my date? Me a lot is, you know, the folks that were kind of using Windows and Mac back in the day and then all of a sudden you had this group on the side that was using Linux, you know, on their computers and anyway, and it was, you know, it wasn't as well supported, but it's faster, it was easier to deploy and if you really knew what you were doing, you were actually going to get much more benefits out of it. That's kind of how I view Mistral and some of these other kind of open models. It's like it's not as mainstream to people that are outside the community as other models like Llama 3 might be more well known. But there is a lot of benefit if you are kind of willing to go through some of those steps. The documentation may not be as good, there may not be as much prior art out there, but if you kind of are willing to experiment, you actually get better benefit out of it.
- [40:54]
Michael Stelzner
One name we have not dropped is Google. And obviously they're putting a lot of money. Their models are not open source, to the best of my knowledge. What's your general opinion of the Google ecosystem?
- [41:04]
Yash Gad
They're not great at what they do. It's unfortunate. We've tried using them. We've had enterprise clients that have actually wanted to lean on them.
- [41:10]
Michael Stelzner
On Gemini?
- [41:11]
Yash Gad
Yeah, because they are within the Google ecosystem already. And they were like, why don't we just stay in that vein? And we're fine with that. But the performance isn't as great as you will find in other private models or other open source models even. And it's unfortunate. And I'm not sure if it's the data they were trained on, the structuring of the model under the covers, or a combination of both. If I was to guess it's both. It's kind of messy data. It's how they Structured the data and it's some of the guardrails they put on the models.
- [41:41]
Michael Stelzner
Okay. A lot of people that use the big models, ChatGPT and Claude absolutely love Claude, especially because of its use of English language, you know, like in the marketing community in particular, they love Claude. Do you feel like Mistral or Llama 3 are there yet as far as their ability to create content, or do you feel like it's just a matter of time before they'll catch up?
- [42:00]
Yash Gad
It's a matter of time. They're getting really close. If you'd asked me this like six months ago, I'd have said it's really, really far apart. The Llama performance was really bad. The Mr. Performance, it wasn't as bad, but it was still not anywhere close. It's getting really close now. And I would say, like in some scenarios, better for really, really specific domains, I think it's gotten really better. I was giving the example of like healthcare or, you know, some, like, you know, technical areas, engineering and things. Like some of these models have gotten really good at those, those things because of what they've been trained on. So I do see over the next six months to a year, a lot of catch up on this front where the private models are really not going to have that giant leap on everybody else. Now, I will say that ChatGPT has a good head start on everybody and it's going to be hard to kind of match some of the newest models that are coming out there. And you know what's being promised, I think, for the next release for ChatGPT.
- [42:51]
Michael Stelzner
Just so people understand this, when you put these local models on your computer and they have an update, is it just like updating software or do you have to like, not do that? Like, let's say the Llama four comes out, Is it just a matter of replacing the model or is it way more complicated than that?
- [43:06]
Yash Gad
Usually it is just I have this, this existing model deployed. Let me just literally rip it out and deploy this new one. Hopefully the existing hardware and everything I had set up supports this new model. Sometimes it won't and I may need to go to something that's, you know, a little bit beefier. But the hope is that everything else, all the other infrastructure I built around it works the same way. It's not always the case. Sometimes, you know, people have made changes on, okay, the model actually needs to see the data this way or it needs, there's some tweaks like to how it needs to now read in things, but largely everybody has now fallen into the same structure where the models are all receiving the prompts in the same way, the context in the same way context window might have changed. Like how much data you can feed into the models may be the thing that's changed. And so those are some of the coding changes that we have to make onto the covers. Maybe we would have restricted how much information we sent to these models before we're like, oh, this new model gives us much wider window. Great, let's throw more information at. And that's a small coding chain. So those are some of the small changes that might have to happen. But largely these things should be hot swappable.
- [44:09]
Michael Stelzner
Yes. Gad, thank you so much for sharing your insights with us today and answering all my litany of questions. If people want to connect with you on the socials, what's your preferred channel? And then if people want to contact you about possibly exploring working with you, where do you want to send them? What's your website?
- [44:25]
Yash Gad
Yeah, so ringersciences.com is our website has a lot of our kind of more recent blog posts as well as just a general overview of the kinds of things that we do. We don't have a lot of our AI case studies up there yet just because it's been such a fast moving space like you know, which what do we put up there? But that's a good place to kind of, you know, to see an array of the things that we're doing. But also on LinkedIn, connect with me there. I do have kind of a newsletter that I've started that just has some basic ideas about the kind of current things, the current thinking on custom language models, but also a great way to just connect with other like minded. So please reach out to me there.
- [44:58]
Michael Stelzner
Yeah. And for folks listening, his first name is spelled Y A S H and his last name is Gad Yash. Thank you so much for sharing your insights with us today.
- [45:06]
Yash Gad
My pleasure. Thank you so much for having me, Michael.
- [45:08]
Michael Stelzner
Hey, if you missed anything, we took all the notes for you over@socialmediaexaminer.com A33. Be sure to follow this show on your favorite podcasting app. And if you've been a regular listener, would you let your friends know about this? And also I would love a review. So whatever you're listening to, if you're on Apple or Spotify, could you give us a review and let your friends know about the show? As I already mentioned, I'm Elsner on Facebook, Stelzner on LinkedIn and ikestelsner on X and check out our other shows, Social Media Marketing Podcast and the Social Media Marketing Talk Show. This brings us to the end of the AI Explored Podcast. I'm your host, Mike. Michael Stelzer will be back with you next week. I hope you make the best out of your day and may AI help you become more successful.
- [45:55]
Yash Gad
The AI Explored Podcast is a production.
- [45:58]
Michael Stelzner
Of Social Media Examiner. Don't forget to get your AI ticket to Social Media Marketing World 2025. Become an AI Enhanced Marketer. Grab your tickets now at social mediaexaminer.com Aicon.