← AdExchanger
AdExchanger
Marketing In The Age Of AI Answers
00:52:36
The fundamental shift from traditional search to AI chatbots has major implications for the entire marketing organization, says Bluefish CEO Alex Sherman. If brands want control over how they appear in AI search results, they must think about the...
Loading summary
Transcript89 lines
- [00:00]
Alex Sherman
Foreign.
- [00:13]
Allison Schiff
Welcome to Ad Exchanger Talks, the podcast devoted to examining the issues and trends in advertising and marketing technology that matter most to you.
- [00:30]
Alex Sherman
Alison.
- [00:32]
Allison Schiff
I'm Allison Schiff, and thanks for tuning in to Ad Exchanger Talks. I've got Alex Sherman on the podcast this week. He's the co founder and former CEO of Promote IQ, which he sold to Microsoft in 2019. And now he's got a new startup, Bluefish, which helps brands track and improve how they show up in AI chat and search tools like Chat GPT. We'll talk about what that entails because there's a lot of nuance there. And we'll take a trip down memory lane to the early days of retail media, before that was even a term. But first, Programmatic IO New York is just around the corner. The whole Ad Exchanger editorial team will be there, and we've put together a great agenda for you that's designed to make you smarter, make you think, and give you a sense of where the industry is going. This Q4 and into 2026, there's still a little time to snag your ticket and you can use the code PODCRUSH for 25%. That's P O D C R U S H all in caps. Thanks for listening and see you there. Hey, Alex. Welcome to Ad Exchanger Talks.
- [01:39]
Alex Sherman
Hi there. Thanks so much for having me.
- [01:42]
Allison Schiff
So where are you now? You're in Paris, right?
- [01:44]
Alex Sherman
I am in Paris, yes.
- [01:47]
Allison Schiff
Are you there for vacation and doing fun Parisian things or work?
- [01:52]
Alex Sherman
Work. I wish. I wish. My team keeps asking me like, how is Paris? And I feel like I'm transitioning from zoom calls in my hotel room to in person meetings and just shuttling back and forth. I've had zero baguettes on this trip, which is like a tragedy.
- [02:08]
Allison Schiff
Yeah, they're not going to let you out of the country, Right?
- [02:11]
Alex Sherman
Right, right. Yeah, I'll do that on the way to the airport.
- [02:15]
Allison Schiff
No, that's. Don't eat your baguette on the way to the airport, Alex. Anyway, what. What is. Because I'm going to still start out. I'm going to doggedly stick to my favorite first question, starting out with, what is one thing about you that not a lot people already know that just googling Alex Sherman in quotes won't tell me or I wouldn't learn by just looking at your LinkedIn page.
- [02:37]
Alex Sherman
That is such a great first question. I love that. So, look, I'm in France, so I will. I will say one thing that not a lot of people know. I'm actually Half French. And so it sort of puts me in an awkward position in these meetings because, you know, they. They see me as like, the American. They're like, ah, la, Mel, he's here. But then I sort of will trot out a little bit of French as my like, party trick to kind of like, break the ice in. In the meetings. And then we end up having this weird sort of Franglish kind of back and forth during. During business meetings. But yeah, that is definitely not on my LinkedIn profile, and I don't think you can find that from Googling. But, yeah, my. My family's half French.
- [03:22]
Allison Schiff
I mean, now people will be able to find it out by Googling because I extract these fun facts about people, and if I include it in the story that we write about you, I guess it will become something searchable. And we are going to talk a lot about search, actually, and how it is evolving greatly. But before we do, and actually before we move on to the next thing, I wanted to ask you. Have you ever been in a situation where people don't know you speak French and they say something in French, like, about you, and you're like, I know what you're saying.
- [03:49]
Alex Sherman
Yes, that happens all the time. I would say probably the biggest offender on that front was my. My late grandmother, who, like many French people, sort of assumes that no one else speaks French. And so one of my favorite memories of her, and she recently passed away. So this is a tribute to my French grandmother. We were on the New York City subway once, and she was basically talking trash about someone else who was standing next to us on the subway in French under her breath. And like five or six people in the car all turned and listened because it's New York, you know, like, there's. There are a lot of people that speak French. And so, yeah, that. That definitely happens a lot.
- [04:32]
Allison Schiff
What was she saying, though, Alex?
- [04:35]
Alex Sherman
You know, it's the New York City subway, basically, like, everything bad is happening down there. Didn't they. Like, they recently did a. They did some study where they took samples.
- [04:46]
Allison Schiff
Are you talking about the swabs where they found bacteria that doesn't exist anywhere else in the world that originated in the New York subway?
- [04:53]
Alex Sherman
Like the plague. The subway.
- [04:57]
Allison Schiff
It's hideous. And I will be taking the subway later and studiously not thinking about that study.
- [05:04]
Alex Sherman
It's what makes New York great.
- [05:05]
Allison Schiff
Yes, sure. That's what we have to tell ourselves. Although I love New York, so I know my first question was, what's something about you that I couldn't find out by looking at LinkedIn. But of course, I did look at your LinkedIn to find out about your background, and I saw that you studied history at Columbia, which is very cool because so did I. Not at Columbia, but I was a history major at Brandeis. So did you focus on any one particular historical period?
- [05:34]
Alex Sherman
So I was a bit of a history mutt in that I couldn't make up my mind. And so, like, you know, at Columbia, they're quite like, gracious with their, you know, like, what determines the sort of focus within a history major. So I was technically a European history major, so I wrote my thesis on European markets. But no, I think, look, I think one of the best parts of being a history major is you get to just sign up to learn about history all over the world throughout, you know, throughout all of time. And, you know, undergrad is about those sorts of experiments. I think it's one of the best, one of the best things you can do when you're 18, 19. You have no idea what you want to do with your career. History really gives you sort of a, you know, a broad base of exposure. And so that's, that's, that's what I did. What was your, what was your focus?
- [06:27]
Allison Schiff
I also was kind of random. I did a little bit of a concentration on Russian history because I find Russia really interesting. But I also took a class, I swear to God, this is what it was called, the History of Victorian Poverty. It just sounded interesting.
- [06:41]
Alex Sherman
That tracks.
- [06:42]
Allison Schiff
So that, that's part of my history major. I took a little American history. It was quite, quite a melange.
- [06:51]
Alex Sherman
Yeah, that sounds fantastic. I still remember my, you know, my South American history classes, my Asian history classes. It's like, fantastic. I'm, I'm, I'm really happy that I did that. And no regrets on that front.
- [07:05]
Allison Schiff
I am going to bring us back to the present day soon and what you're doing over at Bluefish.
- [07:10]
Alex Sherman
But first, I, it's like, tell us the mistakes that you made in high school. What is the one you regret?
- [07:15]
Allison Schiff
What's the worst thing you ever did when you were 18?
- [07:21]
Alex Sherman
That would be a great subject for a podcast. Like, you know, like, bad things we did in high school, we could, we could fill that out.
- [07:28]
Allison Schiff
I, I do really want to ask you that question now, but I'm going to leave it as a question mark and take us all the way to 2008, which is right around the time you graduate from Columbia and you took your history degree and you went and worked at Media Math as a product manager. As one of its very early employees. So you joined pretty early on because Media Map was only founded in 2007. So how did you find yourself there building the early technical foundations of our weird and wacky programmatic advertising industry with your interest in European markets?
- [08:03]
Alex Sherman
Yeah, this is like a very standard career path. You do sort of like random history undergrad and then you go straight to a dsp. Like back in. Back in my day this was like novel, but now this is like standard course. So the story is. And I'll give you the, like, I'll give you the honest and straightforward version. So I graduated in 2008. And the thing that I. So as a history major, you sort of have a few standard tracks, right? You can, you know, go to law school, you go to business school, go work for an investment bank. There were like, there were sort of the standard spectrum of white collar careers and I had my, my little heart totally set on going to work at a management consulting firm. I was like, this is like the dream. Like, this is, this is the perfect place. I'll get exposure to business and I get to work with all these companies. And this is where the best and the brightest are going. And, and of course in 2008, all of a sudden we had this global financial crisis.
- [09:06]
Allison Schiff
I recall it. I was also looking for a job around then and it was.
- [09:10]
Alex Sherman
Yeah. And so there were no jobs. There was nothing. And I was like, well, this is, this is bullshit. Like, this is not what I signed up for. And my friend was working at a startup and he calls me and he's like, hey, you are broke and unemployed. You should come join this company. And I was like, what is it? And he's like, well, it's a startup. And I was like, well, what is a startup?
- [09:30]
Allison Schiff
What does it do?
- [09:30]
Alex Sherman
What does it actually mean? And. And he sort of, you know, like gave me this, this kind of, this babble about, you know, algorithms and media trading and, you know, the future is programmatic. And I was like, okay, well, that is utter nonsense. I have no idea what you just told me, but let's go, let's go take a look. And Media Math had just raised their Series A. I think I was employee, like 30 or something like that.
- [09:59]
Allison Schiff
Wow.
- [10:00]
Alex Sherman
And it was, it was unlike anything I had experienced. It was, it was this, like, this true meritocratic work environment. Everyone was really smart. Everyone came from diverse backgrounds. We were, it was like us against the world. It was, it was like, it was magical. And, and of course, in 2008, if you're at a cocktail party and you tell people, oh, I work at a startup, it was not cool. You know, like you say that at a cocktail party and everyone's like, oh, I'm so sorry, like something horrible has happened to you. How can I help? And so, so, but I didn't care. Like I, I, I loved it. And I was like totally drunk on the startup Kool Aid and building. And it was, it was sort of the programmatic heyday. That was when all of the, you know, I still remember when Invite got acquired by Google and that was like the biggest news in the industry. And um, everyone was like, maybe DSPs are over. Like maybe that's, that's it.
- [11:01]
Allison Schiff
Oh my God.
- [11:02]
Alex Sherman
Um, and so, yeah, that was, those are the circumstances that I joined and I was there for, for about three years.
- [11:09]
Allison Schiff
So then I'm going to fast forward now a little bit more to 2012 because you leave Media Math and you co found Promote IQ, which got acquired around 10 years later by Microsoft and it was a big player in the retail media space. I mean, that's not where it started though. And I wasn't aware of this when I was doing some research for my chat with you. It was more like proto retail media, like the nascent beginnings. Because when you were launched you were called something else. You were Spot Front. Very clever. Right. Play on Storefront, I assume a software platform that let online retailers make extra money by giving brands automated tools to promote their products on retail sites. I'm just like grabbing this, reading it off the tin without using traditional ad networks. And then retailers could also use spotfront to launch and manage their marketing programs. And this was really like prescient stuff. Right? I mean like Amazon was already out there, but I had to look this up. I couldn't remember when the first retail media network launched, but it wasn't until 2015. I think Walmart and then Target Roundel was the next year. And then everyone in the universe, from Planet Fitness to Ulta Beauty to Chuck E. Cheese, like got in on the game.
- [12:15]
Alex Sherman
I'm so sorry, I have to like, apologize for, like, I am partly responsible for this, for this beast, this monster that we've created.
- [12:23]
Allison Schiff
Well, what motivated you to create the monster? Like, what was the spark? What did you see in the tea leaves that led to spotfront which became Promote iq before it was clear that this was really going to be a thing.
- [12:34]
Alex Sherman
Yeah. So there was a stage even before all of that where after mediamath, after spending three years building programmatic technology from a product point of view, when we left, when Peter, Peter Schwartz, my co founder at Promote IQ, and I left MediaMath, it was to basically take this programmatic RTB tech and actually apply it to SMBs. And so the plan was, hey, we're going to take the marketing tech that Pepsi uses and enable mom and Pops to do the same thing. Because you can run a campaign for $10 million, you can run virtually that same campaign at much smaller scale for a hundred bucks. So we're going to do that. And we were basically building technology that enabled large corporates that served SMBs to offer that tech. So we would go. So our first client was JPMorgan Chase, for example. So they served SMBs. We sort of built them essentially like a miniature programmatic DSP that enabled them to offer that to. If you have a chasing credit card, you can offer this to your customers. And we started building spotfront. What became Promote iq, doing that. And it was fine. But what started happening was that retailers would reach out and they would say, hey, you guys are sort of building this kind of ad tech middleware. What if you just brought that over to us and we started to use it to basically build our own kind of ad network thing? And we basically, we basically made the decision to transition and pivot from SMB to retailers because we, like, there was just so much more growth. There was one retailer that reached out and then two, and then five, and then 10. And it became super clear that that was the trend that was going to happen. In those days, the only retail media network that existed was Hook Logic.
- [14:35]
Allison Schiff
Of course, Critio bought them.
- [14:37]
Alex Sherman
Yeah, yeah, yeah, exactly. And so every retailer had basically signed up for Hook Logic and they had sort of that little Hook Logic carousel at the bottom of the page, basically like adsense for retail. And they had no idea what was getting sort of pumped through that pipe and showing up on their website. The merchant teams hated it, but they got a check at the end of every month. And so sort of a necessary evil. And so in the early days, our pitch was, hey, A, you know, that check is way too small. Like you're, you're, you're not getting your fair share. And B, you're a hundred billion dollars corporation and you have no control over this box at the bottom of every pdp. Like, that's crazy. This should be your business. There's no reason why you can't control this. And so that was how we, that was how we got us got our start. And I remember pitching investors this idea. And in those days, this must have been like 2014, you could not have possibly had a more unpopular business than an ad tech business in those days. It was like, it was absolute kryptonite. People would throw rocks at you. It was so bad. And the industry basically felt like Hook Logic that was eventually bought by Criteo. They have locked this up. And the idea that a retailer would ever take control of their own data and media assets and like enter into some business that was as non core as ad tech was craziness. Like people were like, this is the dumbest thing we've ever heard. And we sort of understood those criticisms. But one retailer after another started signing up for Promote iq and we benefited from this incredible marketplace dynamic where every retailer that joined our platform, they basically had 2,000 advertisers that were ready to go. And so all of a sudden it just started accelerating on its own, right?
- [16:46]
Allison Schiff
Well, it became a thing. And then Microsoft bought you guys in 2019 and you stayed there, I think it was until 2022, right, running the Promote IQ business inside of of Microsoft. But what was that transition like going from a startup founder and a CEO, A startup founder at a time when being a startup was no longer like oh, I'm so sorry. And then, I mean you were master of your own domain and then you're working inside of this massive company.
- [17:12]
Alex Sherman
It was incredible. I mean, you know, we were, we were about 50 people when we were acquired at that point. We already powered a decent portion of the retail media programs in the us. We had begun expanding into Europe. And the Microsoft pitch was, hey, this is a platform to grow 100x. And within Microsoft, if you build a billion dollar business, you are a small division of Microsoft. You are not as on the radar as you might think. And so the experience was there was a pretty dramatic transition where all of a sudden you go from this sort of fully independent, high growth startup, really like a pirate ship on the open waters, to everything just has to happen at 1000x the scale that you were previously used to. Everything from functions of the business, hiring operations, building products, go to market. Microsoft is a breathtakingly large organization and even though their advertising business is quite small comparatively, we were sort of as a division, even though we were funneled into the adtech. Org, we had strategic connections to the Azure team and other, you know, really large divisions within the mothership.
- [18:49]
Allison Schiff
You moved on in 2022. But then last summer, July 2024, Microsoft officially shut down promote IQ's on site retail media offering, which seems to be like a strategic shift for them. Not that they're not interested in the retail media category, which is obviously a thing, but instead of running their own ad tech, they decided to have a partner do it and they started working with Criteo to handle their retail media demand, which is such a interesting world we live in. Everything is so cyclical. But I think it is interesting to see the shift in strategy. Like Microsoft, I don't know if they would put it like this, but it seems like they're washing their hands of a lot of their ad tech. Not everything, but like beyond promote iq, they're shutting down Invest the former upnexis dsp and there were some other reorgs in the app nexus side of the business. But I know you've been out for a while, but why do you think Microsoft like lost its taste for third party ad tech? Well, like, not completely, they're doing it through partners, but owning that piece.
- [19:51]
Alex Sherman
Yeah, look, I think that, I think that Microsoft's ad tech business, and I can say this as sort of, you know, as an outsider now, they kind of hibernated for a lot of ad tech. There was, you know, because a lot of the, you know, the Microsoft advertising business was predicated on Bing and Bing had such a relatively limited amount of market share. I think that there was a little bit of stunted growth in terms of the tech that was built on top of it. So there were a lot of like, like programmatic technologies that have sort of scaled over the last 20 years that Microsoft didn't really play in as much. And so I think it was difficult, it's become difficult for them to kind of catch up in sort of the modern martech ad tech landscape. I think that probably matters a little bit less now because AI is a little bit of a reset button for.
- [21:00]
Allison Schiff
That's a lot of it. A lot. A bit of a reset button.
- [21:03]
Alex Sherman
Yeah. And so I think, you know, look, if you're building a, you know, a DSP right now, probably not the right time to be doing that. And, and, and by the way, I feel the same way about retail media. I. Retail media feels very sort of end of cycle in many respects. And that doesn't mean that retail media won't continue to exist, but it's going to exist in a very different way when those ads are embedded within, you know, agentic conversations. And so I think that, that to answer your question, I think for Microsoft they were, they were spread a little bit too thin and they were playing a lot of catch up. And I think that's just a, I think That's a tough game for an organization as large as them to play. And so that's sort of, that's, that's kind of my take.
- [21:57]
Allison Schiff
I mean, that sounds right. And I'm glad you brought up Agentic because we are going to take a break in a minute, but I want to tee up the second half of this episode by segueing us over into generative AI search, which is the focus of your new startup, Bluefish. So in a nutshell, you guys help brands monitor and control how they appear in AI generated search and chat responses. And to do that, you track visibility, sentiment, accuracy across all the major of large language models and AI platforms. So ChatGPT, Google, Gemini, perplexity, others. And then you help brands like optimize content and run targeted AI campaigns to improve their brand presence in generative AI search. And we're going to get into all the technical stuff after the break. But just to take us into that break, a personal question. How have your own search habits changed? Like how much do you personally, personally use generative AI search yourself now more than traditional search? And I'll say that I, I've almost completely segued away from traditional search myself.
- [23:02]
Alex Sherman
Yeah, same. I, look, I, I think for me the big, in the beginning I like, I sort of quickly adopted the chatgpts and the perplexities of the world, but was still supplementing that with Google. And as Google rolled out Gemini and AI overviews and AI mode became a thing, that basically just sort of was the final transition in my like searching behavior. And it's strange now because if I'm on Google and I still get those like 10 blue links below the AI overview, there's almost like sort of a new muscle memory where I don't quite want to go poking through all of those links. There is, I think, sort of a new expectation of an instantaneous synthesized answer. And I don't think I'm alone in.
- [24:02]
Allison Schiff
That experience 1,000% not. And it is remarkable how quickly it happened. It happened and really it was like a blink.
- [24:11]
Alex Sherman
Yeah.
- [24:12]
Allison Schiff
All right, well, we're going to take a quick break and when we're back, we're just going to talk all things Bluefish and Jennai Eye Search. So stick with us. All right, we are back and as promised, the rest of our time together is just going to be like Gen AI, Gen AI, Gen AI. So I explained just the very basics of what Bluefish is before the break, but I want to talk a little bit about how it works. So take me through how this works. What you're looking at, like what you're doing to measure and analyze how brands are showing up in Gen AI search results.
- [25:04]
Alex Sherman
Yeah, totally. So, very simply put, we've basically built a marketing platform for AI that essentially helps large brands gain visibility and influence over how they're showing up in these AI channels. The way we do that is when a brand gets a Bluefish account, they basically designate what they want to track. So what is our brand? What are our sub brands? What are our products? Which customer segments are relevant to those individual product lines? What are some buying criteria for our customers? Our engine takes those sorts of inputs and essentially generates a large list of synthetic prompts that reflect all of those dimensions. We syndicate those prompts to the major AI providers every single day. And then we do a sort of semantic analysis of all of those AI responses where we extract information about how the brand is portrayed and then we render that to the brands in our platform so they can understand are they showing up for relevant consumer journeys? How are they portrayed? Is it positive, is it negative? How are they compared to their competitors? And then they can do a lot of things based on that data to actually go and optimize for the channel. But that's, in brief, how it works.
- [26:23]
Allison Schiff
Make it real for me. Give me an example.
- [26:26]
Alex Sherman
Yeah, so, for example, financial services is one of our largest vertical categories. Maybe a credit card company has a credit card for small business customers. And so they know that when those small business customers are shopping for a credit card, they're really focused on low fees. Or maybe they have a frequent flyer credit card that's optimized for frequent travelers that care about lounge credits. And maybe they have two or three other credit card companies that they see as competitors and they want to track those too. They would basically input those parameters into the Bluefish system. Our engine would generate prompts that reflect how a consumer might search within those categories. So I am a, you know, I'm a frequent flyer. What's the best, you know, what's the best credit card with the most lounge access? If I'm a Delta Flyer, for example, and we're going to get a response from ChatGPT that basically advises that consumer on which credit card is the best fit for them, how they should compare different credit card providers, what the strengths and weaknesses are of each, and then that's the input that we analyze. And we do this every single day. We look at millions and millions of AI responses in order to power the Bluefish engine and provide those brands with that visibility.
- [27:51]
Allison Schiff
So you're Running millions of automated queries through LLMs. Right? That's how that works.
- [27:57]
Alex Sherman
Yeah, we spend an obscene amount of money on.
- [28:01]
Allison Schiff
I was going to say that was my question. Like, isn't that super expensive to send so many prompts and tokens through an LLM? Like when your usage is built on a per token basis, like you're running millions of automated queries, does how much does that snowball?
- [28:15]
Alex Sherman
It's a lot. It's a lot. And I think the other thing is that. So look, AI marketing, especially today, every brand needs to be managing the AI channel. I think that's clear at this point. SMBs, you know, mid level companies. Bluefish exclusively partners with the largest brands in the world. So 80% of our customers are Fortune 500 brands. These are, you know, multinational corporations that manage, you know, anywhere from one to $10 billion a year of marketing budgets. They have multi thousand person marketing teams. And the reason they work with Bluefish is specifically because we track so many prompts. Because the more prompts you track, the more resolution you get in terms of the data, and the more highly resolved the data, the more insight you can glean from those responses, the more effectively you can optimize for the channel. And so ultimately part of why our customers choose us and partner with Bluefish is because we give them the tools to really, in a highly bespoke and customized way to track the dimensions that are relevant to their customers and to their business lines. And yeah, the net result is a high volume of prompts for sure.
- [29:35]
Allison Schiff
But so Geo versus SEO, right? That's what you guys do.
- [29:41]
Alex Sherman
Yeah. I think this is bringing me totally back to the days of Programmatic where there were like competing acronyms and there was sort of a battle for like, what do we call this category? I generally think we're still in that moment with Geo where no one's decided whether it's GEO or AIO or aeo. I sort of don't think it matters too much. What I do think though is that search teams were kind of the canary in the coal mine in that they were the first ones to detect that consumer search behavior was changing because of AI. And so it makes sense that the first few steps that our industry took towards AI marketing was, oh, okay, well what is SEO for AI? But I think what's very quickly becoming clear in our category is that this goes way beyond search. There are implications for the rest of the marketing organization. So for example, this is just as relevant for content teams. Right. So if LLMs are trained on the Internet, then The team that is writing copy for your brand.com or for your retailer PDP, they need to start to optimize and tune that content for LLMs. And similarly the PR and corp cons teams that are tracking brand safety, reputational risk, well now they have a new vector that they need to track and when they issue a press release, they need to optimize that. Same thing is true of paid editorial on third party content. Same thing is true of social. And so all that is to say, I think we think pretty holistically about GEO beyond just the next chapter of search. We really think of this as sort of a fundamental rebuilding of the Martech stack. Whether, you know, today there is this emphasis on GEO as like taking insights and turning that into optimization. But we basically believe that most of the enterprise Martech stack will need to be rebuilt for this next chapter of the Internet. And that's the thing that we're focused.
- [31:47]
Allison Schiff
On like a little bit of a head exploding emoji, right? I mean it is a seed of. No, no, I don't disagree with you. I don't know if this question betrays a lack of technical knowledge, but can prompting an LLM model too much cause problems? I mean like more hallucinations or maybe vendors pushing back if activity looks like scraping? Is that something that you have to think about?
- [32:15]
Alex Sherman
I think that's a really good question. I think there is a lot that is going to change in our industry. So for example, we have this sort of vantage point in that we work with many of the largest corporations in the world and so we can see sort of what each of those teams is learning about in their own sort of GEO tests. So we can see what moves the needle. We can see how you actually optimize for an LLM. Like what is the thing? How does it actually work? How do you create change? And then when you've taken those actions, what is the impact? Like what, what does GEO actually do? And when we look at that and we start to see, okay, if you like create content in this way, here's how that shows up in a response. When you do a press release, here's how that shows up in a response. We see a lot of stuff that we, we do believe that AI providers will have an incentive to manage differently in the future. So for example, right now, if you think about how a model learns, it learns from what it's trained on, right? There's sort of a foundational model that's trained on a text file that represents the Internet that is augmented with real Time information that comes from a web search, and sometimes there's there are other sort of supplementary data sets that feed into the model. As a result, the models are a bit skewed. They are skewed to where they can find content and information. So if a consumer goes to, you know, chatgpt and they ask about an F150, for example, there is a crazy amount of content on sites like Reddit or on YouTube that describes F150s. None of that content was written by the Ford company. It was written by bloggers, by influencers, by paid editorial sites. And if you go to the Ford website, there's much less information. And so right now, the models are learning more from content that didn't come from the brands than content that does come from the brands. That might sound like a good thing, but it actually creates this sort of perverse incentive for companies to try and trick the models by creating lots of low quality content. You can kind of like flood the zone with data that is structured in a particular way on a random blog somewhere that basically is, is, is optimized for something that's going to be consumed by the models. If I work at OpenAI, that's a huge problem because ultimately it means that the integrity of my responses is going to be manipulated and so reliant right on the inputs. Exactly. And so I. All of which is, you know, to say, in response to your question, I actually think that there will be a lot of doors that are closed by the AI providers, a lot of, you know, loopholes where they have an incentive to get high quality content to their models, to train the models on content that has integrity to it. And that doesn't mean that the models need to turn into propaganda machines for brands. But it does mean that right now there's probably an imbalance where there's lots of really low quality content across the Internet that the models are learning from, and there's relatively less from the brands themselves. I wouldn't be surprised that the AI companies try to move that slider to a more balanced position over the next few years.
- [35:51]
Allison Schiff
So, semi relatedly, this is something I've been wondering for a while, but how do you account for the fact that you don't necessarily get the same answer twice when you prompt an LLM, even if the information is kind of the same, it's presented slightly differently and just anecdotally. And to support this question, I did a perplexity search a few hours ago for what is the worst ice cream? And the first response I got was this the quote unquote worst ice cream flavor tends to be subjective. But some commonly cited worst flavors include Superman ice cream and Breyer's chocolate ice cream. Superman ice cream is criticized for its inconsistent flavors. And then I just opened a new browser tab and I prompted perplexity again with the same question. And this time it didn't mention Breyer's until way later in the answer. And the answer was also phrased totally differently. And it just started out with this like the worst ice cream flavor often cited is Superman ice cream, just like definitively, mainly due to its inconsistent and generally poor flavor profile. And then at the very end there was just a mention of Breyer. So Breyers kind of gets off, you know, nicely there because I don't know if someone's going to read that big chunk of text they just see like, oh, Superman. So how do you account for inconsistencies like that in terms of what you guys do?
- [37:09]
Alex Sherman
Totally, and I think that's such a good point because look, there are so many companies that are jumping into this geo category and they're like, ah, we offer AI monitoring and AI optimization. I mean there's like, there are dozens and dozens of startups that are entering our space and when we talk to brands we really try to help them understand that GEO and AI marketing in general is, it's really easy to do badly and really difficult to do well. Because imagine that, that your search wasn't for ice cream. Imagine your search was, hey, I want the best. Like I'm training for a marathon, I'm looking for the best sneaker. And you ask that question three times, you get three different responses. A low quality Geo vendor is just going to take the first response and they're going to be like, well Nike, here's your position, here's how you're showing up. You're top of the pack or at the bottom, here's your rank. It's one of the reasons why we take a really high volume approach to our prompting methodology. Because what we're actually doing at Bluefish is we're asking about the best ice cream flavor millions and millions of times so that we can actually discern what is the more consistent signal in terms of how the models think about ice cream flavors. And so, you know, if you're a big brand, that's the, that is the sort of GEO vendor you want supporting your, you know, your decision making. When you go to optimize, I think the, the other, the other thing that's, that's relevant to what you Said is there's no such thing as generic AI performance. We talk to a lot of brands that are like, well, we like, you know, we figured out this is where we stand in AI, But AI is hyper personalized. It's hyper individualized. You and I could ask the same prompt about ice cream and get totally different responses because, you know, the clear answer is that vanilla is the best flavor and all of the AI for sure. So, anyway, so when we prompt, we basically enable our customers to create specific customer profiles for each of their product lines so that when we're figuring out how the brand is performing, it's not some, like, sweeping generalization. We want to figure out, how does the brand perform differently for their key customer segments, and that's really the only way for you to actually understand your performance is okay. For the 10 to 15 different customer cohorts that are relevant, we actually ask different prompts for each of those cohorts because we want to understand, are we performing consistently across that. That sort of slice, or are there. Are there, you know, are we over indexing with one and really struggling with another and losing share to a competitor with the third? It's really important to take the analysis that additional step. And that's kind of what I mean by, you know, easy to do badly and really difficult to do well.
- [40:16]
Allison Schiff
Well, let's say Superman or Breyers. They were your clients. I don't know. Maybe they are. But, like, considering the results I just read to you, what would you suggest that they do to rectify how they're showing up? And I feel like one answer might be, don't write content about how you're great. Just make a better product. Like, sometimes the Internet is right. Like, I love Breyer's mint chip ice cream, and it's fine if you love vanilla mint chip from Breyers. It's childhood to me. I used to eat it with my mom and watch Dr. Quinn, Medicine Woman on Saturday nights. It's delicious. But the chocolate ice cream is. It's objectively not good. It has a really weird consistency, and it never seems to fully freeze. And I just know this from experience. So, I mean, I guess, like, to rephrase the question with more nuance, like, what would you advise Breyers to do here? And could the answer just be, take what you've learned from the Internet and use it as an insight to improve your product? Like, that's a valid thing to do also, for sure.
- [41:13]
Alex Sherman
Look, I. I sort of feel like anytime a new marketing channel emerges, marketers go through Sort of the like stages of grief. Right. And I think like in the beginning there is this like, okay, like AI marketing is the thing. Where's the big magic button that I can like push to like fix it?
- [41:40]
Allison Schiff
Yeah.
- [41:41]
Alex Sherman
And over time you sort of come to this place of acceptance where you realize that this marketing channel in some ways is like other marketing channels where at the end of the day like if you have a shitty product that's going to hurt you. Like, like step one is make the best product. There are going to be organic tools that you can use, right. So you can, you can start to optimize your first party content. There are going to be third party tools that you can use. So maybe doing some paid editorial with an influential blog like helps helps move the needle. There will eventually be paid methods within this channel. I know that ads are in their very earliest days, but AI advertising is around the corner. And over the next six to 24 months we fully expect that AI advertising will emerge as a real tool. And so for brands that are looking at this channel, ultimately it's going to be like anything else. Where is your product good or bad and have you invested in it? Is step one, what are the organic tools at your disposal and then what are the paid tools? And that portfolio of approaches is how you manage your performance in the channel. Now to answer your question more, more specifically though, I think today when we partner with brands, yes, we are helping them track their performance. So we're tracking all of those AI responses. But the reason we track those responses is so that we can not just figure out how the brand is being portrayed, but why they're being portrayed that way. Our job is to isolate the drivers behind the Breyer's performance across all of those different consumer journeys in AI so that the actions that they take can be hyper targeted to address those problems or meet their marketing goals. So they might say actually we're really looking to win over this particular type of consumer because they're really like mint chip. Is this like, is this rising category? It's back from, you know, back from the past and everyone's obsessed. All the kids are obsessed with Min Chip again. I mean did it ever go out.
- [43:51]
Allison Schiff
Of I'm a real men chip influencer.
- [43:53]
Alex Sherman
Yeah, but so they might say, hey, like that's we want to win that, we want to win that demographic or we want to win that category. And it looks like we're losing share to this other provider. Bluefish is going to help us figure out why. They're going to help us figure out how are the models learning about our brand and what do we need to change if we want to improve those outcomes? So our job is to, is to help them track it, understand the drivers, optimize and take action and go and sort of, you know, redo the content and do the paid editorial and do the social outbound. And then our job is to help them measure the actual impact of those optimizations. So we did a bunch of stuff. What happened? Did it actually move the needle? Did it actually impact our AI favorability or our AI share of voice or our AI influence? And so we're there for sort of the full cycle of tracking the responses, understanding the drivers behind those responses, optimizing and taking action and then measuring the impact of those actions. And it becomes this sort of iterative cycle like that's, that's where it goes.
- [44:57]
Allison Schiff
And just to zoom out and I'm not asking you to have the answer to this definitively, but what can be done to prevent brands or like any entities from just dragging their competitors and messing up with how they show up in searches? You know, like flooding the zone with negative stuff about your competitor. Because like SEO is just, well, I want to show up on top and like, maybe there'll be an attack campaign or something, but an LLM will give you a sentence and it'll say mint chip Briars is bad and that'll be wrong because Superman was spending against them or something. And I don't know, is that something you have to think about?
- [45:35]
Alex Sherman
I think it's like the early days of search where you're going to see all of that. It is A, already happening and B, will continue to happen over the next few years for sure. But I sort of, I really do think that the AI providers, even though today they're taking this sort of dogmatic position on AI advertising and I do think that over the next few years the dust settles a little bit and AI providers have an incentive to create the best possible product for their consumer users. So, right, OpenAI wants ChatGPT users to get a really high integrity response. They are going to need to monetize. And so, you know, and they've launched shopping now. And shopping, as we all know, is just the gateway drug to advertising. So, so shopping and commerce will drive paid options. It will be up to ChatGPT and the OpenAI team to do that in a thoughtful way. They'll probably make some mistakes along the way. And, and like that's, that's a normal part of the process. But brands today are not really involved. Like there is not a really great link between commercial brands and the AI providers. From a data point of view, from a content point of view, that will probably start to change. And I think over the next few years we're going to settle into a sort of more balanced, more mature sense of how AI works from a commercial point of view. And so to answer your question, will there be misbehavior 100%? Like, absolutely. But, you know, as a little bit of an old dog on the ad tech martech side, having seen the rise of programmatic and search and social and mobile marketing history rhymes, there is a pattern, there is a cycle that plays out every single time. And if you've seen the last few cycles, you start to see it and we're in the early stages of the next super cycle. And to be clear, AI is the channel that is eating all the other channels and it will be large and it will redefine the entire lumascape probably. But, but I think we, I think we will get to a more stable, mature place with best practices and a right way of doing it. And, and interestingly enough, I think the marketers already know that. I think at this point the digital marketers we work with have seen the last few cycles too, and they know where this goes.
- [48:09]
Allison Schiff
Well, I, I think, I agree that it'll eventually even out, but until we're there, what's the weirdest hallucination you've ever encountered? This is my final question. While you think about it, I'll tell you what. What I've seen and I haven't seen anything like pizza glue level. And that's so silly, like, it's a joke, you know, the ones I've come across I think are really insidious because they're just really, really very small things that are wrong and they're stated as objective fact. And the only reason why I know they're wrong is because I happen to know that fact. And if I didn't know, then I'd be learning something wrong. Like, for example, I was doing a little research a bunch of months ago about agency holding companies and like how their earnings were doing. And the response, like in one of the responses I got it just said that Martin Sorrell was the CEO of WPP, which he hasn't been since 2018. And that was so weird to me because there are tons of articles about his departure and there are tons of articles about him being in charge of S4 capital. Like, what the heck, right? It's just like black and white. But yeah, like what, what have you seen?
- [49:19]
Alex Sherman
Totally. I. Look, I think this is a huge problem and I know we were talking about it a year ago, but we're not talking about it as much these days, but it is still there. And I think to your point, the real, the real issue is not in the like the really overt over the top hallucinations. It's in this subtle like gray hallucinations where it is credible. It's just wrong. And because it's sort of so believable, I think it's, it's, it's sort of even more, you know, problematic. So I agree with you. Okay, favorite hallucinations by. I have. So I have so many. I, I actually like, I really provides a lot of fodder here. So my all time favorite is definitely the Google Pebbles one. Are you familiar with that?
- [50:08]
Allison Schiff
Oh, there was something about eating rocks, but I can't remember advising users at.
- [50:13]
Alex Sherman
Least one rock per day. Yeah, yeah, I, look, I've worked in this industry for a very long time and I sort of love these little like Google missteps. And their rollout of AI was like pure chaos.
- [50:25]
Allison Schiff
And yeah, it wasn't ideal.
- [50:26]
Alex Sherman
I sort of pulled up some Popcor and really enjoyed watching that.
- [50:31]
Allison Schiff
Well, do, do you eat a rock a day? I mean, do you?
- [50:33]
Alex Sherman
Look, Google told me too, so I know, I'm, I'm pretty sure. Yeah. Look, I think, I think the, the hallucinations that we see. So, so Bluefish basically enables, enables brands to, to track this. And I will. It's not, it's definitely not my favorite, but one of the things that we see often is ingredients. So for a CPG brand or for a pharmaceutical company or for any sort of vertical where a consumer might want to know, hey, what's in this stuff? What we see quite a bit are these synthetic ingredients that are just tucked in there. I'll give you, I'll give you like a personal example. I, I'm like a little bit of a wine geek and I, I was bidding on like a wine auction the other day and there was like, you know, like a thousand lots and I had like no time and so I just downloaded a CSV of all the wines and I plugged it into an AI provider and I said, hey, can you tell me which ones to bid on? Which ones are going to be the best value and what should the bid be? And the, you know, sort of thought for a few moments and it gives me these recommendations. It's like, all right, these are the top three that you should bid on. And it's like, Chateau so and so. Here's the price. Here's where you should bid, here's the value. Chateau so and so and so. It gave me these three options. I was like, oh, perfect. This worked so well. This is, like, exactly what I wanted. None of the three wines existed at the auction.
- [52:15]
Allison Schiff
Well, what are you going to wash down your rocks with? You know.
- [52:18]
Alex Sherman
Yeah. Mystery wine. Yeah.