Loading summary
Transcript39 lines
- [00:00]
A
So I've been trying to record this live read script for. I think this might be my eighth attempt. I keep screwing it up because I'm getting tongue twisted on these S words. There's a lot of S words here. So let's see how I do. This episode is brought to you by ipsos. We're talking today about AI and advertising execution. But let's look at using AI to refine strategy. Right now there's a massive rush towards leveraging synthetic data to simulate human feedback. It's fast and it's cheap, but if you're building your entire strategy on it, proceed with caution. IPSOS research shows synthetic data tends to shave off the edges, ignoring the exact nuances that make for a differentiating strategy. While AI helps us iterate quickly, over reliance on these tools as your only signal risks both creativity and effectiveness. IPSOS creative Excellence takes a more responsible approach by training AI models on their databases of over 10 million human responses. Anchoring AI in real human truth ensures it's genuinely predictive of real world behavior. I think I did pretty good. Now back to our episode. Welcome to this month's episode of on the Pulse. I'm Fergus o' Carroll in Chicago. On the Pulse is our monthly episode we do with Ipsos where we take different subject matters that we think are really interesting and we dig into them with some thought leaders in in that sector. This month we're going to be talking about a report that IPSOS has a report from a study that IPSOS partnered with Purdue or not Purdue, Syracuse University on AI ads. AI generated ads versus Human ads. The headline on the report is AI Ads are Good Enough. And that's the problem. I think that when we look at this conversation, it's important to keep in mind and each of the participants in this conversation are well aware of it, that this is being recorded in are being released in July of 2026. We don't know whether in three months or four months or a year from now that the results of a test such as this would be dramatically different. I think we all predicted that it will. But I think what we're trying to do with this conversation is point to the fact that as we run wildly in our attempts to integrate AI into our creative processes, into our business processes, that we need to be very, very careful about in our zeal for efficiency that we don't for get about effectiveness, the results of this study I think are going to point you strongly in that direction. And if you need substantive examples of making the case to your senior management about how best to use AI and what are its strengths and weaknesses. I think that this conversation can contribute to that broader conversation. So we have with us today Adam Peruta. Adam is Associate professor at the Newhouse School at Syracuse University, where he leads the Media Management Master's program and serves as the Dean's Fellow in AI. And Lisa Zelensky returns as SVP Creative Excellence Strategy at Ipsos. She returns, of course, because she's a part of this conversation every month. So I encourage you to download the report. The link to the report will be on our website@onstrategyshowcase.com in this episode, you can click and link to it. You can also link to it in the description in your podcast episode. In this episode, if you go down to the description of the episode, the link will be right there. Same on YouTube. You can click on the link in the description box and download the report. The title AI Ads are Good Enough. And that's the problem. Enjoy. Adam, great to have you join us. We appreciate having you, man.
- [03:56]
B
Hey, thanks, Fergus. Yeah, happy to be here.
- [03:59]
A
And Lisa is back again. Lisa Zelensky, SVP of Creative Excellence Strategy at Ipsos. Good to have you back, Lisa.
- [04:07]
C
Thanks. Good to see you, Fergus.
- [04:09]
A
So let's dive in. Adam, tell us a little bit about how this project got underway. What was the early thinking and what was the goal?
- [04:18]
B
Well, this actually all started from something that we had our advertising students do in a classroom, which was generate ads about themselves and introducing themselves. And we found that the output that they were getting wasn't great. Like nobody was really like proud of the work that they were producing with AI. But you know, over time the image and the video generation capabilities became really good, really fast. And you know, I think from a, from a higher level we wanted to, you know, really answer the question, like can AI generated advertising work as well as human made advertising? You know, because a lot of the, the AI conversation is always about whether people can tell the difference between human made content or synthetic content. And I think that's just a small part of the, the issue. The more important question I think for, you know, marketers and, and brands and agencies is whether AI generated ads can drive, drive the outcomes that marketers really care about. You know, things like getting attention and are they emotional connection, short term sales potential, long term brand equity. So I think the big objective here was to, yes, show that the AI generation capabilities are scary good, but we want to separate the perception from effectiveness.
- [05:54]
A
So I guess not only are you an associate professor, but it's important for this conversation to point out the fact that you're also an art director. So you have a sense of the sensitivity of creativity and the process of creativity. So I want to note that for everybody, the headline on the report is AI ads are good enough. And that's the problem. Tell us about that.
- [06:17]
B
Yeah, so ultimately what we did was we compared 10 human made ads to 10 AI generated ads, but they were their counterparts, right? So we had this human made AI ad that we deconstructed using AI so that the new ad that we generated with AI was on the same level, like strategically and creatively. And then we sent those 20 ads out to a total of 3,000 US consumers. So that was our panel and we used Ipsos Creative Spark methodology and that allowed us to ask all sorts of questions, questions about, you know, getting attention and memory and Trust and across 27 different measures, I believe. And in the end, what we found was that all of the 10 AI generated ads, like, fell within the acceptable range in terms of, of effectiveness. You know, we can, we can get into the weeds, you know, if you want. I mean, there were some of the AI ads that actually performed better than the human created ads. But for the most part, the human created ads, like, edged out the AI ads overall. And the story really wasn't that like, oh, the AI ads were horrible. No, they were good. They were passable. But does passable, does good enough deliver on those outcomes again, that marketers really care about?
- [07:56]
A
You want to jump in on this, Lisa? Do you want to add anything to that?
- [07:59]
C
Yeah, sure. So, yeah, exactly. What's up? What Adam just said, we, you know, put these through sort of the normal types of solutions that clients are using when they're evaluating creative before launch. Because, you know, a lot of our clients right now are sort of in this, mostly in this hybrid space, right, where we're, we're using human intelligence and AI to create advertising. But there are some signals that we're going to be quickly, you know, as the, as the technology is progressing, that we're going to be quickly moving into a space where we're looking at completely AI generated ads and these things going just automatically in flight with some sort of evaluation in flight. I think we hear a lot of, a lot of conversation about that in the industry. And so that's where we were really wanting to sort of put this to the test and see how these stack up. And as Adam was saying, so we put these through kind of our usual, you know, assessment that clients use when they're looking at this type of creative and looked at some of the KPIs that we know tie to business results. So sales lift and market through marketing mix or long term brand building potential of the ads. And what we saw was that the AI generated ads perform kind of within the average range of what we typically see across our database. So exactly what Adam is saying, that these AI generated ads are good enough. But what we see when we do the validation work is that ads that perform, you know, in the above average range, that's where you want to be. No one sets out to create an average ad, I think when you're a professional in this space. And it's really those above average ads that deliver significantly more return on investment when you put those into market and they make those media dollars work a lot harder. So if you're a brand that is trying to, you know, really grow and stand out, an average ad is really not good enough. So that's sort of the impetus behind the title.
- [09:56]
A
So let's dig into some of the methodology. Were you, when we say we tested them or you tested them amongst a large sample of people, 10 ads, AI, 10 ads, human made. Are you asking the same people to test ad the version that is AI generated and human generated, or what was the, the methodology behind the literal testing and capturing those scores?
- [10:21]
C
It was a monadic test, which means that everyone just evaluated one ad and they were just seeing that ad in isolation. So we weren't asking them to compare AI to the human generated ad. It was 150 people that saw each. So that's where we're comparing ad by ad, rather than asking them directly about that and drive.
- [10:45]
A
So then Adam, one of the things I was super curious about in the early stages of this when we've chatted, it was the methodology for generating the AI versions of these ads. And more specifically why AI was used to sort of retro generate the brief off of the human ad. Why was it necessary to have AI do that versus just going to the original brief or having a human retro and interpret that brief to then plug into AI later to generate the AI version?
- [11:21]
B
Yeah, I'd love to talk about how we made those AI ads because I do think the process was a little, a little novel. So when we identified our 10 human made ads, right, it was very important that the AI ads were, you know, gonna have the same creative strategy. And now we didn't have the ability to go back to, you know, for example, you know, Fiat or their agency and say, hey, do you remember that ad that you made? That campaign from seven years ago, can we have that, that brief? Right. So we needed an easier way to make sure that the AI ads were going to be on that same creative level. And so what we did was we took the human created ads, we ran them through Google, Gemini and asked Gemini to deconstruct the ad to create a brief that could have been used for that ad or for that, for that campaign. And then once we have that creative brief, we fed it back into Gemini and said using this brief, come up with an idea in a shot list for a new 30 second ad. Then we get the shot list. And then at the time we generated these, which was the end of 2025, OpenAI had a video generation platform called Sora 2 and we're able to take that shot list and feed it into Sora 2 and in one shot generate a complete, a completely new ad with no human involvement in the loop. So there was no editing, there was no post production, there was no adding in audio. Sora 2 was able to just look at the shot list and make the ad. And that was it. It was pretty simple actually. So this, this process of using AI to deconstruct and then reconstruct, I think it really cut down the amount of time that it took for us to go to market with this, with the study. Because it was so incredibly easy to generate the AI ads using that, using
- [13:33]
C
that method, we wanted to provide some strategic guardrails around what the AI was generating creative off of. And you know, as we think about AI generated creative in the future, the brief becomes more and more important. Right. It's sort of like the last human handoff in the process of where we're setting guardrails, where we're setting the strategic direction that we want the AI to take. And so I think it's really cool that Adam and team were able to use Genai to kind of back into a brief that worked for the AI generation.
- [14:08]
A
What was the conclusion coming out of it, Adam? In terms of the way that humans were interpreting the AI work and ranking it compared to the original work, the
- [14:19]
B
easy place to jump in is that first and foremost, people could not reliably detect the AI generated ads. I think it was only 25% of the viewers were like somewhat confident that an ad was made by AI. But again, the capabilities are like so scary good. Like, I don't think that's, you know, I don't think that's the headline here. I think, you know, a better reaction to focus on is the gap that was, you Know, there was a visible gap between around creativity and emotion. You know, I think the human ads were seen as like, more entertaining, more unique, more eye catching, more imaginative. But where AI performed really well was on the ads that were very product driven. So, you know, for example, like, hey, here's a, here's a problem that you have. Here's our product that's going to solve that problem. Okay, here's the fabulous result. I mean, I think that's a formula that's been used in advertising for as long as advertising has existed. And so AI, like, knows that formula really well. And those product driven ads were the ones that people really reacted very positively to. These tools and platforms are only generating content based on probability. So can AI tell a good story? Can AI be funny? Can AI be creative? I would argue and say yes. But I think a lot of that, a lot of that depends on the perspective of the human who's creating the prompt or asking for the, for the outcome. You know, I would say that we, you know, we did see some, some ads that, like, subjectively I would say, like, oh, that's funny, or like, wow, that was a really, you know, creative leap that the ad made. But, but overall they still still fall short.
- [16:24]
C
Yeah. I think one of the interesting things that we saw is that the AI used storytelling a lot less than we see just in our overall database in terms of how much narrative is attempted in advertising. So it wasn't really going after narratives specifically. I think a really good example of that. One of the ads that we did in the study was actually the chewy holiday ad, Fergus, if you remember the one that we tested back in the holiday show a couple months ago, where there's the little puppy and then there's sort of this magic moment where he puts the sweater on the puppy's head and then the puppy grows up. And so it's like this linear. It's. The story of that one scene is just magic and it really connects for you. This is the same dog, right? And the AI version of that ad is kind of interesting because, like, it has it, it did generate this like puppy versus senior dog idea, but it's not clear that it's the same dog and the dog is just sort of there with the family around the Christmas tree. But it's not like the focus of the story. So there is sort of this aspect of like, it doesn't get to those like, storytelling elements. And to Adam's point, it doesn't make those like, creative leaps that we see humans make. So like, AI can basically can give you something credible, but only humans can really give you something very compelling. Right. And it kind of comes down to a couple different things. What you're saying first is we have all talked about how AI doesn't really invent, it synthesizes. And I think Adam, you said it best when you said it's going to a probability. Right. And so it's like taking and drawing from the best award winning campaigns that we've ever seen, but then also like all the forgettable digital filler that's on the Internet and regressing sort of towards that meme.
- [18:21]
B
Yeah. And I would say too that, that the chewy ad that was AI generated like it's, it's not terrible. I mean, I think that's an important point to make. It was okay, it was average. It just didn't have the same like emotional precision. I would, I guess I would call it with, with the, the shot selection and the, in the humanity. So like it had the structure, it just didn't quite have the same like underlying human truth of, you know, someone's relationship with their, with their pet.
- [18:53]
A
There's probably two points here. One would be can we improve things by improving prompts? And then secondly, does this just point to the fact that there always should be human involvement in the creation of these AI ads?
- [19:09]
B
Yeah, I don't think it's any secret right now that, you know, the human and AI combination is what really makes this work. But again, in this study we wanted to let AI make, make all the decisions just to test the, the current capabilities. Right. And you know, if you think about, for example, like what Facebook is doing right now with their advertising platform, I mean they just want you as an advertiser to tell you like what, what's the business goal? What's your objective? Okay, great, give us your credit card and we'll figure everything else out. We'll generate the ads, we'll use our data. Right. And like that's happening right now. Right, right now. But most of those ads are just text and images. But soon video is, is, is going to, to follow. So yeah, so we, we know that you know, when the human is driving, you're going to get much better outputs. But just as a, as a reminder, this study was to see like, what happens if we just let AI make all the decisions.
- [20:16]
A
Do you think, Lisa, that generative AI can do performance marketing better than humans? Because if we're led to believe it, most humans hate doing performance advertising so much of their hearts in it. Maybe generative AI is better at performance marketing.
- [20:32]
C
So I would argue that performance marketing often gets treated too much like a formulaic thing. So yes, if that's how you approach, you know, if you approach brand versus performance in a very binary way, which a lot of companies do, then yes, you can definitely use AI to do basically an interns level work. Right. In terms of the advertising that you put out into the world, is that an effective and efficient use of your media dollars? I would argue no, but that's a broader, broader shift, I guess, in how the industry approaches advertising overall. And so, yeah, if we want to see those channels, like where all of the growth in advertising is to AI and to performance marketing, then sure, maybe that's a good future. But I don't think that that's what we should do because then we're basically, you know, taking all of the growth in the industry and putting it towards short term performance AI slop. So I would prefer to not see us go in that direction. Really. The strategy and the brief in this world of AI generated things is more important than ever. The highest performing brand in the entire study that we did across both human and AI versions was the Cheerios ad. And the brief that was generated from Gemini really spoke to this idea that obviously talked about heart health. That's core to what they've been messaging as a brand. But it tied heart health to this idea that you want to be there for your loved ones when they need you the most. The original human generated ad was the daughter was putting the Cheerios on the dad's chest, like on his heart because he.
- [22:24]
A
Yeah, that's a cute ad actually. I remember that.
- [22:26]
C
Yeah, it's such a cute ad. And you know, AI kind of took that, that brief and it did, it did do a decent job of telling a story around that and really tapping into that human emotion. Now, was it as heartwarming as the human one? No. And the results did show that. But it was a really strong performer still in the AI version because it had, you know, human insight to tap into and to like since it was delivering that message. And it's, it's pretty good at doing that, right? And getting that message across. The power of the insight was what really carried that one, even though it wasn't, you know, quite as, as human of a story as the other one.
- [23:11]
A
So I guess that's it. Adam, it's in the short term, with the technology in the state that it's in or the phase that it's in, what we have to do is make sure that what we're feeding into it if we choose to use it, is really tight. And the tighter that brief and the tighter that strategy, the greater the chances are of it delivering something that's close to human created.
- [23:34]
B
Yeah, yeah, 100%. You know, a vague brief is going to give AI room to, I guess, for lack of better way of saying it, average everything out and, you know, a strong brief that really resonates, you know, in the human truth and is driven by, you know, research. It's going to give the AI something better to, to work from.
- [23:57]
C
Yeah, I would have thought, like, oh, the more specific that you can be with the brief and the prompts and the tone, that that was going to set things up for success. But kind of another example where we had a very specific brief, I think was fiat, where it was asking for, like, hey, we want the tone to be defiant and rebellious and have this, like, subtle humor and was a little bit more descriptive and how, you know, it saw that. That idea coming to life and I really struggled with that tone. So it's not necessarily like, oh, we've put all these guardrails on it and we've got all of this direction. And so it's the. It's going to be a great prompt. Right. Like, sometimes the types of things that we're asking of it, it's just not. It's just not good at sort of delving into those spaces. So even though we feel like we're being very prescriptive, it doesn't necessarily yield good results.
- [24:56]
A
So let's loop back to the. To the report and summary here. Adam, what do you think? What do you think the industry needs to learn or should. Should be paying attention to in the report?
- [25:09]
B
I think the biggest question, the biggest point that marketers have to think about, you know, when reading our report, is whether the effectiveness gap is going to narrow at the same rate that that perceptual gap is. Is going to narrow. Like, I am never going to bet against AI in terms of its output, the quality of its output. But again, the harder. The hard problem isn't just like, rendering or generating that, you know, spot. It's going to be like, where do we impart our judgment? Right. Knowing, like, what parts of, you know, the story, like, really need to stay human? Do we go against the grain? Do we go with the grain? Are there, like Lisa mentioned before, the cultural insights, like, what are the ideas that are really going to stand out and deserve people's. People's attention? And so, like, as I mentioned before, I don't think this is going to be like human work versus AI work. It's going to be that, you know, human and AI combination that's really going to win out in the end.
- [26:21]
C
We're talking about this in terms of the capabilities today, but I think as marketers, you know, we really need to be thinking about those effectiveness trade offs because it really is, you know, the business result that we're talking about here. We're talking about, you know, watering down creativity and how this is making everything a little bit more vanilla in some ways. But there's also a lot of risk here that we need to talk about in terms of the return on investment not being as strong. So it's not just about the lack of creativity. It's how creativity drives business performance, performance. And if we're seeing that right now, the effectiveness is not there. Maybe the efficiency trade off is not worth making at this point in time. Now, that could totally evolve, but that's the type of conversation that I don't think that we're having frequently enough. There's a lot of focus on how do I embed these solutions so that I can show that I am getting faster, doing things cheaper, you know, showing my boss that I'm embedding the solution that we've invested in. But when we think about the media risks, when we think about just the overall risk to brands that this could represent, I think we just do need to be mindful of that and think about how we leverage these solutions in the process, but where we still keep humans in the process, both from a strategy perspective, a creative perspective, in bringing consumers into the journey with us and continuing to listen to the voice of the consumer throughout the creative development process. These are all things where we still want to keep that humanity in creative development. And I think this research shows that we're not there yet.
- [28:15]
A
Yeah, I guess that's a great point, because I think what I take from it is similar to what you're saying, Lisa, that this sort of independent, fully automated way of using AI is just not ready yet. It is not possible to do it. I think you've proven that in the research. That may be different, as you said, Adam, in six months, 12 months. But it also points to the fact that, as Lesbinet has said recently, and I've mentioned on the show a number of times, the way he talks about it is that AI will raise the floor, but it won't raise the ceiling. In other words, that it will make it more efficient and easier to do. It'll make average a little better, but it won't create original great ideas that will still be human driven, but the future will be. To your point earlier, Adam, it's going to be a partnership between these tools and the human creative mind or strategic mind to be able to build upon each other. And I think with the evolution of the tool and with smart thinking, I think that it can create those business building outcomes that even human centered ads alone are doing today. It feels like that's inevitable, but it's not. I don't, I hope there wouldn't be. I don't see it being fully independently done.
- [29:33]
B
Yeah. Remember, the only caveat is if we do truly get to AGI artificial general intelligence, then, then, then we, then we're in trouble.
- [29:43]
A
Yeah. Then we all lose our jobs. But I suppose the signal is for strategists in the next five years. We're okay, guys, it's good. Good. We're still going to be relevant and needed. At least for the next three to five years. It is. Lisa Zelinsky, SVP Creative Excellence Strategy at ipsos, and Adam Perida, Associate Professor, Syracuse University. How do people get a hold of this study, Lisa? Is it available? Can we post it on the site?
- [30:06]
C
Yeah, we definitely can.
- [30:07]
A
Yeah. We will post it on our website@onstrategyshowcase.com and you can download it there. Thank you guys for coming on this episode of on the Pulse. We appreciate it.
- [30:16]
C
Thanks, Fergus.
- [30:17]
B
Yeah, thank you.
- [30:18]
A
And we will see everyone on the next episode.