Loading summary
Transcript76 lines
- [00:00]
A
The Voices of Search Podcast is a proud member of the I Hear Everything podcast network. Looking to launch or scale your podcast, IHeAreEverything delivers podcast production, growth and monetization solutions that transform your words into profit. Ready to give your brand a voice then visit iheareverything.com welcome to the Voices of Search Podcast and I Hear Everything Production. In this podcast, we'll share the news, knowledge and strategies you need to navigate the ever changing world of SEO, ready to expedite your company's organic growth efforts. Sit back, relax, and get ready for your daily dose of search engine optimization wisdom. Here's today's host of the Voices of Search podcast, Tyson Stockton.
- [00:46]
B
My name is Tyson and joining me today is Kristen Tinski, SVP of Creative and co founder at Fractal. Welcome back to the podcast, Kristen.
- [00:55]
C
Thanks for having me.
- [00:58]
B
It's been a little while, but second time I've had you on the podcast, I believe. If I remember correctly, the first time was probably back in I guess like 22 or 23. You were coming out with quite a bit of work as far as like some early kind of AI LED workflows. But I want to pick your brain a little bit more on kind of like how you see our industry developing. And before we started recording, we were talking a little bit about like a post transformer world. So what, what does that mean to you and how would you describe what that looks like?
- [01:34]
C
Yeah, so, well, I guess I'll preface it by saying nobody knows exactly the truth here because these architectures are not completed and some of them may require breakthroughs that we haven't achieved yet. So the timelines are not entirely certain, but it's seeming like in the next maybe three to five years we'll have some of these breakthroughs that will allow for remediating some of the issues that Transformers have. And that will be what changes the search and marketing game, I think. So I guess you can start by talking about Transformers themselves and the issues that they have. So a Transformer is basically like a giant matrix. You can think of it like a spreadsheet. The weights are in each individual cell and every time that the training is done or each pass of training is done, every single weight is updated. So it's an incredibly computationally dense thing. And for that reason it's also very. It's a huge energy suck. Right. So like talking about building massive data centers that are requiring even their own power plants, the future of Transformers in terms of scaling them is sort of hitting an upper limit. We know that they scale in a quadratic way. So the more parameters you have and the more training you do, the more compute you need in a really exponential way. So we're sort of hitting the upper limit and I think that's, that's really what's going to require us to, to find this new paradigm.
- [03:13]
B
And how, how do like, and I agree with you, it's like we don't know exactly when, but I think it's like it's less of a question of whether or not this is the direction that we're heading in and more of like that wind. So like if we remove kind of, if it's next 12 months, 36 months, whatever, how would you describe kind of like how we would compete in that? Like what implications does that have in how we're strategizing our efforts within search?
- [03:45]
C
Yeah, so when we get these new architectures, the primary thing that's going to be solved is continual learning. So right now transformers need to be trained and then they can run inference and you get an answer, but they're not learning as you're working with them. They were trained once and then they're doing inference. The next generation of models will be continual learners. So they'll learn with you and they'll learn continually as you go. You won't have to retrain them continually at half a billion dollars or whatever it currently costs to train a frontier model. So when that happens, you'll also start to get hyper personalized agents. So you'll have an individual AI that is growing with an individual person. So that creates a hyper personalized specific AI that is tuned to the intents and the interests and the personality and the background and all of the history that that particular AI has built up with that person and not in like a add on like prompt over an existing model. Like the model is actually integrating information into its system in a continually learning sort of way that's different from the way that transformers are currently doing it. The other thing that I think will happen is we'll see AI move to the edge. So right now everything is on these massive server farms, but next generation AI will be much less compute intensive and will be available I think on phones. And again, not sure exactly how long that will take, but when it does happen, I think what the future really is is again individualized AIs that grow with, with people as they, as they live their lives. And those agents will become over time hyper personalized and in a way that's not really transparent between individuals. So there won't be like A like a core, you know, search system that's ranking everything. Like any search that's done will always inherently be hyper specific to the person that's doing the searching. And then whatever the index is.
- [05:51]
B
No with either of those I think and you kind of hit on it right at the end. Like one of the big takeaways for us then as like practitioners is going to be that high variance in results essentially because if you have that significant personalization then you know, you would expect that like my results are going to differ more from yours. So there's obviously like how are we approaching tracking and measurement and success. But then I think more probably more interestingly too is like how then does that guide our content creation or our like activities? And I assume you'll still have your kind of tech foundation to ensure that, you know, you have a seat at the table, your content's indexed, it's has the potential to be served. But then it's like what you can create and how you create feels like that is going to be more of like the change in our prioritization or our strategy, so to speak.
- [06:53]
C
Right. So I think these models, as they become hyper personalized, you're going to see a shift in the distribution of content that's, that's served to people who are looking for it compared to now. So like now we have, you know, the, the long tail is, is sort of invisible in a lot of cases. So like it's almost a winner take all scenario in terms of like if you're in the top 10 search positions or you're currently mentioned by you know, AI results. But in a hyper personalized AI future you'll get a long tail that gets served much more frequently because the, the answer to a particular person's question is something that's, that's answered not generically for like a whole bunch of people, but for just that one person. So it could be something incredibly specific that was answered only by one person in one place, you know, a long time ago. And I think you also get a situation where like the middle of the, of the tail is also becomes larger. So it depends on your industry. I would say I think you will have some winner take all scenarios where like the answer to the question is universal for almost everyone. Like you know, particular brand or product that serves the entire market, not just one person. There probably aren't very many things like this for the most part. I think you'll have, you know, like a restaurant for instance, could be kind of invisible if they're not well rated. You'll you'll sort, you'll sort of see I think reviews and things like that also becoming much more useful to these engines where they, they can leverage not just the positive reviews but the negative reviews to really understand like who is this restaurant for or who is this product and service for and then match it hyper personal in a hyper personalized way so you won't get like is this a good restaurant or a bad restaurant? It'll be this is an amazing restaurant for this super specific type of person and a terrible restaurant for this super specific type of person.
- [08:57]
B
And in that sense I feel like especially for you know, a smaller player, someone that's not like the clear market leader. Like to me that's just a clear signal of like, yep, pitch. Pick your niches, pick your like core areas that you can attack on and like that would be a way of competing with the bigger players that maybe aren't able to. Would you say that there's like depending on industry, brand like size of market
- [09:31]
C
become about really clearly defining who you are and who you serve and who your customer is and what their, what their intent like gestalt is. So all the intents that they might have that your brand serves, you need to really understand that clearly so that all these hyper personalized custom AIs that exist out in the world can see your brand for what it really is and who it actually serves best. If you try and do too much and try and serve too many audiences or customers, but you do each one in sort of like a half hearted way, then the models are going to think that you're like this non specialist or not really understand you as being a specific thing. And, and for those reasons you'll be more invisible in the long tail. So I think it becomes less about like trying to become popular and more about trying to highly define who you are and understand your customers incredibly deeply. In a way that's not just who I want my customer to be, but who is my customer that I currently can serve well, which and I feel
- [10:41]
B
like again that the smaller, already more niche ones probably like perfect. These are three to five. Very clearly where you're going to get debate is with some of these big players where they're like yeah, but now I'm understanding this is I just need to be really sophisticated in how I specialize to all the different ones. And with that it sounds like yeah, you try to spread yourself too thin and do it kind of halfway, you just lose the overall how would you guide folk like what that focus should be and it's like I know there's gonna be a trade off of like how well can you do it? But I feel like if you pick some of like the bigger, broader, kind of like more generic consumer brands or like maybe like E commerce aggregate websites and things like that, like they may have the data infrastructure to have personalized experiences. How would you look at determining if you're going too wide or too narrow on your efforts?
- [11:54]
C
I think you need to think about trying to become like the canonical best answer or best result for a particular Persona. And that's what you should really be trying to achieve. Not capturing every possible target audience that you think might apply to you. I think that that will be a recipe for disaster. And those that are able to like hyper define who their customers are and understand them and then create content that addresses those needs very specifically in a, in a coherent and cohesive way that represents who that customer is in like a, in like a latent representational space in the AI's mind. Right. So like the AI has a representation of the person and who they are, has a representation of the brand. And those two things have like mathematical shapes in their latent spaces. If those two shapes are very similar, then they'll get, then that will get served or that will be the answer to a question. If they're very different, it won't be. So that's sort of how you have to think about it. It's like you're, you're matching like hyper dimensional objects in a latent space. That one represents a person and their needs and the other represents a brand and its services. And when those two things are very aligned, that's when you get an AI answering that this is the brand or service that would be a best fit. And so that's what you're trying to do is understand the search space and the intent space and then your brand space and your content space and create the mash between those two.
- [13:23]
B
And as more and more of that experience and everything is surfaced within these systems, obviously there is impact to site traffic. How would you be looking at kind of measuring success in this as well as like we know more of the buyer journey is going to be in that, you know, not necessarily to the site and hey, how agents develop, there may even be less people going to make those transactions. How would you be looking at monitoring success and then also being able to use that to yeah, kind of like manage within your organization.
- [14:07]
C
I mean it's hard, I mean it's getting harder right now because these, the current AIs are black boxes. Right. So like we can, we can sort of probe them a little bit and try and understand, you know, what brands they, they answer with for particular things. But I think it's always going to be really difficult when you have a transformer based model that will always be a black box. I think next generation models will be inherently inspectable or we'll be able to understand their reasoning processes in a much more specific way. And when that happens that'll open up a ton of new doors for understanding how, how they come to the conclusions they come to and how as a brand you might be able to better fit that progression. But until we're there, I don't think we'll have great insights into exactly what, what matters most to them. I mean like we have, we have an understanding of what their training sources are and you can to some extent like manipulate how much you're served in there. But again it's, it's, it's a black box and it's always going to be some combination of like how often you were mentioned and then also like the, the spread of how, how well you are understood to be related to a particular topic by the AI. So
- [15:24]
B
and you see like a lot of this has been geared towards like what we are creating and who we're creating it for. Are you finding yourself like re examining or reconsidering like other facets and like yeah, tactics within SEO and thinking like yeah, maybe I want to be revisiting how I was looking at, I don't know, internal linking, backlinking, off page SEO, like what other areas is it kind of like shifting your viewpoint of?
- [16:01]
C
I mean I think linking still matters. I think it will always be a signal that is useful. Right. Like it's a, it's a vote of confidence to some extent but I think it'd be in these next generation models and even to some extent today it becomes more of like infrastructure background, like table stakes that you need to get right in order to be even considered. Right. I don't know in the future really how much manipulating like on site stuff is really going to matter at all. I think for the most part it's going to be either you're included in their training set or you're not. And if you are, then the way that you're understood by the AI doesn't really have too much to do with like the technical specifications of your on page stuff. It has to do with whatever the content, the specific content that was scraped and it's, it's topical understanding and the generalized breadth and depth of what those topics are and how they cohesively fit together. Not necessarily like internal linking structures or things like that. Those were heuristics used by search engines. They're not heuristics really that are used by AI other than I think the link graph will be part of maybe what's added on as like a secondary trust layer or some sort of filtering criteria for what's used in training set training, data sets. So yeah, I think trust signals, which is more than just links. Right. So like I think you'll get whitelisted domains and things like that as well.
- [17:39]
B
Yep. Yeah, it's like general, maybe not like corning it into like just link building but more of just like authority building or management.
- [17:50]
C
Right? Yeah, it's definitely authority building. And these, these models, they understand authority, right. They, they know inherently that you know, a certain publication is well known. Right. So in a general way they understand if a certain brand is referenced a lot by other popular outlets or well known figures or things like that, then then it understands that this is a more salient brand to the general public. So those sorts of authority signals do matter quite a bit to these models, but not in a search engine type way. Not in like a link building authority type way where you can tie a specific like domain authority increase over time to like a specific increase in search traffic. It's a much more generalized. If you could get a lot of coverage that mentions your brand within the context of the service that you offer and the customers that you serve, that's hugely important. Getting a single link from one place, it wouldn't do the same thing it did 10 years ago.
- [18:54]
B
Yeah, yeah. It's not still the old kind of like wild west days of buying links?
- [19:02]
C
No, yeah.
- [19:04]
B
Much more deliberate or targeted in that sense now. So a lot of this has been around kind of like these industry shifts, strategy prioritization. I feel like there's also the how we are doing work as well. And like I know you spent a lot of time within creating agents for it. You've spent quite a bit of time on like the content front. Tell me a little bit about like maybe like before the specific agents, like how are you thinking about that evolution, how we work as SEOs and what's your vision for how you're progressing? Like the utilization of agents in your work?
- [19:51]
C
Yeah, I mean I, I think what's happening with AI is that new abilities are being unleashed for individuals that are absolutely incredible. Right. So like most of the work that we do at fractal is creating data journalism type content for brands. And that process has always been very labor intensive and research intensive. And there's a lot of work in gathering data sets and analyzing data sets and then putting together investigations and finding what's interesting and finding what's newsworthy and doing the statistical analysis to make it relevant and trustworthy. And these new tools and new agentic frameworks and environments that we have now allow us to massively improve our ability to do that work. So not only do it faster, but do it much deeper and better so we can explore much more than we previously could. We can gather data that we couldn't previously because we can write scrapers incredibly fast sorts of data analysis that can now be done is just astronomically better because again, you can have frontier models write code that will, you know, do disambiguation or cleaning or, or collating or analysis or, you know, every single thing that you would do in a data journalism project can be automated or semi automated by these models now. And so it moves a creative or a data journalist from the one who's doing all of the specific technical work to the one who's managing at a top level what's interesting and newsworthy about this story. And so these new technologies, especially agents, have allowed us to accelerate and deepen our ability to do really interesting work. And in a future where, where what matters most is having content that is sort of unique only to you, where you're presenting something original to the information space around in the world that is the real currency of the future, I think. So if you can present something new empirically, some new synthesis, some new distinction, some new mechanism that's data driven, those are the things that these models, especially the newer frontier models, will care most about.
- [22:26]
B
And what is your approach to development? How do you pick? Is it more of like, hey, my team's having to do this at this frequency, this is going to be the biggest opportunity to work on like, how do you go about like identifying what the build is going to be and a little bit on like, what is your process to in that creation?
- [22:53]
C
Yeah, so I mean, because there are so many different types of data projects that you could do and so many different data sources and things, a lot of times it requires writing custom code for a particular new project. But a lot of times there are repeat data sets that we use. So like if we wrote a scraper for a social site, we can obviously reuse that agent or that scraper the next time. So we're always looking for like modularized or repeatable steps within the content production process and then leveraging those across our different creatives to have shortcuts for the next time. And then aspects of our creative process that are just built in, like fact checking and like quality control and final review and things like that. And peer review, we can now automate in new interesting ways and also use agents as peer reviewers, which is incredible. Right. So like we, we used to spend a lot of time having other individual creatives critique other creatives work and we still do that. But the depth that we can now do or the depth that we can now have with AI critiquing our work is so much better. We can be certain that we've explored all the different areas that there are to explore. We can be certain that we haven't misrepresented the data in a critical way. And we do this through using agents that have a test driven sort of approach. Right. So like we understand that, that all current transformers have the problem of confabulation and making stuff up. The way that you mitigate that is by using a test driven approach where you have the agents so the AI is writing code that is actually testing the data itself and then empirically finding a result. Right. It's not just coming up with some answer from its, its weights. So yeah, did that answer the question fully? Not sure.
- [24:59]
A
Time for a one minute break to hear from our sponsor, Pre Visible. So you're looking for SEO help and you got a couple of options. You could start replying to spam from agencies that claim they can get you to rank number one on Google. You could pay an hourly rate for a consultant who will inevitably nickel and dime you with hourly charges. Or you can work with a cookie cutter agency to quickly launch a strategy less project with low success rate. None of those sound very good now do they? Well, that's where Pre Visible's integrated consulting model comes in. Pre Visible draws From a collective 40 years of SEO and digital marketing experience to unlock your organic growth opportunities. They build custom solutions that combine strategy, technical expertise, content and reporting to effectively operationalize SEO for your business. Pre Visible's four stage approach ensures that your SEO programs thrive by starting off with a strategy first approach. Then they support you in your efforts to create quality content, help you identify technical issues, and most importantly, they'll work with your cross functional teams to integrate your SEO strategies to make sure that your SEO budget actually drives results, not just your agency's bottom line. So join brands like Yelp, eBay, Canva, Atlassian Square, all who rely on the SEO Consultants at Pre Visible. For more information go to Previsible IO that's pre visible. P R E V I S I B L E IO
- [26:26]
B
no, no, no. I mean I have a follow up just on like that. Very envy. So if I understood correctly, you're saying, so tackling something like QA or fact checking, rather than just doing like a post kind of analysis, you're adding a layer in, in the creation process that's like forcing it to essentially score, have a certain threshold and then move on rather than just doing the more traditional like post process.
- [27:01]
C
Yeah, I mean ideally you have an agent that has an understanding of what the process should be and what the fact check process looks like. And then it's not just a singular step, it's a co creation process with that agent where you're continually reminding it of what the rubric is or what its understanding of what a good project should be is. And then it's incrementally helping you to keep it on track. And that's, you know, that takes practice. Right? Because a lot of these, these AIs are, they can go off the rails and it takes like a certain amount of discipline to keep them in line and being careful with keeping like a, like a constitution or like a document basically that it has to constantly refer back to and validate its work and then again test driven stuff. So if it's writing code that you write a test first, then write the code, then run the test.
- [28:03]
B
Makes sense. Now within, within these, like I would like to, I don't know, dive a little bit deeper into maybe like a, a favorite or least favorite. Like has there been a particular project that felt the most rewarding or that you were just personally pumped on the most? Like,
- [28:30]
C
so like what a good illustration I guess of, of like the progression. Like a year and a half ago maybe I wanted to do a project on, on Trump vs Biden's speech patterns because like there was all around the election there was all this discussion around like dementia and cognitive decline. So my question was like, could this be measured in their, in their speeches in some way? So I did this project like a year and a half ago and it took me maybe like two weeks to do because it was really difficult to do the scraping of their audio. So like I like there actually aren't that many sources for this and so I had to like go out and find the sources which took a while, write my own custom scraper to scrape the site, use voice to text to do, you know, to get the transcripts from them. And then do lexical analysis on that. So it was like this four or five step process and each step of that process was like, had to write code for, had to do some sort of deep analysis that took a while. I redid this recently with, with Opus 4. 6 in Claude, like a Claude agentic environment. It was able to do almost the entire thing, like start to finish. Like it found the source, it wrote the scraper for the site, which was not easy. Like it was a JavaScript site that had all these dropdowns and clicking things, got, got the downloaded videos, transcribed them using AI, compiled the data sets, then run analysis on the data sets and do all the statistical work that you need to to really understand whether or not these things are just statistically significant. So being able to do a highly complex project like that and even go deeper than I had before was just, you know, ten times easier. Right, right. So like, what does that mean for the future of content when you can do an investigation of that depth in a matter of hours versus a matter of weeks? I don't know. I mean, in my opinion, every brand could become a, essentially like a journalist outlet or journalistic outlet. They could do deep investigations in their industry on absolutely everything they wanted to or could. And I, in my opinion that's what the future of content marketing will be, will be brands that, that make themselves like a canonical fit with a particular service or topic or category because they have helped to broaden the world to broadly understand that category in every way that they possibly can through multiple types of investigations that would matter to their customers.
- [31:03]
B
And it's like, yeah, that brings it back to the personalization front where having that added ability you can come up with something super unique and super insightful for a specific niche that maybe you didn't have data access. Prior, like prior, it just wasn't necessarily like feasible to take on something like that.
- [31:29]
C
Or maybe it was even like a really niche topic that you said this is a cool idea, but the audience for this is too small to be worth the time now the amount of time to do it is so much smaller and also the target audience is going to be so much more reachable. So even if you have a really small niche audience because of the hyper personalization of the future of AI, where you get like almost a perfect parity between the search and the answer, addressing the niche or the super long tail is going to be much more worthwhile.
- [32:04]
B
Which to me that's really interesting because it's like, then again, you're looking at those previous assumptions and trade offs, whether it's like size of market, size of niche, whatever. It's like all of that has to be kind of like resurfaced because like, yeah, now your weights and your kind of differentiators aren't. Aren't the same.
- [32:27]
C
Right. Yeah. And also like all of the, all of the fluff language is sort of useless at this point. Right. So like best, any superlative, like the, the models do not care about this. They will assess you on their own. Like if you call yourself the best, that doesn't make any difference to them. Like they will understand you holistically. So fluff language, marketing language, all of it will decline in its usefulness and it will be about representing the truth accurately, both what you do well and what you don't do. To have highly defined boundaries on who you are and what you do and who you serve. And the clearer those boundaries are, the more likely you will be to be accurately served to the audiences that will want you. And then your metric becomes conversions really as the primary thing that you care about. And your conversion rates go way, way up because you're actually the answer to whatever the search was. You're not a partial answer or you know, sort of what the person wanted. You're exactly what they wanted.
- [33:36]
B
Yeah, yeah, like that, that opportunity and personalization has also that opportunity for significant change to conversion rate, which is a nice counter to the, hey, we're not necessarily worried about just site traffic here. This is ultimately how we want to be. Yeah. Monetizing it. You know, as a consumer, all that sounds amazing. Like I'm like, great, I'm going to get like more specialized content, higher value content, more research, like more accurate content. Like all that is to me painting like a pretty nice picture as a consumer.
- [34:17]
C
Yeah.
- [34:17]
B
Yet there's still a lot of fears and obviously sometimes just with change comes, you know, some level of fear. I feel like the consumer positive seems pretty obvious. Do you see other kind of like trade offs to the consumer that you would be fearful of and it's just more of like yourself of as a consumer that you're like less optimistic about?
- [34:49]
C
No, I mean, I can't. I mean privacy related things. Yes, of course. But that's an ongoing thing. No, I mean I think the major risks and concerns are really on the brand side. Right. For, especially for companies that are just sort of like me too companies and kind of blend into the middle and aren't differentiated, aren't the best at what they do, aren't producing anything new or interesting to the conversation. That, you know, rely on fluff and exaggeration and marketing language to sell. Those will be the ones that I think will suffer the most. But honestly, maybe they deserve to fair.
- [35:38]
B
I was actually kind of thinking that too. I was like, yeah, this also seems like a win for the consumer.
- [35:44]
C
I think brands need to start thinking really critically about what they're contributing to the world beyond just what their service offering is informationally. So these AIs care about that a lot. If you want to have a broader reach, you have to provide more informationally that the model hasn't seen before. And that will really be what the bottom line is.
- [36:10]
B
True. Now, before we kind of jump into the shorter like lightning round kind of portion of the interview, I like asking too, like, what are some of your favorite tool? And it feels like too, it's just things are moving so quickly like and I'm sure you're building most of these tools kind of yourself but like for development creation, like what are some of your favorite kind of go to tools
- [36:41]
C
at least as of lately, honestly I would say I do almost all of my work in Cursor now and then to some extent. Like Claude Code and OpenAI just released like a new similar offering that are basically agentic systems that do coding but can also run code on your computer and can interact with your computer in a few different ways. But in my opinion Cursor is the best out there, at least for developing. You can use any model. It interacts with your code base really well. I don't know. It's getting to a weird point though because like some of these new frontier models like Opus 4.6 are just, are so good. And especially in systems like Cursor where they can launch like multi agent sort of investigations, you're getting like this sort of swarm agentic environment within Cursor where it can accomplish large projects in really short amounts of time. Like you could say like here's a website that I want to spin up and here are the criteria for it. And Opus 4.6 will generate four, four different agents that will each independently work on four different aspects of it in test driven ways and then put it all together. It's, it's getting to a point where it's, there's not that much human in the loop that's required other than what is the final product and is the model, is the model following directions properly, sort of keeping it on track?
- [38:24]
B
And do you see that as being like more and more and I guess we've already seen this in some ways but it feels like they'll likely be that scenario where you have, you know, different specializations between models and it becoming like a little more agnostic to it where it's like you have your system, whether it's curse or something else and then it's just delegating out depending on what the task is to different ones. Or do you see it being more of this kind of like winner take all?
- [39:01]
C
No, I do think you will have specialist models depending on the industry. The moat there is really about the training data set. I think so like a specialist model will only matter like insofar as its training data was special. I don't think like architecturally it will have huge differences. You know, like right now we have 10 or so frontier models that are all sort of like of the same ish level. I don't, you know, people pick and choose from those more more on pricing I think than on like their actual differences. They seem roughly interchangeable for most cases for me at least between like Opus or like sonnet, like the CL, the top cloud models, the top OpenAI models, the top Gemini models. Like they're all pretty similar in terms of what they can do.
- [39:58]
B
And for you then are you picking that more on a pricing
- [40:05]
C
or is
- [40:05]
B
it you just have a favorite?
- [40:08]
C
I don't know. Like I'm still working through that, I feel like, because it can be easy to say, like just let Opus 4.6 do it because you know it'll do it almost perfectly. But again, it's a very expensive model. So like you have to be careful create. Could a much cheaper model do it? Well, maybe. Or maybe it messes it up and you have to do like 10 different rounds of iteration to get it right, whereas Opus would have gotten it right on the first one. So I don't know. It's really, it's a really weird calculus that you have to do for any decision. Like use the best model, use the. Use the third best model. That's way cheaper. Can it do it? Maybe, maybe not. So yeah, if you have a really good feel for what each model is capable of, then I would say yes, go for it and try and switch between them. Otherwise it's probably best to pick like a mid range price model that's also highly performant like Sonnet4.5 and just stick with that.
- [41:03]
B
Makes sense.
- [41:04]
C
Yeah. You can also with cursor and also some other systems, they have like an automatic mode that supposedly does this for you. Like chooses the best model for the case. But they're not transparent about like what that heuristic is, or if there's like some intermediate model that's like deciding which model uses is used. I haven't found it to be that great.
- [41:25]
B
See, I mean theoretically it feels like that would be the direction it goes in, but also like, yeah, I haven't necessarily seen it play out that way, but I don't know, I guess just like to myself it feels like, yeah, maybe it's not immediately, but it seems like that would be the direction eventually.
- [41:46]
C
Yeah, and it could be. It really could be. It just depends on like if that automatic intermediary model, like gets really good at understanding which models are good at what. And also you don't have a case where there's just like clearly one very superior model, which there have been times where that's been the case. But then like a week later someone comes out with a, you know, something at that level.
- [42:09]
B
So yeah, it's like, it's like a release race versus actually like a sustained advantage. Well, I want to move on to kind of the, the lightning round section, so I'm going to throw five different questions at you, shorter form, probably rehead some of the topics from earlier. But the first question I'm going to throw out, and I know it's a tough one because we briefly kind of touched on this, but how long do you feel like we have as marketers realistically before that kind of like post transformer systems? Like, we're completely in that post transformer world and I know we don't know for certainty on it, but like what would be your kind of gut instinct?
- [42:59]
C
Like three to five years, something like that? Maybe like the, the pace of change is clearly accelerating, but transformers are also hitting some upper limit at this point. It's going to require, I think, a few new breakthroughs. But what's happening now is we have incredibly performant models that can help make those next level breakthroughs with us. So I don't, I don't think there's anything that's, that's categorically preventing us from getting to that in the next few years.
- [43:34]
B
And you think the main, like, I mean, I know there's like multiple challenges to this, but you'd say it's like more processing and energy.
- [43:46]
C
I mean it's, it's going to require a paradigm shift in, in the architecture itself. Right? Because like the transformer attention model is not viable at least beyond what we're currently at, just because it's the way that it scales.
- [44:01]
B
Right.
- [44:01]
C
Like it's just incredibly compute intensive to have every Individual cell attending to every other individual cell. So yeah, but there are architectures, you know, that are, that are approaching this in different ways and are getting much closer to a future where that will be possible to have a system that learns in a much more brain like way. We know that the brain doesn't use back prop, doesn't use gradient descent. It learns through heavy and accumulation where it's like you learn something one time and it sticks. You don't need to see a cat 10,000 times to know what a cat is. So we know architecturally our brains work very differently from transformers. And we're starting to get to a point where we will be able to replicate a learning system that is much more efficient than basically just a brute force statistical engine, which is what transformers are.
- [45:06]
B
Which would also kind of pull us back to piece earlier around like some of the value and significance in that like authority and reputation management could assume that would be then a heavy play if it's not just volume but having like greater essentially like authority weights.
- [45:31]
C
I, I mean I think for brands, I think the, the play is the right play has kind of always been the same like even from like 20 years ago. The goal of Google and the goal of AI is the same. It's how can we match a need with the result. And the progression has been to get better and better and better and better at that. And we're going to hit a point where we, where AI models become near perfect at it, where the answer to the question is exactly what you need every time. And it's because it is able to go through all of the requisite steps to find that answer. It does all the investigations that it needs to, that you would have previously done yourself. So that's I think what the future is, where what's most important is kind of what's always been important to create content that really defines who you are, that serves your particular customer very directly and that expands the understanding of your space and how you are important to that space in a really clear and well defined way,
- [46:39]
B
which it's a perfect segue to. Our next question which would be if you had to simplify down and just have one thing to really push and have people walk away with of how content strategy needs to evolve, what would it be?
- [47:02]
C
Start thinking about the deep investigations and questions that you've always had about your industry, your product, your service, your customer, their needs, their psychology and how it could be investigated and how you could learn more about it and then start doing that because you, you can, you don't need to be a coder, you don't need to be a statistician, you don't need to have gone to school for this. You just need to have an idea and to be able to work with an AI to bring it to fruition through an iterative process and through keeping the AI on track by having it do a test driven development process. And if you can do that, then you can do investigations that no one's ever done before and we can expand the knowledge of humanity in a great way. Where brands are contributing to this understanding, they're not just trying to sell to people. The way that they become better and more visible within AI is by becoming more informationally relevant to the world.
- [48:05]
B
It's like, in some ways it's like a true journalist's dream to be able to go into this type of investigation which would have never been feasible or possible.
- [48:17]
C
And in this way I think that brands can rescue journalism to some extent because if their mission is about representing truth and not selling, because AI models care about truth and not marketing fluff, and then, and also because AI enables these deep investigations to be done much, much more efficiently and cheaper, then it's, it's just, it's like a win win for journalism, A win win for, for humans. A win for brands that decide to go down this path.
- [48:49]
B
Yeah. What's the biggest misconception that you're seeing with marketers about how these AI systems are surfacing content? Like what's in your mind? The biggest misconception with where we're at in search and how something is getting surfaced?
- [49:07]
C
I, I think anyone saying anything about understanding truly what these models are doing is, is lying. And tools that try to figure out things that are black box are never going to work well. So like checkers for AI text are never going to work that well. There will always be a really high false positive and false negative rate. I think, I think people really need to consider the drawbacks of the current models and what they're actually capable of versus what they're not. And I think there are, there are misconceptions on both sides. I think a lot of people don't realize what they're actually capable of and I think there are a lot of people that think they're more capable of certain things than they actually are. So like understanding the deficiencies and the capabilities, both are really important and that takes exploration and working with them consistently. Yeah, I mean other, other things like, like visibility within an AI or across different AIs or even within AI search. I think like that that sort of thing is also super inaccurate. I'm not even sure like how valuable it is to really even do it. I, I think you can get like a generalized idea of like you're maybe invisible to them, but if you're like a brand that's existed within their training data sets, you're not going to be invisible to them.
- [50:36]
B
Them.
- [50:39]
C
Yeah, you know it's a weird situation that we're in right now because these AIs are black box and, and because there's really like, even if, even if the model providers wanted to, they couldn't give us deep insight into why a certain answer was given. So it's always going to be really rough heuristics to try and understand that until we get to next generation systems where we can inspect their actual reasoning process, which I think is coming, which
- [51:06]
B
I completely agree on that. And I was in a debate like the other day too because especially if we kind of where we started with this significant rise in like the personalizations, the variance is going to be so much greater and so it's like yeah, a true representation is going to be very limited until you know like those other progression points. For me I think it comes down to like two things and I was making like the argument I'm like basically in my mind you have two use cases at this point. One is just like internal stakeholder management where it's like if you're using it as a tool to lead conversations and kind of like keep awareness for people engaged in it because of how you're managing the channel. Like to me that makes sense. There's maybe a competitor like awareness understanding. It's getting a little bit weaker of an argument but like there's still probably something there. But otherwise yeah, you're probably just spending more time on measurement and reports rather than actual value generating activities. Like to me that's like if you're using that to make your life easier from getting tickets picked up and buy in, great. But if it's just to do it to know how you're doing, it's like that's not really a high value activity in my mind.
- [52:35]
C
Right. It's also not instructive. Right. Like it doesn't, it doesn't tell you like before when you're investigating search visibility. Right. It's at least semi instructive. You can look at link profiles and, and understand where you are versus your competitors and what a certain activity might yield versus a competitor. Now even if you could fully understand what your visibility was within, within all AI results or within all AI conversations that are had. It wouldn't really tell you how to go about improving that. Right. Other than what we already know because these are black box systems. It's about a generalized topical understanding of where you fit, where your brand is in the latent space, what that shape looks like and how that relates to the search. So yeah, for now I think it's not a really very useful activity and you're much better off spending your time, I think, thinking critically about your customer Personas and how to serve them best and what your brand really is and defining it in a very specific, well defined way with clear outer edges.
- [53:49]
B
Yeah, it's like take that time, put it into consumer research. That's probably going to have a net
- [53:56]
C
better like yeah, result for you. At the end of the day, I think that will actually become a much bigger focus of marketing will be consumer research and profiling different individual Personas. That's how you will get a, a greater understanding of whether or not your brand actually fully addresses this. Like the, the search surface by understanding all of the potential Personas in a really deep way, what their, each of their needs are in a really deep way. And then whether or not your content and your service offering aligns with all of those things holistically.
- [54:35]
B
No. Next question I want to ask, we lightly touched on it earlier but like review platforms, do you see this? And it's like I feel like there's an argument to be made on either direction of it, but like do you see the review platforms keeping like kind of the, their space in the market and it continuing to be like an important factor? Or do you see that kind of third party piece is maybe not being as relevant in the future? Like how do you see the review?
- [55:11]
C
I actually think they become much more useful than they currently are. And it's for that same reason that it's about understanding not just like what a brand is good at, but what they don't do or what they can't do or who they don't serve. So instead of like a really rough generic 3 out of 5 stars, you understand instead it's 5 out of 5 for this part of the consumer base and 1 out of 5 for this other part. So everyone that got stuck in the three out of five middle because they were great for some and horrible for others now can be the five star thing that they were always meant to be because they're serving their real audience. So I think, yeah, reviews become much more important with the caveat that they need to be trusted. So like this again becomes, I think maybe a decision of the model creators where they're deciding what they use in their training sets or maybe even AIs that are deciding whether or not to include things in their initial training sets or developed over time the heuristics of a model to understand whether or not something is trustworthy or not. So that that could go much deeper. But in general, I think it'll be about deciding what review sites should be included in the initial training sets that create the models. And right now I think that's just like, I don't know, does it, does it seem legitimate? I mean, what are the processes for onboarding reviewers? You know, something like, like Glassdoor or like Yelp? You know, I don't, I don't know. Like, some are more restrictive and take much more to become a reviewer than others. So trustworthiness and I think will become a really, really important thing.
- [57:01]
B
So with that, thanks for stopping by and we'll see you in the next episode.
- [57:12]
C
Sa.