← Always Be Testing
Loading summary
Transcript94 lines
- [00:00]
A
Foreign.
- [00:07]
B
Hello, welcome to another episode of the Always Be Testing podcast. I'm your host, Ty Derange and I'm really excited to talk to Amro Natty today. Amro, how you doing, man? Good.
- [00:17]
A
Happy to be here. Ty, good to see you again.
- [00:19]
B
Absolutely. Good to see you too. We are ready to dive in. I can't believe we are already in the midst of Q4, which is crazy. This year has flown by. Where did it all go?
- [00:32]
A
Crunch time.
- [00:33]
B
Q4, let's go, let's go. Amro's going to be just an awesome addition to the pod. I'm so happy to chat. We've had some amazing conversations. He's had an amazing run. 20 years experience in E commerce, built out monetization arm of U.S. news & Report, which is no small feat. So lot of learnings to talk about which is near and dear to our heart. Let's jump right in, tell the story of you launching 360 reviews at U.S. news & World Report. I'm dying to hear about that whole journey that you went through to do what you did. It's pretty cool. Yeah.
- [01:10]
A
I mean, where to start? Seven years. I guess it's kind of in a way, maybe one way to structure it is to talk about it being a story with three phases. I think of it as kind of joining. On my first day was Halloween 2018, so late October, I joined with U.S. news and World Report. And the idea, the kind of vague idea was we think that the US News and World Report brand could be extended from the places that's traditionally been like college reviews and hospital reviews and then some of their other areas like motive and car reviews, credit cards. We think that there's more kind of meat on the bone. There's more categories, more areas that U.S. news could help consumers make important decisions. And sort of looking at the, the brand, which is kind of bar none, one of the best in, in rankings out there, which every American knows almost because of the college ranking franchise.
- [02:05]
B
Yeah.
- [02:06]
A
And then sort of looking at the domain authority of I think it floats between 92 and 93, which is about as, as good as it gets. I agreed and I saw a big opportunity for us to extend into search and rankings and reviews. And so it really was kind of a memoir, sort of a concept phase that my boss Chad Smolinski and I developed into memos, financial models, sort of took from concept into I would say one of the top, maybe somewhere between five to 10 review sites on the Internet. And I would say probably top three within the most high priority Areas and some of the most competitive review businesses on the Internet. And so. And I'll qualify top 10 within. Within certain niches, but I think top three, especially within certain niches. So, yeah, I think it's a story that goes from. From concept to. To sort of scale. And I think that there's some of what we talked about in earlier in previewing the episode was there was. There was a startup concept phase where it was like, how do you go get stuff done, see what sticks? 0 to 1. And then there was sort of a second phase which was. Which was sort of growth phase. And it's kind of 1 to 10. These numbers. All the. The business information is confidential, but in a general sort of metaphorical way, 1 to 10 and. And then eventually sort of what I call the. The multiple 10 space. So like portfolio of multiple material businesses, multiple divisions there. And I think it was a fun story that we started to talk about that we'll talk about today. I think the way that you do things in the 0 to 1 phase is different than how you do it. The kind of multiple 10 sort of portfolio business phase. And I certainly don't think I nailed it in every key phase, but I think you learn so much about the 4,000 things not to do, and then the three or four things that may be kind of too.
- [04:02]
B
That's amazing. Yeah. A man that we chatted a lot, and I know we may draw from your whiteboard for inspiration. You did an amazing job of kind of like talking that through. And it's so fascinating because I think we talk about a lot internally in our journey. It's been a long one for a group like ourselves. And so it's a really interesting call out of like, each business has a unique phase typically, and I think it makes a lot of sense. I have to ask sort of off the cuff, is there anything that. Was there a phase that you maybe enjoyed more than any other, or is there? I don't even. I'm curious about that.
- [04:43]
A
Oh, that's a great question. So I'm gonna answer a different question, but it's going to sort of circle back to it. I can tell you the phase I sucked at the worst and the phase I sucked at the worst. I actually did. Introducing the whiteboard here, I did sort of chart out the 0 to 1, the 1 to 10, the sort of multiple tens phase. The phase I sucked at the worst was the sort of one to ten phase. Because I think by its nature, it's kind of a messy middle. It's the middle school of your sort of business education. Right. Like I think of it in team size. Like for us, when I think of that kind of phase, it was when we had, I think at scale at like total peak, we had over 100 full time employees and then lots of contractors and freelancers and the whole nine yards on the back of it, but over 100 people on the team. And that was what I guess I consider scale and was a pretty big milestone. The messy middle for me was when we started to have about like 50 or 60 team members. And the, the part that I sucked at was transitioning from being someone who can run a 10 to 20 person team as sort of the manager and sort of sitting at the center of the business and the operation, having my hands on all the levers, making sure every decision goes through Amro. That approach can work very well in 0 to 1. But at a certain point, learning how to let go of maybe the first one or two levels and empower downline folks in your division or divisions or whatever it is, empower your managers that like when you're in that middle phase, you still have to be hands on, but knowing when and where to be hands on is a little bit, it's like discretionary. It's like a call that you have to make. And I don't think I probably botched that call as often as I, as I made it work. And then I think when you get to kind of some degree of scale, it changes and there's, there's, there's far fewer times where you're sort of in it so much as we're kind of driving accountability. I'm going to use the V word vision, which I think is actually really useful, but it's kind of floated around a bit.
- [06:55]
B
Yeah, overused.
- [06:57]
A
Overused a bit. But, but I think it's like getting back to your question about which phase did I enjoy? I think I really enjoy. I think we naturally gravitate towards enjoying the things we're good at. And I think I enjoyed kind of the early startup phase builder as a tinkerer, as a, like the explosive growth that you see in 0 to 1 and, and that ride which any founder, any entrepreneur or entrepreneur can tell you when you have a good ride going zero to one, it's awesome. Every day is learning, it's big growth, it's great. And I think that maybe the, the scale side, when you sort of got a business that's operating, it feels a little bit like I'm not a surfer, but it, it feels a bit like what I've heard surfing A is like where you just get this, this momentum and you're kind of. You sort of coast and you can sort of like the, the way that you're managing it or leading it is a little bit more light touch. Sometimes things come up dramatic, but it's generally more light touch. And that's, That's a really fun. That's a really fun skill. Like, it's. It's very. It's very fun when you can tackle big problems with a person, team and sort of knock them out. And so I think those, those two are. Are interest were like most gratifying for me. When.
- [08:18]
B
When did you feel like you were kind of surfing? For lack of a better term?
- [08:21]
A
When I was surfing, I think that was. That was phase. That was like the maturity phase. That's like 100 people.
- [08:28]
B
The third phase.
- [08:29]
A
The third phase, yeah. It's like maturity business.
- [08:32]
B
Yeah. Did you feel like that challenging middle was going. If I'm hearing you correctly, it sounds like. Were you going from like manager to managing managers?
- [08:43]
A
I think it was probably another layer above that. It was like managing. I think the first time that I managed another VP or a senior director, and then that VP or senior director has like two layers underneath them. That's where I think, like, when you start managing managers, that's totally a milestone, totally valid and presents itself challenges. But you can sort of see how if you think about General Electric and Jack Welch, that used to be the big, like, business.
- [09:15]
B
Yeah. It's the holy grail of like, business management. Right?
- [09:18]
A
Yeah. In the 80s, that was like the swag. That was like the Google.
- [09:22]
B
Even in the 90s. I remember, like reading about it and like it was talked about all the time still.
- [09:27]
A
Yeah. And like, you think about a General Electric, you think about, like, there's probably 12 layers between Jack Welch, famous CEO and sort of frontline worker. There's this like estate velocity that you hit when it's like a certain couple of layers removed from the frontline work that I think the work can get a little bit abstracted. And so you have to sort of take pains to do things every day. Remember, like, empowering the front lines is really important. Understanding the customer is really important. And that's so obvious when you're sort of doing the work or a step above it. But it gets less obvious. I would imagine for a Jack Welch in a boardroom somewhere, it's less obvious that he needs to sort of empower the front line or focus.
- [10:09]
B
Yeah, no, that's great. That's super. Helpful and I love that story. Just like all that you were able to witness and achieve there. And the learnings just had to have been immense, obviously for you to go through US News obviously has reputation, rigorous ethical, data driven standards for the reviews themselves. Reviews are a whole topic that is probably worth talking about. How did you and the team find that balance? Especially through the growth phases of the common challenge of the age old challenge of that editorial, commercial balance and kind of going into new categories and growing volume and growing headcount and having all these commercial demands on you. How did you balance that? Dying to hear more of your perspective there.
- [11:01]
A
Yeah, I think those are. I think those are like two questions. At least I would think of it as two questions. The first being how do you preserve the wall between editorial and business? And why does it exist? How does it exist? How do you make that happen? I think the second is like where should you extend the brand? I think to the first question of how do you sort of build and maintain the wall between business and editorial? I think there's a lot of aspects of business that are a little bit discretionary and then there's some other parts of business that are like bright line. Like there is a rule. This is that rule. I think editorial independence is kind of the latter camp. It's like there's a bright line and you just don't. But here's how some of that looks like in practice and how I've seen it at media companies and publishers. One topic is whether and at what level editors can see revenue or know about business relationships and partner relationships. And I've seen a number of different companies that do it a number of different ways. Typically. I think the gold standard within traditional media is that if you have a line editor, like someone who is touching words and issuing opinions, sculpting reviews and such, that line editor almost never sees revenue. There are some exceptions, not at U.S. news and what we saw, but there are some exceptions. Maybe sometimes commerce teams will operate a bit differently, but usually most review teams, Frontline editor almost never sees revenue and business relationships. There becomes a level above that of maybe managing editor or section editor or something whose job is more managerial, less likely, less typical that they'll see revenue. I've seen businesses where they do and then you, you probably start to hit like director vp. And I think at the director VP level it becomes more commonplace. Like that director or VP of content or editorial. They're an executive at that point. Like they're, they're kind of helping manage the business and allocating services into priority and making sort of priority decisions. And I think it's more typical that directors and VPs can, content, can see revenue. And I think it's ultimately incumbent on them as the person who runs that all important function. Right. To preserve that within their function. And I think it's also incumbent on business and product leaders, marketing leaders, whatever it is, to keep that information out from the sort of meetings that have editorial in the. And we, we would follow very explicit processes where we, we might have a managers meeting where we're talking about topics that 15 or 20 managers need to talk about. But when it came to talking about revenue or talking about partnerships or whatever it is, we would sort of cleave the meeting in half and say, all right, anyone who can't hear about this stuff, you know, break you guys, take those folks off call and then start getting the partnership stuff. Another kind of aspect that US News is famous for, methodology driven rankings and reviews, which is a little different than some other media companies which will have more like an expert editor has tested a bunch of stuff and their opinion based on testing drives the review for U.S. news. Typically it was math has been done relative to some parameters about products or services, whatever it is, and those, the math sort of dictates the review. And over time within 360, we kind of built more opportunities for editorial testing or editorial perspectives to weigh into the methodology. But generally it leans more towards like a math exercise when you do ratings and reviews. And the nice thing about that is it's sort of, it's sort of purpose built to not have any sort of corruption or any sort of influence from the business because I can't affect the math. Like the, the math is the math. And it, it, it's set by an editorial and research team that are, that are doing independent work. And the fact is my, my take Ty, is you pay an integrity tax where sometimes some, yeah. Credible national brand is an advertiser and they'll pay you a bundle of money if they're ranked number one. And they might be ranked number one in a rating cycle. And they'll say oh Amro, thanks for that. And I'll be like bud, I didn't have, I didn't have anything to do with that number one ranking. And then they'll go down to like eighth place. They'll be like, oh my God, what's going on? We have this great business relationship. Like listen, has nothing to do with the rankings. Yeah, you'll be number one again in six months. Yeah, yeah. And on the whole like that, that hits the business a little bit. Like there's probably low double digit percentage kind of tax that you pay to have that editorial separation. It doesn't localize sense like relative to a category. It's not going to tank the business when these things happen. But sometimes it'll, you'll see a swing that occurs. But to me I'll pay that tax all day long because you have a reputable brand and a reputable commitment to the, to the consumer. And I think that's.
- [16:32]
B
Yeah, love that.
- [16:33]
A
A slice of profit you give up for that.
- [16:36]
B
That's so amazing you were leading up to that. I was starting to think through how questions around it like tenants something that Amazon really, really emphasizes and you can use words like other algorithms and all which I think similar to vision can be misused sometimes. But I think that having that system in place is really liberating and I think helpful especially if it does align with going back to your favorite word, the vision. And it differentiates and is valuable to the user. The thing that jumps into my mind is it kind of reminds me of the cowboy code conversations we've been having talking about just our shared values and things that I grew up on and fun lessons from my upbringing and some things aren't for sale. And I think that notion that you said of integrity tax. Is that what you said?
- [17:26]
A
Yeah, the integrity tax. Yeah.
- [17:27]
B
I love that. That's going to be a good one. And I think that's a really smart learning. Yeah. You must have seen a lot of examples of that throughout the business, I would imagine.
- [17:38]
A
There's a great story that I remember in the early days. This is zero to one days. So this is like January 2019. Okay. I had seen this article. So we were, we were figuring out what categories to expand into in the early, early days of 360. And the first category we started with was home security. Chad, my boss at the time, had sort of gotten the ball rolling on home security when I joined. And then I was sort of tasked with go figure out the next set of categories to expand into. And one of the first categories that I felt strongly we needed to test test doing reviews in was mattresses. And the reason why is this great article that came out. Did you ever read Ty, the Fast Company article about Sleepopolis and Casper? Do you remember this?
- [18:29]
B
I remember it popping up. I don't recall it specifically. I don't know if I actually read it. I remember the law going on in that space and seeing stuff pop up.
- [18:38]
A
Like that though yeah, this is like, who knows if it's true, but this is the Fast Company kind of, kind of article that came out, I think 2017, 2018. And what had happened was something like probably going to butcher it. But Sleepopolis was the kind of top SEO ranked niche publisher doing mattress and sleep reviews at the time. God knows who. Sleepopolis is just one of those like, like vertical specific niche review brands. And so Sleepopolis is reviewing every mattress out there and then it gets to the Casper mattress and it basically says, boy, this thing is gone awful. Don't ever buy a Casper mattress. It's just the flipping worst, okay? And so Casper is obviously not particularly stoked about that negative review. This is like the top thing. Casper sucks. Don't ever buy. And so my understanding from the article is that Casper sues the bejesus out of Sleepopolis.
- [19:38]
B
This.
- [19:39]
A
And they're like, I got. I don't know, under what. Who knows what the lawsuits are. But hey, you. You're not sorry. Whatever it is, they're, they're still. And Sleepopolis is like, oh, how dare you. This is editorial independence, journalism, whatever. The fight was, right? So, so meanwhile, this, like, battle is, is being duped out in a set of lawsuits. And the, the venture capitalists that were bankrolling Casper just kind of looked at the situation and they were like, my God, guys, like, we're wasting all this money on litigation. This tiny little review site is worth like $4 million or something. Let's just buy the damn thing and, and then we'll change the review. And so the Fast Company basically article says, these VCs just bought Sleepopolis. And lo and behold, like 24 hours later, suddenly there's a great review of the, of the Casper mattress on it. And I showed that to the leadership team and to Chad, my boss at the time. And I was like, look, there's a real cleanup that's needed in the online review space. Like, there's. I think there's good work that can happen with niche publishers and niche review sites. But the temptation to not pay that integrity tax and just do these pay to play or corrupt kind of dynamics, it can be a very strong temptation for a small business and to give Sleepopolis its credit. I mean, I don't know the Sleepopolis guys. I don't know anything about it, but to give them their credit, it seems like they wanted to do the right thing. They originally thought the Casper mattress stung and maybe tried to fight it, but they kind of got swept up in it. And I thought isn't it, is it, doesn't it just validate the opportunity that we could come in and help out, whether it's in home security or in mattress or insurance products which we launched or in all sorts of stuff around commerce and home service. So I just thought that was kind of interesting.
- [21:41]
B
Oh yeah, love that, love that. It totally hits on so many things for me in particular with regard to our cowboy coach stick. It all stems from trust. And that's like everything we, we do, we've been working on that years and, and you know that and, and so it's, it's fascinating. I was actually having this conversation this morning as I was sharing. I had a really nice coffee meeting with a business potential client that has become a friend. And it's like health and wellness. We're talking about kind of influencers content and podcast in particular and how a lot of those head tail elite massive podcasts that like Huberman Rogan, whatever, every channel goes through these peaks and valleys and evolutions and changes. Right. And how a user perceives things that you and I know. We live in this trust gap era where consumers, rightfully so, are pretty skeptical. And this is a great example of that situation. And you, for you guys to think, okay, how do we avoid this? How we lean into authentic, real data driven reviews as opposed to biases is really important, especially for everything that you and I, our entire ecosystem is that. And so I think for me it goes back to like, what are you working on that is actually authentically driven versus juiced and gamed and things like that. And so I think for us to be in this distrusting era, it's really important to highlight those things that we are doing that is authentic, real, validated and tip of the cap for you all taking that on.
- [23:17]
A
Yeah, I mean and to that point we, we really tried to push, I think the ambitiousness or the ambition or whatever some of the review content we were doing. Like, and I'll share an example of that. We did a lot of hyper local reviews where we would take national data sets or local data sets and sort of stitch them together with a bunch of other data and try to create thousands or tens of thousands of hyperlocal pages in different areas and stuff like that. I mean it's really hard analytical work. There's no humanly way that a person sitting in Austin, Texas like you Ty, if you wanted to calculate, I'm thinking about some of our best moving companies pages which was a fun area for us to work on. Let me do something like, you know, who's the best mover in Austin, Texas or what's the cost from Austin to those pages that we built or some of the app, like some of the content areas within those, those pages. It's really interesting content. It's data driven. It's hard to get that data. And I think the natural alternative is like okay, Yelp, Google reviews, like and user generated content basically. And I think that some of the, the frontier of what we're seeing on, on in reviews right now that I think is interesting is we see more consumers moving away from Google into ChatGPT and AI tools, which are great, I use them daily. I, I don't cook without my Chat GPT assistant anymore. He's got a British voice, his name is Lee and. Oh, you're talking to Lee again. And I'm like, you know, he's my kitchen buddy.
- [25:02]
B
So that's great. Sous chef. I can picture him now.
- [25:08]
A
I should ask Lee what he looks like. I'd love to get a little, you know, phone call, but so like GPT and these things, they're great, they're great at synthesizing information and tailoring it. And if they just announced yesterday, which late September, since this will probably come out, not today, Gripe is doing this partnership with ChatGPT to make it easier to shop inside of ChatGPT. Pretty awesome. And people can really customize their queries and say I'm Ty and I'm this tall and I have this kind of foot and so what kind of sneaker should I have if I want to do these activities? You get all this customization from Ty, from the consumer. What I worry about is that the underlying data that powers that is the sort of wisdom of the crowds. It's just like user generated content from all over, mostly Reddit and Wikipedia and like a few sources. But it's just kind of ugc and I'm a little bit, I'm a little bit more, I'm more than a little bit skeptical that you can just apply killer math to a crapload of UGC and come up with just the right answer and you're going to get this fabulous thing. I think that what the work that U.S. news and other publishers were doing was not a wisdom of the crowds approach. It was a centrally sort of expertise driven approach, which is how do we bring industry data and reputable sources and create a really gold class or whatever, gold standard methodology and bring it to a category. I tend to believe in that work. I'm drawn to that kind of work. I also see that lots and lots of UGC can be interesting for, I don't know, sneakers or small items or whatever, where you're going to have 30,000 reviews on the Labubu whatever doll or whatever it is. Small, small stuff.
- [27:01]
B
Yeah.
- [27:02]
A
But I sort of. But I think that's an interesting. I think for big stuff, for big decisions where I think US News crushes it. I think that there's a bunch of publishers out there that are addressing these needs. Red Ventures has a bunch. There's. There's some great brands out there. But I think for like big purchases, I don't really want like some mishmash of UGC and Reddit, you know.
- [27:20]
B
Yeah.
- [27:21]
A
John Smith, 1971 says that like GEICO is a great car insurance company man. Like that. I don't know that that data leads to the right answer.
- [27:32]
B
It's so fascinating. It's such an interesting time and I. And obviously that dropped literally yesterday and so by the time this comes out, it'll still be news and our team's talking about it. It's been speculated it was coming. All that. All that. I think the thing that we're. I don't know. I don't. I'm excited because I think there's still such an opportunity. You nailed it. For those things that are more nuanced, require multiple stakeholders, whether that's B2B or a longer term decision like your insurance. And I think there's that opportunity so real. And we still haven't figured it out yet. I sent a snapshot of. It was like a statistic infographic showing of 150,000 queries or something. Reddit and Wikipedia are at the top. Obviously that's getting a lot of buzz. Obviously that's getting a lot of chat. I sent it to the team. Like, how much you want to bet this is going to look very different in one year? Could be wrong if you know what you think. But it kind of underscores what you're saying here. Like, is that really what you want to be hanging your hat on? No offense. Like, there's plenty of information that's factual and helpful, useful. But there's plenty that's kind of. Not too.
- [28:42]
A
Yeah, yeah. I mean, I think the. I have so many things to say about that and we can go down a rabbit hole. I'll go down.
- [28:50]
B
I'll just.
- [28:50]
A
I'll go like an inch deep into the rabbit hole. Which is to say.
- [28:53]
B
Good, good.
- [28:53]
A
I think probably three separate points. The first is I think AI results and LLM appearances is absolutely a metric people need to be optimizing for. I think it's nonsense when people are like oh chat GBT merely gets a billion searches a day compared to Google's 14 and only 20 of them have search intent. I've been seeing that on the Internet. It's like it's a year old. It's going to just yes, Google my, my hot take.
- [29:20]
B
Even though 70 growth rate or something insane of like a very short period. You might want to pay attention.
- [29:26]
A
I think there's. That said, I also understand the pendulum can swing too far, but I think, I think Lloyd traffic and optimizing for it is a trend to pay attention to.
- [29:35]
B
Yeah.
- [29:36]
A
Second, I think that ChatGPT and these others have effectively a stakeholder problem and I've written about this on LinkedIn where they, these LLMs basically ingest the Internet and synthesize it and give ty an answer or an answer to like what to cook in the kitchen, sneakers or whatever to get. And think that the old bargain, the old payment that happened between Google and search engines and publishers was Google's going to scrape your stuff, but you're going to get some traffic back from that scraping machine. And we see that the Google AI mode or the ChatGPT or LLMs or whatever we saw from the Cloudflare data a few months back that like the amount that they're scraping versus the amount of traffic they deliver is orders of magnitude by compared to what Google search used to be. And so I think there's this fundamental like, like exchange problem where the incentive for publishers like, like whoever, like U.S. news or Forbes or whoever it is or Red Ventures, any of these guys, the incentive to go invest in localizing a bunch of industry data or stress testing the methodologies for how to rank the best credit cards, whatever it is to do that, you're just going to suck up that data and present it to the user with no attribution, no pay. I think that's a real stakeholder problem. And I think you see these people saying like AI slop is sort of getting fed back in the machine and it's like eating its own. I see that. And so I had a third point. I totally don't remember what it was.
- [31:12]
B
I know. I think that last one's so strong it's worthy of exploration. So it kind of begets the question of if that model has been inverted, if that model's changing and that bargain is now shifted and maybe doesn't work as it should. Obviously there's talk and speculation of like what is the new structure that's more rebalanced appropriately for the publisher, for the creator, for the writer, et cetera? Do you believe in your mind what is the, the scenario that could, Is there a scenario that could be rebalanced there where the open AIs of the world are like, hey, you snooze, we're going to cut you a deal the following way? I'm not super optimistic that that even remotely makes sense, but I'm just wondering where you see that bargain shifting, perhaps.
- [32:03]
A
I mean, it's a question that has been on my mind for a year and a half. Math, I mean. And I don't think there's a super satisfying answer or a clear resolution to it. But here's some thoughts and. Oh man, I've got a whiteboard, I've got a dry erase marker.
- [32:21]
B
Yes.
- [32:22]
A
Do I do it? Okay, yeah, do it. I'm.
- [32:24]
B
Do it, do it.
- [32:27]
A
So, I mean, I think like if you look at it, there's a couple of options that sort of exist. Right? Like, I think one option is the sort of hey, to scrape. Oh my God, this is a terrible looking blue marker. What in God's name was I thinking?
- [32:45]
B
Sorry. We're working on it. We're always being testing. Let's go. Go with the red. Go with, go with the black.
- [32:52]
A
Yeah, this black one's hardly working. But so like, I think there's the pay to scrape model, which you get from like. Yeah, Claire, I think is trying to do that. And I don't, I'm not super, I'm not super bullish on the, on. On the idea that Cloudflare can sort of be a toll booth between the entire Internet and these LLMs. Maybe it's interesting, but I think where I.
- [33:16]
B
It's one step.
- [33:18]
A
Yeah, I mean where I get stuck on it is like, what's the price of the scrape? What's the price? What's the right price relative to each query? And I think that Google, if you think about Google differently as a search engine and think about it more as a pricing engine relative to we will serve traffic to you and publishers or media companies or whatever can sort of figure out the relative value of that visitor and then prioritize against it. I think that it gets really fuzzy when you have paper scrape, even if you have variable pricing, which I know they've talked about, but it's like if Best Credit Cards is one of the most valuable visitors on the Internet and well, exactly how much is that spray worth? If you're going to mosh it up with 14 other sources I don't know, versus an article that says should I use my Chase rewards card to redeem for such and such purchase? What's the pricing of that? Really complicated question. And I think like, feels good to say that that cloud flare will crush it and just be this auction. I don't know. I'm a little, I'm a little skeptical. I, I look at, there's a really interesting company who I'm kind of buddies with the CEO Noah Greenberg called Stacker, if you know Stacker. And so Stacker, I'm going to make Noah buy me a beer for mentioning him on this podcast.
- [34:39]
B
So there you go.
- [34:40]
A
You're welcome, Noah. So Stacker is this kind of syndication network that helps brands that have, have developed really good original research and syndicates it out to a network of lots of, I think it's like 300 small newspapers like newspapers like the Sacramento Bee or something. And so these small newspapers will, will very, very often pick up this original research from, from these brands. So if like Mint or Intuit or something does a really interesting piece, it gets pushed out to lots of newspapers. They tend to pick it up. What Noah has been writing about on LinkedIn things interesting is that that's like catnip to the LLMs. They love when your brand, when Intuit is picked up by 200 newspapers, they stand up and pay attention and cite Intuit a whole lot more. And I think also the information within that piece of content, the information within those pieces is also getting picked up and cited. So if Intuit mentions like six tools that are really valuable, those tools end up being perceived as more valuable to the LLMs. And so one thing that I think could really be prone to corruption, but could be a model is if you're doing a page like Best Credit Cards and you know that the LLMs are paying attention to the brands mentioned on that page, maybe you are charging a fee to get mentioned. Maybe the publisher finds a way to like charge that fee. But again, it's, that's so prone to corruption because it's like, well, what if the best credit card company doesn't pay to get mentioned on that site? You really get kind of a, a challenge. And I'm trying to think about how do you capture as a media company or publisher the value of not ever getting, but instead just having an influence over what.
- [36:32]
B
Yeah, it sort of reminds me, I recall, I want to say it was Connie Chen of A16Z. I've mentioned her in the past. I don't know, you've probably heard of her before. I'd have to fact check that if I got that right. But her and a number of other colleagues were talking a lot about kind of the explosion of micropayments over the last 15 years in Asia in particular, and how again, not saying this is going to make sense, but I'm wondering if there's maybe a little bit of a shift to that. You obviously saw Meta recently with like, hey, you can do the ad version for the membership and maybe offering more consumer and publisher flexibility of just how that ecosystem works and a little less like either fall into this bucket or good luck kind of thinking. I wonder if there may be something there around micropayments where there's some value distribution, but you're not like going all in on one or the other. It's a really fascinating topic that I think you've kind of touched on together here.
- [37:33]
A
And I think, I think Perplexity is that there's some discussion that perplexity has talked about sort of being maybe the Spotify of the Internet, where Spotify then, or excuse me, Perplexity kind of distributes royalties out the back, akin to a Spotify. I kind of go back to that question though. Same thing like Cloudflare, like, well, what's the value of a scrape? I think that when you get the traffic in and you're able to see how users behavior, what their lifetime value is and whether they convert and all that, that I think that you can tell what the value of that is and you can allocate resources into it appropriately. You look at what Google incentivized, what the Google search system incentivized that I was a part of and plenty of publishers are a part of, is this sort of arms race of updating and improving the pages that are the most valuable. And so I mean I'm going to make up something that I was never a part of. But if it was like Bess. Best travel destinations in Europe. Let's just assume that that's a really valuable query. Best homes. I remember when I worked in real estate tech. Best homes in Santa Monica, California or in Vienna, Virginia. Sure, those are super valuable terms for Zillow and whatever. Real estate.
- [38:48]
B
Yeah.
- [38:49]
A
So there becomes this arms race around people to improve and improve and improve those pages to try to be rent in the number one spot. Whereas a more informational page, because the value of the traffic is lower, doesn't get as much attention and priority. And so like again, in a world where some central resource like Perplexity or Cloudflare is setting the price per scrape that will have a profound effect on how much attention those pieces of content get. And I feel like Google in a way by doing it through visitors, that system became more efficient over time. And I just worry that Cloudflare moderated pricing or a perplexity moderator pricing, does that get more efficient over time or not? I sort of think about that.
- [39:37]
B
Yeah, I think so. And I think we've, we've all. I mean, gosh, we remember the days when it was like wild west beyond belief and there was all kinds of gaming and you could print money if you were kind of falling into gray areas of SEO and not in even semiconductors paid. And I think that's evolving. I think we'll, I think we're going to see something similar here. I like. We're starting to talk about, we're starting to provide real help in terms of this area just candidly of like getting technical knowledge to look at things, to look at competitors across, search across AI and say, hey, here's where you're showing up. What is it going to take to get here? Here's what we suggest investing in. That's a really interesting challenge and I think that's going to be certainly needed and I think counseling people on obviously we've talked going well beyond click. We have to look at exposure. We have to look at those things. So the balls sort of moved a little. We have to kind of monitor that and improve it. I think you're right on. I think it will get better. I noticed, I know there were some folks talking about today even dropping with the news. I know some of the people we're talking about was Andrew Seifert who does a lot on mobile and attribution and the inevitable ad network. He's his voice of like everything is an ad network. The ads are coming to it now in particular with the stripe post stripe announcement. So I don't know what your thoughts are there since we're just going ham on this concept, but I think it's worthy.
- [41:04]
A
We said we weren't going into the rabbit hole and here we are. Yeah, I think that it's an interesting time we're in. I mean, I guess something I've written about on LinkedIn is that OpenAI and ChatGPT have majority share of this LLM market. It's something like 3/4 of market share belongs to OpenAI and to sort of set the rules of the game.
- [41:27]
B
Okay.
- [41:28]
A
Like Gemini is a small percentage. Perplexity is a small percentage. Whatever. Grok is a Small percentage. It's basically ChatGPT's game to call. And I think ChatGPT has not really done a good job about telling the world how it wants outside companies to interface with it. And I'm going to go down this rabbit hole for a second look. Look at the biggest marketplaces on the Internet, marketplaces that govern the biggest transactions that any that that in the world. Look at real estate marketplaces like Zillow or Amazon and commerce marketplaces. Or look, any of these markets.
- [42:09]
B
Ebay was kind of the foundation of the Internet in some ways. With Amazon, I mean.
- [42:13]
A
Yeah, and Zillow, Zillow Red, and each of these. Or credit card marketplaces, there's many. And the point is that all of these marketplaces are optimized for the old world of search. And there's always been three or four or five top marketplaces and then a long tail below it. And they're all kind of grappling and fighting to be in first place. I think what we're seeing, the early stuff that I'm seeing is there's starting to be these MCP model, context protocol or other sort of enterprise partnerships that happen between some of the biggest marketplaces and chatgpt Open Air. And I think of what I had heard of in the last couple weeks. My buddy Daniel Lysot is the CMO of LegalZoom. And LegalZoom did a partnership and connect it in with OpenAI. So you'll see kind of legal zoom results in parts of OpenAI. Zillow is has announced that they're doing some sort of partnership. And so you'll probably be able to Talk to your ChatGPT and get listings is presumably what's going to happen. And so companies are starting to integrate kind of piecemeal. And it tends to be like the dominant marketplaces in how efficient. Like, here's what I worry about is why did Google beat AOL and Yahoo in the early days of search? Well, AOL and Yahoo were doing exactly what OpenAI and ChatGPT are doing now. They would sign a couple of enterprise deals and say, oh, this one company will handle all our travel traffic. This other guy will handle all of this traffic. And those were inefficient systems. Google did something different.
- [43:48]
B
Google said, auctioned.
- [43:50]
A
There's an auction for our ads. Anyone can play. The more you pay, the more traffic you'll get. And it was a competition that made better and better and better ads, and it accrued value to the system and to Google and to the user through the competition. In the organic serps, like the arms race we talked about earlier. There's a competition for the traffic and the value accrued to the user who gets incredible up to date valuable information. Right. Google harnessed the power of the open Internet through search. I worry that ChatGPT and OpenAI have this really real genuine stakeholder problem. They're not really harnessing, they're not creating an open Internet, they're creating a gated community like AOL and Yahoo where if you can go sign some, some amazing enterprise deal because you're the number one marketplace like Zillow, great. What if you're Homestucker? What if you're, you know. Right.
- [44:42]
B
Yeah.
- [44:43]
A
And last thing I'll say is what I see is a more open OpenAI would be one that allows ads and allows multiple people to compete for the organic result and paid results similar to, similar to Google, even if the interface looks different. And so I think with Fiji Simo coming in to, to lead ChatGPT and you're starting to hear in this last week, kind of mid September at least 2025, she's poaching folks from like the meta ad sales teams and I think you're going to start seeing an ad unit built out. ChatGPT Pulse is coming out. It's like a feed of stuff and that seems to be. I saw someone say that's a perfect substrate for ads. So I think we're going to see more. I think we have to see. I think we're going to see more places where the open Internet will be able to interface and participate. It has not been the case in 2025, but I think it will be the case in 2020 26. That's great.
- [45:42]
B
I love that Amro. It's. It reminds me of. I spent a lot of time nerdied out on user experience best practices for a variety of reasons in my performance marketing world and one of the concepts I'm going to butcher the user experience naming of it. But it's like how do you create familiar frameworks and mental models for people that they can fall into. You don't want to necessarily reinvent the wheel on everything. And I almost feel like it's similar to that where the unit economics of business, of network effects of growth that's worked and performed well is really kind of important part of the recipe for OpenAI and for the open Internet to actually exist and perform well and be valuable for all the stakeholders, not just one or two. And so I love where you're going with transparency. More, more open quote Unquote, actually have an ad or a ability to say, hey, I'm publishing the rules of the game for you, similar to how Matt Cutts used to do back in the day with Google and say if you do these things, you're more likely to do XYZ and get value and participate in the marketplace. So what an amazing. This is just. Yeah, this has been really interesting.
- [46:54]
A
Yeah, no, I appreciate, I appreciate we got a chance to sort of riff on some of the biggest. I mean, I think it's the biggest show in town right now.
- [47:01]
B
Yeah, we skipped over the. I mean, look, if people want to message you and talk Amro and say, how did you fix xyz? You have so many learnings. But I think that was, that was fun and juicy to dive into for people and I think, I think it'll be appreciated. We jumped over a lot of questions, so we may have to do a part two in the future with Amro if he's open to it, maybe down the road. But do you want to go down the final lap of just getting to know you a bit and asking a few personal questions? Do you have some time?
- [47:28]
A
Yeah, let's do it. And I'll just say, you know, it's awesome to. Awesome to be on here. You have a great podcast and I was so to get invited to be part of it.
- [47:36]
B
Thanks, man. Thank you.
- [47:37]
A
Sure. I mean, I'll say, I guess the maybe a thing that I'll. I'll impart is that I had a Talk with the U.S. news & World Report interns, me and a few of the other division GMs this summer. And I think one kind of advice that I gave to them, but I think pertains to all of us does, regardless of what level you are in your career, what stage you are in your career is it's so important to be a lifelong learner and keep learning and keep sort of being adaptive and resilient and just going, going where your learnings take you. And I think especially so many, my peer group of like mid level or senior executives feel a lot of fear in this year of 2025 of what's next, how's robots are going to take our germs and whatever. There's always some like pessimistic bear case to innovation and creative destruction and stuff. And I think just the same way I sort of said it to the interns, I would also say to all of us as mid level, senior whatever execs is stay open, stay learning and sort of go where the learning takes you and sort of work with the system that's around you. And I think that's good advice. Stay open to where I think you go to a very different phase of how the Internet works and monetizes and just sort of staying empirical and stay learning is the way.
- [49:02]
B
Stay empirical, stay learning. Very, very aligned with the pods message and my reason for being. And I'm just grateful for you to chat and learn and dive into this stuff. Speaking of which, we've gone over because we dropped so much knowledge, but I have a great final get to know Amro question. Just in terms of family adventure, what's your go to recommendation for the audience that you suggest for family adventure? Maybe like a vacation or a getaway or a trick. Anything fun you want to share with the audience?
- [49:38]
A
Sure. My wife recently, we recently celebrated my wife's birthday and it was kind of a milestone birthday. I won't say the number, but it was a milestone birthday birthday and we.
- [49:47]
B
Happy birthday to her.
- [49:48]
A
Thank you. Yeah, I'll pass that. A long time. And we went horseback riding. And it's not, if you live in the Washington D.C. area, it's not like there's a million places to, to go. So we found in Charlottesville, Virginia, which is where we both went to school at the University of Virginia. We went down Charlottesville, we found this like horse ranch that's right outside of town and we signed up for horseback riding lessons with us and our six year old daughter. And I was assuming that like by lessons they would teach us how to ride a horse. We literally got there and they got each of us on a horse. They gave me the largest horse I've ever seen in my life. It was like the Chevy Suburban of horses. It was enormous. Okay. Its head was like the size of my torso. And they were just like, all right, here you go. They got me up on a ladder. They got me on it. I was like, what do I, what do I say to the horse? Like, what are you. And they're like, yeah, you just say whoa if you wanted to stop. I was like, that's it. That's, that's all the guidance I'm getting. And so we immediately started taking the horses into the mountains and did a trail ride. And I was like, holy cow. I'm in the middle of like a mountain range on horse within like 10 minutes of getting on it. But the short recommendation, it's a great in Sharemore, but horseback riding, totally recommend it. Awesome time seeing my 6 year old ride a horse by herself. Yeah, it's awesome.
- [51:08]
B
Oh man, that's amazing. And definitely on brand to some of the stuff I've been sharing lately and grew up with with that world. I was lucky to. It was. It was kind of funny because my parents were really devoted to that world. So to. To get to expose. It's so cool that you experienced that. We're actually taking our daughter for her fifth birthday in a couple of weeks to do similar. She. I think she did it a couple of years ago. So. Yeah, it's cool to pick up on the communication of the horse and rider and take so bit to get there. But it's cool that you got to experience that.
- [51:44]
A
My horse's name was Sully. I believe that. I think we developed a bond in a couple hours.
- [51:50]
B
There you go, man. We'll have to talk more about that in future conversations because there's a lot there. Emeril, thank you so much for coming on. You were super generous with your time. For those that want to learn more about the journey with you and building out this whole amazing group with US news and what's what you're up to, where can people. What's the best place to find you?
- [52:10]
A
Catch me on LinkedIn. That's. That's where we all are, isn't it? So, yeah. But I'm so, so glad I could.
- [52:16]
B
I could come to thank you for taking the time. Ambrose, have an awesome rest of your day. Thanks, everybody. I.