← Always Be Testing
Always Be Testing
#119 Why 70% of Affiliate Programs Fail Before They Launch — and How AI Can Fix That | Kyle White
00:32:39
Loading summary
Transcript68 lines
- [00:00]
A
Foreign.
- [00:07]
B
Hello. Welcome to another episode of the Always Be Testing podcast. I'm your host, Ty degrange, and I'm really excited to talk to Kyle White today. Kyle, how you doing, man?
- [00:16]
A
Doing awesome. Great to be here.
- [00:18]
B
Good to see you. Beautiful. Calling in from Utah today.
- [00:22]
A
Yep. Got some snow on the mountains finally, so we're stoked.
- [00:25]
B
It's a good area. It's a good area. I love it. I'm gonna have to. We have to hit the slopes at some point.
- [00:30]
A
Oh, anytime. Hit me up when you're in town. I'd love to have an excuse to get up there.
- [00:35]
B
I'm going through withdrawals being in Texas. I love it down here, but it's a little. We're going to try to do our first family trip next year, so fingers crossed. I'll let you know how we survive that or if we actually get to that.
- [00:49]
A
There's nothing better than little kids shredding down the hill. It's the cutest thing ever. So have fun.
- [00:54]
B
Yeah, they bounce around like a pinball. They don't really care. It's just good to go. Love it. You guys are in for a treat today. The audience out there. Kyle is the CEO and founder of Refinery, Leading tech solution that helps people use AI better manage affiliate programs. Something near and dear to my heart and everyone in the audience, I believe, or most of you. And it kind of combines two things that are happening at a very fast and exciting rate right now, AI and affiliate. What was the inspiration, Kyle? Just to. Just to jump into it for the build and for creation of Refinery, because I think it's an exciting and novel thing to talk about for folks.
- [01:38]
A
Yeah. I had spent a good portion of my career in affiliate management and moved over to the product side and had left for a few years and got roped back into some consulting with some big brands. And I was kind of like, why, why are things still this way? Why aren't we using the latest technology? And I kind of searched for some, some tools to handle all this data. I'm pretty data driven manager myself. That's my kind of foundation for starting a program. And I just figured, well, there has to be a better way. There's some great tools on the networks, but there's some missing things too. And we just walked through a situation where there were some obvious low hanging fruits and some nuggets that could be found easily without too much work. And that took a lot of work. And it's like, why isn't this automated? So let's, let's start Building a solution and for the past year we've been getting a lot of feedback from a lot of managers and tweaking things and iterating to just unlock the power of AI and saving a lot of time and uncovering opportunities for affiliate managers that's hiding within their data.
- [02:42]
B
I love it. Yeah, I know you have a really interesting mix.
- [02:46]
A
Right.
- [02:46]
B
We talked a little bit about product led growth, what that means as opposed to like a sales driven SaaS solution, something that we love and love, working in love, helping brands unlock and figure out. You've got this performance marketing background and affiliate marketing background like myself and then you've got this new created software that's super exciting. So I think there's a lot of you come at it from a really good angle for people to understand and for people to learn from you. On let's wind back the clock. Let's, let's go back to the old days of overstock. Some of the learnings that you've had there. You've built out a lot of the tooling and software with impact. Obviously one of the leading affiliate networks. And then obviously now with Refinery, was there a particular aha moment or maybe some aha moments that you might want to share with the audience where there was kind of this convergence of like product led growth or maybe a product led growth learning that you find might be worth sharing?
- [03:49]
A
Yeah, I think the, the crux in all of this was an experience. While I was at Uber, one of the programs I was managing was Uber Ambassador program. We were given an, an order to scale and go across the world and find this subset of affiliates that really didn't understand affiliates. They were like influencers, content creators. This is like 20 what, 16, 2017, like 2018. People are just kind of dabbling in affiliate marketing and influencers. And so we called it the Ambassador program. A little easier to understand than affiliate, less scary than affiliate. And we were losing. We were inviting all these great partners across the world in France and in Asia and we were noticing that after they were clicking on our emails or talking with us and trying to join, only about 7% were joining. And we're inviting hundreds of people that join this program and it's simply a different audience and understanding that situation of they're not used to five pages and a lot of terminology that they don't understand with a lot of questions. So we built a white label experience for this audience and we stripped down all the unnecessary parts, make it much more focused, change language and got them to a promo code at the Time to promote Uber as soon as possible. We went from like a 7% join rate to like an 80% join rate. And looking back on it, there was a lot of PLG principles that applied to the onboarding experience to improve the time to value. The time to value for these affiliates was to grab a code, share with their audience and get paid. And there was a lot of stuff on the onboarding experience and activation that was just sucking out the energy and time from that. So anything that we could do to improve that helped everybody, it helped the affiliate get paid more, it helped our goals get new customers. And obviously the audience got a little discount too at the same time. So it was a win, win for everybody. And it was a changing moment for me to focus more on product.
- [05:44]
B
That's super interesting. If you wouldn't mind. I'm curious about the kind of the removal of the friction giving them the code. Were there other things that you, if you recall that were helpful in sort of getting an audience that's not really clued in and geared towards affiliate per se and thinking about how to promote Uber in this case or promote a particular product in this case?
- [06:11]
A
Yeah, I mean, we broke down the whole situation, even from the introductory email, right. And making sure that every single email had some relevancy to that, to that, to that affiliate. So we would say, hey, it looks like you have an audience. Like, I saw this tweet or I saw this thing. We saw something that we could tie back to how they could promote Uber.
- [06:34]
B
Right.
- [06:35]
A
And that really broke the ice a lot easier and made it much more relevant. And starting even from breaking down the whole process and the step from that introductory email of saying, hey, we're Uber. We saw you have this, this is relevant. You can get, you can monetize this. Would love to have you join the program and jump on a call or whatever it is and just be more relevant to them. And then, then, okay, they want to move forward. How do we make sure that process is as easy and seamless as possible?
- [07:03]
B
Yeah, and just out of curiosity, we talk about it being. We're talking about product LED growth and product and Uber is its own beast in a way. Was there, how much did the relationship play or not play in this case? With this piece, I would imagine there's a number of folks that are going to join without necessarily having a conversation like this. But I'm just curious, like, if it, if it, did it play any role there and, and then when, when did it play a role or not either, like via phone call or via email. Did you find that relationship building was part of the equation?
- [07:38]
A
Yeah, it, it definitely depended on the size of the affiliates or the potential affiliate. Right. I had calls with people who were big at the time, like Jeffree Star and like a French influencer. I remember specifically that was a big deal. Those are definitely. They have a team, they have. It's a little more complicated. And then when we start to track their progress, we could see volume and definitely started to reach out more personalized, more personally to those who are increasing in volume. So yeah, it was an ambassador program. It was kind of. They're easy to engage and easy to disengage in a way. However, there were long standing, almost affiliates that were created from that program and we're always looking for those diamonds in the rough as well. If they can have great conversion rates
- [08:25]
B
and performance, that's great. When you think about kind of defining terms and product led partner marketing or product led growth as it relates to how do we run and manage and think about partner marketing for it, I guess I'm curious maybe for the audience to learn a little bit more about your definition of product led growth and product led partner marketing. What does that look like to you?
- [08:47]
A
Yeah, PLG is eventually all about retention via the product itself. And retention in this sense is defined from my classes with Elena and all these other great folks. Is a formula. Good company, great company. Yeah. Is an activation formula of activation, engagement and resurrection. And there are different flywheels and moments within all of those to improve customer value via the product. If it's done right, it kind of sells itself. And what does that look like in a affiliate program and a brands program? You almost have to think about like what tools, what processes and what experiences can you provide continual value to your customers, which in this instance might be the partners or affiliates who influence your customers and your KPIs. There is this whole process and experience you can construct to have continual win win scenarios. Win win meaning the partner sees that promoting your brand is beneficial to their business. And once you get that value, you can continue in this loop of retention and creating relationships with partners and getting higher quality customers at scale. What does that look like? That's like working backwards from your KPIs and seeing which third parties partners in or out of the program can influence those KPIs in a meaningful way and giving those partners the best experience possible. That means you probably have to do your homework, your research. You can use AI and enabling them to be successful, understanding what's going on with their Data, whether that's helping them monetarily, having them a good experience that they can easily tap into their audience like giving them better ad copy suggestions. At Overstock, I provided a monthly. I didn't know it was plg. I mean it's kind of PLG related, but I provided a monthly report card that included their customer LTV score that they were sending. And I could say if you can improve it, well, we could potentially raise commissions for you. It's this whole kind of value loop or product or service that's valuable to a specific audience. The partner has access to a portion of that ideal audience. We have to help them unlock it and provide value through any means necessary.
- [10:55]
B
Yeah, I love that I, that hits home so, so well for me. Kind of like LTV scoring and giving, giving some type of metric or scoring to a partner to give them a little bit more insight. Not necessarily something that's going to be uber proprietary or something you can't share, no pun intended, but like something that's a value to them that go, oh wow, I'm scoring in the top X percentile or if I get to this level I'm going to earn this bonus. And I love that you're thinking about it in a way that's just empathetic and thoughtful to the person that's, that's partnering with you and building, building the business and teaming up with you to grow.
- [11:31]
A
Yep, yep, that's part of it. That's why I love affiliate marketing because it's, it's, it's, it's a partnership and if you create a nice virtuous loop, everyone wins. And that's when I think real success happens.
- [11:43]
B
I love it. Agree. Obviously you had a great experience with Impact, being able to kind of build out their data, science backed matchmaking tools and a lot of other things going on with them. When you think we think of it a lot like matchmaking, I think matchmaking has been used a lot as a analogy and a really pure example of what it can be and it needs to be done right. What are some of the signals you look for when you think about whether it's SMBs or mid market or larger brands around, how do you think about the learnings from that? And also what signals would you want to see to make sure the matchmaking is where you would want it to be based on your experience?
- [12:25]
A
Yeah, I mean we all don't have data scientists and hopefully with AI it gets a little bit better. I'll tell you what not to do. At first we just followed the money, I was like, oh well, these affiliates are doing very good for these similar types of brands. And we were just following the money of top performers, right. That did not go well. The match rate, the first few tests when we were testing all those out were not great. And then we started to weight relevancy heavier than payout and the model changed quite a bit and we saw much better engagement and match rate. And when I talk about relevancy, what does that mean? That's things like obviously vertical, that's audience types, that's size. So if you're a small brand, that's don't go after big publishers. Look for other mid tier, smaller micro influencers that are hungry that will be willing to partner with you. Focus on contextual partners. You might have to know your audience and do your homework and put yourself in their shoes, walk through their experience, see what other people or brands or channel partners, whatever things they might encounter that would, what kind of searches they do. Do some AI searches, reverse engineer those. Just really get in their shoes and be like, where is a good affiliate partnership here? And just focus on relevancy is definitely the most important thing that I found and the most success.
- [13:58]
B
Yeah, I love that. It's, it's, it's definitely thinking in similar terms. And you know, we're seeing, you want to look at a number of signals and I feel like we could devoted whole episode to this, you know, this discussion. But building that trust and having that relevancy and that audience overlap and thinking about ic, icps. I love, I love the direction you're going there and I agree with it wholeheartedly. We, we kind of both sit at this intersection. It's really interesting, right? You've got product, you've got experimentation, you've got affiliate. What are a couple experiments that have just been cool, interesting, novel that you've seen or you can share a little bit that might have taught you some things that were surprising or interesting to share.
- [14:44]
A
Yeah, if I were to just off the bat do testing, just an easy one to do is tiered commission performance bonuses. So if you understand your affiliates and you understand your audiences, understanding where is a good inflection point and an exciting moment for an affiliate to be at and use the data obviously as a baseline experiment where that inflection point might be and getting qualitative feedback from the partners that would fit in those commission tiers. So once you start testing, you set up these kind of schedules and hopefully minimize variance, as little variance as possible. But you give them an opportunity to maximize their value and to prove how powerful they are. And that can be things like, hey, if you send me five new customers, you're on an average of two and maybe it's a big brand with B2B brand with it's hard to get customers. Right. If you can get me to five, I'll get you a $10,000 bonus. Or it could be more commission the more they do based on their performance. And so this is also relates back to product led in a way because it creates that value. It's a care at the end of the stick kind of motivation that helps all parties. It helps stretch them to be their best selves. And it's a really exciting and fun easy test to do a B testing just to see what happens. And you can really get some great optimized value out of your existing affiliates. Out of that.
- [16:10]
B
Yeah. It's such that direction and that philosophy is I think surprisingly not utilized enough as you'd think. And for me also surprisingly powerful. I've mentioned it. I was chatting with a group essentially writing a post about some of those hidden gems and that was that was one of them. When you think about isolating and making sure it's not to get too in the weeds of experimentation, but are there tips or tricks to remind people of about how do you think about making it a proper test versus looking at it just as a straight promotional effort that either drove X or Y, Is there a way you might guide people on how to test it effectively versus
- [16:59]
A
yeah, you definitely have to set your baselines and understand what the historical data says. Here's the averages, here's what success means. So setting that beforehand, obviously looking at any seasonality, making sure that you're not setting them up for failure if it's a bad time of year. And again looking at those averages and baselines and saying, well maybe it is a bad time of year and we want to improve that bad time of year. Well, what is the baseline? What do we need to reach to show what does success look like? And then obviously things change. When you look at historically, maybe their sites have changed, maybe their ways of promotion have changed. Maybe their look at their tools that they have in their toolbox to help you and understanding those variables and where you benchmark it against maybe there's a similar similar affiliate making sure that there's not a commission increase with them at the same time that might skew the data. So just little simple things like that. Just making sure that priorities are set and baselines are set and there's a clear success metric and you're not looking at weird seasonality and weird offers. Comparing offers and making sure many things are as even as possible.
- [18:06]
B
Yeah, no, for sure. Be surprised how often that people can fall into traps of comparing what is not apples to apples and trying to isolate that as much as you can is super important. So I love that reminder for people to try to isolate for some of those outliers and those bigger variables. When you look back at 2025, which scary. Here we are almost basically in March of 2026. Time's flying by as we expected, sort
- [18:36]
A
of oh my God.
- [18:39]
B
What's your big learning from 2025?
- [18:42]
A
Man, that was a wild year for me personally. We had a baby, started this company.
- [18:46]
B
Congratulations.
- [18:48]
A
I think my biggest learning was fail fast and iterate faster. So if you have a hunch on something, sure, do it and fail at it fast but make sure you learn from it and grow from it faster or else you're going to be just stuck in the mud for a few more months and that's going to delay everything else. So that's my biggest learning. It's been amazing to talk with so many affiliate managers and I came in with my biases of course but it's been humbling and great to learn and to iterate and see the success from basically a group think one day been exciting.
- [19:27]
B
That's awesome man. It's really cool what you've achieved and you've, you're building. Speaking of the building, I. I want to give you a chance to kind of share a little bit of sneak peek of what, what's, what's the, the secret sauce without sharing the secret sauce with Refinery around, what is, what are you finding is like the, the thing that's like maybe the Ahama method. Those that are using the tool and kind of what is it sort of replacing or helping because I think it's easy to think okay with AI like what's it actually going to do? I feel like there's like combination of hype and concern and how do you kind of reconcile it and just share a little bit of like what's the thing that makes you guys special? For those that are not as aware of what you're building at referrals, Refinery,
- [20:21]
A
there's two things and it really does come down to what our name is Refinery. Right? Like we want less is more. The too long didn't read executive level version on finding new affiliate partners. We don't have large lists. We have a refined. We have five to 10 people a week that will Be the best that will insert into your top 20. And we use AI to reverse engineer search and find a lot of great data that would be find that relevant audience like we're talking about. And the second thing is the AI insights and I think the scary part is like oh it's AI. Was it going to tell me wrong? But like it really once we've said the aha moment that you're talking about is someone will say wow, this insight that it provided for me would have taken me multiple spreadsheets, vlookups forecasting hours of time and I barely have time every single week to get my reports ready for my superiors. Right. And so when they see oh wow, that's a thousand dollars, $2,000 nugget or that's going to improve my without doing much work that's like the moment of like wow, this thing is powerful and if I can apply it and test it and we have testing tracking and everything and we can see things up and to the right. That just saved me hours of time and I have much more confidence with my conversations with my affiliates.
- [21:39]
B
Love it. Yeah, the always be testing philosophy coming through is just like the pod we talked a little bit about last year. We talked a little bit about what's coming for this year and predictions. What is your do you have any hot takes or estimates that you want to share with the group?
- [21:58]
A
Yeah, I have probably two if that's okay. Yeah, one is a little bit more doom and gloom but it's more like a cautionary tale. Right. I love what Hanan is doing at Impact and if we don't get a seat at the table and work it correctly AI search may bypass affiliate and either become the affiliate taking the commission or allows for paid visibility. Much like SEM. It could be a whole channel cut out affiliate and take a paycheck for itself. I don't think there's a chance that that could happen and I think that we need to keep pressing and be on top of things for our channel because people are greedy and it's understandable businesses want to make money for themselves and but they need the content from us and they need the content from us. So it's I really love like I mentioned what Hanan is doing and bringing that visibility to the content creators and making sure they're getting paid and I think we need to continue that and partner with these big AI search engines and make sure that we are getting adding value and we're getting credit as well and a little more doom and gloom. But it's like, I think there's a chance, I think we should just stay on top of it and just kind of see that conversation and the industry leaders need to do that.
- [23:18]
B
Yeah. I don't disagree. And. Is it integration with. Can, can they be collaborated with? Is a bold question. If so, how. I'm trying to think of other ways to prevent that. I think, I feel like there's a lot of things that are happening right now which candidly feel a little bit out of our zone of control. Yeah, but. And I think the data we're seeing is like there's a, there's certainly a lot of value and a lot of attention on affiliate as a result of this. There's a lot of players that are legitimately improving rankings and I think there's a historical hypothesis in reality that, well, the affiliates are the innovators so they will figure that out. It definitely doesn't mean they will this time. Yeah, I don't know if, I don't, I don't know how much we're not going to solve it today, but I think no. Is there, is there another thought you have around how we elevate our position and help advocate for a seat at the table?
- [24:20]
A
Yeah, I think it's maybe creating an affiliate relationship with these big companies when we pass back, hey, you can pull the content from affiliates and they're going to get 5% and you're going to get 1% or a half of a percent. Almost like a network in a sense of like, hey, let's all make money in this together. I think that provides a win win relationship and you can continue to find great quality content. You care about quality and obviously the affiliates provide the best. So let's find a good way to do that. And everyone, I think that's, I don't know, I'm just, just a little guy trying to figure it out. But yeah, that's just my thoughts.
- [24:54]
B
Well, I appreciate it. It helps spur the conversation and think thoughtfully about it. And I think the incentive alignment and profit sharing is the direction I'm hearing you say. And there's a lot, heck of a lot more that goes into it. And I, it's hard. It's, it's. There's only so much I can speculate on it myself. But it's interesting.
- [25:16]
A
That's my affiliate hot take. My AI hot take is that with the advent of openclaw and AI bots like, you know, agents, quantity of shiny objects will increase. However, quality will be much harder to find. There are a lot of cool things it's like a bell curve, right. There's a lot of good things in the middle, but on the end of the bell curve in these verticals, like affiliate, it'd be a lot harder to find true value. So as long as we might be lost for a minute and that's okay. But if we keep alert and track and progress, then I think the things on the end of the curve that will really provide value will, will, will stick around.
- [25:53]
B
Yeah, yeah, I think there's something to that. I think that when a new technology is developed, there's, there's probably a number of historical examples where there's this explosion of either activity or investment or hype or content or noise, whatever you want to call it. And I think, tend to think over time things would work themselves out and become more quality regulated signal out of that noise.
- [26:23]
A
Yeah, it's a fun time. It's a fun time. Yeah, it's really fun.
- [26:27]
B
Yeah. Yeah. And you're exciting because you guys are, you're seated right there in the middle of it. You're, you're technology, you're surrounded by technologists and you come from a great background. So it's always fun to talk to you about these things. Something reminded me of that Dan Hockmeyer is also part of, part of the. Our Friends of the Reforged Community wrote an article, I think it came out yesterday around. I think his focus has been marketplaces, if I'm not mistaken, and growth for them specifically. And talked about like the example of like a grubhub not being displaced by an agent where there's a lot of over the last two weeks, especially semi hysteria in market reaction around agents and their impact on everything from credit card companies to grubhub marketplaces. And his opinion was that it takes a hell of a lot more than pricing to get customers excited about a service or into a marketplace and retaining something. We've been beaten into our heads in our learnings and our work. So I kind of tend to agree with that direction that a SaaS company is not just the pricing and the features you get. A marketplace on the consumer side is not just the pricing and the fast food that gets dropped at your door. Whatever burrito that you're ordering do you have to trust. You have to have distribution. You have to be able to contact someone when you, when something doesn't go well. So I think there's an interesting, my, there's my hope, my hope and optimism coming out, I guess. And in quoting that.
- [28:05]
A
Yeah, I think so too. I think the More complex it gets, the harder it is to replicate. There might be little things that could do one little job to be done but like for that whole, like you could the whole suite of things that's like almost impossible for someone just to whip up AI agent that could do all of it. Maybe. Cross my fingers maybe one day. Yeah, maybe one day.
- [28:23]
B
But you know, you see in my newsletter and in some of these conversations it's fun to sometimes talk about like partner types and the types of affiliates that are emerging or on the decline or on the rise or what's the mix look like and we're obviously looking at it every day for us certainly. Are there certain partner types in the affiliate space that you are maybe more excited about maybe seeing rise or improve or maybe some surprises in there for people?
- [28:50]
A
Yeah, I mean maybe it's not too much of a side but pay per performance TV and now audio with audio hook. Like talk about full funnel, right? This is a great targeting. Any affiliates I've noticed that are being successful for all the users on Refinery. They have some targeting factor when it comes to top performers and quality affiliates. If your brand qualifies. I personally seen a lot of great, great things coming from the paper. Performance, tv, audio, those kind of partners and then along the same line technology solutions like payment providers or post purchase targeting. They've kind of had a little bit of a rocky background sometimes, but really understanding that contextual experience in the user is much more valuable than just kind of a last click situation, which I'm glad to see the whole industry is like no more last click, let's move on. And those are two types of partners that keep showing up in that conversation that they're more full funnel and are only going to provide quality.
- [29:50]
B
Yeah, let's keep waving the flag on moving beyond last click. I'll keep saying it. You'll keep saying it. Maybe it'll become a reality at some point. Needs to happen. That's, that's amazing. I love it. Kyle, this has been awesome. Just coming down the home stretch with a couple personal questions to get to know you. What's a recent purchase that you just can't live without?
- [30:10]
A
My Oofos slides, they are like I just walk around my house and them I love them. My feet never hurt. They are, they're like, they're for like marathon recovery I think is when they started but they're like actually have good arch support too. So that's all just gushy. Like they are like okay, I can
- [30:28]
B
feel the help your feet like not check that Out.
- [30:33]
A
Yeah, they help your feet recover. It feels like you're kind of walking on gushy clouds, but they also have little arch supports and I don't know, man. Like, it's just a little thing that you're like. I'm walking all around chasing kids around the house. Okay, that makes my life a little better.
- [30:47]
B
Dude, it's from one dad to another. We'll drop the link in the in the show notes and get your Kyle affiliate code at checkout. That's amazing, man. Besides cruising around on those chasing kids, what's something people might not know about you?
- [31:05]
A
I speak Russian.
- [31:07]
B
What?
- [31:07]
A
I do some film photography as a hobby. So I will develop my own black and whites at home sometime. And big Simpsons fan. If you're running to me anything from seasons like 5 to 8, I will quote it back at you. And then I was briefly on an episode of South park park, not as a cartoon, but me in real life. Me and my wife were in a stock video and they grabbed it for a commercial for some people on a holiday dinner for Integrity Farms commercial. So I've been in a South park episode that was really fun.
- [31:37]
B
Those are some really epic fun facts. Impressive.
- [31:42]
A
How did you.
- [31:43]
B
Russian.
- [31:44]
A
Russian. I did a missionary service for two years in Lithuania, Latvia and Estonia. And I live mostly on the border of Russia and they speak like 90% Russian there. So. Yeah, helping people out and teaching and it was a great experience.
- [31:57]
B
Good for you, man. That's fantastic. How long were you there?
- [32:00]
A
Two years. Almost two years. Yeah. I miss Smetana and Pelmeni and all the time. And I still use some. Yeah. Some spices from the area with my cooking at home. So, yeah, I love smart speaking Russian and love the Baltics.
- [32:16]
B
That's very cool, man. It's always good to chat with you. What a great chat and good conversation, good learnings and we'll talk to you soon.
- [32:23]
A
Thanks, man. Thank you. Have a good one.
- [32:26]
B
Thanks, everybody. Bye.