← The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
#842: Braze Chief Product Officer Kevin Wang on how AI has forever changed product development
00:25:36
Loading summary
Transcript38 lines
- [00:02]
A
And Doug, there's nowhere I wouldn't go to help someone customize and save on car insurance with Liberty Mutual. Even if it means sitting front row at a comedy show.
- [00:12]
B
Hey everyone, check out this guy and his bird. What is this, your first date?
- [00:15]
A
Oh, no. We help people customize and save on car insurance with Liberty Mutual together. We're married. Me to a human, him to a bird.
- [00:22]
C
Yeah, the bird looks out of your league.
- [00:24]
A
Anyways, get a quote@libertymutual.com or with your local agent.
- [00:27]
C
Liberty Liberty Liberty Liberty.
- [00:31]
D
At Strayer University, we help students like you go from Is it possible? To Anything is possible by offering access to up to 10 no cost gen Ed courses so you can reach your goals affordably and fast. Visit Strayer. Edu to learn more. No cost Gen Ed is provided by Strayer University affiliate sophia. Eligibility rules apply. Connect with us for details. Strayer University is certified to operate in Virginia by Chev and has many campuses, including at 212150 in Arlington, Virginia.
- [01:03]
B
With AI promising to accelerate everything, it is the most important role for a leader now to be the one who knows when to hit the brakes. Agility requires a balance between the rapid adoption of new technologies and the strategic wisdom to know which ones will actually deliver customer value. It's less about moving fast and more about moving in the right direction. Today we're going to talk about how AI has permanently changed product development. But while it enables teams to tackle more problems and faster than ever, the real challenge has shifted to keeping pace with rising customer expectations. This new landscape requires a shift in product leadership from simply adopting trends to discerning real value and using human judgment and creativity to decide where AI can make the biggest impact and where it's just noise.
- [01:52]
E
Welcome to season eight of the Agile Brand Podcast. This season we're going all in on Expert Mode, MarTech, AI and Customer Experience, talking with the people and platforms behind the brands you know and love. Again, I'm your host Greg Kilstrom and I help Fortune 1000 companies make sense of martech, AI and marketing ops. Hit, subscribe or follow to make sure you always get the latest episodes and leave us a rating so others can find us as well. And make sure you check out our sponsor, TechSystems, an industry leader in full stack technology services, talent services and real world adoption. For more information, go to techsystems.com now let's dive in
- [02:30]
B
to help me discuss this topic. I'd like to welcome Kevin Wang, Chief Product Officer at Braze. Kevin, welcome to the show.
- [02:37]
C
How's it going? So good to join you, Greg.
- [02:39]
B
Yeah, really looking forward to this conversation. Love it. And yeah, let's. Before we dive in though, why don't you give a little background on yourself and your role at Braze?
- [02:47]
C
Yeah, so I'm the Chief Product Officer at Braze, so running product management, product design, technical writing, some operations functions, user research, partnering really closely with our engineering team to design our roadmap and then actually build and launch the products that we're bringing bring to market. Been at the company for a little bit over 14 years now, since very, very early startup days, which is a whole different journey. Yeah, but you know, you feel very, you know, everything's very, very new with AI and all the change that's going on in the market right now.
- [03:18]
B
Yeah, I love it. Well, yeah, and we're going to get into quite a journey of our own here in a second, but wanted to start kind of at the high level and talk about this strategic shift that I touched on briefly in the intro and just how we redefine value in the AI era that we're currently in. As a product leader. How does a product leader effectively filter the AI hype from the real genuine long term customer impact?
- [03:51]
C
Yeah, I think that that is one of the key questions because if you're spending a lot of time on X or for better or worse, I don't know if it's better or worse, spending a lot of time on LinkedIn, there's obviously a lot of AI hype that's, that's going around and also at the same time the technology is changing really quickly. And I think one of the interesting things is that very fast technology change that's really delivering impact and also somewhat empty hype can sound the same until you actually try things out. And so a lot of what I think really matters right now for AI is getting in there and really testing out and trying these tools yourself. And also, and importantly not just trying them and testing them in sort of a surface level way. Like what I think is really interesting about AI is that what would have been sort of a hello world style first program in the past with AI and the newer models can be very, very fully featured. But a lot of people I think are bringing the same level of dip your toe in experimentation with AI or that same level of light experimentation, but they're bringing it to AI and so they end up with a little bit more in terms of what they're trying to do. Maybe they're trying to build like a little schedule builder or a little tool or a little widget or something like that, but they don't end up actually trying to apply AI to real problems or try to apply AI inside of a real workload. Like I've got a whole bunch of teams that need to do a bunch of work. And that to me is the way that you actually figure out where's the impact going to be, whether it's to customers, whether it's to what you're actually doing. Because this is such a novel sort of technology set, you need to actually see it in action to know where the value is actually going to get created.
- [05:33]
B
Yeah, I mean, to that point there is value in the playing around with it and even doing small use cases. But, you know, I can only imagine there must be a lot of pressure. I mean, I, you know, I go to many of the shows and see, you know, see a lot of the products and to your other point, there's. Everyone's talking about AI, some of it is more substantial than others. I'll just leave it at that. Yeah, but there must be such immense pressure to ship stuff with AI. Just to, you know, to speak broadly, how do you coach your teams to, you know, kind of protect the space for human led creative problem solving when, you know, there's all kinds of, from everything from shareholders to stakeholders and others that are demanding, you know, speed and just that AI adoption you mentioned.
- [06:25]
C
Yeah. So the way that I think about this is that some of the goals that you can have with AI, like speed, speed is always valuable. I think that moving faster always has value in business because if you move fast and it's the wrong thing, you can always throw it away, at least in software. Right. That's kind of the beauty of software. At the same time, though. Yes. The pressure to use AI and to have an AI story and to be able to compellingly show that you are using AI in effective ways, that is very, very high. I mean, even just for one of the simplest stakeholders on all of this, if you're a software builder like we are, our customers are expecting us to help them stay at the bleeding edge of AI, at least within the domains where we're delivering value. And so showing up and not having a story around AI, not having a story about how it matters to them and how we're going to deliver on that value. That's not something that we're able to do. But that I think transitions very nicely to that question of where is the human LED component? Where's the creative problem solving? Because there was obviously a period after ChatGPT3 came out where everybody was taking any text input box that they had in their product, they were saying like, oh well, I'm just going to put a little magic wand on that one. And now you can rephrase whatever you put into the box, like, all right, thanks. That's like really going to save my business or whatever. Those days are over and now it's actually much, much more about how can you figure out the creative ways where you can figure out where more automation, where some real reasoning embedded into your product is actually going to create the value. This is something that we talk about a lot at Braze, where a very important part of our product design philosophy at this point is figuring out how we're going to guide our customers to understand how they can use AI. Because it's such a novel technology set. There's never been something in the past where you can just take sort of random ideas or random unstructured knowledge and get actual processed outputs out of it in the way that AI can generate. I mean, that's kind of what an LLM is doing. And because of that, people are rightfully not very familiar with the capabilities of the technology. And so if you're not going to provide ways for them to get familiar, then they won't use it. And so what we're seeing in product development is that this isn't just a technical problem anymore. There's also this whole element of it actually being a sort of product led education problem at the same time.
- [08:48]
B
Yeah, yeah, well, and that leads to just customer expectations continuing to rise. And you've got to balance that, the customer expectations to see, yes, this uses AI, but to your other point, not just slapping a, a chat interface on a, on an existing interface. So maybe give us an example of, you know, what's an AI powered feature or capability that wasn't necessarily generated by AI, but guided by human instinct to solve a customer problem, even if it used AI to do that, you know, and, and maybe in a way that technology alone couldn't have, you know, come up with that solution.
- [09:33]
C
Yeah, so I think that one recent example that we've had that was just developed by a number of engineers actually just a few days ago, involves two of our newer AI products. We have an agent builder product that's in our dashboard which allows our customers to harness the power of AI by actually creating agents which take different actions directly within a customer journey or within a customer experience. So this is really, really cool. It sort of unlocks all these new capabilities. But like we were talking about before, it can be Tough to know exactly when you should apply these sorts of agents. And also it's a very novel sort of format or a very novel way to be creating customer experiences. A lot of marketers are used to creating ABN tests. They're used to composing messages. What they aren't used to doing is figuring out, how do I build an agent to solve a business problem and then inject it into a workflow? You know, there's a lot of novelty there. We have another AI product that we call the Braze AI Operator, which is designed to help and serve as a direct partner within our dashboard product to create different types of experiences. In line with the education challenge that I just mentioned, one of the most powerful things that we're finding is that the best way to build agents is actually by using a different agent, our operator, that knows all about what the AI is actually capable of. And so we've launched a number of different products. Like, for example, the first version of our agent builder that we've launched. Rather than launching with a number of templates, what we launched it with was a set of rules and context baked into our operator, such that our customers, the people who use our product every day, can talk to the operator and plan out the creation of an agent and then can actually use it within a contextual flow. Because that's something where, you know, you're. You're trying to get people up to speed with how these different tools work, but it's so much simpler if the AI can sort of teach you how to be better at AI.
- [11:26]
B
Right, right. Well, and, and even, you know, in, in reality, a lot of teams are using a lot of different products for many different tasks, or, you know, they may go to like a ChatGPT or something like that for some things, and the bouncing to another thing and another thing. I guess you could say marketing teams have always done that, but it feels like there's a lot of bouncing around. So what you're talking about, with this ability to do all of these things in one place guided by AI, it seems like what was always kind of needed is a framework that not only encourages experimentation, but also prevents just utter chaos, which is what happens when you're switching between, you know, 30 different browser windows and things like that. So is that kind of the. Some of the rationale behind this?
- [12:19]
C
Yeah, I think that that's certainly part of why. What I would say is that's certainly part of why it works. Because ultimately, no matter how human these large language models sound, no matter how friendly they might be and how quirky and fun their personalities might be. They're not real, living, breathing. And so as a result, they are only as powerful as the system that they're a part of. They're only as powerful as the platform they're a part of. They're only as powerful as the tools and interactions that you allow them to actually take. But on the other hand, they really can actually handle a huge amount of chaos. You can throw a huge amount of data context at them. At least as of this filming, the context windows are getting pretty substantial. And they can handle all sorts of different tools and all sorts of different information all at once. Much more so, actually, than a human can, especially before they've had their first cup of coffee. And so a lot of what we find to be really important is just saying, okay, we're going to take these agents, we're going to take this AI, we're going to put it in the middle of a product that can do a lot, that can really create anything that our customers want to imagine. And then we're going to give it all the context and all the tools that it needs that it can play around and it can follow the guidance of humans and who actually have a plan or they actually have an opinion on what direction it should be moving in. And then we'll let it harness all of the different tools. We'll let the AI be an expert in how to create more AI systems. That's been really, really effective for us in terms of how we design and think about these products.
- [13:54]
B
And so then, you know, how, how do we measure success here? And even, you know, to go back to the strategic decisions that need to be made, sometimes, you know, sometimes the best decision is not to pursue something. Right. You know, I would imagine you're faced with a lot of, a lot of choices and a lot of things because even with AI, there's still, you know, we still have to focus and things. So, you know, when you make a strategic decision to not pursue, you know, a AI feature because it doesn't pass that, that human judgment test, how do you measure success of an inaction? It seems, you know, and how do you prove the value of that kind of focus to the rest of the organization?
- [14:38]
C
Well, I'll say one thing, which is that I'm, I'm a startup person first and foremost. And so I'm like, I'm really not into inaction. I'm, I like action. I do not like inaction. And I think that one of the really neat things that AI is making possible is this idea that you can just do a lot more, you don't have to say no as often as people had to in the past because you can go and you can create that little widget that's going to help you in your day to day. You can go and you can build a bunch of these different features that were always sort of a next week, next month, next quarter, sort of perennial backlog resident. And now you can do a lot more of that. And so I think in that sense we are, what we will actually end up being able to measure is you'll be able to measure the cost of inaction more than anything else. Because the companies that really embrace AI and they move faster, I would predict that they're going to outperform the ones that don't. On the other hand, I will say that with all these automated systems, when we're now talking about sort of the actual consumer experience, like how you interact with marketing, we're thinking about the costs of inaction. There's one of the reasons that we were really excited to acquire Startup Offer Fit and build the Braze AI decisioning studio around it is that they have very robust AI reinforcement learning engines. And the reinforcement learning engine that they've built that is really, really heavily geared around being able to quantify both the value and the lift that customers are getting from using AI. But comparing it against business as usual, comparing it against, to the point of inaction, literally doing nothing, because it is so important to understand what are the different dimensions. Because once everything becomes possible, you have to actually quantitatively measure what is the effect of what we're doing versus what we could be doing. And so I think that in that sense what we're seeing is a lot of rising standards for marketers and rising standards, because it's like if I give you a really powerful tool with AI, I expect you to use it in really powerful ways. And it's not enough to just say number go up, I need to know how much why and so much more about that.
- [16:47]
B
Yeah, yeah. And then that, that leads to, you know, what so many marketing leaders are. I, I feel like talking about more. They may have always cared about it, but customer lifetime value just, you know, seems to be a bigger and bigger topic. How does this approach of, you know, both, you know, calculating incremental value and things like that translate into, you know, those kind of metrics that, you know, CMOs, CROs, even CFOs can, can point to like, you know, retention, engagement or lifetime value?
- [17:23]
C
Well, I think that those metrics Are of course incredibly powerful and incredibly important. Retention and cltv, I mean, these are kind of the gold standards of, in marketing of what you're really looking for. And what I think is interesting and if again, if you spend too much time on X, it's easy for people to lose sight of this is that to me, nothing has changed about that. Like those retention and LTV still matter. They are still the gold standard. That is still what we're trying to get to. Because what is AI? AI is a tool and it's a tool that talks, it's a tool that types, it's a tool that paints pretty pictures. And we as humans and consumers, when we see something that talks and paints pretty pictures, we're like, that thing's really smart, that thing's a person. That thing has upended everything, but it didn't upend like capitalism. And so retention, engagement, ltv, those things all still matter. This is just another tool which is opening up the methods and the strategies that you can use to drive those metrics that really matter. Because if you're a brand, you care about ltv and there's kind of nothing, save the sun swallowing the earth that is going to make you not care about ltv. Because that is how you measure the value that you're delivering to your customers and the relationship that you're building with them.
- [18:34]
B
Yeah, yeah. And I mean, to that point, AI should never be the strategy. Right, right, exactly.
- [18:42]
C
That's how you end up with the text input and let's rephrase all the tickets in our backlog or whatever.
- [18:49]
B
Totally. Yeah. So looking ahead and going back to your role as a product leader, as you are continuing to build a team and looking at developing the next generation of product leaders, what are some of the non technical skills that have now become non negotiable and are you starting to prioritize things that maybe were technical skills and other things were higher priority or I guess, what's changed? Long answer, long question, short.
- [19:25]
C
Yeah. So the skill that I'm looking at for my teams that I think is really critical in this era is, is just the ability to be very versatile and to have a really wide array of business understanding. Because when we're talking about what the future of work looks like, it most likely looks like a world where everybody has sort of a stable of agents at their disposal that they're delegating to go and doing different things and that we're there asking them to reason, asking them to set strategies, asking them to propose things. But these agents, until somebody shows up and asks and tells them what to do, or at least until somebody guides them. You know, the agents are made of silicon wafers just like all of their computers are. You know, they're sitting there, they're just minerals. And so as a result, what's. It becomes even more important that as a person, you're able to act with that sort of executive authority to understand what does my business actually need, what are the goals that I actually have here, and whether that's accounting or whether that's an engineer getting better at marketing, a marketer getting better at sales, a salesperson who's getting better at organizational operations or procurement, it's really, really important that everyone starts to develop this much more versatile skill set, because that's what's going to be required to actually harness the agents in a really effective way. Otherwise, I think that what you'll be able to do is, of course, you'll be able to automate more. You'll have to do less typing, like your fingers will touch fewer keys on your keyboard. But that isn't really the big unlock here. The big unlock is that ability to sort of turn yourself into the manager. And one of the things that we talk about internally at Brain is being a strategic conductor of kind of an army of agents. And to do that, you have to have an understanding of the full symphony and what the whole thing is going to sound like and what should all sound like together. So I think that that's one of the most important skill sets, and this is one of the most important new skill sets.
- [21:23]
B
Well, and to me, that just. That seems like that's elevating everyone's role. You know, generally speaking, you know, as you kind of progress in an organization or in your career, you get a little broader understanding of other things. But it seems to me like the. What you're touching on is the time savings or just the focus that you don't have to put on the hands on keyboard stuff will allow you to actually know more and grow more and stuff like that more quickly. Right?
- [21:53]
C
Yeah. I think that that's really the idea is that history shows that empowering technologies tends to just increase productivity generally overall. It tends to kind of take the overall productivity that we have, make it much, much bigger. Shuffle. Everybody, everybody plays a little bit of musical chairs to figure out what the new seats are, and then we go forward as a society. That's just what history has shown, and I think it's probably going to continue.
- [22:18]
B
Yeah. Yeah. Love it. Well, Kevin, thanks so much for joining today. I Got two questions for you as we wrap up here. The first one, if we were having this interview one year from today, what is one thing that we would definitely be talking about?
- [22:32]
C
Yes, definitely be talking about. I think that with the pace of model change, it's hard to say definitely, but I'll say, I think that a year from now to the point of that sort of musical chairs within the broader, you know, ecosystem of society, I think that there will be a lot of impacts to what work looks like and where humans are injected into different aspects of work. Again, I'm, I'm an optimist and so I think that the story is going to be an exciting one. I think if you're a pessimist, maybe you say like, oh, it's just going to be the singularity and you know, the change to our jobs, that we're all going to be batteries like in the Matrix. And so, I mean, I guess we'd be talking about that instead. But I think that is, that's really the big thing is that we're on the precipice of just a lot of change to at least a very large number of knowledge work jobs and I actually think to a number of, to a lot of jobs that don't, that we don't traditionally think of as knowledge work, like, you know, being a plumber, being electrician, working in H Vac or something like that, but that involve a lot of, they're not, they're not desk jobs, but they involve a lot of judgment. Because I also think that what's really neat about these AI systems is that they've encoded such a rich world model that you, you really can fix more things around your house because you have AI available. You can probably solve a harder problem for somebody if you show up and all of their pipes broke because you now have all of this consolidated knowledge. And so I think there's going to be a lot of shift and transformation. But again, I'm pretty optimistic that it's all going to be better for all of us.
- [24:01]
B
Yeah, yeah. Well, we'll have to check back in a year and do that. Last question for you. What do you do to stay agile in your role and how do you find, find a way to do it consistently?
- [24:13]
C
Yeah, so I think, well, I've got young kids, I've got a two year old and a five and a half year old and so that keeps my wife and I very, very busy. And I think that also having young, very, very young kids in your life forces you to be very, very adaptable because they themselves are just operating at a faster speed and a faster pace and you have to adapt to that. I think, though, that in a broader sense I just try to read a lot and I try to find people who are, you know, more experienced, say, product managers than I am, who are working at bigger companies that are working on companies that are working on very different things from me and just learn as much as I can from that. I find that a lot of people don't do that. They don't listen to podcasts that are informative, they don't read a lot of blog posts that are informative. And I think it's a real miss because first off it's really easy and and if you're not doing that, then it's just hard to stay at sort of the cutting edge. And I think one of the things that holds people back is that they'll sometimes feel like I need to fully learn about and fully research this topic. At least for me, I haven't found that to be the case. I think you can just sort of read some stuff, listen to a few podcasts, have a few conversations, just sort of let it osmosis into your brain and that alone is enough to really get most of the way towards being more agile.
- [25:30]
B
Yeah. Yeah, Love it. Well, again, I'd like to thank Kevin Wang, Chief Product Officer at Braze, for joining the show. You can learn more about Kevin and Braze following the links in the show notes.
- [25:41]
E
This episode is brought to you by Tech Systems. They're leaders in full stack, tech services, talent solutions and helping companies put it all in action. You can learn more@techsystems.com that's teksystems.com and thanks again for listening to the Agile Brand podcast. If you like the episode, hit subscribe and drop a rating so others can find the show too. And if you're interested in consulting, advisory work, or if you need a speaker for your next event, feel free to reach out. Just visit GregKillstrom.com that's G-R E G K-I H L S T R O M.com the Agile brand is produced by Missing Link, a Latina owned, strategy driven, creatively fueled production co op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. Until next time, stay curious and stay agile.
- [26:42]
D
At Strayer University, we help students like you go from Will I to why not? For over 130 years we've been innovating higher education to make it more affordable, accessible and attainable so you can reach your goals. Go from thinking can I? To Yes, I can and keep striving. Visit Strayer. Edu to learn more. Strayer University is certified to operate in Virginia by Chev and its many campuses, including at 2121 15th Street north in Arlington, Virginia.