Loading summary
Transcript77 lines
- [00:00]
A
I'd say the beginning of this year we were truly hill climbing, building an assistant. And for anyone out there who has tried to do this, it is the easiest thing to demo and the hardest thing to build in terms of production because when it comes to conversation, the user could ask anything and it is so difficult to succeed and provide good answers across all the topics that could be asked. It's so easy to make a demo and so hard to get right. And so I think that's what we really learned. And one thing that was a big unlock for us, which people don't normally get to see. And I think it's such a good demonstration for how creative PMs have to be in this day and age is how we do our evaluations. We'd create this judge that is like ever evolving, right? You're basically embodying all of your product intuition about how you want this agent to behave and you're putting it in this LLM judge. We would take that and, and we would actually then give it to the entire development team. And so every time any developer sits down to build on Sidekick and every time they say, okay, I iterated the prompt, I added a new tool or I'm testing a new model, they would then run it against a bunch of simulated conversations and then they would have the LLM grade the conversations against that candidate build of Sidekick. And so we do this every time anyone sits down to build on Sidekick. We also run this LLM judge on live conversations when we launched the to see how is this trending based on what users are actually asking. So this becomes the tool that you use to guide the team. It's so much more than a spec because it becomes the measurement of whether or not we are achieving what we want as a product team in this agent, which is fascinating.
- [01:35]
B
Creating great products isn't just about product managers and their day to day interactions with developers. It's about how an organization supports products as a whole. The systems, the processes and cultures in place that help companies deliver value to their customers. With the help of some boundary pushing guests and inspiration from your most pressing product questions, we'll dive into this system from every angle and help you think like a great product leader. This is the Product Thinking Podcast. Here's your host, Melissa Perry.
- [02:12]
C
Hello and welcome to another episode of the Product Thinking Podcast. Today I'm excited to have Vanessa Lee with us. Vanessa is the VP of Product at Shopify, where she leads innovative efforts in agentic commerce and AI integration. I'm thrilled to dive into how Shopify is setting the pace in agentic commerce and what lessons we can learn from their journey. Welcome Vanessa. It's great to have you here.
- [02:31]
A
It's good to be here, Melissa. Thanks for having me.
- [02:34]
C
Yeah, I'm so excited about this topic. You guys have been really full blown. Good going crazy at Shopify with really cool new products.
- [02:42]
A
Yeah, we've been a bit loud about it from all respects, so this is a meaty topic to dive into for sure.
- [02:47]
C
I love it. So can you tell us a little bit about how did you become the VP of Product at Shopify? What do you oversee and what was your journey to this moment?
- [02:56]
A
Yeah, I've been a founder a couple of times, so and I'm engineer by. By education and I did a couple startups. I worked at Microsoft for a little like a hot minute and then when I joined Shopify, I actually joined as a senior product manager. And so I have seen every level of product management at Shopify. I worked on a few different things at Shopify. I started first building our developer platform and essentially our developer ecosystem in 2017. And then I worked on our online store, which is what Shopify is known for. It's our bread and butter. And then I worked also on some of our AI initiatives. So like things like Sidekick and what we do with jet generative AI like media generation. And then so recently I've gotten a chance to pair with, to be honest, like most of Shopify, which has been really fun. And yeah, so it's been a journey. It's been almost nine years, which is wild.
- [03:51]
C
Wow, that's awesome. And that. So what have been like some of the key moments when you look at Shopify's journey for you that define how you think about product management?
- [04:02]
A
Oh, I feel like product management is different in every organization. It's probably the one craft that swings wildly depending on the culture of the of the company. Us in particular. I'd say also a very specific flavor of product management, but one that I personally couldn't imagine not working in. So we are a very in the details product management culture. Like I think it flows from Toby just all the way through our entire company. We are the type of product managers who just absolutely love building and being in the details. And our team is actually quite light when you look at ratios between PM and engineering. So I think that kind of sets the tone for how much we expect PMs across the board, even all the way up to director. I have a bunch of product directors who are way more in the weeds than you would expect. A product director to be. But I think if you love building and you're super curious, Shopify's PM culture really speaks to those types of product managers.
- [05:04]
C
And you are working right now on one of the hot button topics.
- [05:09]
A
I feel like, I feel like it's like the hot button topics of all of tech.
- [05:13]
C
So I feel like it's funny. It's like almost a game now. It's every time you say AI, do.
- [05:17]
A
Something, take a shot. Yeah.
- [05:19]
C
But I know Shopify has been doing a lot of work around agentic AI. Can you tell us what you've been working on lately and what's making you really excited about this?
- [05:28]
A
Yeah, we are the Toby and Harley and folks like to say we are the entrepreneurship company, which means our goal is to make sure that merchants, our merchants succeed. And we believe that there's more entrepreneurs in the world than, than they are out there today that are just waiting to find their passion. And so one of the things that we've really started out in now, I believe a couple years we put out this, essentially this vision for what became Sidekick, which is our AI assistant, to make sure that merchants have someone by their side to help them succeed. So if you think about AI in that respect, this is like a co pilot or an assistant that you can chat with that knows Shopify better than any other assistant out there, knows your business better than any assistant out there, and that knows how to, how to do things on your behalf. So like it's able to proactively create products and collections for you, help you set up your payment methods. Basically, anytime there's a hurdle in entrepreneurship, we wanted Sidekick to be there to help you through it. Knowing how lonely entrepreneurship can be at times. And so that was probably our biggest focus in the early days with AI outside of just smaller projects that we had around product description generation or image generation, we really envisioned a lot more than just the one shot generation. We also envision this like Assistant, this entity beside you. And yeah, Sidekick really kicked it off. And I'd say lately there's been a lot of advancements that we've, we've seen over how Sidekick can go also beyond just someone you talk to and now moving into an assistant that is a lot more proactive that folks turn to in a lot more of a general sense versus just creating products, creating collections. So I say a lot of it in the early days were, was around our soul, our heart and soul as a company, which is how do we make more entrepreneurs successful?
- [07:23]
C
And with Sidekick, I think people could understand it. A little bit and think maybe it's just like a chatbot answering, but it goes beyond that. What are some use cases people will turn to Sidekick for and how does it work?
- [07:36]
A
I think the best? Well, Sidekick is just embedded inside of fi and I think one of the things that we're still working on is really how Sidekick will evolve over the next couple years in terms of how it changes the UI dynamic on how someone, you know using Shopify day in and day out to run their business experiences. Sidekick. But to be honest, Sidekick is actually from a UI perspective, pretty simple. Today it really is just embedded inside the admin. But what was magical is that over the last couple years, in the last, I'd say the last 20, 25 in particular, we saw Sidekick really take off. And I think that's a mixture of two things. So one, we have a audience, a merchant base who is super hungry for this companion. Entrepreneurship is a very lonely journey. It's not easy. The cards are often stacked against you. And so we had people who were just really hungry for help. And so that's, that was one piece was we had just this user base who was ready for Sidekick to come and help them. And then we also just hit it. In terms of foundations, Psykik really improved in terms of answers and what it was able to do. So Psychic's able to do things that you would expect it to be able to do in Shopify. Things like creating a product, creating a collection, creating a discount for you, helping you navigating Shopify's admin easier. But now we've started to see people really leverage Sidekick for I'd say like higher order tasks, things like, hey, tell me about my sales over the last year year. What trends are you seeing? What should I be doing in the next year to help me grow? What's an attainable goal that you would see for my type of business in my industry considering I had X number of dollars of revenue in the last month. And I thought I there's like one really key conversation that's such a good example. There's this merchant who just went back and forth with Sidekick on like a Friday night a hundred, over a hundred times and you really saw them start to like just even talk to Psychic truly as if it was a another member of their team. And I think that was incredible because it meant Psychic for so long. We had worked on the foundations for a long time and we finally broke through as to okay, now we're really providing useful answers for our merchants. And that journey was not an Easy one. But yeah, I'd say like merchants use us a lot for higher order tasks, which was also a surprise. It's like anything like a data analyst, an advisor, a financial advisor understanding what marketing tactics they should explore. I'd say that those are some of the surprising most useful use cases for psykik.
- [10:17]
C
I love that. It's basically an E commerce expert.
- [10:21]
A
Yes.
- [10:21]
C
Embedded into the product, which is really changes the user experience for people. Right. Who are not, let's say e commerce expert. They're small business owners that know their brand or design clothes, but they're like, how do I actually sell this? So I love that you're giving them that whole experience in there too.
- [10:37]
A
Yeah, yeah, it's, it's been such an amazing, probably one of the most rewarding products I've had a chance to work with, to build in my career. Like even just watching merchants, they're like, yeah, I just, I want to talk to Psychic all the time. And the fact that Psychic just knows so much about their business is I think what unlocks a lot of that versus any other assistant that they could reach for that is around the Internet.
- [11:01]
C
Yeah. And it has all the data. So how did you, how did you come up with Sidekick? Tell us a story about like how you built this, how it got started, what, what that process was like.
- [11:12]
A
Yeah. So this is. If you go back, you'll see Toby's original post. I think this was, I can't remember the exact date, but I think it was in 2023 3. And he really set the aspirations for what we wanted this to be. I think like originally we even in Toby's video, I don't think you see exactly where we're going. But he already painted the picture pretty far into the future at that point. So Psychic really kicked off then. I think Once we saw LLM start to kick off ChatGPT, there was a moment, I think that everyone who was working on especially SaaS based software knew, okay, this is a, this is a moment to pay attention to. And I'd say that was like, I'd say at that year, 2023, we were all just finding our way. Like at that point, 2024, we started to be like, okay, you start to see companies truly come out and say, okay, we're going to start to have assistance and assistance inside of our platforms and we're going to give that a go. And so really for a while, I'd say since Toby sent his video out and up until even, I'd say the beginning of this year, we, we were truly hill climbing, building an assistant. And for anyone out there who has tried to do this, it is the easiest thing to demo and the hardest thing to build in terms of production. Because when it comes to conversation, you like the user could ask anything and it is so difficult to succeed and provide good answers across all the topics that could be asked. It's so easy to make a demo and so hard to get right. And so I think that's what we really learned. And one thing that was a big unlock for us, which people don't normally get to see. And I think it's such a good demonstration for how creative PMs have to be in this day and age, is how we do our evaluations. So when you're building an assistant, it's such a intricate system of what models am I using, what context am I providing the model? Are we using several models who are that are working together? What data is feeding that model? Is it a fine tuned model? Is it a model that you're getting off the shelf? And then also how does it integrate with the platform? There's so many variables in that. And every time you change any of those variables, which your developers are doing all the time, every time they do a PR for a new iteration on the prompt, they are changing a variable. How do you know that it's not just like, how do you test it? Right when we build software up until this point, you have unit tests, you would, you have your set of tests that you know, okay, it works. How do you test when it's not, it's not a deterministic product. And so what we did was we also had the cold start problem. Everyone who build, who builds an assistant has that problem where they just don't have the data yet. So we had no data, we had no example conversations. And, and so one thing that we did, which I think is so ingenious and I can't, I definitely can't claim credit, the team really did this. I watched folks build a simulator for a merchant. So you take all these topics and basically ask LLMs to help us generate all of these topics of give us like a thousand questions that merchants might ask in their journey. And we would do it over and over again for different verticals of merchants, different maturity of a merchant, a scaling merchant, a merchant who's just starting. And then we'd take all of those topics and then we'd feed it to another LLM that we have prompted to be a merchant. So we said, you are a merchant. Yes, you are a Merchant, you are a brand new merchant. You are working out of your garage. You are aged 30 years old. You like answers that are simple. Your goal is to build your, I don't know, your, your lip gloss business out of your garage. And then we would do, we'd do many merchant profiles. We'd give them all of these corresponding topics and then we'd have them talk to a candidate of psychic, like our first Build a Psychic. And we get all of this synthetic conversation. We get like these LLMs talking to LLMs, which is also hilarious because you'd be surprised watching LLMs talk to each other. They're just so willing to please that, like they just kept going on and on. So there's a lot of work you have to do to actually get that dissimilar to be a real topic like humans do. But we get these. We got to a point where we'd be able to have these thousands upon thousands of synthetic conversations. And then we would take them and we would actually start to mark them as a group of humans. So they're like, let's say we had five humans and PMs do this all the time. I think this is like one of Those things that PMs don't realize when they build these products. But the way you actually spec out what you want your agent to do is actually through this grading process because you said, hey, this conversation, the agent went sidekick, went too long, did not answer the question, and created a product instead of creating a collection, which is what would have been the right answer. I'll grade this and let's say, Melissa, you're also BM on the team. You would also grade the same conversation. And we would say, okay, what we care about for this product is like on five dimensions. It's like, did it serve the merchant's goals? Did it follow our safety standards? Did it actually use the right tools? And was it grounded in the right way? Did it have the right tone? And so we would define all of these dimensions. Then me and you would grade this particular conversation. And funny enough, even humans don't agree with each other all the time. Okay. And so we would take this overlap and we'd say, okay, Melissa and I, on average, we have a agreement overlap of about 60%, which means 60% of the time we would mark the same conversation the same way across these dimensions. And then we take that and we say, okay, we're going to build an LLM judge. So this is like not proprietary. I think this is just fascinating because it's. If you Read papers out there, you will find this. This is not. We are not the only ones to do this. But I don't think people tell this story enough. We take an LLM judge and we actually say, can we prompt this judge so that it would also overlap at 60%? We want it to be able to slot into the team and be like, vanessa's not here today. So you know this LLM is going to judge. And the way that they judge basically simulates Vanessa enough. So once you have this LLM judge tuned, means you prompted, you iterated on it, and you've ran it a couple times against all of the same markings, and it turned out it would grade it the same way me and you would with the right overlap. We call it Cohen's Kappa. And then from there we would say, okay, now we have our judge. That process that I just called to you, we do that all the time. So every time there is a new feature in Shopify, every time there, we believe that actually, let's revisit. We don't think that merchants are starting to change their behavior with Sidekick. And we need to start to create essentially new ground truth sets for our judge. Because people are asking now about marketing tactics. Oh, we haven't graded marketing tactics before, so me and you would do that again and then we'd create this judge that is like ever evolving.
- [18:16]
C
Right.
- [18:16]
A
So this is almost the new version of a spec. When you think about pm, you're basically embodying all of your product intuition about what you want this, how you want this agent to behave, and you're putting it in this LLM judge.
- [18:31]
C
I think it's so fascinating because you're getting to what I was going to ask about this, which is it's a different tool set and a different way of working, I think, with PMs. For PMs who work with more generative LLMs and agents training agents. Tell us a little bit about. Now it's not a spec. Is it E files? Like, what do we have to learn? What are we doing here? That's with different tool sets. Yeah.
- [18:55]
A
So that LLM judge that we just trained, we would take that and we would actually then give it to the entire development team. And so every time any developer sits down to build on Sidekick and every time they say, okay, I iterate the prompt, I added a new tool or I'm testing a new model, they would then run it against a bunch of simulated conversations and then they would have the LLM grade the conversations against that candidate build of Sidekick. And so we do this every time anyone sits down to build on Sidekick. We do this many times. We also run this LLM judge on live conversations when we launched to see how is this trending based on what users are actually asking. So this LLM judge becomes the tool that you use to guide the team. It becomes even more. It's so much more than a spec, which is so fascinating because it becomes the measurement of whether or not we are achieving what we want as a product team in this agent, which is fascinating.
- [19:54]
C
Yeah, that's really cool. So it's pretty much like designing the agent, like how you want it to think and then embedding that into it and then it's basically. You make the product for it though too.
- [20:06]
A
Yeah, you're basically imparting all of your product intuition into an LLM to then shape yet another ll, which is so mind boggling. But this is the type of creativity that think when teams truly hit an acceleration with AI or building AI products, this is the type of creativity you need. You do not find data just lying around. You do not manually. Okay, let me test this over and over again. That's not going to scale. And so I think that this is one of the fascinating stories that I love telling because it's just such a amazing story of ingenuity.
- [20:45]
C
Yeah, it's really cool. It's extremely creative about how you handle that lack of data that we were talking about at the beginning. And I get that question from newer PMs or people launching products so much where they're saying, hey, if I have no data, how do I even start this? How do I look at it? You guys figured out a whole system on how to generate it.
- [21:05]
A
I think everyone should start that way. LLMs, they have so much to offer in terms of capabilities. I know we are all used to chatting with agents and using them to help us learn faster. And I think those are all great applications of AI and we definitely do that a lot internally where we expect gut checks, sense check ideas with, with agents, especially as they've gotten better, especially this year. But I also think that LLMs we often forget are useful tools in creating data and cleaning data. And that's what with how the team has built Sidekick probably at this point, like this is not. This is tribal knowledge. I think for the teams who have built agents at scale, but I think for people who are just getting started, you don't get to see that every day. And I think that is, that's such a valuable lesson.
- [21:51]
C
I'm excited to share that. I'll be joining Product Weekend in New York City on November 14th. An incredible event powered by Localize. In the last edition of May, I had the chance to meet a few people from the Localize team and some of their clients and it completely changed how I think about localization. It's not just about translation. It can be a real growth driver. In fact, some companies attribute up to half of their revenue growth to localization. Localize helps teams do this at scale with over 3000 companies using their AI powered localization platform to speed up translation, ensure quality and deliver better experiences worldwide. Head over to Localize.com to learn more and check out their localization revenue report to understand more about how localization can help scale your business. For people out there who are trying to get better at doing product management when it comes to generative AI and using LLMs, what types of new skill sets would you advise them to look into to be able to do these things correctly? Like, where do they start? What should they learn?
- [22:51]
A
Yeah, to be honest, this part is. Is hard. I think these stories and why I love telling this one, I tell it internally at Shopify all the time. Because to be honest, even at a company as large as Shopify, this knowledge that I'm telling you, I'm telling you is not uniform. But I think the biggest advice I have is definitely play. Everyone plays with AI. I feel like that's not a new thing. But also see how you can use AI in more creative scenarios. I think that's when you start to just get better ideas. So like, we started using AI even before Sidekick. We went through that loop simultaneously. I had another team who was building our product taxonomy, which is a different application of AI, where we take LLMs and different ML models and we try to basically get a better idea of the products that merchants are selling so we can help them find the right channels to sell those products. And every product is very different and every merchant holds their product data in Shopify differently as well. So sometimes you'd be like, okay, is this a garden hose or is this a lipstick? It's funny because we were able to use LLMs to give us product attributes and suggest it to merchants to be like, all right, it's a garden hose. We can guess from the description and the title and the image. But also garden hoses should typically have a length and it should have a color and it should have. These are all the attributes that you should fill in so that it can actually be sold properly when you connect it to marketplaces, SEO, et cetera. And we used LLMs also in that respect to help merchants basically accurately label all of the attributes of their product, which now have also helped merchants sell more accurately when they go to connect to other channels. So these things where you're like, this is not like a normal application. I'm not just chatting with an LLM. I'm using it on a specific goal. I think once you get that muscle going, then you'll draw a lot more creative conclusions on other applications you could use it for. But I think you just gotta start.
- [24:52]
C
Yeah, you have to see what it's capable of doing.
- [24:55]
A
Yeah.
- [24:55]
C
I guess there's the limitations part of it too, that we talk about with generative AI and LLMs. Is there anything that you would call out that say, hey, this is not well suited for it, or here's some limitations that we found that we try to steer clear of.
- [25:09]
A
To be honest, we use LLMs so ubiquitously across Shopify. I like. I'm trying to think of an application that wouldn't be good. I think that's actually funny enough, that's more challenging nowadays than finding applications which AI and LLMs are good for, to the extent where I'm going to have to get back to you because I actually don't know what that would be. I think everything in Shopify has been completely redefined and retouched in some way or other by LLMs and AI.
- [25:41]
C
Oh, that's cool. That's also saying, like, they're getting better and they're super smart at this point to lean on them.
- [25:48]
A
Yeah. I recently had to onboard very quickly to our inventory system, for example, and I wanted to be helpful to them. And there's this funny story where I'm sure Toby would be fine if I told him this, but me and him were both going back and forth on the design of Shopify's inventory system and how do we make that better? It's a really important part of Shopify. And we started passing each other essentially like chats that we had with AI. But, okay, what if it worked like this? And if we had virtual inventory groups and then we have a ledger and we actually use LLMs quite a lot in how we learn and how quickly we can ramp up in areas that we. We either want a deeper understanding of or. Or we could be new to. So I think that's. It's a fun story. Everyone uses it. There's no ego in Shopify about not using AI to help us do our jobs. That's for sure.
- [26:38]
C
That's cool. That's really cool. And you're operationalizing AI at a massive scale at Shopify. If it's touching everything, you're. You have it across the platform. What do you think? It's such a complex challenge. What gives Shopify a unique edge in overcoming doing this at scale, being able to actually implement AI across such a large platform?
- [27:03]
A
I think thanks to just a lot of curiosity from Toby and from a lot of folks here, we've been willing to be cutting edge on stuff that may or may not have been ready. We were super early on AR VR. We were super bullish on QR codes before COVID And I think that even on things like, you know, stablecoin and then now AI, I think there's. There's a big belief that if we are on the cutting edge, that our merchants will also be on the cutting edge. And so we're always bullish on, okay, let's do a 3D model and make it possible for someone to upload a 3D model in their online store so that you could see the details of this stroller. Like, I remember doing that, and that was way before I think merchants even understood, like, how do I get a 3D model of my products made? But I think we always trusted that if we did our jobs and made it easy on our side of the platform, that over time, a lot of these bets will pay off. I think AI was just one of those where very early we could tell, okay, this is a bet. Oh, this is more than this bet is very likely to hit. And. And then I think we just really encouraged everyone to play with it and to use it and to not be afraid of it. And I think that comes all the way, Toby, all the way down. And I think that's really changed the way that we use AI and then how we build with AI as well.
- [28:25]
C
When you think about the vision in the future for Shopify, and especially when it comes to agentic commerce, how do you see that playing out? Can you tell us a little bit about what agentic commerce actually means as well? And what. Where do you think this is, this is going in the future?
- [28:42]
A
Yeah, I think if you Google agentic commerce, you will come up with many results. So agentic commerce, I think was, is really like this big umbrella term about how are we going to change shopping in the advent of agents? And it goes everywhere from how are we going to change how we discover products to buy and how we live our lives. For example, the ones that I'm personally excited about are things like helping people spend less time searching. I Want to be able to say, hey, it's my five year old's birthday this year. Help me plan it and help me procure everything. Okay, great, that looks good purchase and that just saves so much time. And so I think that there's just this advent of okay, there is an assistant that is helping you with all aspects of your life, including the parts that perhaps require a bridge into physical goods. Right. Purchasing things. How do, how does AI play in that space? This is something that we've been excited about for a long time, to be honest, because we do think that is humans are anything but very smart and lazy beings, which is what makes us great tool makers. And so we trusted that this would happen. So we started planning for this a while ago, a couple of years ago. Even with the taxonomy, the product taxonomy work that we, that I told you about, like understanding what products are what so that we could serve up to an agent. Okay, this is a garden hose. It's good, it has great reviews. It, it's green, it's eight feet. Right. If we could understand our merchants products better then we could better help integrate them in a future, not even that far in the future where agents were helping to actually purchase on behalf of users. So there's the discovery portion of this and then there's the actual execution and checkout. We have one of the most amazing checkout infrastructures. I think that has been our bread and butter. We had some of the largest flash sellers in the world come to Shopify because of what we can handle at scale. And then also we're the only checkout platform that have millions of merchants on a single platform.
- [30:46]
C
So cool. I like, I love that I can just go to a random clothing website or something like oh, let me check out and do you want to log into your Shop account? Yes. I don't want to create 18,000 other accounts. That's amazing. I remember the first time I saw it I was like this is brilliant.
- [31:02]
A
Yeah, it's so good and funny story inside story. That was a hack days project in Z a long time ago.
- [31:08]
C
Wow.
- [31:08]
A
Which was clearly go like a great idea that has then just, absolutely just exceeded all of our expectations. It's been such a great thing for merchants to have that and such a great like everyone that I talked to is oh, I love like logging with Shop. Shop Checkout is the best. So we have this great checkout. But checkout is so much more than just a payment. Like when you log in with Shop, you don't need to put in your payment credentials. You have Your address field. But our checkout, what has been so fascinating is over the last half decade we've invested in making checkout modular. So our checkout, if you are a merchant, let's say you're someone really large, I have a lot of gmv, I'm all birds. I can use Shopify's checkout and I can customize it. I can say, you know what I need to show you. Like, you know in Canada right now we have a strike on our postal service. I need to tell you, I need to tell all my Canadian customers, hey, there might be delays in shipping or our shipping costs are different or whatever. So this is Albert all the way to a furniture store where they need to do a white glove delivery. And so they're like, hey Melissa, when do you want this chair delivered? And so we have all of these extension points we call extension points in our checkout. But what's great with us investing in that for the last five years is now we're at this point where our checkout is a single platform that works across millions of businesses. That in a dime and we launched this in August as a checkout sheet you can integrate into your agent and it can come up and be styled to be, let's say your background is Bing and your agent, you can style it to feel native to your agent and it will work across millions of businesses. And I think that is just something that is like incredible that people don't normally think about when they think of agentic is like how do you check out? And so that's something we've invested in for the last half decade. So it's like everything from discovery all the way to checkout I think is like something that you'll start to see more agents adopt over the next couple years.
- [33:09]
C
And then when you think about AI to commerce, especially with small brands compared to like large global brands, how do you think it differs? How does your product philosophy differ to serve the one person entrepreneur versus the large global brands? Like we're talking about allbirds, like huge company, huge sneaker company.
- [33:28]
A
Yeah, I think that there are, I think the one thing that we always try and do at Shopify is we always try and make sure that we are democratizing access to customers and helping all entrepreneurs succeed. Whether it's a small merchant or a really large merchant. I do see differences in terms of how they reach customers and consumers. We want to make sure that's pretty uniform. So we're going to do our job to make sure that even the two person company trying to Build something is going to show up in all the right places, no matter what channels happen. We had social media on the rise, and now we have Agentix. This is new and not new at the same time for us. We've seen so many rises in search being the biggest thing when we first came out as a platform in like 2010, being the primary way then social media. And so this is just the next wave. And every time we try and make sure that we're at the forefront, we see a large difference in terms of how AI is leveraged, to be honest, by those folks, which has been fascinating. Yeah. I think that there's just this scrappy resilience that smaller entrepreneurs have, which is fascinating. And so they actually are some of the most hungry users of our AI tools. So, like, they're generating media using Sidekick for their ads. They're generating media to help them with their product images. Super. Into leveraging Sidekick to change all of their SEO tags to be more optimized. And so you see them use it a lot for larger merchants. I think it's a lot more like what that specific person is doing. So this is like internal to the organization. A marketer will use Sidekick for their specific job. Tell me about the data of the store.
- [35:08]
C
But.
- [35:09]
A
But we actually see more uptick, I'd say a lot of uptick in smaller merchants. And then when it comes to really large merchants, it's really on specific tasks. Right. Like specific tasks are on data analysis and trying to understand what should my marketing tactic be coming up. So they're different. I think there's more holistic usage coming from smaller merchants. And for larger merchants, you just see it more and. Okay. They have a very particular role that they're using Sidekick for.
- [35:36]
C
That's fascinating. Yeah. Do you think that has to do with the. You were talking about the scrappy nature of the entrepreneurs, but they don't have a whole team around it. And do you think it's more about, like, the specific roles that people play in larger companies where it's like, how do I use this to help me? Rather than how do I look at it globally?
- [35:52]
A
100%. 100%. Like that one entrepreneur needs to do all the things.
- [35:56]
C
Yeah.
- [35:57]
A
Right. Whereas that marketing manager is trying to do their best job in their role in marketing. In marketing. And the fascinating thing is how many. I'd say, like executives at larger merchants, they have been using it more holistically.
- [36:12]
C
Oh, interesting.
- [36:13]
A
Yeah. Because then they're able to oversee like a larger area. Okay. Like help. Help me brainstorm what what customer segments should I go after next quarter that we haven't tapped into yet? Build them for me. So there's, there is some stuff that we're seeing, but yeah, for sure, the early entrepreneurs, they are, they're a different breed of a person. They're just so stubborn and just scrappy and resourceful. And we see that in the way that you. Sidekick, for sure.
- [36:39]
C
That's really cool. And then you have been partnering with OpenAI's agentic commerce protocol. Can you talk a little bit about that partnership and how that came about, how you're using it?
- [36:50]
A
Yeah, I mean, we've worked closely with OpenAI for years, not just as a partner, but also as a customer, to be frank, because we use a lot of their models. We use a lot of models for, from a lot of different partners, but they for sure are one of them. And last year actually, we started talking about, okay, how do we get, how do we get product discovery in. And so earlier this year, we started getting together to talk about, okay, how do we bring commerce to ChatGPT? And right. I think this was just announced. We're super excited to start to roll this out. But essentially we partner with them to make sure that our merchants products are discoverable and that they're able to check out directly inside of ChatGPT. And so that's been, yeah, that's been super fun to work with them on. And there I think that commerce will start to take over, not just. It's going to become ingrained in every agent. And so I think you will start to see a lot of activity across the board. But we are super pumped to be working on this with OpenAI.
- [37:46]
C
Yeah. When I saw the launch about this, I got really excited because I was like, man, I am using ChatGPT to brainstorm all these gift ideas and stuff. Yes. And then they're just linking out to it. I'm like, I just want to buy it here. I don't want to create like 18,000 accounts.
- [37:59]
A
A hundred percent. Yeah. And I think that's part of why we decided, okay, let's work together on this and let's start to work on how do we make that possible for so many consumers.
- [38:08]
C
Do you think that's also going to change the way that, that people think about marketing their shops as well? Because now it's going to be like, how do you show up in these searches? What, what do small business or merchants or anybody, really large businesses too, have to think about now? About how we talk about our products and what we offer and how they're marketed to different people.
- [38:29]
A
I think that is probably the top question that I get. Not just about this particular integration with ChatGPT, but also at large. I think merchants could smell this, having gone through SEO, then social media. Like, they are very. They are very in tune with where consumers are changing behavior. Yeah, I do think that will change. So a lot of that has been, okay, how do we help merchants make sure that their product data is correct? Because a lot of the models currently, they're just starting to get awareness on, okay, what are these products that I can search for? Can I search for it accurately? Am I pulling the feet versus 10ft? Because that's the type of nuance you get with agents, right? You're like, hey, I'm trying to set up a sprinkler system in my front yard. It's about 10ft long. What's a good garden hose for me? And it'll give you. It'll give you the exact one that you want. But in order for that to be the case, it needs to know that this product is 8ft. This product is 10ft. And so we really started with the foundations. There is like, how do we create a standard product taxonomy? Now from there, every merchant's gonna be like, how do I make sure that I stand out? And this is where we as a platform have tried to make sure that all of the stuff, product descriptions, reviews about products are all within Shopify so that it can then be served, like, in one shot, in one integration with all of these partners. We always say, I think one of the superpowers of being on Shopify is the fact that we're millions of merchants. And so we can go and talk about these partnerships with folks like OpenAI, Microsoft, all these, like, all these partnerships that we've brokered over the years because we represent millions of merchants, and it's one integration that they can build, and then all of these. All of these businesses can get access to it. And so I do think that we will invest a lot more in, okay, how do we make sure that you continue to show up properly? And how do you want your brand to show up? We launched something in quietly called, like the Knowledge Base app, where you can install that application. You can see what questions are being asked of your products, of your store, and you can provide grounding data so you can start to understand, okay, I want to be like, I am. I do not have any tariffs or duties on my product, because that's a question that I get asked a lot. So I want to make sure, that's clear. So we are experimenting, but I think we'll continue to see more of it in the next six months.
- [40:50]
C
When you think about the future of product management, what do you think are some emerging trends that are going to have a significant impact on it and like our whole discipline in the next couple years?
- [41:00]
A
Oh, I think I am so excited to see where product management will be even inside Shopify. We started a Vibe coding interview. Pairing interview. Yes. Because what I want product managers in Shopify to do, and I think it's so exciting for the industry at large, all product managers, is we can now prototype stuff that we would typically have to rely on the dev team being free and we can just tinker with it and we can understand how it feels. And that process, that feedback loop is so tight. And so I think that that is such a superpower. Of course, like engineers, not all of this code is actually ready to be actually put into production. But even that alone, being able to gut check ideas is huge. And I expect PMs to do this all the time to be able to communicate better. PMs don't. It's fascinating. I always say this. We don't actually do any real work. Our work is actually in communication and helping to make sure that everyone is building the right thing at the right time. But that goes. Building the right thing is not an easy thing to do. You can't just throw requirements on. There are a million ways to solve any problem. And in being able to do a prototype quickly, you can quickly see is it working out? Is it actually how I want it to feel? Especially for like very high interaction products where you want to understand, like, does this going to. Is this going to feel right? You think about like the swiping gesture that was introduced by Tinder and others and then now that's very native in a lot of different UIs. But that you don't. You wouldn't come up with that without having this tactile experience and feedback loop. And I think that PM's now getting the ability to do that independently and then now taking. Okay, team, here's a more clear, crystallized view of what I'm thinking means that their teams can now be more effective when they're bringing it to production. So I just expect this to be a huge force and function for us to move faster as a whole. Discipline.
- [43:01]
C
Yeah, I like, I am super excited about the whole Vibe coding thing. I'm worried because I think people are gonna put junk code into their systems.
- [43:08]
A
But in general, do not merge your PRs without someone reviewing it for sure.
- [43:12]
C
But I love the fact that we can now prototype, test with users, test with ourselves, just like communicate things to so much more easily instead of just waiting and trying to translate it through developers. And it's okay, we can just look at it like you're saying and then see if it's feeling right. If it's not feeling right, get it in front of people. Like it's huge to me that we.
- [43:33]
A
Can now do this 100%. It is, it is magic.
- [43:37]
C
Amazing. Vanessa, my last question for you. If you could give one piece of advice to your younger self, what would it be?
- [43:43]
A
I think one of the things that I realized, and I wish I realized this earlier, is that not only just as a pm, it's what led me to, I think PM and staying in this role for this long is instead of chasing titles or like particular companies, think about how you want to spend your time and then go find the place where you're going to get to do that. And so if you really enjoy collaborating, go find the place where you're going to get to collaborate with people the most. If you enjoy building, then go find the place that you're going to get hands on building. I think sometimes I talk to a lot of PMs and PM has grown very quickly into quite and when I started it PM wasn't that much of a thing. It was just getting started. I think a lot of people are coming into PM because it's starting to become something that is known to be a good job. But I think PM is actually quite difficult for anyone who's been in the industry. It is a Slogan. It is 80% slog, 10% the best, 10% the worst. And I think you have to know why you love it. So if you love talking to people then okay, then maybe you're being, maybe you be in a PM environment where you're having to talk to a lot of customers. But I think it's how you spend your time because I think more than anything PM is actually being a source of energy and courage to your teams. And I think to have that and to make sure you have a good career doing it, you have to know which environments you're going to be able to supply that energy because you get the energy from essentially building. So to have a really good idea of what gives you energy so that you can give it back to your.
- [45:26]
C
Teams, I think that's great advice and man, great advice for people out there too. Thinking about do I want to work at a small company, a large company. Like what types of company to. Because I've seen so many people like let's say from enterprise companies go, oh, I really want to work on a startup. And then they get in there and they're like, not like, never mind, not this one. So it's honing in on those factors. I think that's fantastic advice. Thank you so much, Vanessa, for joining us. If people want to learn more about you and your work and they go.
- [45:54]
A
You can find me on X. I'm not there all the time, but I do lurk and I do pay attention. Yeah.
- [45:59]
C
All right.
- [46:00]
A
Thank you for having me. Yeah.
- [46:02]
C
Thanks for joining us. So we will put the links to reach out to Vanessa on X and also to Shopify and more information about their age on tick commerce on our show notes@productthinkingpodcast.com thank you so much for listening to the Product Thinking podcast. We'll be back next week and in the meantime, if you have any questions for me, go to dear melissa.com and let me know what they are. I answer them every single week and make sure that you like and subscribe so that you never miss an episode. We'll see you next time.