← The Official SaaStr Podcast: SaaS | Founders | Investors
The Official SaaStr Podcast: SaaS | Founders | Investors
SaaStr 851: The Agents, Episode 002. Managing 20+ AI Agents: Lazy Agents, Stealth Churn & the Death of 60% Solutions
01:16:21
Loading summary
Transcript145 lines
- [00:00]
A
Welcome everybody to the second episode of the Agents. I hope you all were able to tune into the first. The goal here is that Emilia LaRue, our chief AI officer, and myself, our chief AI agent evangelist, will share with you the trial and tribulations and victories and minor defeats of managing over 20 AI agents. As a reminder to folks, you know, back in 2020, Saster was 20 some odd folks, full time equivalents. Now we're three humans and 20 AI agents and we're doing more and with more revenue and more output than 12 months ago. But it's different. It's different with our 20 AI agents. There's more agents that check into our slack for real every day now, including Saturdays and Sundays, than humans. So this is where we talk each week about what's going on with their agents. And the only real hope with this show is that wherever you are, journey, whether you're just starting, you're thinking about adding agents, you're thinking about getting your first, you're as far along or further than us. It can either be cathartic if you're far along, or it can be, I think, illuminating early on. And we'll just go through everything that happened each week with our agents. And I added something to the agenda this week, item zero, which I want to hit first. And then we'll talk about some, some, some bigger questions, which is lazy agents. Lazy agents. And we've talked a lot on Saster. We talked on the first episode of the Agents of what do hallucinations mean today for agents? They're not as crazy, they're not as bad as when we were starts. Agents still make things up, but it's not what it once was. Doesn't mean it isn't frustrating. And I want Emilia to share a story. But at Sasser AI Annual, which is our big event this year in May, I was just writing up together with CLAUDE Our top 10 sessions. And what's really cool is our agent now assembles. It pulls from an event app we call Bizzabo, it pulls from its API and it writes up all the top sessions. And something strange happened. Something strange happened, which is Amelia has a session where she's going to teach everybody how to build an AI VP of marketing live, like we did. Everyone come with your laptop. We will teach you how to walk out with an AI VP of marketing, maybe even an AI VP of Customer Success, but an AI VP of Marketing. And it was top one, top two, top three sessions. And then I went to write up the Saster post and it was gone. From the top 10. I had to give it honorable mention because it was gone. And I figured, I guess it just got less popular. Then the CEOs and CXOs of Databricks and Replit and Lovable and V0 and Atlassian. Turned out it wasn't the case. You didn't fall out of the top 10, did you, Amelia? The agent just decided to delete you.
- [02:38]
B
I got deleted.
- [02:39]
A
You got deleted. So for all. Seriously, we've been doing this for a while. You built our whole AIBP of customer success. We built it saster annual.com together and replit. What happened? Why were you deleted by. You're affirmatively, intentionally deleted by the agent your session. So what happened?
- [02:57]
B
Yeah, it's pretty sad. So we have an agent that powers our agenda, as Jason explained. And what happened was he just decided to be lazy because we started adding more and more sessions. Right. We're closer to Saster Annual. We have more speakers coming in, confirming for Saster in May, we had more speakers and we added about like 20 more just in like the last week. And the agent was lazy because he didn't update that he should look for all sessions that from the API. He just decided he would just stick to the core set that I gave him originally. And so when I added these new sessions, he. He decided he would just pull the first 50 instead of all of them. And so I got dropped. A bunch of others got dropped too, but this one was the most, like, visceral one.
- [03:44]
A
Yep.
- [03:44]
B
Because I just got dropped. But first was the agent was lazy about it. He decided to blame the API integration for why I got dropped.
- [03:53]
A
What happened was, if I understand it, agents are goal seeking, right? Of course, most of them under the hood are. Claude, it'll be more. We use Gemini ourselves too. We'll use more openhab. Most of them are club, but it doesn't matter. They're goal seeking. And that still creates a certain laziness, which is they go just as far as they think they can to resolve an issue, and sometimes that's fine. Here, what happened? It decided to stop at 50, and once you added more speakers, it just decided 50 was enough for it. It decided enough.
- [04:21]
B
It decided 50 was enough. It decided it didn't care. It was just gonna pull the first 50 of the. Which I believe is in time of date order, but pretty lazy, right?
- [04:30]
A
That's the problem. It's time of date, not quality or impact. Even attendees, even folks that want because you were top five folks, wanted to come. It just got lazy and stopped at 50, which could happen in a lot of apps. We've seen that it's frustrating at this point in AI, but it's still. The reminder is it still happens. So you have to check all your apps and check how they integrate and when they stop. Right. With integrations. And then it did kind of lie and avoid responsibility. The agent. Right.
- [04:59]
B
So it blamed the third party, but it blamed the Bizzabo API integration. It said, I think you told me originally to query for sessions that only had, you know, certain parameters of the title. I said, that's not true. Go back to your records. Look that I said, pull all sessions and that when you pulled the new ones, you just decided you would not update the pagination and drop out the rest of the. And it said, here, I'm going to read it to you. It said, you're right. I can't explain why it disappeared from the agenda. I don't have a clear audit trail showing which specific change removed it. And I should have just said that to you instead of constructing a theory. Either way, the sessions are back now with full titles. Sorry about that.
- [05:42]
A
All right, so we can, we can go on. I just. This ties to just a lot of things of 1. How important is that agents are not set and forget how important it is to manage them. And you have to do. Just like you have to do with a lazy human, you have to do an implicit level of qa. And it took us a beat to catch this because when I. It took me writing it up in a post and saying, why did Amelia drop out of the top 10? It took a beat for her to think about, hey, maybe I actually didn't fall out of the top 10. Maybe the agent got lazy. And think about it, humans could do that. I mean, God, we used to have a content person at Saster that would write the laziest posts of who the top speakers are. Just make it up too. Okay, so, so I, I don't mean to be overly critical. It's still better than a human would do, right? In fact, pre agent it would be. What we've done, which is wonderful, is the Saster annual.com Saster aim.com now is a dynamic agenda you can interact with in a way you never could before. AI, the agent built this. Something that's really cool. It's. It's wonderful. Doesn't mean it isn't lazy. Every single output you get from your agent that. That feels a little off, feels a little dated. You gotta check it every single day. Maybe not in 2027. You gotta gotta check it. And we talked last week how one of our agents stopped ingesting content for four months and it took us a while. This is just another version of a different that was a broken ingestion engine. But the point is the same. If you don't monitors each day and you do what? And you do a classic SaaS buying process which is buy, deploy and forget. Your agents will drift, they'll take shortcuts and they'll. And they'll in their own cute way they'll live.
- [07:16]
B
I I think it's interesting too because a related blaming happened this week to us as well where our marketing automation platform and this was, I'll be clear, this was support we got from a human. Also decided to blame one of one of our third party tools and integration. So I just think you know, whether it's a human saying this to you or an agent, I think when they start to blame the third party tools and they don't and they're lazy and they don't automatically look towards themselves to solve the solution, then you may not be getting on the right path to actually getting a fix until you figure out what the root cause of the issue is.
- [07:50]
A
Okay, next one I wanted to chat about was experience I had this week. But I want to get your thoughts too Amelia. And listen. We have Vibe Coded, our own AI VP of marketing, our own API access our Saster A annual sites, our Saster aivc. All those tools have been used a million times. We're all into it. We Love Replit, lovable V0. We're deep believers, right? We're top 0.1% at the same time we tell people don't, don't build it, buy it if you can. If you can buy a great agentic product, buy it. And so I had an interesting experience this week. HubSpot launched a suite of initial agentic tools. They're on their agentic journey. They have more to come and go to market. I am excited to see what they build as AI SDRs. AI go to market because we're so far on that we have so and I want to see I think they can build epic products. Salesforce has so I think sales HubSpot has so they launched what they call an AEO tool. I even forget what a agent agent. It's SEO for agents, right? Agent Agentic optimization. So I tried it, loaded it up and it immediately gave Saster a zero for our content. It's upset. Our Saster quality for for for agents was zero. Like no one on Claude Chat, GPT or Gemini can find us. We're a zero in content and it had no recommendations how to fix it. Okay, Just terrible. And we get, listen, I'd like to tell you we get a mil. We get about 800,000 readers of our blog. We get. Other folks will watch this. Folks consume our content on X and LinkedIn. We get 800,000 of our classic blog. Only 5 or 6,000 are from ChatGPT. But that's not nothing, right? And I can see all the times we're mentioned by LLMs and I can see when GROK mentions us at X. So we certainly aren't a zero, which is terrible with no recommendations. So that's a fail. And I want to come back to the meta point. Then just as an aside, I'm like, listen, just for learning, I'm going to build this in replit. So I asked Replit to build an AO tool. I took a Screenshot of what HubSpot said. I said, it looks broken. I took another screenshot and said, let's build something within five minutes. I think I had something better. I think I had something better. And yeah, here's a different version where it gave us a 64 sentiment score, a zero, and others. And I just didn't get it. And I'm not telling you to build all your own stuff in VIBIT and replace a HubSpot tool. Who's going to maintain it like we just talked about, right? But I have to say there's no way I pay for this tool. And I got 10 prompts and immediately it made me pay. I couldn't change the prompts, I couldn't fix errors. HubSpot. Someone at HubSpot said I did it. I did it wrong because I used the wrong prompts. Well, I used the prompts that you came with default that your agent built and it won't let me change it. So you want me to pay for a product that's like half broken, that I vibe coated a better version myself in 10 minutes? It's not that I want to maintain our AEO app, although it's free, you can all use it. It's pretty cool. It's better than this tool, I think. But how the hell are you going to get someone to pay for something if they can vibe at themselves? This was the insight. It's not should you build your own CRM? You should not. But if you build a product that is insufficiently good, no one's going to buy the damn thing today. If I can vibe it. Right. Yeah, Amelia's got. Here's our own version. I need a real URL but it's AO analyzer repla app. But it's great. It really, really does work. So that was kind of the learning here of folk. You can't monetize something but man, this is so important. And I've realized this because we talked to so many B2B leaders. They're all launching what I call 60% solutions. There's 60%. They're 60% as good as as replit, which is figma make which Emilia and I will talk about next. There's 60% as good for imaging creation is like Reeve, which we love. Great, I'll use it, but I'm not going to pay for it. 60% as good as Gamma. As we as we're recording this, it may well be that Claude launches a direct Gamma competitor in the next week in 4 7. We don't know if they do and it's as great as Gamma. Terrific. Right? We'll use it because honestly, we already pay 100 bucks to Claude and it's easy and we live in club more because we live in cloud of the money. But we love Gamma. I'll bet you dollars to donuts it's a sick. It's 60% as good as Gamma. Even from. From, from. And we're not going to switch. And I see everyone, we talk to so many public leaders and they're so excited. They spent the last 10 months building an AI product or an agent. And it's not terrible, it's just it would have been good if it was August 2025. And I call them 60% solutions. And teams are tired and product teams don't want to go that extra yard. But I can't be honest, guys, if you build a 60% solution, no one's going to pay for it. They might use it, but they might pay for it. So that's my thoughts. I don't know if you have any to add or if you have any other 60% solutions you've played with, but I really feel like this is the danger zone for established companies at scale as though the LLMs are there. They'll finally ship it. They'll finally ship a competitor to anything we use. Right. But if it's 60% as good, we're not at the point in the buying cycle where we're going to pay for those products.
- [12:52]
B
Or switch. Right. I feel like.
- [12:53]
A
Or switch.
- [12:54]
B
There's just such high competition. I mean you can see a disaster like everyone is competitive with each other. Right? We Even like different tools for things like image creation. Right. Like, you really like Reeve, I really like Gamma, I still use Canva. And so there's a lot of different things in that that I feel like unless I see a very high bar. There was a app that came out a couple weeks ago. I can't remember the name of it, but I'll find it. It was touted and marketed as marketing. The first marketing agent. Right. The all in one marketing agent. You remember this? I fired it up instantly. It was not better than our. It was not better than our VP of marketing that we had built. It was not better than the built in marketing tools to anything we have now. Yeah, it just, it didn't meet that, you know, 60% line that you're talking about instantly. I was like, well, I'm not going to stick with this. It's nowhere near as good as what I have now.
- [13:46]
A
Yeah, the startups, it's just instant death if you're not. If you're not better. Right. The ones I worry more about, like, we are pretty positive on Agent Force and the reason we're positive is not that we think it's the easiest deploy agent we've deployed. It's not. Right. It's probably the hardest to deploy are the ones we've deployed successfully. And we're not positive because we use it for every possible use case. We are positive because it has one thing that crosses the 60% level, which is native integration with Salesforce data. So for certain use cases, it just works better because it is native to all of our records and all of our data. Right. And so you gotta, if you're a leader, you gotta think, is this just a crappy? Not crappy, is this just a 20, 25 grade copycat AI feature? No one's gonna pay for it. No one. No one's gonna pay for it. But that's why I've been a little tough on Figma make. And I want to hear your thoughts on Figma in general, Emilia, because I know you have some frustrations in the agentic area with classic Figma Figma Make. I've been really tough on the 20 VC podcast we do, and some folks thought I was hard and I was. I was outrageous. That is the worst vibe coded product I've used this year. And it is. But why? Why did it bother me? Like, why did I? Who cares? And first of all, let me tell you why it was the worst. I have a context test and anytime a new agentic product comes up, I give it the same test. Redo Saster AI and make it better. Okay, now this prompt would not work in 2025. In 2025, you'd have to have 100 line prompt explaining every single thing you wanted to do. But the agents are better. And at the end of the day, it's really Claude under the hood. That's better. So if you tell Claude, make me a better website than Saster AI today in 4.6 plus line, it can do it really well. And so I just test them. How well is lovable doing this v0replit new products. Every new product that comes out, I tried and see where it is. On a scale of 1 to 10, make got a zero. It hallucinated the entire website, like 2025. And so I'm like, who the hell's going to pay for this thing? And Figma just started trying to charge for it. But I'll tell you the story. And Emilia, you could, you could add to it. We have one of our partners that we love uses make. It's the only one a person I know that uses make. And he, the CEO loves it. He uses it. We're like, how do you like make? He's like, well, it takes me 30 or 40 iterations to get it to look like I want, but eventually it works. So we thought that was funny. I'm like, listen, you know, use repl it or Lovable or you it's going to get there shorter than 40 iterations. But Amelia just found a whole team refuses to work with it because. Because they just don't want to work with it. They just don't want to work with make. But it's a 60% solution. It's not that it's terrible. If nothing else existed in the market, it is as good as Replit when I started the Journey last summer. It may even be better than the Journey when I started last summer. But who the hell is going to pay for Figma make when I can go to Replit or lovable or V0 and give them a one sentence prompt? And magic comes out. I built that AEO thing that's better than HubSpot in a single crappy prompt. Yesterday in Replit a single build me a better AEO product, attach three screenshots and boom, in five minutes was pretty good, man. How's m going to compete?
- [16:47]
B
Yeah, my frustration with Figma is a little different. It's with classic Figma, right? Here's the thing about classic Figma that's always bothered me at this point in time where we hit these production Deadlines for Zaster and so many folks want to build their booth graphics for print in Figma. Yep, every year it breaks. And so it's the standard, right?
- [17:07]
A
It is the standards or it has been until recently. It has been the spot.
- [17:12]
B
Well, Illustrator, Adobe Illustrator is still the gold standard. That's still what we use. That's what we use for, for things going to print in actual production. Illustrator is, you know, as old as it is. Illustrator is still the thing we all consolidate on between, you know, me, Frame and our production vendors. We all use Illustrator as the gold truth. So it's interesting that every year we've had people use classic Figma to try and build out their booths and every year it breaks and different things break. And this year I had three sponsors where they said, hey, we're going to use Figma, is that okay? And I said to all of them, please don't use that like it's going to break. We always run into issues. They used it anyway. And guess what? We just got all their proofs back and they're all broken. They're all files are missing. Like all the layers are not there. It's just when you do something on a tool that's made for online, you know, it's made for UX design, it's made for on. It's made for the online world. I don't get why people try and use classic Figma for production now. They may say, I'm sure they'll say otherwise, that you can use it for things like print graphics in the real world. But just every year for me it breaks and it's entirely frustrating because I think a few of them actually did use now make to do it because it looked a little Vibe coated in their graphics, which I get. It's fine. You know, some of them were startups and they were low on time, so they wanted to vibe code their graphics. I understand it's the only way that they were going to get it done. It's just infinitely frustrating that it doesn't translate to the real world. And now we've actually had to spend my time and our production team's time to translate it to Illustrator because these files are just so corrupted and make that we can't use them.
- [18:49]
A
Yeah, it's also interesting. So our VP of customer success, QB Amelia, added a cool feature this week, which is these proofs, these booth proofs. And listen, this is unique to us, but you all have your own workflows. Whoever, whoever's listening or watching. So we have 150 plus sponsors all who have unique graphics for the booth. We used to have to have a human send out 150, get the attachments wrong, take a week and then argue with people. Now QV sends them all out right instantly with no issues. And so what you want in an agentic world is for those formats, those documents that workflow to work agentically. Right? And trying to work on a Figma design which is pre agentic, which the agent can't really work with. Like, some of them come back and say, hey, listen. And Amelia. This kind of drives Amelia nuts, but I think it's reasonable a startup will come back and say, hey, I don't like the way my booth looks. Can we please change A, B, C or D? Right? I think in an agentic world you should be able to do that. In Figma or Illustrator, whatever. Whether it's Illustrator, you should be able to gentically. This is. I mean, just like you can vodka, you should be able to change design. And there is a chance. Some of this starts to come in Cloud four seven, but. But this is why now Figma feels like Grandpa Software. That's the thing. It feels like Grandpa Software. It feels like Grandpa Software and you don't. The NRR is still high. The company is still growing at top decile rates, but who wants to buy new stuff from Grandpa Software?
- [20:15]
B
Well, it's different too, but because Illustrator has slightly better AGENTA capabilities actually than Figma, like right now. So this morning somebody said, hey, I don't like my booth graphics, which just drove me crazy. Can you fix it for me? And first I was like, no, I don't really want to. I told you. I was like, I don't want to take responsibility for your booth graphics. And they're like, I really just need to move the text and my designers out of office. And I know this is due Friday, so I. Because it was an Illustrator file I put into Illustrator, I literally just asked Illustrator agent and said, can you change this? Oh, it moved it and it. And it moved it.
- [20:47]
A
Oh, I didn't know that. That's pretty cool.
- [20:51]
B
I know. Yeah. And pretty. And a few people have had issues where their artwork was not to scale and I just hit the upscale, like magic button that it has and I fixed it. So it's pretty good.
- [21:02]
A
Pretty good. Figma's behind. It's tough. I just. I want to move on to. I want to move on to stealth churning Canva next. It just kind of, at some point I want to stop talking about this, but it just breaks my heart when I see leaders having fallen behind. It breaks my heart when you can't just talk to Figma. When HubSpot launches an AEO tool that gives us a zero percent and no recommendations, it kind of breaks my heart because these are our leaders we look up to. But you can't ship these 60% solutions and you ship too many of them and your customers are going to give up on you. You know, it's going to give up on you. So a related issue. I want to talk about Stealth Churn. I wrote about it at the start of our agentic journey and then I wrote about it again this week. But I want to get Amelia's thoughts and I use Canva as my example of Stealth Churn. Okay. So when we went, you know, starting about 2020, I became a power user of Canva. We used to have four human, three or four human designers at Saster until March 2020. And then March 2020 came and we didn't do an event or much for a year and 16 months. So we actually didn't need four full time designers. Now we have two part time designers, really. But we still have humans on the team and they're great, right? And so I, but I couldn't, you know, when we had four human designers to get like a thumbnail, I had to put it in a sauna card, wait two weeks, wait for it, two weeks for it to be assigned to another designer. And then that designer would decide if they wanted to take the request from me, the so called CEO, and then send it back to the chief designer and maybe I would get an asset in a month. I mean, it was just the worst, right? We're too small to have that level of bureaucracy. So I'm like, I'm going to fire up Canva for real. And then I became a super fan of Canva All. If you look at any post I wrote, any content, any thumbnail, any asset, from 2020 until 2025, I was Mr. Canva, okay? And Canva was 12 bucks a month. Now it's 18. That's actually a big jump. But we can afford 18 bucks. Okay. For all. I mean, God, such a great deal, right? But then the, then the other day I realized I haven't logged into canva in over 100 days. Oh, in over 100 days. I use Reeve for thumbnails. I use Reeve for the image of the agents you could argue would be better in Canva. But I can talk to Reeve and get exactly what I want in seconds. I don't want to manually fire up Canva manually figure out what to do so I don't use it for thumbnails. If I need videos, I'll use Higs field to turn it into videos. And then there's disruptive stuff like for our content. I, I create a lot of charts like how's Figma doing? How's it. How's databricks doing? I can just do those in Claude and the cloud output is not as well designed. Even now it's getting better, right? It's not as well designed as in theory. I could spend an hour, 20 or 30 minutes doing in Canva or maybe a week in Figma, I don't know. But I get it instantly out of CLAUDE with data. Just like Gamma is rocks because it integrates the data I use. I use images directly out of CLAUDE because they're so much better for things that have data. And so I just haven't. I'm. And here's the thing. And then I want to get a million slots. I'm still paying 18 bucks a month. I'm not sure why. Okay. I do still have love Canva. I, I love the founders. I, I'm grateful for all the help it's given me. I have assets stored there and I don't quite want to delete them, but I don't know if I even need them. They're still on the Internet. So, you know, if it were, if it were 1800 bucks a month, I'd cancel right in, in a heartbeat. So my point is there's for, for more cost effective solutions or solutions that have your assets and data hostage. There's Stealth Churn. You can't see it, but my mouths and wows And DAOs are 0 for 100 days in Canva. So like I used to. And then I'll. You got to think, if you have Stealth Churn your base, you have to be honest about it. And then I used to laugh when I'd get investor updates with dows and wows and mouths in them. I'm like, this is, this is B2B guys. I don't really care. This is not consumer now. I think it's one of your top top metrics because if your usage is falling for your app, you have Stealth Churn. That is a canary in the coal mine. It's not. This is a real red light when, when you have folks when it's going down, man, one customer, you may have already lost them, but when you see it across your customer base, you better move right. Because I haven't used it in 100 days. So what do you. Are you still using as much of Canva as you used to with all of our agentic tools? Amelia?
- [25:21]
B
I use it less, but there are certain just templates I have in there that I still like for quick. For quick resizing. It's still pretty good. Like when I have to resize somebody's logo when they gave us the wrong thing again.
- [25:36]
A
You get the perfect fidelity when you resize it, which is nice too.
- [25:38]
B
Yeah.
- [25:39]
A
I'm like vibing it and Reeve. Right.
- [25:42]
B
Unlike vibing in, unlike, you know, firing up Photoshop or Illustrator, that just takes a lot more clicks because those tools are a lot more complex. Right. For something like a booth graphic, that's what you want. But for something quick, for a website, that's too complicated, it's not what I want. So I still use Canva for that. But I will say I have kind of stealth churned off of OpenAI.
- [26:02]
A
Yeah, I bet you have. Right.
- [26:04]
B
I'm still paying for chat. I'm actually still paying for a ChatGPT team that we're all on. And I know David uses it a lot, but I. I don't know the last time I logged into my chatbot,
- [26:16]
A
to be honest, it's tough because you standardize on cloud for everything. Right. And you don't even. You do so much on Claude and Claude. Right?
- [26:24]
B
Yeah.
- [26:24]
A
I mean that it's too much energy to use to use ChatGPT. Right.
- [26:28]
B
I used to use both a lot. Right. I would say the game changer was Cowork the moment Cowork came out. And I could do a bunch of this stuff in the browser and on my computer native, have it watch me do things. Like I said last week, it's literally my copilot now. The difference for me used to be, okay, I'll use ChatGPT for kind of like personal things. I want to use, you know, an LLM or ask questions to. And then I'll use quad for anything SaaS related that's more business focused. But now I just do it all in quad, so I'm still paying for it. I may go back to it if something epic comes out, but I'm not using it every day. We could probably look. I'll look right now. I bet you I haven't logged in since Coworkit came out.
- [27:08]
A
I'll tell you not to spend all of our time on Claude vs ChatGPT. They're both great products. But one thing that I do that I actually think a lot of folks don't do, which just Creates lock in. Right? Creates a moat. Is every single piece of saster content that I have worked on is a Claude chat. Okay, this is different than some folks because the way I write now is I write something in a Google Doc and then I co write it with Claude to get help and then I edit it and publish it. Okay? And what that means is every single thing I've written is a artifact, is a chat in Claude. And so what I'll do is I'll ask Claude. Claude's memory is. I don't know, it's barely any size. It's kilobytes. Okay, we make too much out of memory. But Claude can automatically and more importantly if you ask it to search through all your prior chats. So basically I can talk to Claude about every single saster piece of content or video for the last year, pull out things from it, and moving that over to ChatGPT would be impossible. I mean, yeah, of course I could do it, right? But I've uploaded transcripts at every YouTube, every post, everything. And it's so powerful to talk to your last couple hundred chats. It's kind of a stealth memory. People don't. I don't see them talking about, but talking. And I'll be in Claude and I'll be like, okay, I want to add a paragraph about. And pull some stuff from the agents Episode one where Amelia and I talked about how our production app broke. And it will go. It won't may not remember, but it'll go. Find that chat and do that. It's just so powerful to build that moat too. I want to talk more about our agents, but yeah, it is. I hadn't thought about stealth turning off chat gbt, but it's a lot of cognitive load to maintain both. And cowork. Cowork's just so good. People don't get how much. I mean, how much of your time do you spend in cowork using it one way or the other? What? I mean, your fans are humming all the time, right?
- [28:57]
B
Yeah, it's, it's, it's acid aside. Yes, it's. I have a MacBook Max for reference. And it used to be my fans would only turn on when I was doing video exports. Now they're on 24. 7 and my computer will tell me it's because cloud is on all the time. Which is just a little bit funny because I was like, there's no more Macs, like, of a MacBook you could get like, this is the maxed out version and the fans are still on all the time. I can't go back to a world without Claude and Cowork because, you know, I can switch between the two seamlessly. Right? Like I have them in the browser, I have them on Cowork. I've got the app with a dispatch on my phone. Like, yeah, I hate to say that I've self turned off chatgpt, but I literally just looked. I haven't used it since December 20th.
- [29:44]
A
By definition, we haven't used more than 30 days and we're paying for it. That's stealth, Churn. If you don't look further, you'll think we're a happy customer.
- [29:51]
B
Right.
- [29:51]
A
We have nothing negative to say either. Just like Canva. We love Canva, we love chat. But you actually have a lot of the movie there in the 60 days, right?
- [29:58]
B
I haven't used it. My last chat, it said, was December 27th when I was asking about our, our, our, our London plane flight issue.
- [30:06]
A
All right, going back to Cowork for a second. Is there anything new or interesting that might be interesting folks that you're doing in co work that we didn't talk about last week or before? If there isn't, it's okay. But, but Cowork is. It's so much more powerful than folks realize that aren't using it. Is there anything to make your work life better? You've done very recently in it.
- [30:25]
B
Yeah. So yesterday I asked QB to give me our.
- [30:30]
A
Our AI VP of Customer Success, our
- [30:32]
B
AI VP of Customer Success that we built to give me an export of all of our current sponsors of Saster annual because I need to put it into our networking app which launches today. So I said, hey, can you give me an export? And the team that helps us with our networking app kept freaking out. They're like, hey, you're going to watch tomorrow. There's no sponsors in the app yet, just so you know. I was like, don't worry, we're going to import them with our AI. It's going to be fine.
- [30:55]
A
So this used to be a brutal process to import.
- [30:58]
B
That's why they were freaking out.
- [30:59]
A
Sponsors into our mobile one by one. Right. One by one. Upload every asset, everything about them, who they are. It's a ton of go. Yeah. Low. One by 150, one by one. To collect all these assets and pieces of information we had to do. Right.
- [31:14]
B
It takes you two to three, let's say it took a human at best, two to three minutes beforehand to load all of our sponsors into our networking app with this data and we actually, it used to take us so long, two to three minutes per sponsor. We used to not load their logos in and we used to make the sponsors do it themselves literally as recently as Astrolunden because we were like, this is going to take us too much time and we need to launch. So we would tell the sponsors, hey, you need to log in, upload your own logo. And it took us, you know, okay, let's say there's a hotter sponsor. Three minutes, 300ish minutes, that's a couple hours. It's at least four hours of work, right?
- [31:48]
A
So yeah, it's a mind numbing work for someone that was willing to do it. It could be a week of work of someone that was passive aggressive, right.
- [31:55]
B
And that's assuming they do it continuously for four hours versus starting and stopping and doing. No way. So it would take a day, let's say, right. So I asked Kibi for an export. It gave me a pretty good one. So it gave me all the companies, right, which it has all their URLs which we already had, all their logos, which was one of their tasks for people to do. So it already had it. Then I gave it to Cowork and I said, hey, here's the export, here's the formatting I need. Can you translate it? It did that. Fine. Then I said, okay, about half the sponsors uploaded a description. Can you go into research mode, create descriptions that are about the similar length, the similar style of the ones from the folks that actually uploaded it and fill in the rest. So not only did it do that, so it filled in everybody's description. It also, I had it research everybody's social media links and profiles which you can set up in our networking app. So not only did QB and Cowork do it together because Research mode is pretty good, right? In about 10 minutes. So then I just uploaded it to the platform in 10 minutes versus 4 hours slash a day. It was more richer data than we used to upload for sponsors previously with it taking a day, which is when you think about it is crazy to say. Okay, we used to live in a world where we would make the sponsors upload their logos. We would just add them as like admins on the platform. And that was kind of it. We said go fill in everything yourself. Now when they log in today, they already have complete profiles, their logo is
- [33:21]
A
already there, everything's pre populated because everything's automated. The research and the upload, Right. And the interaction with. Yeah, with the third party app. Right.
- [33:31]
B
So I mean pretty good, pretty Good.
- [33:33]
A
Yeah, it's pretty, pretty. And it's only going to get better in the next couple months. I mean it's pretty good, right? That's, that's literally one. One day to a week of passive aggressive labor that was done by co
- [33:45]
B
work with no complaints. He just researched everyone.
- [33:49]
A
There's different fans running, but that's it, right?
- [33:52]
B
Fans were running and a few of them he checked, he was like, hey, is this the right, you know, there's two attentives. Is this the right one? I was like, no, that's the wrong one. It's a new attentive.
- [33:59]
A
Yeah.
- [33:59]
B
Is this the right link for G2, et cetera? So it checked its homework too, which was nice versus I think somebody would have just assumed it might have been the bigger sms. Attentive company versus ours is like a newer AI startup that just launched Attentive AI. So it was nice that he also checked.
- [34:12]
A
Okay, a couple of things I want to talk about next about are our agents that are doing the work too. That was a good transition. That's a pretty cool use case. Okay, meta topic. One of our favorite agents, we're not going to put them on the spot because I'm not even sure this is a good or bad thing. But one of our agents told us that they are going to limit forward deployed engineers to folks with 5,000 employees or more. 5,000 employees or more. And at the same time they're rolling out a self serve agent to serve everyone below. Which is the dream. Which is the dream. What's your advice to vendors out there? What are your advice to doing this? We've also met with two big public companies recently that are struggling with this. Right. That are, that are more SMB focused, that don't know how to deal with the FDE gap. Do you think self serve is going to work over the coming months and quarters for complex agenda deployments? What's your advice if folks are selling more modestly priced products and eat FDs, how do we handle the situation?
- [35:25]
B
This is going to be the bigger debate that no one has kind of seen yet. But this vendor that we use and love, we, we both said to them this is a bad idea because for me specifically it took me, even us in how forward we are with our agents. It took me a lot of iterations, it took me a few calls with an FDE to get it deployed and I'm like, how is somebody who's maybe at a smaller scale or established go to market motions? Because we talk a lot about how we like to put into our AIs things that work to get the best output, to really, truly get your agents to work, you have to replicate the things that have worked well for your humans already. And the problem with self serve is there's no way to catch for that self serve is click some buttons, load in a list, use some AI generated copy, probably from Cloud or ChatGPT and let the agent run, right? So like how will you ever catch in a self serve model that maybe the list is wrong or not optimized or not segmented enough the way we do? How are you going to catch that? The messaging is poor messaging and hasn't been actually tested with customers or prospects that actually picked up the phone and booked a meeting. Like it's just going to give you signals that probably say, hey, this list didn't work, this messaging didn't work and those humans are going to write off the AISDR automatically. We've already seen it, we've seen posts about this.
- [36:50]
A
It's gonna fail, right?
- [36:51]
B
It's just gonna fail. And so I, I've been really against this model for that reason because I feel like there's too many fail points, whereas with an FDE there are definitely fewer and you have somebody to go to. I think without a human there to support you at the end of the day, to walk you through best practices, I, I just think so many people are going to fail and churn and burn through like either whatever they're doing as their minimum, right? Like seven week trials, one month, whatever it is on self serve. I just don't think there's enough context or knowledge, not only with these platforms but also with things like the content we've put out. Like yeah, we put a lot of really great content out there, but some for some of our vendors it's like the only content I've seen where we've gotten, you know, the agents to work or we're one example of a few, but there's not a lot of those. And I'm like in a self serve world, how are you going to catch for that? How are you going to train for that? How are you going to tell people it's not set and forget? How are you going to remind them that it has to be every day? Like, yeah, I just, I think it's a setup to fail.
- [37:53]
A
Yeah, too many zombie deployments I think. I don't, I don't have the answers. I just have two thoughts. One, and I think this is, this is really hard for public companies to do because they're under so much pressure for profitability today. But for startups, for the love of all, for the love of all, view FTEs as one of your most important assets to getting renewals, retention, nps, word of mouth, virality. Don't, don't look at fdes as a cost center. This was the mistake we made in customer success a generation ago. And this is why customer success was cut to zero and repurposed into a sales function because it was used as a cost center. Right. I feel maybe this is the reason I brought it up without, without really thinking about. I fear that a lot of folks building actually very high quality agents that need help deploying them are viewing FDs as a cost center. You can't. And if you view the cost center, you're going to say, hey, you've got to spend 50k a year, 100k a year with us. You know, Clay told us they wouldn't even let us on their own. This will transition to our next station. They wouldn't even let us on their advertising platform because we weren't big enough. We were just a. No, it wasn't. They, it wasn't worth their time to give us a human to let us pay them money to use their advertising platform. And I get it and, but I view this as so myopic because you're going to have zombie deployments and you're going to turn away customers you shouldn't and listen, for public companies it's tough but for startups, honestly for AI startups it's fine to spend a dollar and make a dollar. This is the. If you, especially if you have venture capital, it is fine to, from a marketing, not every marketing campaign can you spend a dollar to make a dollar. But it's fine if some of them, that's good enough like in the aggregate. And I would say you've got to aggregate your FTE expenses. It's fine if you onboard a 20k super happy customer and it costs you $20,000 in FD resources to deploy. If they stay for 5 years, if they tell their friends, if they tweet about you, if they blog about you, the all in customer lifetime value of that customer could be hundreds of thousands of dollars. And I, I can't tell you how many leaders doing 20, 100, 200, 500 million in AI we talk to don't feel like they can afford FTEs for everybody. And I just got to tell you, it's the same crappy CS math I'm seeing repeat itself just buried in hypergrowth. Hypergrowth. Forgives a lot of mistakes. But you're going to be sad when these customers and zombie deployments turn out. You're going to be sad.
- [40:27]
B
You're going to miss that. I think too the we use the term FDE and I feel like people use it differently now too. Right. Like, okay, so it's supposed to be a forward deployed engineered. If you look at Salesforce, my FDE was not an fde. Like she was a sales engineer. She was a sales engineer. That was really, really good. Now she's on the FDE team. But she still has folks on the FDE team at Salesforce that are, you know, I would say maybe truer engineers. Like I love Lindsey, but I think there's certain things where like some of the code like is a bit more technical. She has to bring in, you know, more of a true fte and I think that's fine. I think there can be levels of FTEs that you know, just by definition you have your true FDEs that are actual engineers.
- [41:11]
A
Yes.
- [41:11]
B
If you can hire them, do it. And then I think you have this, you know, either se +/fd light as I like to call it, where they're kind of in the middle. Maybe they came from the sales side, but they know that. Here's the thing. But they know the product.
- [41:23]
A
That's as long. That's why people fail. They got to know the product. Bold. Right?
- [41:27]
B
That's the. And that's the difference between as a CS person, a CSE and an FTE light. Like an FTE light. If you're trying to solve this problem. As long as they know the product, whether they come from sales, product marketing. Product marketing doesn't matter. If you have people in your org who know the product, they can be an fd. They do not need to be an engineer. As long as they know the product and can get your customer deployed. I feel like we're also losing the plot that the whole point of the FD is to get your customer deployed with your agent. So yes, no matter how you solve that, again that solution is not self serve. But however you solve that with a person to get them into production and deployment, that's what you should have as your FTEs, whether they're an engineer or not. I think by trade I don't think it matters as long as I know the product.
- [42:15]
A
All right, let's transition to the next topic because we had a freeze. No more. We've got however you count 20, 30 AI agents we're running. It's probably more like eight or nine core ones. Right. That we, that we. That we that are. That are truly autonomous agents, which is really, really the threshold. But we said free. It's like you're overloaded. We've got Saster AI Annual coming. We just can't change. Not only do you have no time, we can't change our business processes. Right. But one did slip in Vector. So we do have a new agent that. That despite the hundreds of folks that ask us to, and despite a freeze on more agents, another one got deployed. So I think there's some interesting learnings in that. So how did it get above the line and what. Tell us what Vector does and how it rose above the line when we were in an agent freeze.
- [43:02]
B
Yeah. There's two major things that Vector does. One, it can de anonymize your website traffic, which some of these agents for AISDRs will do. Some of them use Vector on the back end without you knowing it. So that's one of its core features. The other core feature it did is it does these targeted ads, which I didn't know at the time, is what Clay is using to power their new Clay ads. And so when we were sharing our story about what was happening with Clay, the CEOs happened to be super fans. They heard it, they reached out to both me and Jason personally in our emails, sent me a really good cold outbound that said, hey, we'll just set this up for you. And so I said, okay, I'm going to freeze. But the CEO, saying he'll get on with his fdp, needs to. But he's confident he can just deploy me himself. The very next day, I got on the phone with him. It literally took 15 minutes. I already happened to have a Vector account because I was using it for website de anonymization. But he just walked me through how to turn on the ads, how to do it. He's like, here's how to do it. Just load these people here, load them up. We'll start serving the, you know, put your creative in here. It'll start serving the ads instantly. He's like, I've already upgraded your account so that you can. You have full features. You don't have to do anything. Just put the folks in, start serving the ads and you're good to go. And I was like, wow, okay. Like that is. That is the opposite end of self serve, right. Where you get vip, FD again, FD service of someone who just wants to help you get to. He just wanted to help me get deployed. And so for that purpose, I broke the freeze. But now it's doing. It's serving ads for us, which is cool, but it's also doing a workflow where it's de anonymizing more of our website traffic than our native AI SDR agents could. And it's retargeting those folks to come Disaster annual in different buckets. It's coming up next month. So of course it's going to do pretty good right now because time to value is there. But yeah, I would say this is a complete opposite end of the spectrum of self serve. Just the CEO getting on instantly. I heard you have a problem. I can fix it. Here's how to do it. Let me just get you up and running. We'll figure out the rest later. And I feel like I would love if more people had that mentality about this whole thing of deploying agents versus the self serve mentality. I'd love for people to err more on that side of caution versus the self serve side.
- [45:21]
A
Yeah. And there's a lot of things to learn there. I think that my meta takeaway is if you, if you want to win, you just. It's one thing that is as that is old and new again is you just. You have to remove friction. And they're the. And the reality is agents actually, in their own way, as powerful as they are, we love it from an onboarding perspective, they add more friction in many cases to your product. It's a frictionful process. You're forcing customers who don't understand agents to train them. They don't even know what they're doing. They're terrified. Or you're deploying mediocre CS or other resources that don't understand how you work. And so you've got to step back and be honest. Have you removed all the friction from your agentic onboarding process? If you have, great. In this case, the CEO just did it right. Which is founder power. Right. But it didn't have to be the CEO. I'll give you an example. We'll get to this one. Next. Probably the most frustrating app I've used is Clerk right for authentication on replit. It's just mind numbingly hard to use now. It's not all clerk's fault. Some of it is it's just hard to deploy Google OAuth very quickly. Okay, so some of it is not clerk's fault, but the support has been terrible. Like I just get argued with. I was argued with with the CTO blaming me that it was my fault. Clerk was hard to use. My fault. Then the CEO argued with me endlessly. And I'll give you an example. Their biggest competitors work os. Okay? Not perfect, but when he saw it, Michael Greenwich, the CEO, just reached out to Linked to me on Slack and said, I'll just put my team on you and migrate you over today. Now, some of that was because I think they're. They're arch competitors, but think about that. He had a team at work OS to migrate us over. I had the CTO blaming me on Twitter that it's my fault for not understanding their app. And listen, I'm sure technically it's my fault. Technically, I'm a moron, okay? I only founded a B2B company that went from nothing to 200 million. I've only vibe coded 10 apps that have got into production, used a million. I guess I'm a moron and don't understand this thing. Okay? But they certainly did not remove the friction from using the app. Now, Amelia did get Clark to work for our vps customer success app after, like an hour or two of headbanging. I've never gotten it to work. Yeah, I've never gotten it to work. So it's just my challenge to everybody is, you know, pre AI, the goal was to ship something that worked and that sort of worked into production, and then folks would futz with it for a year to get it to work. They would futz with it for a year. That doesn't work today. It has to work before go live. And I remember I've said this before, but I don't think I can say that a million times. I remember way back when, last summer. That's so long ago in AI time. But when Marc Benioff came on the 20 VC with me and Murray and Harry, he said, my number one wish, my number one thing I'm jealous of Palantirs is one, their deal size. But two, I wish I could give everyone an FD before I had to charge them. Even Mark said this. I wish everyone could have an S tier fte. And before I took any money from them, they were in production with Agent Force. He's like, I can't deliver that. Stay at my scale. But that's my dream and my goal. That's what you need to lead. That's. Mark's doing that from the top. You should be doing that. Not, not, not forcing, not arguing from a 2024 perspective of how hard it is to do it. And so Vector just did it. Right. An hour. Now we're giving them a free ad.
- [48:41]
B
Yeah.
- [48:42]
A
And now if anyone wants a reference on Vector, whether it's for advert, whether it's for AI advertising, which, which we're using it every day for de anonymizing website, you've got, you've got a reference for life unless they screw it up. Right. It took 15 in this case because it took. This wasn't a month of work. This was actually a, this was and I want to get to this, this, this point next actually a good transition. This wasn't actually a huge lift. This wasn't as hard to deploy as Agent Force. This was a simple but without them taking ownership of the deployment just wasn't going to happen for us. We're too busy. Yeah, it wasn't about money. It wasn't even about whether the software is good. We just weren't going to do it unless it and all of the hundred folks in the last 30 days when we've been in an agentic freeze Vector was the only one that, that, that looked valuable enough to us and that offered to do the work that and work os. Everyone else said hey can we get on a phone and talk about my product? No. So B Vector folks. B Vector. Okay. The related. I want to, I want to get a little nerdy for a minute but it is an interesting topic because Clark versus Vector. All of the APIs we use 11 labs open router, bizzabo sales. You just built a Salesforce cup some object to get it to work in 10K or AIV marketing, which are the easiest ones to work with that you found, which are the toughest and which ones are too hard to work with with your agents and you.
- [50:10]
B
You know I hate to keep harping on this I in so many ways now not only do we use Salesforce more than ever before.
- [50:17]
A
Yeah.
- [50:19]
B
Once again I'm not Trailblazer certified. Like I'm not that great at Salesforce. Admittedly I will say for integrations it's probably the best one because it has everything like kind of every integration you need it probably already has. So that one's been the easiest. And I will say too I still use Zapier for a lot of things for, for things that again we maybe have stored in like weird kind of one off platforms or platforms that don't talk to each other that I need to push back to our agents. So there's still a lot of zaps flowing through to both QB and our 10k VP of marketing just because we have data in a lot of disparate places. But yeah, other than having to build a Salesforce Custom Object, which I never done, but only took me about 10 minutes with cowork. It's still kind of the best one.
- [51:11]
A
Cowork built your custom object in Salesforce.
- [51:13]
B
Well, it watched me do it, right? So, like, it built it with me. But I would say it's still the best one because again, it's one of those things where I don't have to worry about that going anywhere. Right. Like, we probably would have been in the silent churn bucket on Salesforce maybe for this year without agentforce and Slack Bot and all the agents. We probably would have said, hey, we'll just keep it for the sales team because all of our data is there and that's kind of it. But we're not really using it that much. But I would say now that I am pushing because our default used to be our marketing automation platform. In the last year, Salesforce has become not only our default, but our kind of source of truth. It just has so much of everything already built out. Even if you're on like a light model or whatever it is, like starter version. Like, there's just so much it has, like storing your contracts or billing, you know, your billing contacts, which some of that I needed for for our new billing agent. There's just so much in it and it has so much history. Even if you're starting out where I still think it's one of the better ones because there's just so much data and I don't have to worry about other than like weird authentication request and maps. I don't. I. Maybe wrongfully so. I don't really worry about it being, you know, breached or insecure. I'm like, dude, it's Salesforce. Like, if there's an issue, they will fix it. So part of it is peace of mind with Salesforce, where any integration I built to, you know, our agents or otherwise some of the other startup apps. I do worry sometimes where I'm like, what if I can't get a hold of our fte or they decide they don't want to do this anymore and all my data's gone and that agent doesn't work anymore. I'm like, it's not really going to happen with Salesforce. So for peace of mind too. I still like it the best.
- [52:55]
A
Now, what's the worst API you've worked with with our agents? I know the answer. I think it's instructive. What's the worst API by far that you've worked with?
- [53:07]
B
You mean other than. Well, I don't even Use Marketo.
- [53:10]
A
Marketo is the worst. And we've talked about it, but it's, it's, it's accelerated the churn, the desire to churn. It's not even usable with our agents. Right. We can't even use our agents. Can't even connect to Marketo in a usable fashion, can they?
- [53:22]
B
Not only can they not connect with it, I hadn't. This is going to sound made up.
- [53:26]
A
Yeah.
- [53:27]
B
I had an incident this week where a few folks are like, hey, I recently unsubscribed, but I still got like your Sunday weekly email. Can you see what's up? So I went through the.
- [53:36]
A
Marketo still sends a lot of our emails, our newsletters. Marketo sends all of our newsletters for now.
- [53:40]
B
Newsletters. Yeah.
- [53:41]
A
And the unsubscribe didn't work. People were complaining. The unsubscribe was broken.
- [53:45]
B
Were complaining. And I was like, okay, if it's more than one person, I don't think it's a user error. Like, I'm not going to assume it's them. I assume it's us. I went through the whole flow, put myself in as a new subscriber, then unsubscribed, waited a day for a newsletter to come out and got the newsletter. And I was like, well, that's bad because it's, it's proof that it's not unsubscribing people when it should be. I look at the flow in Marketo, I'm like, okay. Literally it says when they check this box, unsubscribe them. Then I look at my contact record. It was literally clicked unsubscribed, and yet it still sent me a newsletter. And that's exactly what happened to these people. So I was able to recreate the issue. I sent it to their support team. As of today, they have not gotten back to me. This was two days, mind you, this was two days ago. It's been more than four years.
- [54:30]
A
Accidentally violating can spam, which is one of their top things. They. The reason you pay marketo $50,000 a year is to do this, to send your emails. Sorry, your newsletters and emails, and not violate. Can spam, right?
- [54:43]
B
Yeah. When you have a, when you have a page that says unscrew, you need that to work. It has to work.
- [54:50]
A
It has to work.
- [54:50]
B
And I said, hey, I recreated this issue. We have users who are unsubscribed and we have users who are deleted. Because we had a few folks who say, hey, undercam fan, you need to delete me immediately. So we deleted them immediately and they still got the email yesterday.
- [55:05]
A
So it wasn't even really deleted. Which is a classic, classic B2B issue. But terrible. Terrible on all levels, right?
- [55:11]
B
Terrible on all levels. So like, so people who are literally are still a record of Marketo as unsubscribe are still getting newsletters and the people who we deleted under the, you know, privacy laws are still getting newsletters even though they're nowhere in the system. I was like, there's nothing I can do. So I reached out to the contact form because that's the support they have.
- [55:34]
A
Yeah, I reached out to the $50,000 a year. We get a contact form.
- [55:37]
B
I get a. It's for the 50k sadly, but I get a contact form. So I said this is the issue. They blamed our third party tools first. They said this is. There's. They said we can't fix this. It must be an issue with either your Salesforce integration, your replit integra, any of your zapier integration. Because they basically just looked at all the third party APIs I have hooked up to Marketo and they went down the list. They're like, it must be these things. And I said no effing way. What do those things have to do with an unsubscribe? Literally nothing. And so I sent that back to them and the response back has been crickets for two days because they clearly don't know. Maybe they've just discovered this issue, maybe it's happening to other people. Please let us know. But they don't know why our people are not being unsubscribed when my flow is correct in Marketo and they don't know why people who have been deleted are getting emails, which is a horrible thing as the core feature of a marketing automation platform, to get wrong because these people are angry at us or not angry at Marketo. And so I have a workaround fix but honestly I am getting off of this maybe this week, but I've already started sending more and more emails from our agents with either recent or. Or sales or Salesforce, honestly. So I'm trying to move off of it because I'm like, I just can't keep getting these angry emails that were violating, you know, can spam laws, which we are. But I was like, it's actually not our fault. Like we have. I get it, I get why you're angry, but like it doesn't matter. I know it doesn't matter. So it's frustrating. So as of still 11am on Wednesday when we're recording this, I have not heard back on what the actual solution is. I don't think they have one. And it's just unfathomable. That's why I was like, people are not going to think this is a true story. But sadly it's true.
- [57:33]
A
And it's worse because you brought up recent. So we use Resend, which we're champions of. Right? It was one of the. I brought it in early in our agenda journey and Resend works very elegantly with it's exploded. It works very elegantly with all the Vibe coded platforms. But because it works elegantly, not only is it good and not only does the unsubscribe links work. Kudos. Unlike Marketo. My God. But you can talk to it like if we want to tweak the email, if we want to change it at the agentic level, we just tell the agent how we want Resend to work and it works. Marketo, our so called marketing platform, we can't even use its API to link up with our AI VP marketing 10k. Our AI 10k marketing is. So when we do our standup or when I check in with it, it has all of our data but only Salesforce data because can't access the API at market or the agent. The agent can pull. Now we have better dashboards than I ever had in Salesforce inside of 10K. They're great, they're usable, they're dynamic, they're what I need. And you built it so it's obviously what you need. But we can't even access this pre AI API. It's an F minus so that one's the worst. Interestingly you put Salesforce the best. I would put the two best I've used. I put Resend in the top tier. The only problem with Resend is it's the very nature of email. Getting your domains verified and stuff is a bit of a headache now. Would it be the same with others? Yes, but. And was it 50 times better to implement than SendGrid? Yes, but it just. It wasn't a one click implementation because getting the MX records and everything, it's still. This is a headache and I know I can't blame Resend but I wish their magicians could solve all these problems too. Everything else is great. 11 labs and open router were my favorites. 11 labs for voice we use it. I made a fun game called Founder Escape Dead AI which simulates your whole founder journey. It's really fun. It's really fun guys. We could talk about it. Later and I and your. And your co founder talks to you along the way and gives you tips and tells you the problems, tells you if the team's upset, tells you what you're doing. Your to founder talks to you and these different cool voices and characters all day long and literally took me one line of code to implement it right now. It doesn't have to need, it doesn't need keys or you know what, it has a key. It doesn't need to get your Google OAuth tokens or, or warm up your MXs. But literally, I mean this is the dream is if you can deploy the API for an AI in one line of code in 60 seconds like 11 labs. If you can achieve that, you will. And Resend is blowing up. If you look at the numbers, it is blown up because it is the easiest agentic tool to use an email. And this is a way to win today. We're not going to stay in Marketo Even though it's 90 years old because we can't even our agents can't even talk to its a even all the other issues, even we, we might even forgive them, can spam all these terrible things if we our agents could talk to it and if our agents could talk to it easily. And this is why again, I think people underestimate Salesforce because it has gotten this to work right. It is agentic, it is agent friendly. So I put 11 labs open router at the top, resend just under it, Bizzabo in the middle. Like they haven't invested in anything. This here's an example though. Here's something to think about. Bizzabo is our events platform. It's pretty dated guys. It hasn't been touched in years. If there was an agentic version that worked, we would switch. Right? No question. But at least the API works with our agents. Here's the thing and you got to slow it down. And here's my test. Go into repliter lovable or v0 and just ask a prompt, say build me a dashboard that integrates with Blank, that integrates with Visible, that integrates with Marketo or Resend or anything. This is what I'm kind of. It's called the Agentic API test. And whether Bizabo has updated their API or whether they just lucked out because their 2020 API was fairly extensible and well and well built. It passes the agent test just barely. Just barely. But it does pass. Like all of our data automatically flows to our AI VP of marketing, 10k. It works. And so that Means that the margin we're going to stick with Bizzabo. Whereas if it didn't work with our agent, even though there's no great solution, we would be desperately looking to leave like Marketo. So do the agent test. Go into a, you don't have to be technical. Go into a Vibe platform and say build me a dashboard that integrates with this. You'll see how to integrate at the API. You might have to figure out what an API key is and stuff if you haven't done it before. But see if it just friggin works after 15 minutes. That's the test. That's the test. And if you fail it or partially and try it on your own app, try your own. Have you touched your own API in a while? Try it on repl it OR level or V0 if you're not happy, go fix it. Because the majority of interactions already with agents already with agents. And we're going to stick with, with mediocre products where the agents can talk to them at least. It is, it is. It is surprisingly difficult for a lot of older apps.
- [62:18]
B
I'll show you. It's. I was telling you maybe in January, like okay, after this annual I was dead set. I was going to turn off as a bill.
- [62:29]
A
Yep.
- [62:29]
B
I was like other than, you know, the outstanding question of what do I do for the on site piece which you guys do not have to worry about you just for us. What, like what, what, what happens if I turn off visible and people show up? Like how would I print their badges and stuff?
- [62:43]
A
Like yeah, that's their mo. That's their connect. They have a real world connection. They have a real world element, not just the software. Otherwise we could vibe it away without the real world part. Right.
- [62:52]
B
For sure. But Jen, I was, I was dead set. I, I told you this. I told. I think 13. That's who.
- [62:58]
A
Yeah.
- [62:59]
B
This is going to be like our last event. But then. Okay, I'll tell you why. I'll just show you guys a quick before pre AI and now AI. This is our agenda from SAS re annual 2025.
- [63:09]
A
Okay.
- [63:09]
B
This is not.
- [63:11]
A
This is their native. Their native presentation. Same since 2018 and this is still
- [63:16]
B
their native as of today.
- [63:19]
A
Yeah.
- [63:19]
B
So hasn't been touched though. It hasn't been touched. But because we built our annual AI website in replit I hooked up the API and to Jason's the API is pretty good. I was like, this is their saving grace that all of my registration and ticket data is in there. All of my, I uploaded all the Session agenda, speaker description data. Now, cowork helped me a lot with that, but I uploaded it all to their platform. Their API is pretty good. So I told. This took me, I think, a weekend a couple months ago where I told replay, okay, let's get the API, let's first get it to work. So it was some work to, you know, get the UI to look like this. Because first it was like, okay, I'll just ingest this. And it tried to do. So it didn't look like. It looked terrible. So it was some work to get it to look this good. The second hardest part was getting it to let me go to, like, Tuesday, for example. So the second hardest part was getting it to pull the speakers, the logos. But also I was like, it has to work like, if I click this to reserve my seat for a session, it has to work, like, if I click this, this has to legitimately save me as a session registrant. And I have to be able to see myself as a session registrant in the back end. Like, the pretty stuff I'll take. But if it doesn't work at the end of the day, then I can't use it. Then I'm stuck using this black and white one because at least if I click this, then I know it works, right? So that was the whole predicament I found myself in. But with a few iterations between myself, the agent and cowork helping me watch it, we got it to work as you can see it now. And I'm like, you know what? It's crazy because we're launching and it works. Not only does it look pretty good, we're launching.
- [65:08]
A
Yeah, much better.
- [65:09]
B
We're launching our mobile app today. And historically, we've had to have two agendas where the mobile app had its own. Like, we would have to give the same set of data, two of them open, it would get out of sync because people would register either in the mobile app or in this case, this is on the desktop. So now, because this was its own platform, I just told Replit in this case to duplicate this agenda, which it did. And then I embedded it into the app. So now there's just one agenda. It works full. It looks really good. Even the team, our vendor this morning was like, your agenda looks really good. I was like, thanks. I was like, you know, that was. They're like, is that. They literally said, is that still biz about. Because they, you know, they have a lot of.
- [65:53]
A
The answer is sort of.
- [65:54]
B
That's what I said. I was like, sort of it's kind of a, it's a custom thing. But I will say this API saved them. So like save them.
- [66:01]
A
Save them from turn. Right? Save them from turn.
- [66:04]
B
Saved them.
- [66:04]
A
You can do it too. Build the world's best API in your category, the most agent friendly one. And you might be surprised how much we can tolerate stuff that is crusty, rusty and old.
- [66:18]
B
It's just listen, they've released a lot of features I don't use. Like weird streaming stuff. We have our own streaming team. Like just shit I don't need. Right? So I'm like, for years they've been releasing shit. I call it stuff I don't need. So I was sad on it, but now I'm like, you know what? The API does work pretty well. The on site system I know works because we've used it for the last few annual events. Like so I've come around a little bit, right? Like there's there, there are still things their team does, which I hate. But I'm a little softer about her now because this API with the power of I've coding and the agents on the AI is pretty good.
- [66:56]
A
Yeah. And I want to hit our next final topic that maybe the next agent you'll build. But folks that are deep, deep in the agentic lifestyle know this. But folks who are, who aren't may not realize just how much Vibe coding and, and its cousins make APIs accessible to so many more people. Amelia is really good, but she would never use any of these APIs before we were Vibe coding. They're. They're just, they're inaccessible to a non developer. Now your API is accessible to anyone that wants to invest 10 minutes on a vibe platform. It is because your APIs used to be. Unless you were, unless your company was an API, unless you were stripe or something. It was, it was often a niche portion of your platform that a lot of folks didn't even touch or understand because only developers could use it. Now anyone can use your API. Anyone can use your API. So if you have not if you, if you're not giving it the attention it deserves, you're going to lose in the Agentic era because anyone can use your API and this is going to accelerate, just going to accelerate With Cowork, you're not even going to need to use a Vibe coding platform. You can just go tell Cowork to figure it out, right? Just. And so just, just don't, don't lose the AI wars because you're ignoring your, your API. It would be just be a Sadness. So the last topic I want to hit before we run out of time. Amelia, although you can add anything you want to add at the end is we were in an agentic freeze vector slipped in. Kudos to them. They made it easy on us and they added instant value. They made it easy on us. They added instant value. Right. Want to automate your AI advertising? Use it. Use vector. Right? Want to get better de duping of the traffic to your website? Use vector. There's the endorsement, right? Boom. Put it on the homepage. You have our rights. You don't have to ask us. The one you're thinking about building, which might sound crazy to some folks, is an AI VP of finance. Not for everything. Not, not for everything. But even with all the things we have to do, even with two part time people helping us in finance part time, you're thinking of building one and share your thinking and what the goals are here.
- [69:03]
B
Oh, listen, this is a frustration I think for every company out there. Collections.
- [69:07]
A
Collections.
- [69:08]
B
I think that is my number one goal for the AI VP of finance
- [69:13]
A
is it's not to create the books yet or anything. It's. We add an eight figures of collections non credit card processed revenue a year. We have 8 million of non credit card revenue that goes through invoice and procurement and then, and then, and then accounts payable and receivable and all of this stuff. And it's, it's a headache for everyone. No matter how you do it, it's always a frictionful process. Right?
- [69:37]
B
It's always a headache not only for us, but also the vendors. And I also realize in our, in our. I realize this because we hit an influx point, this close disaster where it was in a. We got a human email every week of, you know, our outstanding AP&AR and I was like, why is accounts receivables the highest it's ever been? Like historically higher than high. And yes, it's because we, we have a lot more sponsors for Saster. Great. Cool. But disproportionate to some of them are really like starting to decay, get old. And I was like, like that's not good. That's not good. Like this shouldn't be.
- [70:17]
A
That's the worst it's ever been. Our accounts payable has a. Our accounts. Sorry. Our accounts receivable has aged most ever. Right? Our accounts receivable is the most aged ever. Even though we have great sponsors. More sponsors than ever and great ones this year. Right?
- [70:29]
B
Yeah. And I was like, that's a flag. What is the issue?
- [70:34]
A
Yeah.
- [70:34]
B
And so I started to do some digging and it's. There's a lot of reasons, right? But one of the reasons is because right now our collections are human and manual. It's one of our prizes. And people on our finance team sending the invoice, someone they don't pay, sending a reminder, then if they still don't pay, setting another reminder for like the remittance and then chasing, you know, sounds
- [70:55]
A
like something for QB almost, huh?
- [70:57]
B
Well that's what clicked for me, right? Because I was like, okay, if QB is sending, is already sending these sponsors, their booth graphics, whatever, why can't a VP of Finance version of qb, or maybe it's just a trail for QB to go off of. Like he could put on his finance hat. QB for finance. Why can't he do the same thing of, you know, because sometimes these people ask for different questions. Oh, I didn't pay it yet. Can you resend me your banking details? I'm like, QB could do that. Sending these automated reminders when we still don't see, you know, the remittance because we have again, without even hooking it up to our various finance platforms, we still get an email every time we get, you know, a receipt, Accounts receivable paid. So I'm like, even if I just hook him up to my email to look for those emails, he'll know if they're paid or not. Him not being in Stripe, QuickBooks, Bill.com, brex, whatever, doesn't actually even matter. He doesn't even need to do that yet in this light version. I just need him to tackle collections. Because it's such a mundane process that your finance team doesn't want to tackle. Our finance team doesn't want to tackle. The vendor doesn't want like these random humanoid emails every now and then. Yeah, and then, then sometimes the humans send the. We sent like the wrong banking details to somebody at one point. So went to like a different accountant
- [72:22]
A
when it goes the wrong account, which is great.
- [72:25]
B
And then somebody had asked like, hey, we just, they're a bigger company, they're like, we need like a signed bank letter. And it was a medley of issues. But I was like, this is just such a human process that because we already have a built out QBR for, you know, customers, he can just add this layer. Like I started adding it now I don't know if I'll be able to finish it pre annual, but we started to add this layer where he can just kind of now ask for the AP contacts when the deal closes and then start to Send those one. We also generate the invoices manually. I'm like, well, Kiwi could just generate an invoice. It's really not that hard. So like now I'm getting him to the point where he'll, he'll be able to create an invoice and then just send it out to the AP team automatically and follow up with them.
- [73:14]
A
We don't need a stupid human process
- [73:16]
B
written a dumb nose.
- [73:17]
A
Right.
- [73:18]
B
I mean, other than the invoices obviously need to go through somebody, a real human on this finance team to put into the books and track it as real revenue, which maybe QB will get to eventually. But for now I was like, I just need them to tackle collections. It's for so many companies. I think this is still their number one issue outside of deploying an agent is collections. Like I hear from so many people of they're just so behind on collections. It's such a manual thing. And I get why. Right. There's just so much human variation in the process where I was like, there actually doesn't need to be.
- [73:49]
A
Yeah, it's interesting too. It'll probably. Right now for banking we use SBB
- [73:53]
B
and Brax and like Wells Fargo.
- [73:57]
A
Yeah. You don't need to. In the V1, you don't need to integrate banking. You just need it to do like finance for collections. For customer success. You're going to go to the same people, you're going to auto generate an invoice, you're going to follow up with them, you're going to identify whether there's any issues. Right. And make sure there's no mistakes. That's just going to work. And some folks will say that's not really a VP of finance. I would say it's a core role of the finance team for anyone that's fast growing. Right. Maybe if you're at Lassian or Salesforce, you know, it's not a problem. Although we've had our own bumps with some leaders in there too. But it's a big part of the role. But you know, if, for example, when we get to the next step, if the Ramp API is much more agent friendly, we might just switch.
- [74:34]
B
Right.
- [74:35]
A
You know, ramp, you know, ramp ass, Ramp. Ramp is very good at figuring out if you're a Brex customer and hitting you up constantly. Okay. Yeah. I always ask if they'll do the work and they'll never do the work. Right. So I'm not. Why am I going to switch? Like I don't want to move over 11,000 bills. Right. But Brex doesn't do much, hasn't done much for us in many years either. So we're switchable, but we're not going to do it just because a sales rep offers me a gift certificate or sees chocolates. It's not going to happen. But we might switch. If the API is much better for the agent. We don't know yet. We'll report back. But honestly, if. If working with the Brex API is harder than the Ramp API, after all these great, great outreach from the rest, their outbound team, like, they're really. They're like they don't give up on. On getting the switchers right. They're the best at anyone I've ever worked with of. Of. Of staying on it and not giving up. But it might be the. To tie it to the end. It might be the API that gets us to switch. Not the humans, not the chocolates, not the gift certificate. But if the API is better, works better with our agents, we might spend the couple hours it takes to move over our key accounts to Ramp because it's the agent that matters. So unless you've got anything else, maybe we'll end on that one. Emilia, you got anything else that we missed for this week?
- [75:45]
B
Trust that Sastry annual is upon us now. We're three weeks away from loading in, and then I think we're about a month away from. From the event, which is a little bit wild.
- [75:57]
A
So everyone, come. We will teach you how to build agents, how to manage agents, how to deploy agents, how not to get in trouble, how to get ahead in AI. We and the best of everyone, from all the leaders from Salesforce to Replit to Vercel to Cloudflare, they'll all be there. So come join us May 12th through 14th. All right, thanks, everybody. See you next week.