SaaStr 850: The Agents, Episode 1: Who Maintains All This?

Summary

SaaStr 850: The Agents, Episode 1 – "Who Maintains All This?"

Date: April 15, 2026
Host: SaaStr
Guests: Amelia LaRue (Chief AI Officer, SaaStr), SaaStr’s Chief AI Evangelist (“A”)

---

Episode Overview

The inaugural episode of The Agents explores the realities of building, deploying, and – most critically – maintaining AI agents and “vibe-coded” apps within B2B SaaS organizations. Drawing on SaaStr’s direct experience rolling out dozens of internal and customer-facing AI agents, the conversation focuses on key lessons learned, recurring challenges, and tactical advice for SaaS operators considering the build-vs-buy dilemma.

The central question: In an era where non-technical people can roll out remarkably powerful, customized AI-powered applications, who is responsible for sustaining, troubleshooting, and keeping these agents accurate—and what does real-life, week-to-week maintenance look like?

---

Key Discussion Points & Insights

1. The Build-Versus-Maintain Dilemma

• Anyone can build now: Thanks to rapidly maturing AI models and no-code/low-code platforms (Replit, Lovable, Vercel V0, etc.), non-developers can build powerful custom apps.
  • “I don’t think you have to be technical to create your own amazing vibe-coded app today. ...But you need someone pretty damn product savvy to maintain it, if it’s at all complicated.” (A, 00:01)
• The real work starts after launch: Most organizations underestimate the ongoing lift of fixing, debugging, and retraining their agents.
  • “Just like when you push an app out that finally works, that’s finally pretty good. Every week you have to maintain it. And who on your team is going to do it?” (A, 06:23)
• Build-vs-buy calculus: Even when building is cheap, the hidden cost is product-savvy talent for maintenance (not just functional owners like marketers or ops).

2. Real-World Bumps: Three Maintenance War Stories

A. Preview Environment Outage (07:22–10:39)

• Problem: Multiple preview instances had database connection issues – production was fine, but preview (where iteration happens) was unavailable for hours.
• Insight: Without clear ownership, teams end up in a Spider-Man meme scenario (“everyone points at each other”).
  • “It’s the spider man meme where you all point at each other and you’re like, well, whose fault is it, right?” (B, 08:54)
• Takeaway: If no one is clearly in charge of agent maintenance, small bugs can go unnoticed for days; those without a technical foundation misdiagnose and compound issues.

B. Persistent Micro-Hallucinations in Marketing Analytics (14:25–19:37)

• Problem: AI marketing agent ("10K") delivers fluctuating, inaccurate “fun facts”—comparing data to the wrong year, making up numbers.
• Process: Requires daily oversight and correction, or data starts to “drift” and becomes less reliable.
  • “Our AI VP marketing is very good, but...it has these micro hallucinations every day. ...I’m kind of maintaining our AI VP of marketing for maybe 15 minutes a day.” (A, 17:02)
• Caution: "Set and forget" yields data drift. If no one tends to the details, even a great agent slowly diverges from reality.

C. AI Model Regression: Silent Breaking (19:37–22:53)

• Use Case: AI Pitch Deck Analyzer tool—stable for months, then post-model update, stats hallucinate and overestimate startup metrics (e.g., defaulting to "$100K ARR, 500% growth").
• Root Cause: Silent changes to underlying AI models can destabilize workflows, even if your code doesn’t change.
  • “It was, interestingly, probably because of subtle changes to the model introduced in this complex workflow hallucination.” (A, 21:41)
• Remedy: Ongoing vigilance, not just in app code, but model monitoring, is mandatory.

---

3. When Vendor Agents Fail: The Clay.ai Pricing Fiasco (22:53–31:51)

• Situation: Clay agent (Sculptor) gave inaccurate, confusing, and more expensive credit usage and upgrade recommendations after the company changed its pricing.
  • “I literally told the Clay team, this is the first time I’ve wanted to rage-quit an agent.” (B, 22:53)
• Root Causes:
  • Agent not updated with latest documentation/pricing logic.
  • Defaults to more expensive actions unless challenged by a savvy user.
  • Recommends unnecessary upgrades.
• Broader learning: Vendors themselves often don’t keep their own AI support/training up to date, leading to customer frustration.
  • “If you don’t constantly train your agents, they will give your customers wrong answers.” (A, 30:10)
• User trust impact: Errors—especially those that feel “upsell-y”—can drive customer rage or churn.

---

4. "No Lead Left Behind": The Real Go-To-Market Impact (34:47–42:28)

• Breakthrough Realization: The true value of AI agents in GTM isn’t their technical wizardry; it’s that everyone, every lead, every customer, is touched and gets a response.
  • “If all...you deploy some of what we do with AI agents...here’s what you know: anyone that has an issue...gets to talk to an agent. And then a human, if needed. ...No one gets left behind. I think it’s that simple.” (A, 37:09)
• Human bottlenecks persist: Even with agents, bottlenecks or “idling” emerge when humans don’t keep up. The most scalable organizations find a balance.
  • “All of our agents end up idle...we have an order of magnitude more capacity than we had pre-agents and pre-humans...how to fully exploit that is [unclear].” (A, 40:20)

---

5. The State of Big Vendor Agents (Salesforce/Qualified) (44:31–50:31)

• Salesforce flips switch on Qualified: Now live as the homepage support/chat interface—faster to deploy, more accessible for non-technical teams than Salesforce’s broader AgentForce.
  • “It’s pretty cool that the day the deal closed, it’s in production—it’s on Salesforce’s website...” (A, 47:58)
• Lesson: Verticalized agent solutions (e.g., for GTM) are much faster for customers to deploy at scale vs. generic mega-stack AI overlays that require heavy configuration.
• Advice: For quick AI wins, deploying these plug-and-play tools (like Qualified) as native CRM/sales overlays beats overengineering.

---

6. Deep Dives: Saastr’s House AI Agents—QB & 10K

a) Integrating with Salesforce—It’s Not "One Click" (53:36–56:45)

• Hurdle: Native connectors (in e.g., Replit) can be misleading—tokens expire, corner cases abound.
• Pro Move: Creating custom objects in Salesforce can solve issues like 24h token expiry, but takes admin-level access and some “reading between the lines”—e.g., using Claude AI for step-by-step handholding.
• Time required: For SaaStr’s team—about 30 minutes; could be much longer for those unfamiliar with Salesforce permissions and API setup.
  • “You may not have all the right permissions...just by nature that may take you a bit longer.” (B, 56:45)

b) Multilingual Localization—20 minutes in a Car (57:54–61:03)

• Need: Last-minute sponsors from China/Spain found onboarding friction in English.
• Agent Solution: Within 20 minutes, prompted agent (via Replit + OpenAI) to localize and create toggle between languages, then QA’d via Claude and Cowork.
  • “We did it in 20 minutes in the car…That’s an AI story.” (A, 61:03)
• Learning: Automation isn’t always perfect on first try—more nudges required for menu content vs. deep text; agent should ask clarifying questions about scope/priority to manage costs and efficiency.

c) Games with Agents: Catching Human Workarounds (62:13–66:33)

• Human behavior: Teams/sponsors sometimes try to trick the system by uploading placeholder assets before deadlines.
• AI to the Rescue: QB checks assets instantly, flags incomplete/fake submissions neutrally, and emails all relevant contacts—more consistently and less emotionally than most humans.
  • “You might have been able to trick a human previously, but you can’t hide from QB.” (B, 66:32)
• Meta Point: AI agents can enforce processes with relentless consistency, without drama or emotional charge.

---

Memorable Quotes

• “It’s the spider man meme where you all point at each other and you’re like, well, whose fault is it, right?” – Amelia, 08:54
• “Set and forget does not work.” – A, 21:36
• “You might have been able to trick a human previously, but you can’t hide from QB.” – Amelia, 66:32
• “All of our agents end up idle. ...We have an order of magnitude more capacity than we had pre-agents and pre-humans.” – A, 40:20
• “If you deploy some of what we do in AI agents...no one gets left behind...I think it’s that simple.” – A, 37:09
• “I literally told the Clay team, this is the first time I’ve wanted to rage-quit an agent.” – Amelia, 22:53

---

Timestamps for Key Segments

• [07:22] – Real bug: database issues in preview environments
• [14:25] – Hallucinations in daily marketing metrics ("fun facts" gone awry)
• [19:37] – Model regressions break workflow in Pitch Deck Analyzer
• [22:53] – Clay.ai "rage-quit" story, agent misleads on pricing
• [34:47] – “No lead left behind”: How agents transform GTM coverage
• [44:31] – Salesforce/Qualified integration: What it means for AI-enabled GTM
• [53:36] – Building robust Salesforce integrations for AI agents in practice
• [57:54] – Localizing agent workflows into Chinese/Spanish live from a car
• [62:13] – Agents catch human “workarounds” on deadlines and assets, enforce policies neutrally

---

Closing Takeaways

• Maintenance, not just "build," is the unglamorous—but essential—work of deploying agentic applications.
• AI agents magnify both opportunity (scale, coverage, accountability) and risk (data drift, silent breakage, poorly trained vendor models).
• Organizations need a new breed of product-savvy owners—not just functional users—to oversee, debug, and retrain dozens of business-critical agents.
• Automating follow-up and enforcing consistency beats human “drama” and error-prone gaps—but requires daily vigilance and thoughtful escalation/escalation design.
• The next leap: using agents not just for internal process but truly for context-rich, adaptive customer experiences—across all channels and languages.

---

For future episodes: Expect more “bump” stories, tactical breakdowns of SaaStr’s AI workflows, and a pragmatic, sometimes humorous look beneath the AI hype. If you want to avoid the pitfalls and accelerate your own “agentic journey,” keep tuning in.

Transcript

[A]

Welcome to the official Saster podcast where you can hear some of the best Saster speakers. This is where the cloud meets up today on the Saster podcast. I don't think you have to be technical to create your own amazing vibe coated app today. You do not need to be technical, but you need someone pretty damn product savvy to maintain it. If it's at all complicated and we talk to a lot of folks and maybe this is the biggest meta takeaway, either they, they don't really plan on anybody maintaining a lot of agents or vibe coded apps, right? Or they push it to someone who's maybe very good at a line of a functional line. They're a decent marketer, they're a decent sales ops person, but they have no intuitive sense of how software is built or made. I don't think they could fix most of the issues we see. I don't think they would understand how to fix these production issues because they wouldn't be able to describe or identify them to Dagen. Hey everybody. Saster Annual will be back May 2026. The world's largest SaaS and AI gathering for executives with 68 VP level and above attendees, 36% CEOs and founders and 25 were AI first professionals. It's the very best of S tier attendees and decision makers that come to Saster Annual and AI Summit each and every year. Lock in your spot today. Use my code Jason100 for exclusive savings. Get your tickets at podcast.saster annual.com or just use code Jason100 when you check out. See you there. Saster annual and AI summit 2026. It will rock. Okay. Hey everybody. Welcome to the first episode of the Agents with amelia LaRue, Chief AI Officer of Saster, and me, Chief AI Evangelist at Saster. And this is a bit of an experiment that we hope work. But those of you who've been following us for the last 10 months on our agentic journey have consumed a lot of our content. It's exploded. Our page views are up 5x, our attendance disaster. AI annual this year will be up 40%. Folks are very interested in our agentic journey and we're constantly asked to do consulting calls and meetings. Literally in the last week I've done it with two CEOs of public companies. They want to know about our agents, how they work. We are not savants. We are not better than you. It's just we're like six to nine to 12 months ahead of a lot of folks. That's it. And so many folks ask me ask us probably 50 people a week if we can help them think about their agents. We can't do that. But what we thought we would do is have a little fun and maybe once a week share everything that's happening with our agents on the agents, every, all the issues and bumps we have. And if you're on this journey, whether it's just rolling out your first AI sdr, whether you want to build your own AI VP of marketing or customer sets like we have, whether you want to think about adding AI for support or whatever, maybe if you listen to this every week or when you have time, we can help you avoid some issues, think through things you wouldn't otherwise think and accelerate your success in the agentic journey. So that's the goal. If we have a tagline for this, it's, it's too long, but it's going to be accelerating your success on the agentic journey. And as part of that we are going to talk at least on the first one about some of our bumps. Because if you hear about our bumps, I think, I think you can avoid them. And so we'll do like six to 10 things that we've seen this week. Some of them may be very mundane, some of them may be topical, but hopefully it's helpful. The first one I wanted to hit, hit talk about and then get a lot of Emelia's thoughts on and which is okay, now you can build it, but who maintains it? Now you can build it, who maintains it. So if you've been following our agent journey since the last summer on the leading vibe code platforms and we love them. We love Replit, we love Lovable, we love Vercel V0. They're all exciting, you know, at first when I started it was tough to get an application out the door that worked in so called into production. And I struggled at first, even even having co founded a SaaS company that went from 0 to 200 million in revenue, I still struggled. As last 2025 went on, the platforms got better, the models got better, it got easier and easier and then toward the end of last year it got great. And if you're not a developer but you're on social media or you talk to your own teams, you'll see everything kind of exploded. When Claude Sonnet and Opus 4. Five came out in December or so of last year, productivity exploded everywhere. Every. It was just magical. You can see it because anthropic went from 9 million to 39 billion to 30 billion revenue earlier this year. It exploded the quality. But even if you're not a developer, if you're using Replit, Lovable Vercel VCR and others, you're benefiting from those models because they got better. And so what happened now is it's not that you can magically in one prompt, build your own CRM in five seconds. Like that was always marketing over reality, but now you really can finish your apps. And I kind of handed the baton to Amelia at the end of last year. I built rebuilt Saster AI. I started our new Saster annual website. I built our AI valuation calculator startups and used a million. I did all that and kind of struggled and went through the bumps. Amelia took over probably in January. Amelia with took over. Yeah, and built our AI VP of marketing on her own, which is very, very good. And then our AI VP of customers says qb, which is even better. So we're going to talk about 10k and QB like every week. All the bumps we have because they're part of our team, they check in with our slack every day. They are part of our team. And so you'll hear us. But my slightly long setup is now if you're persistent, you can build a lot of B2B and AI apps in these vibe coded apps without an engineer. You really can. You really can get them out for real. And the meta question is who maintains them? And this is what is under discussed. And I want to talk about three things that have happened this week for us and it's just something you're going to think about that a lot of folks, vendors don't really discuss it, people don't really talk about it. Put getting an app into production, get building your own app, vibing your own app. Is this, you know, just like sales, when you close a sale, it's the start of a customer journey. Just like when you push an app out that finally works, that's finally pretty good. Every week you have to maintain it and who on your team is going to do it? Who on your team going to do it? And I want to, and I want Emelia to talk about a few of the things we had. But I think this is the meta answer to the meme on social media of oh, we're going to kill Salesforce because I can vibe it. Okay, Even if you could like, which you can't, you can't do a sales source or a work day today. But even if you could, who the hell is going to maintain it? You, You? But you can buy a lot of these apps for like 5 to 100 bucks a month, you're. So Amelia and I are worth a lot more than five to $100 an hour. So this is so. So we're going to share our bumps maintaining it just so you can think about how you will. If you choose to build it versus buy it, how are you going to maintain it? So first one, just to talk about. And this happens, but just today on several of our apps, the preview instance had database issues. They didn't connect. They didn't connect. It's going to get fixed. Bugs happen, issues happen. It happens with lots of vendors. But just from an end user perspective. Amelia, just describe in general kind of the issue and what ran through your mind.

[B]

I mean I literally was exasperated at the agent. I think it's tough, you know, when, when one thing goes down or your site goes down, it's frustrating. And, and of course if you haven't built it yourself, you automatically go to the person, right? Like, oh, if my Salesforce isn't working, I'm gonna go to Salesforce. Is my website down? If did I. If maybe you have a Squarespace or WordPress, like you're gonna go to those folks first when one of those things is down. But then when you vibe something or if you have an agent that is slightly custom that you've done kind of yourself or you have your own hookups yourself, it's like, well, shoot, who do I, I was like, well, who do I go to? Like, like I tried to problem solve it first because I was like, well, I, I guess I can go to the kind of root agent and, and see if I can see if it's something I can just fix in the platform. And, and interestingly, the agent couldn't resolve it in this case.

[A]

Yeah. Sometimes when you're going to have bumps, you're gonna have to maintain. Right. And to be clear, this all happened on the preview environment. And for folks that have built software, you know, I'm oversimplifying, but there's basically preview environments, there's staging, we'll put it aside and development and production environments. Production. We had no issues on the production environments. But Preview is where you tweak your app. You iterate your app until you push it into production and that we were enabled access. It was unavailable for hours. Right. And not only was it down, but if, if that happened at Shopify or Salesforce. Right. Or even Squarespace, it's on them. Right. The difference here is that there's millions and millions of unique apps. So whose fault is it who's, who's the database Administrator.

[B]

I mean, in this case, I was like, it's all of us, right? Like, I literally told you, I slapped you right away. I was like, oh, I think one of the. I think some of the agents are down. Like, is it. I was like, first I thought it was just me. I was like, oh, it must just be me. Maybe you can still access it. But then when we learned it was you, it was affecting you too. It wasn't just me then. Yeah, you know, it's this. It's this. It's the spider man meme where you all point at each other and you're like, well, whose fault is it, right? Like, is it the customer's fault? Like, is it something that they. I kind of assumed it was our fault at first. I was like, okay, well, we built this. I'll just assume it's our fault and I should try and fix it. Then I asked the agent what to do. It started doing some lines of code, but then I realized in that process that some of my other agents were down right. In the same way. So I was like, okay, it's actually not just this one instance. So that was super interesting too. So then I went to our vendor that we use for this. But again, it's one of those instances where it brings up yet, like, who maintains this? And also, if. If you're not as in online as we are, right. We're. We're trying to keep up with our agents 24 7. Like, I notice this pretty much right away. I feel like other folks, if you don't have a database manager or somebody who's in charge of all the agents, you may not notice. It may lapse. Like, you may not notice for a couple days, especially if the front end is live, that some of your agents on the back end have broken. And so I think it's an important nuance now to start to think about who owns all this in perpetuity, not just. Again, to your point earlier, the hard part does seem like getting it, you know, fully baked and specked out the way we like it to do. We like to always use our agents for processes that already works. Like that is a lot of run up and lead up of work to get it already to that point. But then you have this whole other beast. Once you get your agents actually into production, we're like, oh, shoot, what do I do now?

[A]

Yep. So Amelia has now surpassed me in terms of her ability to vibe code apps. No, mostly she has. But in terms of this particular issue, it's interesting. I still have some Domain expertise again, having at least co founded an app to 100 delivery that I still understand some issues out there better than she does without me. Her initial conclusion of the issue was wrong. As good as she is her initial assessment of what was wrong. Then we asked the AI agent and it's pretty fun. When I started doing this, when I started vibe coding, you'd ask the agent, it would hallucinate. The answer it would make, oh, the answer is your, the answer is your, your DNS. It would just make it up. Now the agents. And we'll talk about this in a different context with a little bump we had with Clay next. But the agents are pretty self aware. They can analyze all the issues and it's caught you. They can help you debug like 98, 90%. So when you go into an app like Lovable or Replit or View zero, whatever, and there'll be that little prompt, that little agent, not only can you come up with ideas on how to build the site and features, but it can also help you debug issues. But it's not perfect and it doesn't have access to something. And so it tries. But all agents are goal seeking. They want to make you happy. So at the end they blame somebody else. They'll try to fix it. They always try to fix it. Now and then they're going to blame Claude or, or Lovable. This one.

[B]

Blame some apps. I had hooked up to it, right? It was like, oh, must it must be qualified.

[A]

Salesforce. They then qualified, which was not the issue. Immediately they blame it. Blame qualified as why it couldn't come up, right? And Emilia's like, well, we've been, we didn't change the code. We've been using Qualified for four.

[B]

I was like, nothing changed.

[A]

Yeah, yeah.

[B]

So I was like, are you sure, Agent? Because I was like, I, I haven't even like, you know, when I make updates to qualified, I made them in qualified. It doesn't really change how it integrates with this particular agent. And I knew they had just integrated with Salesforce this week. So I was like, maybe there's an outside shot they changed something on the back end. Like that was the only thing I could think of. But then I was like, okay, let's, let's for a second try and turn qualified off and see if that fixes it. But of course that didn't fix it. Then it started to blame other third party tools. I was like, yeah, it's not qualified. I think it just blamed that one because it's one of our beefier agents that this one hooks into. So it was like, let me just find the most common denominator and blame that instead of maybe looking for the actual problem.

[A]

Yeah. When we talk to folks, I don't think you have to be technical to create your own amazing vibe coded app today. You do not need to be technical, but you need someone pretty damn product savvy to maintain it if it's at all complicated. And we talk to a lot of folks and maybe this is the biggest meta takeaway. Either they. They don't really plan on anybody maintaining a lot of agents or vibe coded apps. Right. Or they push it to someone who's maybe very good at a line of a functional line. They're a decent marketer, they're a decent sales ops person, but they have no intuitive sense of how software is built or made. I don't think they could fix most of the issues we see. I don't think they would understand how to fix these production issues because they wouldn't be able to describe or identify them to the agent.

[B]

I think for somebody like myself, right, like, you've built multiple companies into, like, success and like, for me, having not had that background, right, like, came up through like, more of a marketing and sales go to market motion for somebody who's more so in go to market. When this agent started failing this morning, I was like, okay, if it's not our third party tools, which I don't think it is, that broke it, which we determined with the agent, it wasn't. If it's not the agent itself, I'm like, maybe it was me. Maybe I did mess this up, right?

[A]

I was like, or it could have been me because I did a. I did a little release on this app without telling you. I could have inadvertently. It could have been a regression, right?

[B]

Yeah, it could have been a regression.

[A]

I built a Friends of Jason little page that reused some of Emelia's discounts on my own little page. Now, how could that possibly break the entire site? It shouldn't. But when you get a little deeper on how software works, you understand that regressions, when one thing can break another, that don't even seem connected, right? It just something you got to get zen about. Maybe that was the issue. Maybe I broke. Maybe Amelia broke it. Maybe I broke it. Maybe the agent broke it, maybe the vendor broke it. And you got it. You. Someone's got to own this. Okay, two other ones on this. They sound minor, I would say, for the most part, related to this, generally speaking, hallucinations in our App are relatively minor compared to the old days. The old days, I would say of summer of last year, they're pretty minor. When we started using a lot of these Vibe code and agentic products, they would use more immature models, the threes in Claude and it would hallucinate a lot. It would just make everything up. But it was very frustrating and it made it very. It might have been one of the toughest reasons to finish anything, whether it was to get an AI SDR to work because it would hallucinate or to Vibe code an app because it would hallucinate. But they're still there. And I'll tell you just two stories from this week that were super frustrating. One is on our AIBP marketing 10k. So to describe what it does, Amelia took five years of our revenue data for attendees and sponsors for all of Saster annual $100 million worth of data, every data point and put and gave it and created an AI VP of marketing. So it has all these hard data points. It can compare year over year performance, week over week performance across any metric that you want. It's proactive now. It tells us every day where we're behind and what we should do. It has beautiful graphs and analytics. It's great. We've demoed it and we'll, we'll keep showing more. But I built this fun fact feature where it's to kind of push us. It has a fun fact. Each day pushes into Slack and it pushed and it sends us an email of a fun fact. It keeps getting confused what year it is and what data it's comparing. So yesterday it said we were 44% ahead of plan. And then this morning it said 11 and I asked the agent what what happened? It's like, oh yeah, actually I'm comparing it to the wrong year. And because I didn't have the right year, I made up the data.

[B]

Yeah, this happened before with dates, right. Like we had another agent before where it was telling folks still about London and London had already passed because it didn't know what day it was.

[A]

But this is making up a corner case. And then I congratulated myself on Saster a couple weeks ago is I built only Amelia and I can use it, but I built not only just 10k work, but I built a little bot, like a little fin like intercom type thing where we can talk, I can talk to it, I can ask questions. And it started to hallucinate too because it didn't have a full and updated answer to the question. So not only are Hallucinations still an issue. I guess the point of the story on this and I'll add one more and then we'll move on to the next topic is this is something we have to fix every day. Our AI VP marketing is very good, but it's not as good as our VP of customer success that Emelia built after for a number of reasons. And it has these micro hallucinations every day. And for a while, Emelia built it and I didn't really touch it. And I think she gave up on it. Whatever. Like, it's still directionally correct. But I really wanted to go deep. I wanted answers to questions every single day. So I would push it and these micro hallucinations would creep up every day. And so now I'm kind of maintaining our AI VP of marketing for maybe 15 minutes a day. But I didn't even do it two weeks ago. So Amelia's got her maintenance now. I have these micro hallucination 15 minutes a day. It's not massive, but if no one on your team is doing it, you see drift. The agents can become. Can drift more and more from reality and from your data. Inadvertently. You think they shouldn't, but they do. So this was my little issue. We're kept comparing year over year data to the wrong years. And literally in the same day it said we were 44 over last year. And then 11. I'm like, this morning I wrote which one is it? Like it. It can't be both, right?

[B]

I think this is why, you know, some folks are a fan of letting. I think of letting their agents do stuff on their own. Like, okay, like I'm gonna, you know, just queue up a bunch of stuff for my agent. I'm gonna let it run. I actually don't like doing that. Like, it makes me nervous if I don't see exactly what the agent is doing. Again, because of the goal seeking. I never. All these things we've built, you, I've built that I'm now in like all of our 20 plus apps we have for agents, whether it's in Replit or Salesforce or Slack Bot or anywhere else. I don't. Or Artisan. I don't let any of them run without me looking at it for its like initial execution. Like now obviously can send some emails to do some other stuff autonomously with a lot of guardrails on its own. But when it actually comes to me deploying like new features or fixing something in the app with the agents or putting in a new sequence of artisan, like, I don't let the agents do that autonomously at all. And I know a lot of people are a fan of the dispatch you can use on Claude or like openclaw. Like that's why I don't use openclaw. This is why I don't really use dispatch unless I can watch it on my phone closely with Claude because I want to see exactly what it's doing. Because a few times to your point, it has started to drift and go off the rails to start to do something I didn't want it to. And if I hadn't been watching the like timeline, we wouldn't have seen it.

[A]

Okay, let me just give you the third example from this week. This was a learning for me. This is a slightly unexpected one. It's model based regressions. And what I mean is this is. If you follow us, we have a whole suite of tools to help founders with fundraising. Go to Sasser Data AI, click on aivc. We've done almost a million valuations of startups and we've done almost. We've done like a thousand intros to VCs and we've done. Probably our most popular tool though is you could upload your pitch deck and we analyze it, we tell you how good it is, what, how well you're doing compared to all your peers and competitors and we tell you what your odds of getting funded are. Okay. It's called our, our Pitch Deck Analyzer. We just crossed 4,000 startup pitch decks. It's very cool. Maybe it would be simpler if I built it today. I built this a while ago, but it's pretty complicated. It requires two different passes through claude. Okay. And then after each pass it has to do a lot of error and data correction, data analysis and extraction. It's. It's the most complicated analysis that we do. And it took me like a month of uploading deck after deck after deck and manually looking at it and manually seen. And then I personally graded and made sure we were in lineman and they got all the numbers right. And it was pretty stable for a long time until maybe January and then again in February and then again like a week ago I started seeing errors. And what it kept saying is it kept saying all these startups had a hundred thousand dollars in revenue growing 500% again and again and again. I would get notified startups wanted to meet me. Once again, you know, whatever AI we're doing a hundred thousand five hundred, like this can't be right. And what I figured out is when there was an update to Quad and it probably wasn't just like 4, 7 or 4 or 5. It was probably a dot release or a silent release. All of a sudden, hallucinations creeped in, and the models decided that when it didn't have data, it would just decide. Every startup was at 100,000 or a million dollars in ARR, growing 500%. And it drove me, and I kept fixing it, and it would break again and again and again. And now I have to constantly work on this again. And one, I left it alone for a while and it worked, but then it broke, to Amelia's point. But also, this wasn't because I added a new feature or changed the code. The code did not change. It was, interestingly, probably because of subtle changes to the model introduced in this complex workflow hallucination. So again, 4000 decks, but it went from working perfectly to maybe 5% of them producing completely anomalous results. And so who's going to manage that? Right? Who's going to manage that? So three examples of why set and forget does not work. I don't mean to be too critical, but I want to set. I want to talk about something related to this. Not only do our own apps require maintenance, right. And have these micro hallucinations and drift. We had an issue with clay. And listen, we are big fans of clay for the most part. We use it for what a lot of you do. We use it for enrichment and to track where our attendees and folks have gone. And we blew through all our credits. And we're power users of clay. We support it. But its agent lied to us. Yeah, it's agent lied to us. There's a medicine story. So what happened, Amelia?

[B]

Yeah, I'll tell you guys a story. And it's all. Listen, it's all resolved now because this was a couple weeks ago. So I'll say, you know, all's well that ends well. But in the moment, I was so mad. Like, I was mad and frustrated at a point where I literally told the clay team, like, this is the first time I've wanted to rage quit an agent, which has never happened with all the 20 plus. I mean, I've tried more than 20. We have 20 now in deployment. Right? Like, I was like, this is the first time I've legitimately wanted to rage, quit. Because the answer that the agent gave me was so unacceptable, was so bad. And when I reached out to the humans at the first time, I couldn't get the help I wanted. I was like, that's a double handicap.

[A]

What happened? So walk back from a. What? What did the agent so Clay would announce cheaper, better pricing. For me, they made, they just made the pricing more complex and confusing. But they, they claimed it was confusing. But they claim cheaper for many use cases. Right. What do they tell you? Their own. So in other words, their own. You went on Clay, it has basically a support agent, right?

[B]

It has an agent Sculptor. It's to their, to their point. Yeah, they said this to me afterwards and made this point. It's still in beta, but I was using the Sculptor because I'm used to using agents and you know, the built in agents and all my tools. I'll give you guys the full story. So for the last couple of weeks we have these VIP summits we have for Saster Annual. I was using it to do lookalikes. I've talked about this before on Workshop Wednesday. I think Clay is a really good place if you're trying to scale up your AI SDRs or anything like that, where you can make really solid look alikes. With Clay, it works really well for this. And we were seeing success with it. And so I was using it for that, doing the same thing I had always done. It was kind of late at night on the Sunday. I was building a new list of, you know, VIPs to go out to for the summit. And suddenly Clay wanted to charge me about 5x to run this list than it did the week previous. And so I asked.

[A]

Hold on just for a second. Just so that Clay wanted to. Or its agent told you you had to pay five.

[B]

Its agent told me it was going to be five times the amount of credit.

[A]

Clay, the software application didn't necessarily charge you five times as much. It's a. I agent told you you had to pay five times as much, Right?

[B]

Correct. Yes. It told me I was going to have to pay five times as much. And so when I pushed the Sculptor agent.

[A]

Yeah.

[B]

I said, how can that be? I just did this on a very similar size list, a very similar arrangement, basically almost the same parameters. And it was, I don't let's say something like 2500 credits, but now it's trying to charge me 11,000, which is a huge, a huge spot. I was like, it has to be wrong. Right? It just has to be wrong. Because that doesn't make any sense. And so I asked it a few questions and first I caught that the agent related to their price increase changed how it, how it build, but it also defaulted to the most expensive action versus a cheaper action that would get me to the same outcome.

[A]

Let me. Maybe I misunderstood what Happened. Did the. I thought the agent wasn't properly trained on its, on the pricing of its own app. Both.

[B]

Okay, it's both. Yeah, yeah, it was both. So like, as I was talking to it, I was like, okay, one, I caught that it was trying to use this higher priced, you know, model to get to the same goal as last week. And so I said, hey, use this other action instead. It's cheaper. And so that lowered it by about half. Went down to like 5000 credits to run the same thing. I was like, okay, we're, we're halfway there. And that part of the issue I thought was problematic because most people won't do that. Most people are going to look at the 11,000 that the agent tells you and say that's too expensive and they would give up right there.

[A]

The first issue was the agent told you to use the most expensive model. The most expensive.

[B]

Correct.

[A]

So the upsell. The agent, the agent told you you had to use the most expensive version of the product that you didn't have to use.

[B]

Correct. I didn't have to use it. So I called him out on that.

[A]

The agent told you you had to, right?

[B]

He recommended. Yeah, yeah, yeah. He was like, well, if you want to achieve that outcome, this is how you should do it. Are you ready to deploy and run the waterfall enrichment? I was like, no, not for 11,000 credits. There's a better way to do this.

[A]

Do you think that was intentional or you think that was poorly training the agent? You think that was intentional to monetize the base better or do you think it was bad training?

[B]

I don't know because it honestly happened the same time they announced this price increase. So I was like, I don't know. I don't know if somebody as a human at clay, unintentionally like that was the consequence of them putting in this release for new pricing, that the agent took it and ran with it. Right. Like, I can totally see that as it wasn't malicious. It's just something that happened with their agent because they loaded in all the training for the new pricing. I was talking to the agent and the agent was like, well, let's use this super expensive, luxurious model versus like the normal, you know, Prius model you were using.

[A]

Yeah, because I have a rough. I know. And listen, we, we do use clay and it's a great product. But when they announce all this new pricing, I have a rule, a classic Saster rule that I think is almost always true when a company introduces more complicated pricing. Now, even if they say it's a better deal. It's always a hidden price increase. I've never seen it be the like. There are, there are occasions where companies literally say, hey, listen, we've just decided to go cheaper when. But, but. And it's usually when there's something brutally competitive, but usually when pricing gets more complicated, it's a way to disguise a price increase, even if that sometimes not everybody pays the higher price. But by adding complexity to pricing, you can essentially make segments of your users pay more. So it feels like that's what the agent was trying to get you to do. You try to get you to pay two to five times more than you had to, at least. Yeah, I didn't like it.

[B]

Right.

[A]

That was the first.

[B]

And it led to the set. Yeah, it led to the second issue, actually. So once I caught it in this, like trying to use a more expensive model, trying to burn through all my. At the time, I was like, I don't know, you know, I have like 3,000 credits a month. You're going to burn through my five,

[A]

five months of credit.

[B]

Yeah, basically I was like, no freaking way. So then I got it down to about half once I caught it in the agent, but then I could really only get it down to half. And I was like, you know what, I'm so tired. Heck it, like I'm just gonna, even though this is more expensive than it was a week ago, I'm just gonna push it through because I need to get this done. I was like, I just can't keep. You know, I had already been arguing with the agent for like half an hour at like 11 o' clock on a Sunday. I was like, I just got to get it done. And so I asked the agent to, you know, push the enrichment live with the 2x credits because I just had to get it done. And it said, okay, you're not going to have enough credits to finish, so you're going to have to upgrade. Because this is again, this is around the time they announced the price increase. So at that point I was like, you've got to be kidding me. So I just hit the upgrade button again. I'm tired. It shouldn't be on the customer to figure out tired or not. It shouldn't be on me to figure out that the new plan was going to be worse for this outcome. And I should have just stayed on my legacy plan at the time and just bought more credits, which in hindsight, yes, of course I would have done. But again, when the agent was suggesting to me, hey, if you just upgrade Your plan, you'll have enough credits. I'm just going to click that button. I literally, that's what I did. I. I clicked the button from the agent, I clicked the. One of the new plans or whatever and then I finished the enrichment and I went about my day and I loaded up with my aisdr. So it was a, it was a double handicap.

[A]

I get that. But just. And then we can move on to the next topic. But I thought the second issue when, when you reached out to the Clay team was that. And I think this is something we all need to learn about agents was they had actually not properly trained their beta agent on how their new pricing worked. Like they had not uploaded the documentation, the pricing. And we've seen this ourselves. If you don't keep, you know, digital. Jason, the digital version of me, it broke for four months. The old version, it didn't upload anything for four months. I didn't know it silently failed. So if you don't upload, if you don't constantly train your agents, they will give your customers wrong answers. So I think the second part was Clay changed their pricing. They didn't train their agent with the new pricing. So. So that's one of the reasons they gave you bad answers, right?

[B]

Yeah. Because when I, when I flagged this to the Clay team, I was like, hey, just so you know, like, obviously we use a lot of agents. Obviously for fan. I use a lot Varuna's in London. Like, we're fans. But here's why I almost rage quit, right. And so I sent it to both, like all the folks I know there's. And was like, you know, just this double handicap of it trying to charge me more, use a more pricier model then you raising prices. Then the agent telling me I had to upgrade to get something done, which wasn't necessarily even true. Right. It just told me it just made that recommendation. It's also telling me to use credits I didn't have to use. I was like, you may want to look into this because I don't know if other people would be as patient. I don't know if they'd say, hey, this is mission critical, I got to get it done. I'm going to use the credits and pay the extra costs. Like, I feel as a customer, like, I stuck with it, but I think other folks would be fairly frustrated. And so it's just a reminder for the folks listening, right? Like, I don't want you guys to ever. And we're sometimes at the situation too, to Jason's Point like we're sometimes on the reverse side of this where with our agents, it is a lot of maintenance, it is a lot of training to keep it up to date. But because they hadn't trained their sculptor fully through every scenario, which I know is tedious, right? It's totally tedious. Like, that's why. But that's the job. But that's why it was trying to get me down this upsell upgrade path when it didn't need to.

[A]

Yeah. I was another example. Then let's move on to the next topic. I was. I was using the AI agent on HubSpot's website on the homepage. Right. Massive traffic, right? Yeah. Trying to get a pricing quote for what HubSpot would cost at our scale. I just could not get an intelligent answer.

[B]

Oh, yeah, I remember this.

[A]

I don't know whether it was now. Now it might have been because they. They really want you to talk to a human. I think that was part of it. But it could have been better trained. And so all these stories are that some of the best folks, Clay here, to a smaller extent, HubSpot to others, like, AI support is great. AI agents are great. But someone, you gotta read a segment of these chats every day, like forever. You have to read these interactions with the agent because they're gonna drift, you're gonna forget to train them, they're not gonna ingest documents properly or you're gonna think they say things that they don't. Or your testing worked fine. And in reality it didn't. So it's just these are endless stories of agents not being as well trained as they could be that our customer facing.

[B]

On the flip side, though, we had somebody this week reach out to our events team, frustrated because they thought the answer they got from an AI was not good enough. And it's funny because I was like, sir, actually the answer you got was from a human being. We flip from.

[A]

Not from mistake. Or was it. Well, it was from a human. Right.

[B]

It was too. They made a mistake when they bought their ticket on Saster and then one of our humans on our side, minor mistake, super minor mistake. They. They. When they bought their ticket, they accidentally bought two. And so that is a human thing we have to resolve on our end. Like, I don't let the agents right now, any of our agents touch our back end of ticketing and stripe. Right. Like that's something we have to do and it should be something we do. So this was a flow where he. He said it to Amelia AI. Amelia AI escalated to a Human as she should have down this path. One of our humans, our events team, initiated a refund as they should have, the way they should have. The human on the attendee side said, hey, you refunded me, but not with the ticket processing fees, like, fully. It's this weird thing in stripe. There's like a 3% ticket processing fee.

[A]

They didn't get the 3% back.

[B]

They didn't get the 3 percent back because that's our. That's our standard is to. Usually when we do a refund. Now, this was a case where it was our fault because the website glitched. He got two tickets of the same. We should have just refunded that 3% in the first case. The team didn't want to bother me and be like, hey, can I have an exception for 3%? And so they just refunded him under standard. And this guy complained after the fact and said, thanks for my full refund now, but if this is how your AI support is, I'm really, like, worried about the content. It's answered. And I was like, sir, I understand, I understand. I was like, yes, sir, I understand the frustration, but it was us. It was a person, it was multiple, it was people.

[A]

Like, it would be interesting if Digital Amelia could have identified the issue of the strike fees that a human might not have thought through. The agent might have done better here.

[B]

Probably.

[A]

Right?

[B]

Yeah. And that's the irony. Yes. So it is.

[A]

Okay, the next topic I want to talk about. Sometimes I'm slow, guys. Sometimes it takes me a little while to figure things out. I want to talk about. I want to share a story from this week, but a lot of folks read what we. What we Write, especially on LinkedIn, where people. People, you know, really like to. To be Yahoos. And they're like, oh, Saster's results with its SDRs and GTM agents. Those don't work at my company. Those are unique to Sasser. Saster is such a strong brand. Or Saster does events and tickets. It's not the same of me. Or Sasser is exaggerating or whatever. Now we're just. We're just trying to help. But I see this. I see this all the time. And what I basically tell people is like, we're just doing the same thing you should be doing, and you might get better results than us. You might get worse, but try it, right? It's working for us. But I didn't completely get why it works for us until I had a meeting this week. Even after all these months, even after Agent Force, Monaco, Artisan, qualified, Momentum, all these. Why does it work? But I was meeting this week with the CEO and CRO of a great public company with thousands of sellers, thousands of sellers. And we were talking about agents and AI and you know, some folks, some folks do get a one on one version of this, of the show. And the first question the CEO and the CRO had is that, what do you think we should do first? And they had some Magentic products, but they said, what do you think you should do first? In order. And I went through a series of what I thought they should do first. First do something so that everyone can instantly get great quality questions answered on your website. Going to what we've just talked about, right? Make sure they can instantly get an appointment set with sales, right? And I kept going through different things and then I realized I went through all the things, I said, you know what? You know why all of our agents work in go to market and sales? It's because we lead. There's no lead left behind. This is the flip side. We talk about how agents give you more coverage. We talk about how agents don't mind the B leads, but it's simpler than that. If all, if you deploy some, just some of what we do in AI agents and Amelia can share some details better than I can. Here's what you know, Anyone that has an issue like with that 3% stripe fee gets to talk to an agent. And then a human, human, anyone that wants to know if they can get a discount, whether they get one or not can find out in real time. Any, any sponsor that clearly doesn't have enough budget to be at Saster can still find out if they can sponsor. But a human might be like, they raised $2 million a demo day. Like we don't want them. But the agent, the agent doesn't judge Bootstrap. The agent doesn't judge any issue. And then, you know, and then for our use of agent force or others, you know, we were inspired by Marc Benioff to go back to our prior leads and prior ones that no one had touched. Right. And so clearly we have evidence that in many cases our agents do better than a mediocre human. I know it's a triggering way to say it, but it is true. But even if it, even if it's not, it may be simpler than that. It may be just if you touch every single lead, every prospect, every customer, the way they want to be interacted with in real time, you have to do better. That's all these agents have to do to add value to the organization is that there's no lead, no prospect, no customer left behind. Whether it's in pre sales, sales, customer success. We can talk more about qb, our AI, but no one gets left behind. I think it's that simple. What do you think, Amelia? I know I'm slow, but this was my aha moment. And then with this public company we just talked about all the holes in their, in their funnel and they're as well as they're doing. There's so many ways to not get touched.

[B]

I agree wholeheartedly because I've, I've said now like there are times even with the volume we're doing between all of our agents, our agents still sit idle. Right. We do a lot of outbound, you know, all the normal, quote unquote normal go to market motions. That again, I think some, sometimes people look at it and say ah, that doesn't translate from staffer to me. I'm like well revenue is revenue and customer customers, like that's still the same across the board. And we still have a lot of the same go to market motions you guys are using. They're just sometimes employed a little bit differently or sometimes they have deadlines a little bit differently because we have events that are tied to it. But the behind the scenes motion is still very much the same. And even at our scale, I have not touched every single lead. I'll give you a good example. Just this week we have been in a production sprint because there were a lot of deadlines leading up to today and there were a few on Friday and there were a few on Monday basically for SAS re annual, that's coming up in May. And so we had these big production deadlines and so sprinting between myself, our designers, our production team, our customers, our sponsors to get some of their graphics and like we were sprinting to meet some of these deadlines that we had to meet because things take time to print and sometimes you can't pay rush fees to get these things done. But I'll give what it led to was a major idolization across our agents because I just didn't have time for them. This is one of those extreme examples too where like no lead left behind. Yes. But also if you have somebody who gets a little bit busy in a way a human can that agents you know are working 24 7, don't you run into this issue. I've run into this issue over and over again and that's why I'll keep talking about it. It's my Agents will start to sit idle. And this last week it's the most they've been idle since I've gotten them, you know, all up into production, leads behind. We're leaving some leads even with our agents.

[A]

But our agents are ready.

[B]

They're ready. Yeah. But I, I, I was literally looking at it today. I was like, okay, I have a tiny bit more time today and I've got to go get my agents out of idle mode. I've got to get them back into go mode because I have leads left behind for sure. A hundred percent.

[A]

Yeah. It's funny, it's a different topic and, but maybe one that this is worth continuing in subsequent conversations. All our agents end up idle. Right. It's an interesting topic, which is, you know, every human thinks they're so busy, even if they're really working 20 hours a week from Miami, but they think they're so busy they don't have time. They don't want to follow up with the B leads or the C leads. Agents are so efficient and we have so many. They're all of them are idle 90% of the time. They're sitting there waiting to do more. They're happy to do more work. They're sitting there idle. And it's a, I, I haven't fully thought through where that will lead over the next 12 months. But you know, in a sense we have an order of magnitude more capacity than we had pre agents and then we had with humans. Right. How to fully exploit that is the best way you can exploit it now is just make sure everyone that hits your website, everyone that contacts me, everyone in your database is followed up with in real time. But it's amazing how much idle capacity we really have. Right?

[B]

Yeah. And I think that's a, the website one is a new workflow we added, which we could talk about more later, but we added a newer workflow where when folks hit our website in particular for Saster annual, now they get like an automated follow up, but it requires some human steps to get that data from the website to the agent one because some of our, again, because of how we're built, we have to have this step. You wouldn't have to have this step necessarily, but because we get so much traffic to saster.com, we get not. That wasn't a braggy thing. It's just we get a lot of high volume traffic to saster.com, we get a lot of traffic to Saster manual and we have to filter it out. I can't to Jason's point. Could our agents hit up every single person that hit the website if we de. Anonymize them? Yes, but do I want to. No. This is the human orchestration layer of. Yes. No lead left behind. If I think they're a lead. Right. Like, there's still some qualification that needs to happen there. And I don't have an agentic workflow yet, but it's one of those things right now where we have to do some cleanup of that data because it's just so much data and kind of push people into different buckets. And so could we. Could we have no lead left behind on the website? We could, but I think that there's also an air of we have to filter out some of our stuff because not. Not everybody is a qualified lead. Right?

[A]

Yeah, I think there is. It's interesting when you think about what agents can do over the next 12 months. In marketing, I feel like marketing is pretty far behind sales. Right. Sales is behind coding and support. But marketing, man, we. The reason we had to build our own a VPN market is we couldn't find anything that's any good. Right. And I know folks are trying and kudos. And we're. If you literally can do better than we're doing, email us. But it has to be better. We'll use your product if it's better than we're doing. But we don't need something that just is a geo or SEO scraper or writes content. Like, we have a lot of content guys. Like, a lot of folks reach out to us as, hey, we can write social media content with you with our AI market. Like, we don't. We got 99 problems. That's not one of them. But it's behind. But, you know, so we get maybe half a million uniques to our saster web properties. I mean, people read content on LinkedIn, they read it on Twitter, they watch YouTube, but we get about half a million uniques to our websites, and those aren't leads. Right. But traditionally, if we could use agents so that everyone truly has a customized experience to them, we can convert more of them to leads. Right? Like, we need a. We need an agentic lead magnet where everyone, just like, now we have agents like digital Amelia that you can talk to if you want on our websites. But we should go further where. And you don't even know you have a lead magnet because everything is like, who are you? Okay. You know, Jason would love to have an ebook on how to do our agents. Amelia's pretty advanced. She's a Chief A officer, maybe what she would like is to invite to our CMO summit or our AI summit or something like. But if we had this hyper ability not just to. To use some of the tools we use which are really exciting like all of our tools, you know, whether you think it's. It's. It's. It's ideal or not, they can de anonymize your web traffic. They can identify better than ever these agents who's coming on your properties. But that's not as actionable in many ways. Right. As you might. Unless you're a spam farm. Right. You don't want to just collect people's random email, address them weird LinkedIn notes. I mean people do that stuff. But I, you know, I block people do that. But. But I think I am excited. But you could do something in the middle which is it's. You don't spam them, but you provide a truly contextualized value add during their experience or you push them at least into the very top of your funnel. These 500,000 people where they otherwise might. That would be that. That once you once. Once that. That's a. Maybe our current agents can't do that. But it's a meta idea of, you know, no, you know, no champion left behind, no visitor left behind, no anything that. That's the next. You got me thinking. Okay, next one seems small, but I think it's just interesting to note how are the big guys doing? And you know, we've been relatively close to Salesforce because we're relatively early on Asian force on deployment. We've done a lot of stuff together with Salesforce. It's been fun. And then I think maybe we even played a small role when Salesforce bought one of its partners qualified to accelerate its agentic efforts. We thought it was a great idea because we used qualified and we thought, you know, it has scale. So what the deal closed. What did you see actually happen at the product level this week, Amelia?

[B]

Yeah, I thought it was interesting. So if you go to salesforce.com, just the homepage or any page without logging in, you'll now see a qualified agent versus it used to be a agent for like support agent. Now they've swapped it over to qualified, which I think for a lot of reasons was the right call. Right. It's. I think it's, I think it's part of why they bought it. Right. Like in many ways, some of this, some of the features and functionality that qualified had just went a lot deeper than what AgentForce kind of natively had like that you could get there, but it would just take a lot of. A whole lot of like extra steps. And so again, I think that's. I think they were so smart to buy it because it has all this and because you had to be a Salesforce customer to be on Qualified anyway. I think it was. And. And a lot of the team came from Salesforce. Right. But I think it's interesting because when that module was on qualified.com, it was a real person Blake that they modeled, you know, Piper after who's their. Who was their mascot. And now it's like a 3D avatar version which again, I think is interesting. We've talked about it a little bit of like, do you have, you know, the rights to that person? If you use a person who's not the CEO or founder, there's a reason why, you know, Jason AI and a millionaire us, like we're not going anywhere. Like we have all the rights to it. Like we can still use our agents and our likeness. But I think it was interesting that probably for a cya, if I had to guess that Salesforce changed it from the actual human avatar that it was to a modeled like cartoon version of the same person. Probably for a lot of different reasons there. And also just for speed. I think for essence of speed, like they already had the other video model, but I think to also signify that it's now Salesforce and you know, they have a lot more customers and a lot more security layers that they have to go to. They changed it to a cartoon version of the same person. Again, I think it was smart to get qualified on the website. The day the deal was pretty much done, they flipped the website. There's so many deep features and qualified that now the Salesforce team can action

[A]

on the interesting takeaway. Just we're trying, you know, Salesforce built agentforce and they've got thousands of people working on it. We use IT and it and it and the most important thing for folks is it does work. It does work. But because it's so extensible and so broad, I mean Salesforce has like nine different clouds and so many uses. It is work to configure it and it is. They bought qualified, which is just a GTM tool and classically was just to qualify. That's the name of the company Inbound. Right. So it's a very narrow set of now does some outbound and other things.

[B]

Yeah.

[A]

But it's just a GTM agentic tool. So for their. Especially for their CRM customers, which is still almost 20% of the revenue. Now they bought a tool they can roll out to the entire customer base. Does it do as much as Agent Force could do? No, but it's pretty cool that the day the deal closed, it's in production, it's on Salesforce's website, and now there is an option to buy a Salesforce native agentic product that is much simpler to deploy. And if your goal is gtm right, you could have it up and running probably in a couple weeks rather than maybe a longer period it would take to map out and figure out a full Agent Force implementation.

[B]

Oh, for sure. Like, I'll say, like, for my, for myself, I had a much steeper learning curve with Agent Force than Qualified. Qualified. The UI feels more like that. Right. So I think part of why they did the deal too is okay, if you want to unlock this whole pool of people where Agent Force historically has been, you know, success is the number one use case of Agent Force. But now you want to go more go to market. I think that's why they've got both qualified and momentum together. Right. Because now you've got rev ops for salespeople and you've got this whole GTM kind of suite that is easier for folks to use, better to use out of the box. I think for GTM folks, like I talked about last week, I had to build this like Salesforce custom object for one of our agents. Pre Agent Force, we weren't using our Salesforce that much. But also, I'm not like Salesforce Trailblazer qualified. Like, I was like, I'm not that good. Like, I will confess to anyone listening, like, I wasn't that good at Salesforce. I've gotten a lot better in the last nine months because of how we use Agent Force. But a lot of the times I am, I like look for like, I look for either loopholes or other things that can integrate into our Salesforce to make it easier because there's just so much data in there and there's so much you could do with it. That's, that's why now I use a lot of Slackbot too, because it can do some of it a little bit easier. But again, I think that's why it was smart of them to get qualified up and running instantly. I think it's smart of them to expand and this kind of like go to market area where it's, it's easier for folks maybe to understand, you know, qualified from Salesforce. That's just your go to market agent like where it blends with Agent Force. Who cares, right? I don't care. I told the team to just. Just let people pick what they want. I was like, if you can do it in a way where people can pick either the qualified UI or the Agent Force traditional UI that it has, like just let them hit a toggle and say, I'm going to use the quality. Yeah. Like let the customer, you know, again, go to your customer. Let them pick what's easiest for them of how they want to set up the agent so that they actually get to deploy to agent in production.

[A]

Yep. So anyhow, I want to move on to our. Sort of. Our last and final topic that maybe we'll come back to every week. But. But I think the meta thing is, I think, listen, there's qualified has. There's other folks like Qualified. There's a number of vendors that can do a great job with Inbound and Outbound, but we've used it. Right. And so now there is an option that's natively in Salesforce. It's on their homepage where you can deploy this much more simply than Agent Force. If you're deep on Salesforce, like, go for it, get this thing up and like it's. It goes back. Like, whether it's the perfect vendor or not, it's native, it's built in. And our theme of no prospect, no customer left behind. This is one thing you can do now, get it deployed. And this is, you know, just like a lot of CMOs have patted themselves on the back the last 12, 18 months rolling out clay. Right. And kudos to them. If you're in rev ops or sales Ops or sales and you want to build it, roll out an agentic product and Salesforce just. Just do qualify like this. What it's still called qualified. Do they change it to qualify from Salesforce, Qualify for sales support. But this is. I think there are, there are. And then maybe the competitors are better. I'm not. I'm not picking sides at the moment, but this is a quick win. This is. This is something to think about. Okay. The last thing I want to talk about is our team members and if. And maybe we'll do this each week at the end. QB and 10K, what's up with them? As a quick reminder, QB is kind of a play on QBR. It's our AI VP of Customer Success that Amelia built. 10K is our VP of Marketing. Her initial job was to get us to 10,000 attendees in the first 10 million of revenue this year. So we call them 10K. Now they've all joined slack. They're up early on slack. We now have more daily check ins on Slack with agents than humans. Which is the future here today. If you go in our Slack like our Slack, just like our Salesforce almost died before agents, our slack almost died before agents. Then we added Slack integration for agents. Now they're talking on slack almost too much. But they're better check ins than any human ever did. Right. Any check ins. Okay, so a couple of things on QB and 10K. First one, because you brought it up, you built and maybe just describe the challenges and the opportunities for our. For you added Salesforce integration to 10K for sure. Maybe maybe 2B2. Right. And it was interesting. So we did it in Replit. And Replit has a native Salesforce integration. Right. But that alone didn't quite do what we needed to do. So you had to build a custom object in Salesforce which if folks have been in the Salesforce ecosystem, you'll understand what custom objects do. But this was some learning, a learning curve and work for you to figure out how actually to get the Salesforce integration to work. I want to hear about it. But to give folks context. For me, I don't log into Salesforce, but when I log into 10k, I see all of our pipeline and everything over history and time presented a way a normal human would want to see it. So a Salesforce is now abstracted away from me. I see all of our Salesforce pipeline opportunities, everything done beautifully and visually. And I can drill into it right inside of our AI VP of marketing. Which for me is great. Okay. But you had to build it. So it was different than we'd thought. Right. With the current integrations.

[B]

Yeah. It's just listen the learning with any of the. I think any of the Vibe coding platforms with Salesforce in Barticular again now why it happened. But it was frustrating for a few days because I just used the native because again, why wouldn't I. Like if you're Vibe coding, one is

[A]

you click on a Salesforce connector, you put in your credentials or your login. Then you go to. You just log in once. Right. And it gets a token. Problem is the token expired every 24 hours. Right.

[B]

Every 24 hours. Which I didn't know. And so. And the agent didn't really know it either. It like took it a couple days of it failing.

[A]

Probably it's not a core feature like the clay thing. It probably wasn't well trained on how the integration worked because it's a. It's a corner. It's not a high volume integration.

[B]

I was like, okay, there's gotta be a way where the token doesn't expire every 24 hours. And it was just building a custom object app because in a custom object app, which I asked Claude how to do, super easy. Like, Claude gave me really good instructions. And then I loaded up Cowork so it could watch me do it. Right. I was like, just watch what I'm doing. Tell me if I go wrong.

[A]

Cowork was watching you manually build the Salesforce custom object?

[B]

Yeah, it was watching the screen in the browser. Yeah. Yeah, it was watching.

[A]

That's pretty cool. Yeah. Did it catch errors while you were doing it?

[B]

I didn't do well. It was pretty easy, so. Well, he gave me really good instructions, so nothing bad happened. It's.

[A]

So Claude gave you the instructions to build a custom auditor. Then Claude, Cowork watched you while you did it Again, you're ahead of me.

[B]

She was like, you're doing great.

[A]

I wouldn't have thought to have Cowork watch my work while I'm doing it, but that's pretty cool.

[B]

Yeah. You know, it's funny. I haven't had Cowork watch replit or anything. I should do that. I'll do that next time before or next episode. So, yeah, I was. So I deployed this custom app object, and Claude and Cowork were walking me through, okay, here's how you can set it so that you don't have to. Because I did get. I got stuck a couple times. I was like, you know, Claude, what you told me to do is not exactly where you told me. Like, it's using just kind of like, you know, older UI data. So I'd be like, is this the right button? Is this the right box to check? Is this the right field I want? And it was like, yes, yes, yes, that's fine. That's the one you want. And I was like, shouldn't I do this one? It was like, yes, you're right. Do that one. That's why I'm watching you. And so I did it. And it. And then I found in building the custom object, a way that we could have our token refresh every year versus every day. And now our Salesforce integration has not broken since I did the custom object.

[A]

Cool. Well, it shows. So the good news is it worked. And it's epic. It's cool, right?

[B]

It's epic. Yeah, it's fun.

[A]

But it was harder than we thought. Like, not every. There's some integrations, like, my two favorites, vibe coding are 11 labs and open router. They're so easy. Like you can integrate products in like 30 seconds instantly.

[B]

Yeah.

[A]

And it's a credit to both vendors to 11 labs, but it's also a credit to the fact that how they work, like it doesn't have to be as complicated as Salesforce or Clerk just by nature. But, but, but, but realize that not all of these are one click simple, even if they look like it. Right. They are complicated. And this was probably. Was it a couple hours of work or. How long did it take you to really get the Salesforce integration working in 10k?

[B]

Half an hour. Once we figured out, like, I would

[A]

have thrown my mic at the monitor because, because I, I am. I don't want to build custom objects. But, but, but again, for you it's half an hour. For someone that has never really used that side of Salesforce, it could be a bigger project though, right, to figure it out.

[B]

Yeah. Or you might. Or you might not have access. Right. Like I'm, I'm our Salesforce admin and so I have all the right protocols and permissions to do that. Now if you're building this, you're not one of your Salesforce admins or you have certain like company restrictions for, you know, because it needed API keys and off. Off credentials and it needed a lot of stuff to get those custom app options. It may take you longer just because you're going to hit blockades. Right. You may not have all the right permissions or somebody else may have to do it for you. And so just by nature that may take you a bit longer.

[A]

Yeah. Okay, Last one that we had with QB and 10K, when you were vibe coding in the car and we were taking a waymo to an event this week and you were saying how we actually have a bunch of Chinese sponsors this year, which is great for Saster AI Annual and they were complaining that qb, our customer, AI customer success, which is great. But it was a little hard to interact with the app in English, so we asked Replit to localize it into Chinese. And so what, you've never done this before, Right. So we did it in the car. I figured this was one that could be done on the phone and it was partially true. What'd you learn from the localization project to localize it into Chinese and Spanish. We had replied do both of them, right?

[B]

Yeah, I had them do both because, you know, I'm gonna pick all the like top. Obviously English is native because I built it in English and I Was like, okay, I'm going to pick because this is becoming a frustration point because we have last minute sponsors, we have multiples of them are, you know, native Spanish speakers, some of them are native Chinese, Mandarin, you know, speakers. And I was like, okay, I want to make this as seamless as possible for them to get up to speed. I can't do that, but QB could, right? So we were vibe coding the Waymo and I said to Replit, hey, can you make a toggle so that you can translate everything I've put into Qubi so far into Spanish, Chinese? And then with the toggle they can switch from English to the, you know, non native languages that I don't know. And so it took it maybe 20 minutes because there's, there's a lot of content in Qubi and so it had to, you know, translate everything it used, I think it used OpenAI for all the translations. And then after we got out of the car, you know, I took some screenshots and checked it with Claude to make sure nothing was inappropriate or weird with its translations. But translations, that was pretty good.

[A]

Pretty good. Yeah, yeah. So you took screenshots of what replit built with OpenAI and then you took screenshots to QA it into Claude to see if it thought that the translated app was good enough.

[B]

Yeah, actually what I'm running now too in the background is I'm having Cowork just now run through all of it, just spot check, make sure nothing is weird. But now Cowork is doing the full thing, so that's also an interesting learning. But yeah, I think it took about 20 minutes in the car. I'm about to roll it out to those sponsors who need this, but my guess is that it will ease some friction because, you know, again I'm using our native language, they're having to translate. They're also, you know, all of these folks last minute sponsors and so they're trying to get caught up really fast. And so this is something again, we wouldn't have been able to accomplish with the tool we were using previously. Like, there's no, like, we would have had to get a person to translate it manually and then program that all

[A]

into our worse because you have to build a harness or a system so that each chunk of text is dynamically translated to different languages and can be updated when it's changed. It's not even, it's not because if you really understand how localization, I mean, I'm not an expert, I've done it like, you know, you do Need a. You do need a framework of harness because everything's constantly changing and being updated

[B]

and you're adding things.

[A]

Yeah, yeah. So it's not, it's. It's not a one and done kind of. Kind of deal that adds. And so, so the agent replit, I mean it seemed to have done it all in. In the waymo for the most part now. At first, at first it was as Asians are. At first it was a little lazy and only translated something. Right. And so you had to put. Keep saying mocha like it's the school.

[B]

Keep going.

[A]

It was hoping it was enough to just translate the core stuff and not the deeper text. He had to push it. You had to push it actually multiple times to go deeper. Right?

[B]

I did. But you know what? When I. When I think back about that now, I think it's okay because I see so many people flag the costs on replit where like, you know, maybe if you were cost conscious, you would only want to translate the menus and something

[A]

like the task list to save 20.

[B]

Yeah. But still. Right? So I was like, okay, to its point. Maybe that's why it didn't do the fortune get one.

[A]

This is a complex project. Right. Relatively speaking. So to, you know, to. If 25% of the cost gets you there, that for most people, that might be a good place to stop. Right?

[B]

Yeah, yeah. But the agent should have asked me, okay, I did these menus. Are you okay with me keeping. You know, to keep it would have

[A]

been nice and we can wrap it. I was talking with another public company CEO this week about agents and he asked me, what do you have you guys used any of your agents for localization? And I'm like, I get the problem. You know, I lived it in my day. But no, it hasn't really come up. And then it came up in the car. I'm like, oh, we got to do it. Like, we got it. We got it. I think this is a pretty awesome use case for the agent. But let's see if it breaks. Let's see if it can literally do something that seems simple, but it's not to fully. I mean, Shopify, I think just last year rolled out localization for a lot of its product. Okay, Shopify, I mean it's pretty. Shopify is at what, 13 billion in revenue and they have global commerce. And they just did this. And we did it in 20 minutes in the car. I know. In a waymo on replit, man. That's an AI story. Maybe we should end with that one. Do we get everything.

[B]

I think it's a good. I think my only thing that. My only other thing that happened this week with QB that I think is interesting, but I don't want to derail us.

[A]

Oh, add it and then we'll wrap.

[B]

Okay. Is I. There was this interesting moment, right. So I talked a little bit earlier. We have this big production headline, not only for ourselves, but also for some of our customers who are sponsoring this Astra annual. They had a print deadline this week. A few people tried to, in a very human way, trick the agent. It's a way where from? From the agents. I just think it's always an interesting reminder. You can't hide from the agents, right? And so a few people tried to. Whether it was malicious or not, it doesn't matter, right? A few folks sent us their graphics and it was. I got a couple placeholders.

[A]

They were placeholders, right?

[B]

Placeholders. Either I got placeholders or I got render income, demographics, deadline.

[A]

So they uploaded whatever the hell they had on their. On their desk.

[B]

Yeah, they were like.

[A]

They pretended they met the deadline, right?

[B]

And they were like, okay, great. Like, I sent this to you. We're all good, right? I was like, well, actually between me and QB in a way that I couldn't previously because I didn't have enough scale or output. But now that I have the agent helping me, we did it in a way where not only did we. And they got an automatic email. We got an automatic email. And it would send to all contacts. Because sometimes I would get like a random designer sending us, you know, the booth graphics. And it would say, here's what you uploaded. And I would, between me and Claude, we would check it pretty much instantly, and we would catch these things, right? We'd be like, okay, in the age of AI, you really can't hide like, we can. Cat, we can see it. You didn't. You uploaded a placeholder or you uploaded or a lot of times they would upload something that was incomplete. Things you might have been able to hide back in the day, but that you can't hide now in the age of AI, because it just checks everything. I was like, it's check between me and it. It's together. We're checking everything. Like, maybe I wouldn't have got this, some of this a year ago, but we're checking everything now because I've got my agents running and so I can see if your. If your graphics are incomplete or fake or real or not. And so it was just an interesting

[A]

thing that people didn't fake the Deadline didn't fake the asset. And it can follow up in a kinder fashion than a human might be. So that. Because they would, they would pretend they did it. Right. And QB would.

[B]

Yeah, they'd be like.

[A]

Would objectively say, no, I appreciate it. But that's not actually what, what the traffic or assets that you're supposed to. Yeah, right. In a way that a human. That, that we, we do get yelled at a lot by, by folks. Right. For. I don't want to go too deep. We can wrap it up. But, but, but a lot of folks when they miss deadlines or when they don't have the assets or things they need, they. They sort of try to blame us. And we're zen about it now, but it's not action. Right? Oh, that's not the deadline that David told me. Well, it's in the contract. It's in the. Here's the 28 emails. David said I could, I could just sketch it on my iPad and that you or David Melia said this and people would just make it crap up. But QB's calm. Like QB doesn't get, get, get, get. And Cuba says, thank you. I sent you 28 emails saying this is what we needed. It's late. Understood. Could you please do it by tomorrow?

[B]

Yeah. And that's. And that's the thing, like, in a way where I was like, I don't know if I would feel comfortable pushing back to some of our customers like they are paying us money. But in a way that's super neutral. Just not caring. QB and I would check the graphics. Yeah. Better than a human. And if it was. And if there was an issue, it would just tell that person. It would just tell everybody in the org. Hey. Because maybe, you know, maybe the CMO didn't know that their designer just uploaded a placeholder. That happens sometimes. Like, you know, it's fine. It happens. And so in a way that it was super neutral, it would say, hey, there's an issue with your graphics. Here's the issue. I'm going to email everybody that I have from in the org as a customer, you know, in this customer. Org, I'm going to let them all know. And I'm going to also say in that reminder, either some of them were like already past due. So it would say, you're past due. You need to upload this now to avoid a late fee. Or not. Or not so. And for some of them, when it was approaching the deadline, it would just tell them, hey, we need your fully revised graphics. This doesn't count as a submission again in a very neutral AI way. Thanks for trying. This wasn't it. This doesn't count. You might have been able to trick a human previously, but you can't hide from qb.

[A]

Yeah, maybe we'll wrap on that. When I was talking with another public company CEO about AI, a great one, he said, well, your qb, your aav, that sounds like support on steroids. I'm like, no, no, no, let me explain it proactively follows on every possible deliverable there is and make sure it happens. And so my commentary, maybe we can wrap. I'll close and we'll wrap for this week. If you require any deployment for your product, for your company, for your application, if you have any checklist that you have to do, any training, any alt, do you have any onboarding for your company? If you have nothing, you don't need to listen to the rest of this. If you have any. Your humans are probably not giving you the 100% coverage on this. They're probably being argued with. There's probably issues. It's probably somewhat your fault or someone's fault that's not getting done. I don't think there's a lot of great off the shelf tools for this. Build your own qb AI vp Customer success. Amelia gave you the whole playbook. It's on saster.com, you can see our whole prompt and how we built it. Build your own one. Because if you do nothing else but completely automate the onboarding right of your customers with no drama, no complaints, no issues like we did, your life is going to be better. Right? And so that's why we'll keep talking about QB and 10K and we'll wrap. Hopefully folks find this useful. If we find that on YouTube you all churned after 60 seconds, we may do this less often. If we find out people listen longer than average, we'll keep doing it, but we know this is the number one thing we're asked constantly. What's going on with your agents? So this is the agents. Amelia, thank you for doing this one and we'll see you guys soon and continue the conversations.