Podcast Summary: SaaStr 844

Title: The Top 5 Issues Managing Multiple AI Agents in Production
Host: SaaStr (Jason)
Guest: Amelia (SaaStr's CEO and Chief AI Officer)
Date: March 4, 2026

Overview

This high-energy episode dives into the hard truths and practical lessons that SaaStr's top execs have learned after almost a year running nearly 30 AI agents in full-scale production. It systematically breaks down the top five pain points, surprises, and unresolved challenges in scaling from one or two agents to an entire "agentic" team—offering real-world tactics, memorable stories (including getting roasted by your own AI!), and insights into where AI management tools are still lagging. If you’re considering deploying or scaling multiple AI agents, this is essential, battle-tested advice.

Key Discussion Points & Insights

1. Context Switching & Management Complexity

• [03:15, 08:20]

  • With 20+ agents, context switching becomes overwhelming—each agent behaves differently, has distinct input/output expectations, personalities, and interfaces.
  • It's not just “managing tools”—you need to view agents as team members, each needing daily check-ins, context, and unique instructions.
  • Even with tools like Salesforce and custom-built agent directories (e.g., AgentForce, 10k), the ecosystem is a hodgepodge—no single "chief agent" tool exists.

  Quote:

  “You have to almost think about it now as managing 20 different people… each agent almost as its own entity, as your own employee that speaks a different language, has a different personality, needs different things from you.” — Amelia [08:20]

2. The “Blackout Period” for New Agent Onboarding

• [11:55]

  • Adding a new agent demands massive attention, typically causing two weeks of chaos where other agents CPU idle, missing out on inputs and degrading performance.
  • Onboarding timelines have improved (from 4-6 weeks down to 2 weeks), but each new launch means the rest of the agent fleet will go stale.
  • There is a trade-off: rapid scaling is not free, and human oversight never goes away.

  Quote:

  “Each new agent means you’re not spending…that daily time I just mentioned…they’re literally idling, sitting there, not doing anything because I haven't had the time to give it new contacts. That is, I think, just a huge nuance folks don’t really talk about.” — Amelia [12:50]

3. The Agent Succession Planning Crisis (“N=1 Problem”)

• [17:23, 24:11]

  • Most orgs have one person deputized as chief agent wrangler; if they leave, the whole AI stack collapses. There’s no graceful hand-off process or master agent that can self-document, train successors, or orchestrate all job functions.
  • Backing up knowledge isn’t enough—agents are wired around individual workflows, credentials, and implicit context that’s hard to transfer.
  • Best practice: pair two humans in agent management for resilience, and test for hands-on tool-building experience when hiring.

  Quote:

  “If you actually find the person that can deploy these agents for real, that can actually be success…immediately they should recruit. You need to recruit somebody else on their team and divide and conquer. Make sure you have two.” — Jason [24:11]

  Memorable Moment:

  • Amelia asked her agents what they’d do if she got “hit by a bus.” 10k replied about transferring docs and knowledge, but noted: “don’t get hit by a bus.” [26:44]

4. Agents as Relentless Truth-Tellers (and “Roasters”)

• [28:54]

  • Unlike humans, agents don’t soften feedback—they confront users with brutally honest data and missed goals, sometimes to the point of emotional “roasting.”
  • This constant accountability can motivate but also demoralize (“you’re 56% behind target!”), and it's hard to get an agent to sugarcoat their feedback.
  • Agents don’t perceive time/fatigue; they just keep pushing for outcomes, regardless of whether it's 6am or midnight.

  Quote:

  “These agents… will always tell you the truth and they have all the data. Let’s get into the last one… Compliance and security issues.” — Amelia [34:28]
  “I said, hey you’ve kind of roasted me a lot lately… and it said, looking through the transcript, I haven’t really roasted you.” — Amelia [32:30]

5. Security & Compliance: The Hidden Operational Tax

• [34:28]

  • As agents touch more customer data and systems, security review and maintenance balloon. Most agent platforms are less secure than enterprise-grade software like Salesforce.
  • DIY or “vibe coded” agents present even more fragility; enforcing best practices like frequent security audits is essential but time-consuming.
  • Carefully balance risk by limiting sensitive data exposure and interrogating third-party vendors’ compliance standards.

  Quote:

  “No agents, no third-party agents are more secure than Salesforce. They’re all less secure is the reality…Once you vibe code yourself, which we have done a lot of, they are inherently less secure.” — Jason [34:49]

Bonus: AI Agents Are Changing How Humans Manage…Humans

• [37:14, 43:00]

  • As AI agents become more reliable and “always on,” expectations for human collaborators shift—sometimes unfairly.
  • There’s a risk of “losing patience” with human teammates when agents can deliver answers or work instantly; may impact leadership style.
  • The rise of “agentic” management could change team dynamics (“do I need them, or can I build an agent to do this?”).

  Quote:

  “Now that I’ve flipped the switch to managing more agents, I think I’ve probably become slightly worse at managing the people… I need to remember they are not agents.” — Amelia [43:00]

Special Topics / Notable Audience Q&A

Orchestration Layer: Still a Missing Piece

• [13:23, 43:03-44:54]

  • Despite industry hype, a true universal “orchestration” tool—one platform to rule all agents and integrate their workflows—does not exist. Everything remains siloed, making management manual and time-intensive.

  Quote:

  “No matter how much people talk about on social media about an orchestration layer, we haven’t found one yet…as soon as we master the orchestration layer ourselves, we will share it with you.” — Jason [43:03]

How Should You Hire for Agent Management?

• [46:35]
  • Key test: practical “build and ship” capability. At Replit, engineers are given substantial credits and told “build anything you want, come back with it, and then we’ll do the interview.”
  • New hires need curiosity, initiative, and a willingness to build—not just theory.

Measuring Agent ROI

• [47:26-47:53]
  • Agent ROI is measured in closed-won revenue and productivity—the ability to do the work of multiple humans, reliably.
  • “It is very easy to attribute revenue to each agent. It’s very simple…That’s the main way we measure ROI.” — Jason [47:47]

Timestamps for Important Segments

| Timestamp | Segment | |-----------|----------------------------------------------| | 03:15 | Key issues overview by Amelia | | 08:20 | Managing agents as “team members,” daily routines | | 14:06 | Problem: Lack of a true “Chief Agent” tool | | 17:23 | Blackout period and trade-offs for onboarding| | 24:11 | Single point of failure & succession crisis | | 28:54 | Agent “truth-telling” and human psychology | | 34:28 | Security audits and the “tax” of compliance | | 37:14 | Agents changing how we manage humans | | 43:03 | No orchestration layer exists, Q&A begins | | 46:35 | Hiring tips for agent managers | | 47:26 | Measuring ROI on agents | | 51:14 | Why agents don’t (yet) do more advanced marketing | | 54:10 | Closing and SaaStr Annual preview |

Memorable Quotes

• On daily management:

  “You can have a one-on-one with your agent every day. Even if you don’t want to do the work, you got to do the work to see one-on-one with every agent every day.” — Jason [14:35]

• On agents “roasting” their boss:

  “You’re 56% behind target. Now, maybe you don’t think this is roasting me. I thought it was though…it also told me to block 3 hours. I’m like, I don't have 3 hours to do this and catch up. Like, thanks, but now you're just stressing me out and roasting me versus being helpful.” — Amelia [29:25]

• On security:

  “No agents, no third-party agents are more secure than Salesforce. They're all less secure is the reality...custom is less secure than a Salesforce or something. So all of the apps like us you’re going to run are going to have customer data through them.” — Jason [34:49]

Key Takeaways

• Effective agent management at scale is more akin to managing a team of 20+ “quirky” digital staffers, not just software tools.
• There’s always a “blackout” cost when onboarding a new agent, temporarily degrading your existing stack.
• Agent operations are often functionally a single point of failure—backups and succession plans are severely lacking industry-wide.
• Agents provide relentless, sometimes draining, accountability—and that can shift cultural expectations for both AI and human coworkers.
• Security, maintenance, and compliance remain under-discussed, laborious, and often fragile for in-house or “vibe coded” tools.
• There is still no true orchestration layer on the market—expect lots of manual effort and context switching.
• Success requires intense, daily active management, strong agent wranglers, and practical, hands-on problem solvers—plus a willingness to “divide and conquer” the workload.

For additional resources and SaaStr’s agent directory: Check SaaStr AI Agents (not an actual link).