Today on the Salesforce Admins Podcast, we talk to Joshua Birk, Senior Director of Admin Evangelism at Salesforce.

Join us as we chat about how the human in the loop is key to building reliable, predictable AI.

You should subscribe for the full episode, but here are a few takeaways from our conversation with Joshua Birk.

Understanding the guardrails around AI

It seems like every week, there's a new headline about an AI agent doing something it shouldn't. As Josh explains, that's because we're still in the process of understanding AI as a tool. That's why we sat down to discuss how to build predictable, reliable solutions in Agentforce.

When an agent behaves non-deterministically, it's usually because there weren't enough guardrails in place. The thing is, if you're building an AI agent to do everything, it's hard to control what it can and cannot do.

Josh's advice is to narrow the scope of your agent and build it for a very specific purpose. This makes it easier to build guardrails and also allows you to test it thoroughly before release.

A QA engineer walks into a bar…

When it comes to testing, there's an old programming joke that comes to mind. A QA engineer walks into a bar. He orders a drink. He orders five drinks. He orders zero drinks. He orders infinite drinks. He orders a horse. However, when the first real customer walks in and asks where the bathroom is, the entire bar bursts into flames.

As Josh explains, it's important to test for all sorts of weird edge cases and make sure your agent performs predictably. But it's even more important to think things through from the user's perspective so you don't miss something that should be obvious. AI can do extraordinary things, but you still need a human in the loop.

The first part of testing is planning

Josh emphasizes that the first part of testing is planning: "What are the Ifs? What are the Thens? What are the things you absolutely don't want it to do?" The more specifically you can answer these questions, the easier it will be to build and test agentic solutions that are predictable and reliable.

The most effective AI agents aren't autonomous solutions. They're tools that give the humans who use them superpowers. You still need a human in the loop to make sure they're used for good.

Be sure to listen to my full conversation with Josh for more about testing and building in Agentforce. And don't forget to subscribe to the Salesforce Admins Podcast to catch a new episode every Thursday.

Podcast swag

Salesforce Admins on the Trailhead Store

Learn more

Salesforce Admin Blog: How Agentforce Service Assistant Helps Salesforce Admins Become AI Leaders
Salesforce Admin Blog: Metadata Is Your Admin Blueprint for Building Better Agents
Salesforce Admins Podcast Episode: Exploring Agentforce Vibes Through Real-World Admin Use Cases with Daryl Moon

Admin Trailblazers Group

Admin Trailblazers Community Group

Social

Full show transcript

Mike: This week on the Salesforce Admins Podcast, we're welcoming back our good friend, Josh Birk, to kick off February with a conversation. Well, it's part podcast, part social experiment. Josh and I sat down to talk AI, specifically how admins can plan, test, and build with confidence using guardrails in Agentforce. We cover everything from deterministic responses to chaotic desktops and why designing with trust, maybe a little touch of humor, matters most. So whether you're rolling out your first agent or refining your AI game plan, this episode's got insights for you. So give it a listen and let me know if you like the format or not because, hey, maybe we just might make this a regular thing. With that, let's get Josh on the podcast. So, Josh, welcome back to the podcast.

Josh Birk: Thanks for having me, Mike.

Mike: Continuing the theme, we kicked this around... No one's heard this conversation. We're just going to start here... of having two evangelists talking for a podcast.

Josh Birk: Yeah. Yeah.

Mike: And so we're going to start February off with that. And then we're going to do a social experiment, and for people listening, if they like this, then maybe we repeat it once a month.

Josh Birk: Oh. I like these social experiments. Let's see what happens.

Mike: I know. It's on the listeners. Also, Josh, we have to keep this under 30 minutes so that Daryl can make it from his house-

Josh Birk: Can make it... Right. Right.

Mike: ... to the dock and get his boat in the water.

Josh Birk: Yeah. I remember when I first did my first 40-minute episode and I felt guilty because one of my guests was like, "I like the 20-minute episodes because that's exactly how long I walk my dog." So I'm like, "Well, okay, now you can walk your dog twice," question mark.

Mike: I know. Nope. Once you hit pause, they're gone forever. I was the same way. Selfishly, that's how these podcasts are as short as they were is I wanted something I could listen to when I walked my dog.

Josh Birk: Yes. Love it.

Mike: Anyway, it's February, and things that are still happening in the world are artificial intelligence and Agentforce.

Josh Birk: Amazing that this is still sticking around. It's almost like it's not a fad.

Mike: It feels like it's not going to go anywhere for a while.

Josh Birk: I think it's got legs. I think it's got a few legs.

Mike: Yep. It's not a fly-by-night thing.

Josh Birk: No. No. Whole ecosystem and industry being built up around it.

Mike: Uh-huh.

Josh Birk: Yeah. Yeah. I think we're going to be talking... I have a feeling this won't be the last time we talk about it.

Mike: Probably not, for reason.

Josh Birk: For reason.

Mike: So it's funny. The other day, somebody asked me something, and I was like, "Oh, that's a really good question. How soon do you need me to call you back?" Because I was thinking to myself, I don't have my phone, and my phone has a lot of AI tools on it, and I really need that to answer this. And I thought, wow. Two years ago, if somebody were to ask me that, I would just Google, make something up, and call them back. And now I felt like I'd just walked outside without my shoes on.

Josh Birk: I mean, it's kind of crazy. So when I was out in Arizona for Cactusforce, and my wife and I stayed the weekend, and she's like, "Well, what do you want to do this Saturday?" And I actually have a whole bit. One of my early AI talks was about how AI can kind of be dumb from time to time. And so I did this exact same thing. I'm like, "Hey, Google," or whatever, I think it might have been ChatGPT, but anyway, "give me an itinerary based on these things that I can do." And it gave me like, "Here's your itinerary," and it was like 17 things, all of them about three hours apart. And I'm like, "That's not an itinerary. It's a travel guide. I don't know what you're thinking here." But now it's like, I'm like, "Hey..." I think it was Gemini. I'm like, "Hey, Gemini, my wife would like to go shopping, we would like to go eating, and I have an interest in museums and aquariums," and it's like, "Here's your day." And I'm like, "Can you add that to my calendar?" And it's like, it's added to your calendar. And I'm like, it's moving so fast. My dark humor side of me just can't keep up because I don't know what I can make fun of AI anymore because it just keeps getting better.

Mike: Right. So speaking of which, you did a couple talks around AI at different conferences last year and probably this year. And I think it's important because we're going to talk about guardrails and security stuff, not to scare people away. It's a fun conversation. But it's needed because I also think it's worth understanding why we ask you to do what we ask you to do when you set up Agentforce and why you do certain things. I recently, coming off of, I think it was last we...

The Importance of Human in the Loop for Agentforce