AI can build almost anything now. That’s the problem.

Summary

Insights Unlocked Podcast Summary
Episode: “AI can build almost anything now. That’s the problem.”
Date: June 15, 2026
Host: Mike Mace (Director of Solution Marketing, UserTesting)
Guest: Katie Robeli (Product Operations & AI Strategy Leader)
Producer: Nathan Isaacs (Senior Manager of Content Production, UserTesting)
Length: ~43 min

---

Episode Overview

In this episode, Mike Mace interviews Katie Robeli about the rapid acceleration of AI’s capabilities in product development, particularly the rise of autonomous AI “agents.” Their discussion explores how the democratization of building with AI has changed team dynamics, workflow, and risk—highlighting new challenges such as role confusion, feature bloat, and the heightened importance of customer discovery. Katie offers practical insights for organizations eager to harness AI without losing strategic focus or customer empathy.

---

Key Discussion Points & Insights

1. AI’s Impact on Product Development (03:43–06:18)

• The “Builder’s Dilemma”:
  • AI allows more people to build rapidly, but many teams skip crucial discovery steps and neglect to evaluate whether what they’re building is truly needed.
  • Quote (Katie, 04:03):
    “Now that everyone is a builder, organizations think, ‘oh, we can just build more faster.’ But instead they’re not asking themselves, is it right for us to actually build what we’re going to build?”
• Changing Team Roles:
  • Rapid AI adoption is outpacing organizations' ability to define new workflows and responsibilities. Teams are self-organizing without clear best practices, leading to inconsistency and gaps.

2. The Rise (and Risks) of AI Agents (06:18–11:21)

• Defining Agents vs. LLMs:
  • LLMs (like ChatGPT) are chatbots; agents are autonomous, act with memory/context, and can perform multi-step tasks.
  • Quote (Katie, 07:47):
    “What an agent can do is over time learn patterns and make decisions on its own... instead of needing a deterministic tree of ‘if this, then that’ the agent can make its own decision.”
• Managing Agents is New Territory:
  • Individual contributors now “manage” agents without prior training or frameworks—akin to managing junior staffers.
• Cultural Example: Meta’s AI Push:
  • Mandates to use AI (tracked by token usage) led employees to “game the system” by spinning up redundant agents, showcasing unintended consequences.
  • Case (Katie, 09:44):
    “Some teams gamified the system, created a leaderboard... employees were spinning up agents to show they were using AI... But that doesn't mean they provided any value.”

3. The Need for Guardrails and Training (11:44–14:28)

• Organizational Guardrails:
  • Training on what AI should (and shouldn’t) be used for is essential, especially concerning sensitive data and internal privacy.
  • Quote (Katie, 12:07):
    “Just because our HR director has access to all this personal information… should they be able to put that into an LLM or use an agent to gather that information?”
• AI as Research Partner:
  • One of AI’s greatest values is synthesizing voluminous customer/market feedback to guide what gets built—not just building for its own sake.

4. Avoiding the “Feature Factory” Trap (14:28–16:48)

• Building ≠ Progress:
  • The allure of finishing every backlog item is misleading—quantity leads to feature bloat, customer overwhelm, and maintenance debt.
  • Quote (Katie, 15:37):
    “Just building more stuff doesn't mean that it’s stuff that people want or that people are going to use... There is a cost to anything that you build.”

5. Elevating Human Roles: Strategy Over Tactics (16:49–18:53)

• AI Handles Execution, Humans Handle Strategy:
  • AI frees up product managers to focus more on strategy. However, loss of “friction” (exposure to raw customer data & grunt work) could undermine strategic acumen in future leaders.
  • Quote (Katie, 17:31):
    “The best use for the humans that are building on these teams now is to think about strategy. The best use for the AI tools is tactical execution.”

6. Training the Next Generation—And the Agents (18:53–25:59)

• Human-in-the-Loop Is Critical:
  • To prevent “context collapse” and misuse, junior PMs should validate agent outputs, modeling and teaching “what good looks like.”
  • Training agents is more like “raising a puppy” than traditional programming—requires careful, ongoing, example-rich feedback.
  • Quote (Katie, 22:54):
    “To me it sounds more like raising a puppy than doing coding.”
• Point of No Return:
  • If agents are fed bad context, it’s nearly impossible to correct later—reinforcing the importance of thorough upfront discovery and guardrailing.

7. Discovery is Non-Negotiable (27:54–29:13)

• Upfront Customer Discovery is More Important Than Ever:
  • AI is only as good as the foundation you set—errors in early design or agent training can’t be easily undone.
  • Quote (Katie, 28:15):
    “It’s [discovery] much more important than the actual building phase.”

8. Everyone Can (and Should) Build — But Mindfully (29:13–38:00)

• Lessons from Building an Agent as a Non-Coder:
  • Clear goals, precise natural language instructions, and constant pushback/simplicity checks are key.
  • Memorable Moment (Katie, 29:44):
    “I came in and said, I want to build an agent. How do I do that? … I really think that everyone in an organization, from the CEO to a junior HR person, should be building something just to understand this is how the technology works...”
• How to Get Started:
  • Use AI chat to help scaffold and guide the building process; don’t fear “dumb” questions.
  • Tap into learning communities and support groups for agent builders.
  • Quote (Katie, 36:11):
    “Talk to me like I’m 2 years old and I've never used a computer before. How do I do this? And it will dumb it down… go slow, build something really small.”

9. Practical Organizational Takeaways (38:00–41:35)

• Summary of Key Points:
  • Don’t throw people (or orgs) into AI chaos—education, guardrails, and processes are essential.
  • Human-in-the-loop “evals” should prioritize impact and real value, not just mechanical correctness.
  • The cost of building is now less about engineering and more about customer clarity, maintenance, and internal complexity.
• Final Advice (Katie, 40:28):
  • “Be very clear on what your goals are… only build what’s necessary to achieve that objective. Just because you can build doesn’t mean you should.”

---

Notable Quotes & Memorable Moments

• “Just because you can build something doesn’t mean you should.”
  — Katie Robeli (05:42)
• “Without AI, 40% of code on average is not executed in any given application.”
  — Katie Robeli (15:29)
• “I came in and said, I want to build an agent. How do I do that? … really anybody who has an idea can build a thing and there's really not a lot of friction there anymore...”
  — Katie Robeli (29:44)
• “If you get the wrong thinking in there at the start [with agents]… it's not hard to fix. It's impossible to fix at a certain point.”
  — Katie Robeli (26:41)
• “It’s much more important than the actual building phase.” [on discovery]
  — Katie Robeli (28:15)

---

Timestamps for Key Segments

• 03:43 — “The Builder’s Dilemma” and changing friction
• 06:18 — Teams, agents, and skill gaps
• 09:44 — Meta example and unintended consequences
• 11:44 — The need for AI training and organizational guardrails
• 14:28 — Why building more isn’t better
• 16:49 — Elevating the human role to strategy
• 18:53 — Training junior talent and agents (human-in-the-loop)
• 22:54 — Raising a puppy: training AI vs. traditional coding
• 27:54 — Why discovery is now critical
• 29:13 — Lessons from building your first agent, even as a non-engineer
• 36:11 — How to get started with agent building
• 38:00 — Practical takeaways and summary
• 40:28 — Final advice: clarity of purpose and restraint

---

Final Thoughts

Katie urges organizations to slow down and emphasize intentionality: Clear objectives, robust customer discovery, and incremental, human-guided agent building must precede unleashing AI’s full creative force. While AI can synthesize and execute, it’s up to human leaders to direct, evaluate, and maintain the strategic clarity that drives meaningful value for both organizations and end customers.

Katie invites listeners to connect via LinkedIn or her website, Scaleyze Systems, for more on product ops, AI adoption, and scaling processes. She also hosts a monthly Lean Coffee for product operations leaders.

---

Transcript

[Nathan Isaacs]

Welcome back to Insights Unlocked. In this episode, Mike May sits down with product operations and AI strategy leader Katie Robeli to explore how artificial intelligence is transforming the way products are built. They discuss the rise of AI agents, why customer discovery matters more than ever, and how organizations can avoid the trap of building faster without building smarter. If you're navigating AI's impact on the product development lifecycle, this conversation is packed with practical insights. Enjoy the show.

[Podcast Narrator]

Welcome to Insights Unlocked, an original podcast from User Testing where we bring you candid conversations and stories with the thinkers, doers and builders behind some of the most successful digital products and experiences in the world, from concept to execution.

[Nathan Isaacs]

Welcome to the Insights Unlocked podcast. I'm Nathan Isaacs, principal Content marketing manager at UserTesting, and joining us today as host is Mike Mace, Director of Solution Marketing at User Testing. He's a longtime tech industry veteran and one of the areas he works on at User Testing is helping companies build effective AI products. Welcome back to the show, Mike.

[Mike Mace]

Hey, it's nice to be back. Thanks.

[Nathan Isaacs]

And our guest today is Katie Robley. Katie is a product operations and AI strategy leader focused on how emerging technologies are reshaping the way teams build, scale and optimize products. She's been exploring the growing role of AI and agenic systems in product development, from accelerating discovery and prototyping to transforming how product managers make decisions and drive innovation. Welcome to the show, Katie.

[Katie Robeli]

I'm so happy to be here. Thanks for having me.

[Mike Mace]

So, Katie, man, this is such a fun area and it is so fast paced and stuff like that. I just can't wait to jump into it, I guess. But to start though, to frame this, talk to us a little bit about your background. You know, like the, the experiences you've done, the types of roles you've worked in, and in particular you've. You've been having this AI journey over the last year and I'd love to have you talk about that.

[Katie Robeli]

Yeah, sure. So my background is in technical program management and product operations. And over the last several years I've been focused more on investigating AI and how that can be used in organizations. What this really came down to was with one of my consulting clients, they asked me to help build out a new product using a bespoke LLM that they had trained on Canadian legal data and moving it into the UK. And I realized during that engagement, building tools with AI is completely different than my SaaS background. So I learned how to build this human in the loop evaluation system. And we'll get into that later. I'M sure. But that really showed me that if I'm going to be able to offer my clients the best services, then I really needed to dig in and understand more about how AI worked and what we might be seeing in the future. So I've taken the last year to really get my hands dirty. Built some agents, talked with product leaders across a variety of different size companies and have seen remarkably similar outcomes from all of them where they're having the same issues. So I hope we can dive into what some of those look like.

[Mike Mace]

Oh yeah, let's definitely dive in and in fact, let's, let's talk about, you know, what, what you're seeing in terms of the ways, the ways companies are reacting and the, the issues that you're seeing them have.

[Katie Robeli]

The friction. Used to be waiting for an engineer to build something, but now that everyone is a builder, organizations think, oh, we can just build more faster. But instead they're not asking themselves, is it right for us to actually build what we're going to build? And they're not checking in with customers to make sure that those decisions that they're making are for the customer. So what I'm seeing is that organizations in general aren't prepared to start jumping in and building things, but they're jumping in and building things anyway.

[Mike Mace]

So what is that, what does that do to the people involved? I mean, if they're just jumping in like that, are there, are there typical patterns or typical mistakes that they're, that you're seeing them make?

[Katie Robeli]

Yeah. So especially within the product triad, we have seen that roles and responsibilities are completely different today than they were even a year ago. And because organizations haven't taken the time to understand what the jobs are that need to be done within the product triad, now the teams themselves are the ones who have to negotiate it amongst themselves. And so it looks different for different teams across the same organization. Each team is having to come up with what workflows work best for them and are often skipping steps or things are falling through the cracks because there are no best practices within an org that have been outlined.

[Mike Mace]

Wow. So, okay, so you've got this, this world where, I mean, I guess the, the theory that I hear from people who are enthusiasts for, you know, AI building or whatever, whatever phrase we want to use for this say, hey, it, it frees up everybody to create. And so now you can just be fluid and create stuff. And it sounds to me like the way it's working is more like a bunch of people running around and stepping on each other's toes and stuff like that. Am I misreading it or is that what you're, that what you're seeing that

[Katie Robeli]

is, is actually what's. What's going on? And it's, you know, just because you can build something doesn't mean you should build something.

[Mike Mace]

Yeah.

[Katie Robeli]

And organizations aren't necessarily taking the time to set up some sort of foundation, make sure that everybody has the education that they to understand what AI can and can't do, what it's best at, where humans should be involved. And in a lot of cases now you have individual contributors who don't have experience managing people, but they're being asked to manage agents, which is a very similar skill set, but they don't have any training to do that.

[Mike Mace]

Got it. So, so talk to me a little bit more when you say agents. I think I can picture that. But let's make sure we're bringing everybody along. So what, what would those typical agents that they would be that they would be driving and what would they be using them for?

[Katie Robeli]

Maybe we can start with define what we mean by AI.

[Mike Mace]

Okay, good.

[Katie Robeli]

Several. Several different things. Right. So most folks are familiar with what an LLM is because that's a chatbot that we use. So Claude or copilot or chatgpt, those are all LLMs or large language models that have been trained on a massive amount of information and can answer our questions for us. What an agent is, is it is able to act autonomously and maintain context and memory. So right now, if you go into a chat, it doesn't have any context for what you talked about in the previous chat, unless you've asked it to maintain that memory. What an agent can do is over time learn patterns and make decisions on its own. So you can have an agent answering customer experience questions, interacting with customers, and then understand over time patterns that evolve. And instead of needing to have like a deterministic tree of if this, then that the agent can make its own decision to determine where it's best to send the customer.

[Mike Mace]

Gotcha. So then people are than managing a suite of agents that are doing tasks for them. Is that what it's like? It's like having a bunch of assistants?

[Katie Robeli]

Yeah, you can, you can think of it like that. And those assistants need to be told what to do and what good looks like and what bad looks like. And they need to be trained on what the expected behavior is and why they're being asked to do the thing they're being asked to do.

[Mike Mace]

Got it. Okay. So, and, and let me be sure I'm, I'm understanding it all because I'm putting together the picture in my head. So, so there's, number one, it sounds like there's a learning curve that needs to happen around how you use agents and what they can do for you and stuff like that. Is that correct? And then another thing is how does the product development process work? How does the lifecycle work in this new world where we're AI enabled, like who's doing what roles and all of that sort of stuff. Got it. So we're disconnected about both of those things.

[Katie Robeli]

Yeah. If we think about the jobs that need to be done just from a product manager's perspective.

[Mike Mace]

Yeah.

[Katie Robeli]

Where we might have had a PM writing JIRA tickets before now, we can have an agent write those for us. But the question is, who builds the agent, who maintains it, who writes an evaluation to make sure that the output that it's giving is correct, and then who's responsible for fixing things when they don't go Right. And there's no clearly defined responsibility for that. He might say, oh, well, that belongs to an engineer because it has to do with writing code. Well, now that everyone can write code, is it really an engineer?

[Mike Mace]

Yeah. Yep. So is this, Are you seeing the chaos? Is this mostly like a small startup thing where they're not getting it, or is it happening in big companies as well?

[Katie Robeli]

No, it's happening everywhere. And one example is lately in the news, we've heard there were a lot of rounds of layoffs at Meta that just happened. And Meta has said everyone needs to use AI for everything. It is used as a performance evaluator and they base the usage on tokens. So tokens are. Think of them as like a couple of words. Is one token. Well, the number of tokens used is telling Meta ostensibly how much AI is being used by each individual person. Right. What happened was with the. Some of the teams gamified the system, created a leaderboard, and I believe they used something like over 30 trillion tokens within a 30 day period, like across the organization. Because employees were spinning up agents to show that they were using AI and then they would. Other people would have to spin up other agents to find the agents that their coworkers built. More agents were needed to be spun up to do evaluations on those agents to make sure they were acting appropriately. But that doesn't mean that they provided any value to their customers or to the business internally. It just means that people were typing away building agents that may or may not be useful just for the sake of Spending these tokens to show that they were using AI?

[Mike Mace]

Got it. So, so what do you think companies should be doing in terms of how. If it, if it's, if this is pervasive, that we've kind of got this idea of, hey, AI is great, you know, it's a, it's a goodness in itself to be using it. Here it is. Go use it. How should we be setting up folks that they'll be successful?

[Katie Robeli]

So organizations need to have some sort of training program to make sure that everyone understands what AI is used for. What it's not great to be used for some sort of guardrails in the organization to say, just because our HR director has access to all of this personal information about people, should they be able to put that into an LLM or use an agent to gather that information? Because if they do that, are there any guardrails keeping it from being exposed to other parts of the organization? Or how are we protecting customer data? If we are using AI to come in and determine, you know, what customer, what customers are looking for, is there any chance that this can be exposed to the outside world? Or are employees inside who shouldn't have some of this data able to get that data? So those, those are just a couple of lightweight things that organizations can do just to make sure that everybody has the same vocabulary and that there are some basic safety structures in place. But the main thing that organizations should do is really lean in where AI tools are best used. And in one of those areas is really understanding what customers want. Instead of using AI just to build things, figuring out, taking this massive amount of information that's available with customer feedback and bugs and any customer conversations. These AI systems have the capability of holding a huge amount of context together in one place that a human brain can't, and then understanding how all of these pieces are interrelated and how they work together and then making recommendations.

[Mike Mace]

Got it. I like that. So kind of feeding all the information in, all the stuff that could be old research, could be recent, should be recent research. So it's not just old stuff, but also all the other data points that you get on customers, whether they're interviews or things that they wrote or, you know, in any sort of stuff you can find, the analytics you've got on them, whatever, putting that into kind of a guru which can, or coach or advisor or something like that, that can sort through it and tell you, okay, here's how it all fits together, or maybe here are some trends that you can see across several things. Is that the Sort of stuff you'd query it on.

[Katie Robeli]

Yeah, absolutely. You can do market analysis, competitor analysis. All of that information should help inform what you should build. So instead of building all the cool things that you want to build, you're actually building what your customers need you to build.

[Mike Mace]

Yeah, yeah. You know, that really resonates with me sometimes in some of the enthusiastic literature that you see about the effect of AI on the product development life cycle. What they really focus on is just pure building productivity, you know, and you'll see things like, hey, you can eliminate your backlog. You know, every, every engineering team has a backlog of requested features that would take 50 years to complete. And now you can actually complete all those features. I think what I'm hearing from you is, okay, fine, but that's going to be a lot of features appearing all at once, maybe overwhelming to customers. Maybe you don't want to just do it that way. Maybe you want to be a little selective about what you do. Is that, is that right?

[Katie Robeli]

Absolutely. I mean, without AI, 40% of code on average is not executed in any given application.

[Mike Mace]

Yes.

[Katie Robeli]

So just building more stuff doesn't mean that it's stuff that people want or that people are going to use. Then you have the cost of maintenance, you have the cost of context switching for customers. If you start throwing all these new things at them, it's overwhelming. Right. And then your product team has to, to think about how all of these are interrelated. You have support teams who now need to support all of these new features and functionality, documentation that needs to be written, videos that need to be made. There is a cost to anything that you build.

[Mike Mace]

Yep, yep, got it. So, so the nirvana vision was, hey, we can, this is great. We can just check off all these things and it's going to be wonderful. The reality is that actually creates its own set of problems. You know, if, if you're, if you're able to do everything, it doesn't mean you're making brilliant decisions. It means you've got a faster way to make mistakes. And so you still, you still need to take responsibility for making sure that you're doing the right things. You've just shifted kind of the dynamics around those things, but you haven't shifted the underlying problem.

[Katie Robeli]

Absolutely.

[Mike Mace]

Got it. Okay. About eight months ago, you wrote a series of posts that you, that you put on LinkedIn, kind of the start of your journey. You've been going on for the last year of looking at this stuff, and you talked about how AI can free up product managers to become more strategic. So I'm interested in, do you still view it that way? How have you seen things evolve? And do you, do you still think that's the goal of what we should be trying to achieve?

[Katie Robeli]

I do, yeah. I do think that the best use for the humans that are building on these teams now is to think about strategy.

[Mike Mace]

Yeah.

[Katie Robeli]

The best use for the AI tools is tactical execution.

[Mike Mace]

Yeah.

[Katie Robeli]

So for the humans in product management, they need to understand how they can manage these agents that are being built, that are doing the tactical execution and, and direct them effectively on strategically what the organization is trying to accomplish. The problem, though, is that product leaders, better leaders today, it's great because they had to go through all that tactical execution and the friction that comes with talking to customers and reading these voluminous reports on. Yeah. Bugs and, and feature requests and feedback from customers. Now that we have agents that can do all that work for us, more junior product managers are not going to be able to think in that strategic mindset because they haven't slogged through the same way that past product managers have to gain the strategic insights that they have today.

[Mike Mace]

Yep.

[Katie Robeli]

So my question is, how are we going to start training a new generation of product leaders now that they don't have that same level of friction in order to gain the strategic capabilities that current leaders have?

[Mike Mace]

Yep. That I can see that problem. And I, I actually, I suspect it's not just going to be product managers. I think it's loss of context across a number of roles in the product development life cycle. So do you have thoughts on how you'd go at that? How would you, how would you fix this?

[Katie Robeli]

So in my view, adding some of that friction back in is really the best way to do that. So just because you do have capabilities of agents to take all of this information and synthesize, it doesn't mean that humans shouldn't, alongside the agents, also do that work. For one thing, you want to make sure the agent is synthesizing things correctly. I mentioned evals earlier or evaluations of how the agents are working. And that's one area that more junior product managers can really lean in. And there's I mentioned human in the loop, which is a way of saying, okay, this is the output that the agent has given me. Is this good? Have we defined what good is? And then if it's not good, then giving it feedback to say, okay, you said X, Y, Z, but really the correct answer is abc. So by telling the agent, you did this wrong because of this, this is the right Answer. You're training it now to think about what good looks like. And so one of the ways that you can have more junior product people understand what that friction looks like, what that work looks like, is to have them independently validate what the agents are coming up with and training them for what you actually want the outputs to look like.

[Mike Mace]

Got it. So check the work of the AI. Go do manually a check, you know, using the old traditional ways of gathering information to check that the AI is doing the right thing. I like that. So I'm picturing kind of two issues here that I think the, the, the tech industry, when it talks about this stuff, tends to munch together and I want to kind of separate them a little bit and ask you about them. So one is kind of what we started talking about, which is how AI can change your process for iterating on traditional software, traditional SaaS type software, and how it can accelerate that. But you can't just throw people in. You need to train them on, on how they're going to use it and you need to give them the right guardrails in the right context. And I think especially for agents, that one really, really sounds interesting because I was just looking the other day on LinkedIn on I wanted to see if there were posts about people who had problems with agents. And Katie, I don't know if you've tried this one, but the number of stories that pop up of so and so tried to have an agent do X and it erased their entire Y or it sent their entire Y to their entire mail list or something like that. I mean, it doesn't sound like it's the same three incidents that are just going through the echo chamber. It sounds like it's happening a lot, which is kind of like, okay, so that's an indicator that agents are very powerful. That usually means it's a technology with great potential. But then lots of rules about how you use it, right? So, so a lot of like, okay, let's, let's not just throw people at it, let's do the right training on how to use that. But then there's the second one that you were just starting to get into, which is really interesting to me, which is training the AI as opposed to training us on how to use AI as a tool. Developing AI sounds like a really different process. I mean, to me it sounds more like raising a puppy than it, than it sounds like doing coding. Am I getting that wrong? Or is, or is that that really an issue?

[Katie Robeli]

No, it's really an issue. I mean, you you wouldn't hire a junior developer to come into your organization and hand them all the keys to your repositories and say, okay, go nuts.

[Nathan Isaacs]

Right? Yep.

[Katie Robeli]

You would want to make sure that they knew what they were doing, start them with small files, not give them the ability to push into development. You know, some of the basics. But when organizations have started using AI, they haven't thought through, oh, this is a junior developer we have, that needs to be trained and we need to make sure that they know what they're doing before we give them access to these things. Instead, what we're doing is giving access and then realizing, oh, you need to be trained.

[Mike Mace]

Wow. Okay. When you put it that way, that's really, really scary. So, so talk to me about how, how should you be approaching this, this process of teaching AI to, to do, to do the right things and develop in the right way? What do you, what do you need to think about and what are the steps you need to go through?

[Katie Robeli]

There are a variety of different ways to train any models or agents. There are some tools out there that are out of the box evaluation systems. Those are like the equivalent of using spellcheck and expecting it to also tell you if your content is valuable for your customers. Oh, just because you have an evaluation system, it just means that you have a full sentence. It doesn't mean that the sentence is relevant to anything.

[Mike Mace]

Yep.

[Katie Robeli]

Right. So you need to be able to use. This is where humans are, are not going anywhere. In the process of building software. You need a human to be able to tell the agent, here are some examples of what I want the output to look like. Here are some examples of outputs that are not so good. And then continuously feed them that information and give feedback that this is good, this is not good, this is what we should see. And that is true even when you have agents training other agents. Because we've, I don't know if everyone is familiar with the different evaluation systems, but there are, so there's human in the loop, which is what I was just describing. And then there are other systems that are multi agent systems where you have groups of agents learning from each other. They still need to be trained initially by a person. You need to give them the correct context for your organization, for your use case, for your customers, with examples of what you want them to do and then they can learn from there and, and, and help each other learn from there.

[Mike Mace]

Got it.

[Katie Robeli]

So it's kind of like a, if you have a cohort of new hires. Yes, right. You, you train them all you let them discover things on their own and learn from each other, but they're all starting from this singular place of not having any context or any information, and you need to give that to them.

[Mike Mace]

Yep, got it. So then if you. Man. Because the way these ideas propagate between the agents, if you get the wrong thinking in there at the start is that sounds like that's a really bad thing. Like it's. It's going to be hard to fix later.

[Katie Robeli]

It's not hard to fix. It's impossible to fix at a certain point. There's a point of no return with agents because you can't just yank out a file and be like, oh, forget that. Yeah, it's not possible. So you've probably experienced this yourself. If you've been in a chat with, with any LLM and have given it instructions to do something, it does it wrong. You say, no, don't do that. Do this, still does it wrong. And there's a point that you close out of the chat and start a new one because you can see it's just not going to get it. And you continuing to tell it what to do is not going to make it do what you want it to do. There's a point of no return because there's memory in context that persists throughout that singular chat with an agent, you can't just unplug it. That memory and context persists for the agent. So with an LLM, easy enough. Just start a new chat with an agent. You need to start building a new agent. There's no, there's no unseeing what you've already shown it.

[Mike Mace]

Yep. Wow. So that sounds to me like upfront discovery, you know, what we'd call in traditional development, you know, the discovery process upfront to understand customer problems, what are we trying to solve, all that sort of stuff that sounds like that becomes incredibly important. Is that right?

[Katie Robeli]

It is, yes. It's much more important than the, the actual building phase.

[Mike Mace]

Wow. Okay. All right. So a lot of our audiences, folks who are, you know, in research roles or design roles and discovery is one of the things that they, that they're responsible for and that they do a lot of. And so this is interesting to me that this is actually what you'll hear sometimes is AI frees up everything and makes everything easy. I think in this case, AI is actually putting more stress on the development. Pardon me, the discovery process, making sure we do it right, making sure it's really comprehensive because it's like a foundation. You're not going to be able to extract that out of the, out of the agent later. So lately on LinkedIn you've been writing about creating your own agent. So talk to me about why that's important and what you're, what you're learning out of doing that.

[Katie Robeli]

This is probably the most useful exercise that I have ever done. It's been really building my first agent was an incredible experience. I expected it to work like every SaaS product I've ever built, which is just dive into the middle and, you know, figure out from there. So with traditional software development, I can ask a front end team to start figuring out what the UI should look like and you can have a back end team stub out endpoints and connect them later. It doesn't really matter the order in which you start building stuff. For building an agent, it's completely different. You really need to think through what the end state is that you want, what the architecture should look like and, and make sure that you have a strong foundation so you don't run into that issue of not being able to, to go backwards. So I didn't realize how intricate the process was, just the upfront planning stages and figuring out what the architecture would look like. And I started from a point of I don't code. So this is not me coming in as an engineer saying, oh yeah, it's super easy to build an agent. I came in and said, I want to build an agent. How do I do that? Question mark. So it's shown me that really anybody who has an idea can build a thing and there's really not a lot of friction there anymore because we can use natural language to describe what it is we want to build and have, you know, these nice little robots go in the background and write all the code for us. Which is great. But it also showed me how clear we need to be on describing the thing that we want to build and also being in control and managing whatever's doing the building for us. So some of the things that I realized were I was having an issue where I was asking the agent to go into my email and pull out a URL and go to a website. And it was unable to do that and said, oh, you need to install this, a headless browser and do this. And I was like, that seems a little ridiculous to have to do all of that to get you to click on a link. So I looked at how the link was formulated, I looked in the code, I saw that it was written incorrectly and I said, oh, I think you just need to strip out you Know this. This ending. And sure enough, that was the solution to just strip out an ending. And the. The LM was. Hadn't thought through it that way because it didn't have the same context that I had about what I was trying to accomplish. And so it showed me that you can't just trust the output that you're being told this is what you need to do and will often over complicate whatever it is you're asking it to do. And so push back. Say, you know, is there a simpler way to do this? I want this to be. I want to use as few tokens as possible to do this or whatever it is. The human needs to direct the agent. The human needs to direct the AI into doing what the human wants it to do and not the other way around, because that's what I was talking about managing agents before. It's really up to us to make the decisions and then for the technology to execute them. So I think everyone should build. I got to this aha point where I was like, oh, this is the zone that engineers talk about when they're completely fixed and they need to stay focused. And I have so much more empathy for engineers now and how they work. And I really think that everyone in an organization, from the CEO to a junior HR person, should be building something just to understand this is how the technology works and to figure out if there are possibilities to build something that will help you make your job easier.

[Nathan Isaacs]

Yep.

[Mike Mace]

So it's kind of like this is this. This needs to. The idea of being able to build an agent needs to be as familiar to us in the future as how to use a word processor is today. It's a fundamental building block of productivity, which is super enticing to me and super interesting and at the same time, scares the heck out of me, Katie, because you were talking about headless browsers and other stuff like that. I mean, for me, as a. I mean, I'm a marketing guy, for goodness sake. It's. It's throughout my entire career in tech, I've been told to stay away from code, that I'm a danger to myself and I shouldn't touch it. So if I want to learn this, you've convinced me that I need to learn this. I need to do this and figure out how to make my own agents and how to make them good. How do I get started on that?

[Nathan Isaacs]

How do.

[Mike Mace]

How do I even approach that?

[Katie Robeli]

Well, to be clear, I didn't know what a headless browser was. I just knew that sounded like more complexity than I wanted to deal with. Is there an easier way to do this? Let me see if I can figure it out. Sure enough, when I went back and asked, can we do this instead? I was the. I was told, oh, yeah, you can do that.

[Mike Mace]

Yeah, sure.

[Katie Robeli]

So you don't need to know what all of these things are. If it sounds crazy complicated, it probably is and you should push back. So the best way to get started, I'm just going to say Claude for to mean any LLM that you interact with. So open up Claude. Just cloud chat. You don't need to even open cloud code and say, I want to build an agent. How do I do it? Tell me what the best practices are from an engineering perspective and walk me through them step by step. And if it skips any steps or it's. It's too. If step one is too much, say, simplify that for me. The greatest part about interacting with Claude is that they're not going to judge you for how dumb your question is. Yeah, I will ask some super dumb questions that I probably would be too. Not ashamed, but too embarrassed to ask.

[Mike Mace]

Too embarrassed. Yeah.

[Katie Robeli]

Someone else with Claude, I don't care. Doesn't matter. So I can say, talk to me like I'm 2 years old and I've never used a computer before. How do I do this? And it will dumb it down to as. As much as you need it to. And go slow. Build something really small. If you don't have any ideas, then ask Claude. Say, I want to build an agent. Something incredibly small that I can just understand how the pieces fit together.

[Mike Mace]

Nice. Okay, so you can start with regular AI chat and then you want to be kind of demanding and directive with it. Don't be embarrassed about asking dumb questions or any of these things. Really force it to act like your helper and coach and servant and you can kind of bootstrap yourself up that way. I like that a lot. I think. I think I may even have the courage to try that this weekend. That sounds really cool. So if I accidentally erase my entire hard drive, I'll let you know. Not that I'm expecting that to happen. I think I'll give it some guardrails as we do it. So. So, Katie, let me. Or go ahead.

[Katie Robeli]

If you're. If you're too nervous to do it on your own or you think this is too much for me to take on by myself. There are so many groups out there of builders for various levels. You could be in like a purely beginners group where you're just learning with each other. You can join a group of next level people who are, are there to help you if you need that. Look on LinkedIn, do a Google search for, you know, agent building groups that, that you can join and then that way you can have other people to bounce ideas off of or build something together or start your own business.

[Mike Mace]

Yep. Cool. I like it. Okay, let me summarize back a few of the key points because this is really good stuff. So I think one of the things you talked about was don't just throw people into AI and expect them to be successful. They need guardrails, they need processes, they need training, they need to. They're not. I'm sure there are some people who can discover it for themselves, but most people need to be guided toward what they should be doing. Sounds like that's especially important as we start to get to agentic situations where you're not just doing queries to AI, but you're asking it to do tasks for you. And you've got to make sure you've got the right guardrails around for that, for things like privacy and information that has to be retained and other issues like that. We talked about the process of educating and training AI versus developing software and how that's different. You talked about evals, you know that the evaluations that can be used in different types of evals. I thought that was the whole idea of the, the human based evals to me was really, really good. Because if you just look for evals, you get, I've done this. You do a Google search for AI evals, you get a lot of stuff that's very mechanistic, focused on how to evaluate the correctness of AI answers as opposed to human impact focused. So, so doing a search for. What was the term? It was, it was human something evals.

[Katie Robeli]

Human in the loop.

[Mike Mace]

Human in the loop evals. If you look for that, you'll find the right sorts of evaluations. And especially if you're a researcher designer, you're worried about discovery and not just discovery, sorry, you're worried about how does this stuff apply? How does human insight apply to AI training? That's a really critical one to look at. We talked about discovery being really, really important. You got to do discovery even better than before because the AI is being trained, won't forget. And then we talked about agents and we've all got to learn how to make agents the same way as we know how to use a word processor or a spreadsheet. And you can do it by bootstrapping yourself. Up with the help of an AI bot. Are those the key points you wanted to hit? Is there anything else that's like, oh man, there's this burning thing I wanted to mention that I want to be sure I get across to people.

[Katie Robeli]

You know, I think one of the main things that organizations can do to be successful is to be very clear on what their goals are.

[Mike Mace]

Yes.

[Katie Robeli]

And before AI came along, organizations weren't good at this. It's not like all of a sudden they're good at it now. But in order to use this technology to the most advantageous way for them is to be very clear about this is the objective we want to hit as an organization and then understand how customer needs play into that, do the research, use these AI tools to do that research and then only build what's necessary to achieve that objective. Again, just because you can build doesn't mean you should.

[Mike Mace]

Yeah.

[Katie Robeli]

And that everything you build has a cost. It may have a smaller engineering cost, but it's going to have a higher cost for your customers, for you to be able to maintain and for you to be able to grow your, your customer base if it gets too muddied,

[Podcast Narrator]

if it gets too.

[Katie Robeli]

All these different things.

[Mike Mace]

I like that a lot. Thank you. So Katie, so you're consulting now. Do you consult on these sorts of issues? What, what should people call on you for if they, if they, if they need help?

[Katie Robeli]

I do. My focus is really on pre seed through series B startups working with organizations to understand what kind of lightweight processes will help them grow. So I help scale and optimize all kinds of processes for efficiency and predictability within an organization.

[Mike Mace]

Nice. So if people have follow up questions, how do they reach you?

[Katie Robeli]

You can reach out to me on LinkedIn. You can get in touch with me through my website, Scaleyze Systems. S C A L I z e systems.com book a free 15 minute consultation with me or send me an email. I'm open to answering questions from, from anyone. Send me a LinkedIn request with a note. I also run a monthly product operations Lean Coffee for any leaders in product operations that are running the discipline within their organization that meets monthly. So reach out to me for an invitation for that as well.

[Mike Mace]

Okay, cool. Well, thank you much Katie. This is fascinating stuff. Really appreciate your time today and good luck training the AI of the future.

[Katie Robeli]

Thank you so much, Mike. It's a great pleasure.

[Podcast Narrator]

Want to keep the conversation going? You can find the show [e-mail] podcast if you haven't already. Don't forget to follow us on Apple Podcast, Spotify, Overcast or Google Play, so you never miss an episode. And if you enjoyed today's show, please share it with a friend or leave us a rating and review on Apple Podcasts. And until next time, this is Insights Unlocked, an original podcast from User Testing.