The trend that most marketing leaders are missing

Summary

---

Podcast Summary

MarTech Podcast ™ // Marketing + Technology = Business Growth

Episode: The Trend That Most Marketing Leaders Are Missing
Date: September 24, 2025
Host: Benjamin Shapiro
Guest: Steven Roach, VP of Ecosystems and AI at Qualified Digital

---

Episode Overview

This episode explores a critical, commonly overlooked trend in marketing: the need for intentional and strategic integration of artificial intelligence (AI) within organizations. Benjamin Shapiro and Steven Roach zero in on when and why human oversight becomes non-negotiable in the AI stack, and examine why so many AI implementations miss the mark despite high expectations.

---

Key Discussion Points & Insights

1. The Danger of Treating AI as a "Magic Wand"

• Timestamp: 01:40
  Steven Roach emphasizes one of the biggest pitfalls: organizational leaders expecting AI to be a “magic wand” that instantly solves problems without strategic planning or human oversight.
  • Leaders often rush to "throw some AI at it" without clear purpose or structure.
  • The success of AI depends on intentional integration into workflows—not superficial adoption.

"One of the first red flags is our leadership may treat AI like a magic wand. Throw some AI at it and it's just immediately going to fix it itself. You have to be very strategic about how you implement AI into these tools."
— Steven Roach [01:40]

2. Understanding Deterministic vs. Non-deterministic AI Tasks

• Timestamps: 01:45–03:49
  Roach explains that not all tasks are suitable for full automation.
  • Deterministic tasks (clear, binary, yes/no answers): ideal for automation.
  • Non-deterministic tasks (judgment or subjective decisions): require human oversight to avoid pitfalls like bias or misjudgment.
  • Not recognizing the distinction leads to failed AI projects and security risks.

"If, if there is a very binary output, a yes or no question, does this exist or does it not, automate away. AI the crap out of it. But if you're saying, hey, artificial intelligence, I need you to make a judgment call on something and you're expecting it to get it right all the time, you're setting yourself up for failure. That's where you need humans in the loop."
— Benjamin Shapiro [03:49]

3. The Hidden Failure Rate of AI Agents in Business

• Timestamp: 02:40
  Roach references a recent MIT study showing that 95% of AI agents failed in organizational tests, not because of model limitations but due to knowledge gaps and poor implementation.
  • The challenge lies not in the AI models themselves, but how businesses deploy and interact with them.
  • Enterprise adoption is at risk if organizations don't address this disconnect.

"The most recent example of that is the MIT study, I believe 95% of AI agents have failed in organizations... It really didn’t come back to the models failing. It was just that gap of knowledge."
— Steven Roach [02:40]

4. How to Avoid AI Implementation Pitfalls

• Integrate AI with a clear strategy and understanding of its strengths/limits.
• Maintain “humans-in-the-loop” for non-deterministic, judgment-based, or high-risk tasks.
• Recognize that over-hyping "AI for everything" leads to leadership disappointment and wasted resources.

5. Enterprise Software and AI Are Not Mutually Exclusive

• Roach references commentary from the CEO of Box, noting that enterprise software will always require careful AI integration.
• The future isn’t about AI replacing systems, but enhancing them with thoughtful design and oversight.

---

Notable Quotes & Memorable Moments

• On AI Overconfidence:

  "You have to be very strategic about how you implement AI into these tools."
  — Steven Roach [01:41]

• On the Deterministic/Non-deterministic Divide:

  "If you are, let's just say, throw all deterministic factors into the non-deterministic, you're going to run into issues, you're going to run into security issues. Definitely."
  — Steven Roach [02:00]

• On Human Oversight:

  "If you're expecting [AI] to get it right all the time, you're setting yourself up for failure. That's where you need humans in the loop."
  — Benjamin Shapiro [03:49]

• On AI Implementation Failures:

  "It really didn't come back to the models failing. It was just that gap of knowledge... The implementation of those agents is still, you know, a massive drawback for a lot of organizations."
  — Steven Roach [02:45]

---

Timestamps of Important Segments

| Segment | Timestamp | |:---------------------------------------------------|:----------:| | Dangers of "magic wand" thinking | 01:40 | | Deterministic vs. non-deterministic tasks explained| 03:49 | | MIT study on AI agent failure rates | 02:40 | | Necessity of human oversight in AI | 03:49 | | Recommendations on robust AI integration | Throughout |

---

Conclusion

This episode offers a candid look at the realities of AI in enterprise marketing, urging leaders to avoid the hype and approach AI implementation with discipline, humility, and a clear understanding of when and where humans must remain actively involved. The trend most leaders miss? Believing AI can independently solve all problems—when, in fact, durable success stems from smart, strategic collaboration between tech and people.

Transcript

[A]

The Martech Podcast is a proud member of the I Hear Everything Podcast Network. Looking to launch or scale your podcast, I Hear Everything delivers podcast production, growth and monetization solutions that transform your words into profit. Ready to give your brand a voice? Then visit iheareverything.com.

[B]

From advertising to software as a service to data, across all of our programs and clients, we've seen a 55 to 65% open rate. Getting brands authentically integrated into content performs better than TV advertising.

[C]

Typical lifespan of an article is about.

[B]

24 to 36 hours. We're reaching out to the right person with the right message and a clear call to action. Then it's just a matter of timing.

[A]

Welcome to the Martech Podcast, a member of the I Hear Everything Podcast network. In this podcast you'll hear the stories of world class marketers that used technology to drive business results and achieve career success. Here's a host of the Martech Podcast. Benjamin Shapiro.

[C]

I'm Benjamin Shapiro and joining me today is Steven Roach, the VP of ecosystems and AI at Qualified Digital, which helps Fortune 500 brands transform customer experiences through data driven solutions. And today Steven will reveal exactly when human oversight is non negotiable in your AI stack. What's the first sign that a company's AI implementation tool is about to fail spectacularly?

[B]

That is a very loaded question because I think one of the first red flags is the our leadership may treat AI like a magic wand. Throw some AI at it and it's just immediately going to fix it itself. You have to be very strategic about how you implement AI into these, into these tools. And I think a lot of the warning signs have always been there. You just have to be able to integrate AI in a very strategic way and making sure that it's actually purposeful. You know, again, going back to deterministic and non deterministic factors, you know, if you are, let's just say throw all deterministic factors into the non deterministic, you're going to run into issues, you're going to run into security issues. Definitely there's logica is built in. I actually saw a recent podcast from the CEO of Box. He made that very clear. Enterprise software is not going to go away because of it. And we need to make sure that, you know, it is fully integrated into each step. There's layers to this. So you need to be very mindful again in how you integrate your AI infrastructure and making sure you're not falling flat. And the most recent example of that is the MIT study, I believe 95% of AI agents have failed in organizations. In a test study of like more than 600 organizations or 300 organizations, that's a very telltale sign. And it really didn't come back to the models failing. It was just that gap of knowledge. The models are so good. The implementation of those agents is still, you know, a massive drawback for a lot of organizations. So again, making sure that you are mindful of how you're actually implementing those agents or the full AI implementation across.

[C]

The board, it comes back to deterministic versus non deterministic. Very much right. If, if there is a very binary output, a yes or no question, does this exist or does it not automate away AI the crap out of it? But if you're saying, hey, artificial intelligence, I need you to make a judgment call on something and you're expecting it to get it right all the time, you're setting yourself up for failure. That's where you need humans in the loop. All right, that wraps up this episode of the Martech podcast, thanks to Steven Roach, the VP of Data Ecosystems at Qualified Digital. If you'd like to get in touch with Steven, you could find a link to his LinkedIn profile in our show notes or on martechpod.com or you can visit his company's website, which is qualifieddigital.com if you haven't subscribed yet and you want a daily stream of marketing and technology knowledge in your podcast feed, hit the subscribe button in your podcast app. Or Visit us on YouTube and we'll be back in your feed next week. All right, that's it for today, but until next time, my advice is to just focus on keeping your customers happy.

[A]

Thanks for listening to the Martech podcast and I hear everything. Production. Looking to launch or scale a podcast like this one for your brand? Then visit iheareverything. Com.