The Analytics Power Hour: Episode #266 – AI Projects: From Obstacles to Opportunities

In Episode #266 of The Analytics Power Hour, released on March 4, 2025, hosts Michael Helbling, Moe Kiss, Tim Wilson, Val Kroll, and Julie Hoyer delve deep into the intricacies of managing AI projects. This episode features Kathleen Walch, Director of AI Engagement Learning at the Project Management Institute (PMI), as the guest expert. Kathleen brings a wealth of experience in AI project management, particularly through her development of the CPMAI methodology for AI projects.

Kathleen Walch’s Journey into AI

Kathleen begins by sharing her longstanding engagement with AI, noting, “I’ve been in the AI space since before gen AI made it popular” (02:43). Her journey transitioned from marketing to data analytics and eventually to running a boutique analyst firm focused on AI. This diverse background provided her with unique insights into the challenges organizations face when adopting AI technologies.

The Current State of AI in the Industry

Moe Kiss introduces the episode by highlighting the surge in AI discussions within the analytics community. She notes, “Lots of organizations are also struggling to figure out how to actually identify, scope and roll out AI projects in a clear and deliberate manner” (00:56). Kathleen echoes this sentiment, emphasizing that while AI is a powerful tool, its successful implementation requires a strategic approach rather than a one-size-fits-all solution.

CPMAI Methodology: A Structured Approach to AI Projects

Kathleen introduces the CPM AI methodology, a step-by-step framework designed to guide organizations through AI projects. She explains, “We always start with phase one, business understanding – what problem are you trying to solve?” (03:23). This foundational step ensures that AI initiatives align with business objectives and address genuine needs, rather than being driven by the allure of the technology itself.

The Seven Patterns of AI

A significant portion of the discussion revolves around the seven patterns of AI that Kathleen and her team identified:

1. Hyper-Personalization: Treating each individual as unique, enhancing experiences in marketing, education, finance, and healthcare.
2. Recognition Patterns: Making sense of unstructured data through image, gesture, and handwriting recognition.
3. Conversational Patterns: Enabling human-machine interactions via AI-powered chatbots and large language models.
4. Predictive Analytics: Using historical data to aid humans in making better predictions.
5. Predictive Analytics and Decision Support: Identifying patterns and outliers in large datasets to support decision-making.
6. Goal-Driven Systems: Utilizing reinforcement learning and optimization for tasks like traffic flow management.
7. Autonomous Patterns: Removing humans from the loop entirely, as seen in autonomous vehicles and business processes.

Kathleen emphasizes the importance of matching the right AI pattern to the specific business problem, stating, “Large language models aren't great for everything, and generative AI isn't great for everything” (08:23).

Common Pitfalls in AI Projects

The conversation highlights several common reasons why AI projects fail:

• Over-Promising and Under-Delivering: Organizations often set unrealistic expectations, leading to disappointment and project abandonment. Kathleen cites Walmart’s failed autonomous bot project as an example where the ROI didn’t justify the investment (07:55).

• Scope Creep: Expanding the project’s scope beyond manageable boundaries can dilute focus and resources, resulting in suboptimal outcomes.

• Data Quality Issues: AI is fundamentally a data-driven endeavor. Kathleen underscores, “AI projects are not going to be free. And you really have to understand that” (13:36), pointing out that poor data quality and insufficient data can derail projects.

• Misalignment of Use Cases: Teams sometimes focus on what AI can do rather than what problem it should solve, leading to solutions that don’t address core business needs.

The Importance of Data in AI Projects

Kathleen passionately asserts that “AI is actually a data problem” (29:28). She explains that while technology plays a crucial role, the success of AI projects hinges on data quality, accessibility, and proper management. Kathleen warns against the temptation to fix AI challenges solely with technology, emphasizing the need for robust data-centric methodologies.

Power Skills in the Age of AI

A pivotal theme in the episode is the role of “power skills” – creative thinking, critical thinking, collaboration, and communication – in leveraging AI effectively. Kathleen discusses how these skills enable professionals to interact with AI tools more intelligently, ensuring outputs are accurate and aligned with business goals. She remarks, “Critical thinking is really now critical” (43:32), highlighting the necessity for users to question and validate AI-generated information.

AI Adoption Across Different Industries

The discussion also touches on varying levels of AI adoption across industries. While tech-centric regions like Silicon Valley embrace AI wholeheartedly, other sectors such as healthcare and construction adopt it more cautiously. Kathleen notes that industries with stringent regulatory environments or those relying heavily on human expertise, like healthcare, often implement AI in augmenting rather than replacing human roles.

Strategic Recommendations for AI Project Success

Kathleen offers several strategic recommendations for organizations embarking on AI projects:

1. Start Small, Think Big: Initiate AI projects with manageable scopes to demonstrate ROI before scaling.
2. Iterative Approach: Use short, iterative sprints to continuously evaluate and refine AI applications.
3. Focus on ROI: Clearly define the return on investment in terms of money, time, and resources before committing to AI initiatives.
4. Avoid Proof of Concept Traps: Instead of controlled proof of concepts, deploy pilot projects in real-world environments to gauge effectiveness accurately.
5. Embrace Power Skills: Foster critical and creative thinking within teams to better interact with and leverage AI technologies.

Kathleen underscores the importance of aligning AI projects with genuine business problems, stating, “Don’t try and fit that square peg in a round hole. You don’t want to shoehorn your way just because you want to use AI” (07:55).

Conclusion and Key Takeaways

The episode concludes with Kathleen encouraging listeners to adopt a strategic and data-centric approach to AI projects. She reiterates the necessity of power skills in maximizing AI’s potential and warns against the pitfalls of over-promising and scope creep. The hosts express their appreciation for Kathleen’s insights, acknowledging the depth and practical applicability of her expertise.

Notable Quotes

• Kathleen Walch: “AI is not, set it and forget it.” (03:23)
• Kathleen Walch: “Critical thinking is really now critical.” (43:32)
• Mo Kiss: “You don’t want to shoehorn your way just because you want to use AI.” (07:55)
• Kathleen Walch: “AI projects are not going to be free. And you really have to understand that.” (13:36)
• Kathleen Walch: “Large language models aren't great for everything, and generative AI isn't great for everything.” (08:23)

Final Thoughts

Episode #266 of The Analytics Power Hour offers a comprehensive exploration of the challenges and opportunities within AI project management. Kathleen Walch’s expertise provides valuable frameworks and practical advice for navigating the complex landscape of AI implementation. Whether you’re a seasoned data professional or an organization looking to harness AI’s potential, this episode delivers actionable insights to help you turn AI obstacles into opportunities.