Behind the Numbers: What GenAI at Work Taught Us This Year, and What We Can Expect From It In 2025
eMarketer Podcast Episode | December 9, 2024
Introduction
In the December 9, 2024 episode of Behind the Numbers: an EMARKETER Podcast, hosted by Marcus alongside guests Dan Van Dyke, VP of Generative AI at eMarketer, and Henry Powderly, SVP of Media Content and Strategy, the discussion centered on the evolving landscape of Generative AI (GenAI) in the workplace. The conversation delved into current AI adoption trends, the challenges faced by organizations, future expectations for GenAI by 2025, and insights into optimizing AI productivity.
AI Adoption Trends in 2024
The episode began with a reference to a November 12th Salesforce article highlighting a slowdown in AI adoption among U.S. workers. According to the Slack Workforce Index, AI adoption rates in the U.S. increased marginally from 32% to 33% between March and August 2024, a stark contrast to the six-point surge from January to March.
Dan Van Dyke expressed skepticism about the report's validity, stating:
"I'm not convinced that this is anything but statistical noise." (03:10)
"If we still see that this plays out again, I'll believe it's real." (03:10)
He noted that global AI adoption continues to rise, with only the U.S. and France showing minor increases, suggesting possible seasonal influences affecting the data.
Henry Powderly acknowledged the potential factors contributing to the slowed adoption:
"There's a lot of effort that needs to go into working with these tools, learning how to best communicate with them, or finding outputs." (05:16)
He highlighted issues such as insufficient training and the perception of using AI as "cheating," which may deter broader adoption.
Challenges in AI Adoption
The conversation delved deeper into the barriers hindering AI integration in workplaces:
- Training Deficits: A significant number of workers have minimal training in AI, with 61% reporting less than five hours of learning and 30% receiving no training at all.
- Perception of AI: Many employees feel uncomfortable admitting to using AI for tasks like client emails or internal communications, fearing it may be seen as a shortcut.
- Adopter and Age Gaps: There's an implicit suggestion that early adopters have already embraced AI, leaving a challenge in encouraging the early majority and older employees to integrate these tools effectively.
Marcus highlighted the discrepancy between leadership's strategic focus and employees' interest in using AI to handle administrative tasks:
"Executives wanting employees to prioritize upskilling and innovation. Employees expecting to have that time saved by using AI to catch up on busy work and existing projects." (12:18)
The Future of GenAI by 2025
Looking ahead, both Dan and Henry shared their visions for GenAI's trajectory:
-
Enhanced Corporate Data Access: Integration of intranet data could significantly improve AI output quality by providing context-specific information.
Dan Van Dyke remarked:
"Access to more corporate data... the output would be much stronger." (09:11)
-
Long-Term Memory for AI: Advancements in AI's ability to remember user preferences and past interactions could streamline workflows, reducing the need for repetitive task inputs.
Dan Van Dyke added:
"Memory being the key dimension to get everybody over the line and start using the tools." (10:34)
-
Agentic AI: The emergence of AI agents capable of operating computers and automating tasks is expected to gain traction, with platforms like Claude and OpenAI leading the competitive landscape.
Dan Van Dyke noted:
"Claude and OpenAI compete to be the platform of choice for developing these agents." (10:40)
Henry Powderly emphasized the rapid evolution of GenAI tools:
"Gen AI hasn't been a constant thing all year that works at a constant level. These tools have updated at a very fast rate." (08:02)
He also pointed out the diminishing enthusiasm among workers who, after initial experimentation, face challenges in refining AI outputs to meet their needs.
Overhyped Aspects of GenAI
The discussion shifted to the misconceptions surrounding GenAI's capabilities:
-
Ease of Use: The belief that GenAI tools can effortlessly deliver high-quality outputs with minimal input is often overstated.
Dan Van Dyke criticized this notion:
"The idea that like, if you prompt it, like you're just going to get a great Output simply doesn't measure up." (13:55)
-
Productivity Gains: While AI promises significant efficiency improvements, both Dan and Henry warned that real productivity requires substantial effort in learning and integrating these tools into workflows.
Henry Powderly added:
"We should be doing the work right now to learn how to use these tools and to experiment with them." (15:23)
Dan Van Dyke concurred, suggesting that productivity gains are a means to an end rather than an immediate deliverable.
Quantifying AI Productivity Gains
Addressing how companies can measure AI's impact on productivity, Dan Van Dyke shared eMarketer's approach:
"We've run about 23 pilots across the organization... estimate the amount of time that they save every week." (17:23)
This method involves:
- Implementing AI tools in various departments.
- Tracking time saved through self-reported data.
- Aggregating these metrics to present a collective view of productivity gains.
However, Dan acknowledged the limitations of self-reported data, noting potential inaccuracies but emphasizing its current status as the best available method.
The Transformative Potential of GenAI
Concluding the discussion, Henry reflected on GenAI's overarching impact on information retrieval and business processes:
"I really think things like ChatGPT search and perplexity and the generative approach to gathering information is going to change that a lot." (19:23)
This sentiment was echoed by a Wall Street Journal piece referenced by Marcus, where AI is described as a general-purpose technology akin to the steam engine or electricity, driving substantial economic growth and innovation.
Eric Rinjolfsson, professor at Stanford Institute for Human Centered AI, articulated:
"General purpose technologies are responsible for most of the productivity and economic growth." (19:52)
Conclusion
The episode underscored that while GenAI holds immense potential to transform workplaces, realizing its benefits requires overcoming significant challenges related to training, perception, and effective integration. As GenAI tools become more sophisticated and accessible, the next wave of adoption in 2025 is poised to drive substantial productivity and innovation across various sectors.
Notable Quotes:
- Dan Van Dyke (03:10): "I'm not convinced that this is anything but statistical noise."
- Henry Powderly (05:16): "There's a lot of effort that needs to go into working with these tools..."
- Dan Van Dyke (13:55): "The idea that like, if you prompt it, like you're just going to get a great Output simply doesn't measure up."
- Henry Powderly (15:23): "We should be doing the work right now to learn how to use these tools and to experiment with them."
- Eric Rinjolfsson (19:52): "General purpose technologies are responsible for most of the productivity and economic growth."
References:
- Salesforce Report on AI adoption trends
- Slack Workforce Index
- Google Workplace Survey
- Wall Street Journal article by James Millen and Eric Rinjolfsson