Big Technology Podcast: "Are 95% of Businesses Really Getting No Return on AI Investment?"

Host: Alex Kantrowitz
Guest: Aaron Levie (CEO, Box)
Date: September 17, 2025

Episode Overview

This episode dives into the provocative claim from a recent MIT study that 95% of businesses are seeing no return on their AI investment, despite massive enterprise spending on generative AI. Host Alex Kantrowitz sits down with Box CEO Aaron Levie, fresh from the BoxWorks conference, to unpack the nuance behind the headlines. The discussion covers the real-world state of AI adoption, why internal builds often fail, the evolution and promise of AI agents, the economics driving the industry, and what’s next for both business and consumer applications.

Key Discussion Points & Insights

The MIT Study: "95% Get No ROI from AI"

• Headline Claims
  • Recent MIT study (as cited in Axios) causes a stir by reporting that 95% of organizations see no return on AI investments.
  • Enterprises spent $30–40B on generative AI, yet most pilots and projects are claimed to fail.
• Aaron Levie's Take
  • "I'm shaking my head on actually like seven dimensions." — Aaron Levie [03:05]
  • The discussion around AI ROI is polarized: Wall Street alternates between fear over wasted investments and utopian enthusiasm that AI will eventually upend all software.
  • Reality is more nuanced: AI adoption is still in early days; many "failures" are a natural part of experimentation.

Why So Many Projects "Fail"

• Proof of Concept Phase
  • Most current projects are pilots: exploratory, trial-oriented, and naturally high-risk for failure.
  • Early adopters are still sifting through what works and what doesn’t with new tools and vendors.
    • "By definition, you're kind of in the Wild West." — Aaron Levie [03:23]
• DIY vs Off-the-Shelf
  • Internal AI builds (DIY) have a much higher failure rate than partnerships or off-the-shelf solutions.
  • Many non-tech companies tried to assemble custom AI stacks, only to end up with unmanageable, complex architectures.
    • "You might have 10 or 15 different pieces of software...before a single user could actually interact with AI." — Aaron Levie [04:08]

The Value of Applied AI, Not Custom-Built

• Tailored, Purpose-Built AI
  • AI delivers the best ROI when tightly integrated into specific workflows (e.g., document analysis, data extraction).
  • Companies like Box build on open-source, but with internal expertise, not expecting every law firm or business to invent their own stack.
    • "Open source is insanely valuable, but not in the sense where a law firm should go off and build their own AI project." — Aaron Levie [08:21]

Change Management & Workflow Reengineering

• Not a Plug-and-Play Solution
  • Existing workflows often require redesign to harness AI's potential.
    • "You do have to reengineer the work...We will actually have to reengineer some of our business processes to make agents effective." — Aaron Levie [10:46]
• AI Coding as a Leading Example
  • Software engineering workflows have already adapted, shifting coders into more "manager-of-agents" roles.

Pilots vs. Production Use

• Misinterpretation of Value
  • Many failures reflect pilots or small-scale efforts—not final verdicts on AI's value.
  • Survey data may miss the broader informal use of AI tools (e.g., employees using ChatGPT personally).

Individual vs. Enterprise Use

• Personal Adoption Is Rapid
  • Only 40% of firms have official large language model (LLM) purchases, but 90% of employees reportedly use personal AI tools daily [13:45].
  • "It's empirical that we're choosing to use these technologies on a daily basis because they're adding that productivity." — Aaron Levie [14:14]
• Still Early for Enterprise Readiness
  • We're only beginning to tap into advanced enterprise use cases—most still in R&D or pilot stage.

AI Agents: Buzz or Breakthrough?

• Defining ‘Agent’
  • "It is now the new term of art for talking to an AI system that is doing work for you." — Aaron Levie [31:30]
  • Agents don’t merely answer questions; they run multi-step processes, loop through models, and can do minutes or hours of work before human intervention.
• The "Decade of Agents"
  • 2025 marks the start of serious discussion on AI agents, but the real transformation will take a decade, echoing historical tech adoption patterns.

Concrete Business Examples from Box

• Box Automate and AI Agents
  • Box is releasing tools enabling businesses to create end-to-end workflows powered by AI agents (e.g., contract review, data extraction, onboarding).
  • Agents can be deployed as components within larger processes, leveraging enterprise context and content.
    • "Box Automate lets you basically build these agents on demand...in a workflow that leverages your existing content." — Aaron Levie [37:25]

The Trustworthiness of AI Output

• Grounding in Reliable Data
  • With proper context and high-quality inputs, Aaron says, hallucinations and model inaccuracies can be minimized.
    • "You could nearly eradicate all of, if not the vast majority of hallucinations or accuracy issues." — Aaron Levie [20:19]
  • Human review will remain part of knowledge work: people will increasingly serve as reviewers/editors of AI-generated content.

Industry Economics: Big Bets and High Stakes

• Massive Spending, Strategic Losses
  • OpenAI’s projected $115B cash burn through 2029 is strategic; the prize is automation at scale for all knowledge work.
  • "If you believe like I do...this is the single biggest technology that we’ve probably ever had access to. And so if you think about this as sort of a third industrial revolution...That’s a very small number when you think about the size of the economy." — Aaron Levie [41:47]
• Startups vs. Incumbents
  • Early adopters/innovators will reap outsized returns; laggards risk disruption, though some entrenched giants can afford to wait.

Notable Quotes & Memorable Moments

• On Pilots & ROI Claims
  • “Lots of attempts...will absolutely fail, because by definition they're pilots and we're still in the early phases.” — Aaron Levie [03:22]
• On Traditional Companies Building Their Own AI
  • “A law firm should not go off and build their own AI project. That's a recipe for disaster.” — Aaron Levie [08:21]
• The Inconvenient Truth about Workflow Change
  • “If you don't do all of that work, you're probably not going to get a 2x or 5x gain from AI...Some of our business processes [must be] reengineered to make agents effective.” — Aaron Levie [10:46]
• On Employee Behavior Outpacing Enterprise Readiness
  • “90% of employees use personal AI daily...It’s because they're getting value from it.” — Alex Kantrowitz & Aaron Levie [13:45–14:14]
• On the Role of Human in the Loop
  • “We will be the reviewers of the AI agents' work, we will be the editors, we will be the managers, we’ll be the orchestrators. That’s how you get the productivity gains.” — Aaron Levie [22:18]
• On Industry Betting Big
  • "This is the single biggest technology we’ve probably ever had...$100 billion loss...that's a very small number when you think about the size of the economy." — Aaron Levie [41:47]

Timeline of Important Segments

• [03:05] Aaron Levie critiques the MIT study and headlines about wasted AI investment
• [04:08] Discussion of why internal, custom-built AI projects flop versus applied solutions
• [08:21] Open source, DIY, and the real winners and losers in AI development
• [10:46] Need for workflow reengineering; AI is not a drop-in solution
• [13:45] Data on employee-driven AI usage vs. official enterprise deployments
• [20:19] Context engineering and alleviating hallucinations
• [22:18] Paradigm shift in knowledge work: human as agent manager
• [31:30] What exactly is an "AI agent" and why the term matters
• [37:25] Box’s real-world deployment of agents and practical AI automation
• [41:47] Economic rationale for AI industry’s massive investments and losses

Final Thoughts

The episode paints a nuanced, realistic picture of where AI sits in the enterprise landscape: exciting ROI is happening, but only in focused, well-integrated use cases, and mostly among early adopters. The term "agent" is quickly becoming central, marking a shift from AI as a tool to AI as an autonomous collaborator. Yet, the journey for most organizations is just beginning; success will come to those who systematically rethink their processes and embrace the hard work of change management, not to those waiting for a plug-and-play panacea.

For more on Box’s AI offerings: box.com
Listen to future episodes of Big Technology Podcast for continued analysis of AI’s ongoing impact.