HBR On Strategy: Episode Summary – "Great Strategy Starts with Experimentation"

Release Date: April 16, 2025
Host: Harvard Business Review (Hannah Bates)
Guest: Stefan Tomke, Harvard Business School Professor
Co-Host: Kurt Nickish

Introduction: Embracing Experimentation in Business Strategy

In the episode titled "Great Strategy Starts with Experimentation," Harvard Business School Professor Stefan Tomke engages in a compelling discussion with Kurt Nickish about the pivotal role experimentation plays in shaping effective business strategies. Hosted by Hannah Bates, the conversation delves into why relying solely on intuition and experience may hinder innovation and how a scientific approach to testing can unlock significant value for organizations.

The Power of Controlled Experiments

Stefan Tomke opens the conversation by challenging the traditional reliance on gut instincts in decision-making. He underscores the importance of controlled experiments by sharing a transformative case study from Microsoft’s Bing search engine:

"An employee working at Bing came up with an idea on how to display ads. The manager didn't think much of it and they kind of shelved it, but the employee insisted... that little change... generated more than $100 million of additional revenue in that year alone."
(Stefan Tomke, [01:02])

This example illustrates how empowering employees to run experiments can lead to breakthrough innovations that purely managerial decisions might overlook.

Barriers to Effective Experimentation

Despite the clear benefits, Tomke identifies several barriers that prevent organizations from fully embracing experimentation:

• Misconception of Experiments: Many businesses equate experiments with mere trials rather than rigorous scientific tests.

  "People are doing it at scale. Especially some people are sort of running simple experiments because they refer to an experiment as something like a trial."
  (Stefan Tomke, [02:25])

• Resource Constraints: Lack of infrastructure, tools, and financial resources can impede the ability to conduct meaningful experiments.

  "They may not have the infrastructure to run many tests, they may not have the tools... it may be too expensive to run it."
  (Stefan Tomke, [02:57])

• Fear of Risk and Unknowns: Managers often overestimate the risks associated with experiments and underestimate the potential upside.

  "Sometimes they are too overly concerned about the risk of running the experiment for good reasons."
  (Stefan Tomke, [03:06])

Additionally, Tomke emphasizes the psychological barriers, such as the humility required to acknowledge uncertainty and the challenge of overcoming deeply ingrained intuitive decision-making processes.

Implementing a Culture of Experimentation

The discussion transitions to practical strategies for fostering an experimental culture within organizations:

1. Building the Right Infrastructure:

   • Early adopters like Amazon, Microsoft, and Netflix developed their own experimentation tools to gain a competitive edge.
   • Today, third-party tools have simplified the process, making experimentation more accessible.

   "It's gotten a lot easier than say, if you wanted to start, say five or 10 years ago."
   (Stefan Tomke, [05:52])

2. Developing Organizational Models:
   Tomke outlines three primary models for organizing experimentation teams:

   • Centralized: A dedicated group manages experiments as a service organization.
   • Decentralized: Autonomy is given to individuals or teams across the organization to run their own experiments.
   • Center of Excellence: A hybrid approach combining a core group that develops experimentation capabilities with distributed members supporting various units.

   "The center of excellence is kind of a hybrid model... it's actually sort of a very good compromise."
   (Stefan Tomke, [15:43])

3. Encouraging Hypothesis-Driven Testing:

   • Combining creative intuition with rigorous testing ensures that hypotheses are both innovative and scientifically validated.

   "But they do it for generating hypotheses which are then rigorously tested versus other organizations that generate the hypothesis and go directly from hypothesis to launch."
   (Stefan Tomke, [13:46])

Overcoming Cultural Resistance

A significant challenge highlighted is the resistance to experimental findings that contradict established beliefs. Tomke recounts an incident where an experiment's results surprised a stakeholder so much that they questioned the experiment's validity:

"He was really disturbed by the finding. And so he called them and complained that there's a flaw in the tool."
(Stefan Tomke, [07:07])

This underscores the importance of fostering a culture that values data-driven insights over personal biases or preconceived notions.

Leadership in an Experimental Organization

Effective leadership is crucial for nurturing an environment where experimentation thrives. Tomke outlines key roles for managers in such settings:

1. Setting Grand Challenges:

   • Leaders should define overarching goals that guide experimentation efforts.

   "First role, I think of a manager is to set a grand challenge."
   (Stefan Tomke, [19:57])

2. Providing Resources and Infrastructure:

   • Ensuring that teams have access to the necessary tools and systems to conduct experiments efficiently.

   "You need to invest in tools, you need to make sure that you've got the right organizational design."
   (Stefan Tomke, [20:03])

3. Modeling Experimental Behavior:

   • Leaders must embody the principles of experimentation, demonstrating humility and a commitment to testing hypotheses.

   "They need to be a role model. They need to live by the same rules."
   (Stefan Tomke, [21:03])

A successful manager in an experimental culture is one who not only encourages testing but also actively participates in and supports the experimentation process.

Conclusion: The Imperative of Experimentation for Strategic Success

"Great Strategy Starts with Experimentation" compellingly argues that embracing a scientific approach to decision-making is essential for modern businesses seeking to innovate and stay competitive. Through detailed discussions and real-world examples, Professor Stefan Tomke elucidates how experimentation can lead to substantial revenue growth, like Microsoft Bing's ad display changes, and streamline operations, as seen with IBM's website consolidation.

Organizations that cultivate a robust experimentation culture—equipping their teams with the right tools, fostering hypothesis-driven testing, and championing data over intuition—position themselves to navigate uncertainties and capitalize on emerging opportunities effectively.

By integrating experimentation into the strategic framework, businesses can move beyond conventional planning, enabling dynamic and evidence-based pathways to success.

This summary encapsulates the key discussions and insights from the "Great Strategy Starts with Experimentation" episode of HBR On Strategy. For a deeper dive into the topics covered, listeners are encouraged to access the full episode through HBR's podcast channels.