Cultivating an Experimental Mindset in Your Organization

Summary

Podcast Summary: HBR On Leadership

Episode: Cultivating an Experimental Mindset in Your Organization

Release Date: October 1, 2025
Host: Amanda Kirsey, Eric Jorgenson
Guest: Stefan Tomke, Harvard Business School Professor
Topic: Building a culture of experimentation to drive innovation and better decision-making in organizations

---

Overview

This episode focuses on how leaders can foster an experimental mindset within their organizations. The conversation, featuring Stefan Tomke—author of "Experimentation: The Surprising Power of Business Experiments"—dives deeply into why disciplined experimentation leads to better business decisions, what blocks widespread adoption of experimentation, and how companies of all sizes can practically embed experimentation into their culture.

---

Key Discussion Points & Insights

1. The Value and Power of Business Experiments

• Experiments as Engines of Value: Experiments can deliver surprising and often substantial value by testing assumptions rather than relying solely on intuition.
• Example from Microsoft Bing:
  • A minor change by an employee resulted in over $100 million in additional annual revenue, demonstrating the unexpected benefits of empowering employees to experiment.
  • "It was the most successful experiment that was run at Bing. So what made the difference? The employee had the power or the authority to run the experiment." — Stefan Tomke (03:36)

2. Barriers to Widespread Experimentation

• Cultural Resistance: Many managers are risk-averse or reluctant to admit uncertainty.
• Lack of Infrastructure & Skills: Some organizations lack tools, resources, or encouragement to test ideas rigorously rather than running informal "trials."
• Overestimated Risks, Underestimated Upsides:
  • Managers often worry more about potential downsides (losing traffic, negative customer reactions) than the possible gains.
  • It requires humility to say, "I don't know," and let experiments speak for themselves.
  • "The higher up you go, the more you get paid...you want to be a decision maker. But we get it wrong most of the time." — Stefan Tomke (05:49)

3. Rigorous Experimentation: Not Just for Tech Giants

• Applicability Beyond Digital: Controlled experiments are critical in offline settings too (e.g., retail opening hours at Kohl’s).
  • "Calculating the cost savings is easy, but the big question is, what's actually going to happen to our revenue?...We won't know until we actually do it." — Stefan Tomke (06:38)

4. Tools and Infrastructure: Doing Experimentation at Scale

• From Custom Tools to Third-party Solutions:
  • Large companies (Amazon, Microsoft, Netflix, Booking.com) built their own experimentation platforms.
  • Now, scalable third-party tools make robust experimentation more accessible, even for brick-and-mortar organizations.
  • "It's gotten a lot easier than if you wanted to start five or ten years ago." — Stefan Tomke (08:17)

5. Building a True Culture of Experimentation

• Symptoms of a Poor Culture:
  • Results of experiments are ignored or rationalized away if they don't fit management intuition.
• Cultural Transformation:
  • At Booking.com, experiments are a daily routine, running over 30,000 per year, and the entire organization is empowered to generate hypotheses.
  • Small companies with less data can still experiment—just with larger, riskier changes to achieve statistically meaningful results.
  • "At Booking.com...running experiments is like breathing. You do it every single day." — Stefan Tomke (11:10)

6. Experimentation in Unlikely Industries

• Creative Industries & Entertainment:
  • Even highly creative fields can benefit, e.g., Netflix uses rigorous testing to guide content and user experience decisions.
  • Experiments add transparency and challenge assumptions—even when "creative genius" dominates.
  • "Creative talent is really important, but that doesn't create certainty...experimentation allows us to create hypotheses and then rigorously test them." — Stefan Tomke (14:01)

7. Counteracting ‘HiPPO’ Culture

• HiPPO: "Highest Paid Person’s Opinion"
  • Too many organizations default to following whoever argues most convincingly—rather than real evidence.
  • "We call them hippos. And everyone knows hippos are very dangerous animals." — Stefan Tomke (15:44)

8. Scaling Experimentation Across the Organization

• How Much Experimentation is Enough?
  • Most companies under-experiment: The concern should be increasing testing, not avoiding excess.
  • "Right now, most organizations test too little. So I don't think you should be too worried about testing too much." — Stefan Tomke (17:33)
• Organizational Models:
  • Centralized: A single team runs experiments for others; good for starting out and maintaining expertise.
  • Decentralized: Empowering everyone; fastest for scaling but can dilute best practices.
  • Center of Excellence: Hybrid—central capability-building plus distributed experimenters; found to work best at scale.
  • "Companies found that's actually a very good compromise..." — Stefan Tomke (19:58)

9. Recognizing Cultural Success

• What Does Success Look Like?
  • Experiments become routine—like financial analysis in meetings.
  • "You know it's working when someone asks, 'Where's the experiment?' Or when someone actually walks in and says, 'Here is the experiment.'" — Stefan Tomke (20:30)

10. Tips for Individuals at Any Level

• Start Small, Create Awareness:
  • Run small experiments within your area and share the results; culture change starts with grassroots proof.
  • "I've talked to organizations that actually started this way and then got bigger and bigger...but you got to get started, don't wait." — Stefan Tomke (22:02)

11. The Modern Manager’s Role in an Experimental Organization

• Grand Challenge: Set ambitious strategic direction to focus hypotheses and experiments.
• Systems & Resources: Build infrastructure that makes running experiments easy and routine.
• Role Model: Lead by example—embrace humility, act on findings, and democratize decision-making.
• "Managers need to live by the same rules...That's a different style of leading." — Stefan Tomke (25:06)

---

Notable Quotes & Memorable Moments

• On experimentation over intuition:
  "Rather than trying to follow our intuition or our opinions, why not just run the test and let the test tell us what works and doesn't work?"
  — Stefan Tomke (03:51)

• On humility and leadership:
  "Walking into a meeting and...just go into the meeting and tell everybody, listen, quite honestly, I don't know what's going to happen, so let's just find out."
  — Stefan Tomke (05:33)

• On organizational change:
  "You empower people to do things on their own. At the same time, you actually have someone who centrally owns this capability as well."
  — Stefan Tomke (19:57)

---

Timestamps for Key Segments

• [02:42] — Why businesses must experiment more
• [03:50] — Microsoft Bing example and power dynamics
• [04:52] — The risks managers fear and humility in uncertainty
• [07:16] — Physical world experimentation (Kohl’s case)
• [08:37] — The evolution from internal to third-party tools
• [10:01] — Booking.com’s experimentation culture
• [13:02] — Creative industries and Netflix’s experimentation approach
• [15:44] — The danger of ‘HiPPOs’ in decision-making
• [17:14] — How much (and where) to experiment in organizations
• [18:09] — Centralized, decentralized, and Center of Excellence models
• [20:29] — Cultural test for experimentation
• [22:43] — Grassroots experimentation and change
• [23:06] — The evolving role of managers in an experimental company

---

Conclusion

The episode makes a powerful, clear case: experimentation should be foundational in modern organizations not just for tech companies, but everywhere decisions and innovations occur. Building a true culture of experimentation demands humility, structural change, and leadership by example. Individuals at any level can help by starting small and spreading a results-focused mindset. Managers must evolve, setting direction, building infrastructure, and modeling data-driven curiosity for their teams.

“You wouldn't make a big decision without running the numbers—you shouldn't do it without an experiment either.”

---

For further resources, check out Stefan Tomke's HBR article "Building a Culture of Experimentation," or his book, "Experimentation: The Surprising Power of Business Experiments."

Transcript

[Amanda Kirsey]

Know the feeling when AI turns from tool to teammate. If you Rovo, you know. Discover Rovo by Atlassian and streamline your workflow with AI powered search, chat and agents.

[Eric Jorgenson]

Deal's not just another payroll platform, it's one your team might actually enjoy. Hr, IT and payroll together finally built in house built for peace of mind. Visit deel.com HBR.

[Amanda Kirsey]

Welcome to HBR on leadership case studies and conversations with the world's top business and management experts, hand selected to help you unlock the best in those around you. Hi, I'm HBR senior editor and producer Amanda Kirsey. As a leader, you face uncertainty all the time. Experiments offer a way to test assumptions, but it's not enough to simply run them. Their value comes from designing them carefully and being willing to act on what they reveal, even when the findings upend your expectations. Here's HBR IdeaCast host Kurt Nickish with a conversation from 2020 about what leaders need to know to design rigorous experiments and then put the evidence to work.

[Eric Jorgenson]

In science, the need for experimentation is cut and dry. You come up with a hypothesis, whether it's about how storm clouds move or how cells in the body die, and you set up an experiment to test it. There's a method, it's called the scientific method, and you test it over and over again until you're sure that it's replicable and and your answers are right, or at least as right as they can be until new variables come to light or the landscape changes. In business, there isn't currently as much experimentation. Value has been placed on experience, on the intuition of managers and leaders, and that's a bad thing, says today's guest. Even in the most innovative industries we can think of, more can be done to set up experiments, test the results, and deliver better products and services to customers. And this goes far beyond a B testing at tech giants. Our guest today is Stefan Tomke. He's a professor at Harvard Business School. He's the author of the book Experimentation the Surprising Power of Business Experiments, and he also wrote the HBR article Building a Culture of Experimentation. Stefan, thanks for coming in.

[Stefan Tomke]

Thanks for having me.

[Eric Jorgenson]

Just to start, pretend I'm a business leader, make the case for me. Why do we need to experiment more in business?

[Stefan Tomke]

Well, first of all, it can generate a tremendous amount of value. Let me give you an example. Microsoft's Bing, which is its search engine. An employee working sort of at Bing came up with an idea on how to sort of display sort of ads. You know, the manager didn't think much of it and they kind of shelved it, but the employee insisted. At some point, the employee decided just to launch an experiment, to run a test, a control test. And when he ran the test, that little change, a few days of work generated more than $100 million of additional revenue in that year alone. And of course, more revenue going forward. In fact, it was the most successful experiment that was run at Bing. So what made the difference? Well, the difference was that the employee had the power essentially, or the authority to run the experiment, to launch it and to test it. It's the test that actually told you what works and doesn't work.

[Eric Jorgenson]

And not the manager.

[Stefan Tomke]

And not the manager. The problem is in a lot of innovation, especially sort of when you're trying to predict customer behavior, we get it wrong most of the time. And so rather than trying to follow our intuition or our opinions, why not just run the test and let the test tell us what works and doesn't work?

[Eric Jorgenson]

And what's the answer to that? Like, why aren't people doing it?

[Stefan Tomke]

Well, there are lots of reasons why not. People are doing it at scale especially. So some people are sort of running simple experiments because they refer to an experiment as something like a trial. We're trying something that's not really an experiment, sort of in the scientific sense. And they don't do many of those because they either don't have the infrastructure to run many tests, they may not have the tools to do. So it may be too expensive to run it. And then they may decide that, listen, we run a test and we get some results and then nobody listens to us anyway.

[Eric Jorgenson]

Right. Do managers overestimate the downside to experiments and underestimate the upside?

[Stefan Tomke]

I think sometimes they're too overly concerned about the risk of running the experiment for good reasons. You have a lot of traffic. You may not want to launch something that results in a loss of customers visiting your websites, for example.

[Eric Jorgenson]

Right. If it goes down.

[Stefan Tomke]

If it goes down. And so if you don't have good stoppage rule kill switches and things like that sort of in place, and there may be a risk aversion, it's also stepping into the unknown. And quite honestly, it takes humility to admit that. I just don't know. Walking into a meeting and we're launching this thing and everybody has some hypothesis about what the outcome is going to look like and just go into the meeting and tell everybody, listen, quite honestly, I don't know what's going to happen. So let's just find out.

[Eric Jorgenson]

Even though I get paid more, we get paid more and I'm in charge. Yes, I don't know either.

[Stefan Tomke]

Exactly. And the higher up you go, the more you get paid, the more senior you get. You get paid to make tough decisions, and you want to be a decision maker, and you create sort of an organization that ticks a little differently to do this sort of thing. By the way, it's not just the online world, it's also the physical world where companies are running experiments. And even there, we have to make big decisions, sometimes very expensive decisions. And it's the experiments that can in fact, adjudicate whether we want to do something or not. Kohl's big retailer and so forth, Coles hires a consulting company, and the consulting company basically does a cost analysis. And they go to senior management and tell them, listen, we figured out that you can save a lot of money if you open your stores an hour later. Now here you are, you're running this company, and you have to make a decision. Should we do that? Calculating the cost savings is easy, but the big question is, what's actually going to happen to our revenue? Are customers going to buy less if we open an hour later? So how do you make these kinds of decisions? We can analyze and analyze, but we won't know until we actually do it until we run the test. And in this case, they did. And so they ran controlled experiments in which they sort of set up these tests, opening an hour later. And lo and behold, at the end, the result was that it didn't make much difference.

[Eric Jorgenson]

Just so we're on the same page, how do you go about setting up an experiment? Are there Playbooks for this?

[Stefan Tomke]

Well, first of all, there are tools. A lot of companies that I describe in the book built tools, their own infrastructure, build their own tools, because when they got started many years ago, the tools weren't around. So you look at an Amazon, a Microsoft, a netflix, a booking.com, i mean, you go through them, and it was about a dozen or so. They decided to do it themselves.

[Eric Jorgenson]

So they knew that they had questions they wanted to answer, and they just figured out a way to do it.

[Stefan Tomke]

They figured this is going to give them a competitive advantage if they can kind of go out and just test a lot. And they knew that they often get it wrong. And so they started to invest in infrastructure. And so at a place like Microsoft, for example, you have a very, very large group that basically runs the infrastructure. Something like, the last time I checked, it was something like 85, 90 people or so that are just sort of doing infrastructure but the good thing that happened a few years ago is there are now third party tools as well that can do this, both in the online spaces and in the brick and mortar spaces, which do sort of a lot of the heavy lifting for you, a lot of the statistical stuff and so forth. And so it's gotten a lot easier than say if you wanted to start five or 10 years ago.

[Eric Jorgenson]

Developing a culture for this is probably a little bit different.

[Stefan Tomke]

I think it may be potentially harder than getting the tools and building the tools because now we're dealing with behaviors, with beliefs, with norms and all sorts of things.

[Eric Jorgenson]

How does this show up in companies if the culture for experimentation is not working? What do you actually see and observe?

[Stefan Tomke]

Well, the classical example is they start running experiments. We have an experiment, we hand over the result to the group that asks us to run the experiment and then nothing happens or they will start to challenge the experiment. Something must have gone wrong. I remember a story where an angry person actually called one of the tool vendors sort of in this space and complained about the tool being wrong. The person ran an experiment that actually showed if you give customers less choice in his setting, you get better performance. That was kind of just counterintuitive because everything that he believed in up to this point, he says you should give people more choices. And so he was really disturbed by the finding. And so he called them and complained that there's a flaw in the tool. Something in the tool must be wrong because the result doesn't match the experience that he's had. And he's been doing this for a long time. So you run into that sort of thing.

[Eric Jorgenson]

Which kind of underlines your point that experiments bring new insights that you just can't develop on your own.

[Stefan Tomke]

Correct. There's a company called booking.com, which most of us use. In fact, it's the biggest accommodations platform in the world. More than 1.5 million room nights are booked on the platform each day. It's a two sided platform. This is what we call. It's got suppliers on one side which are hotel operators, for example. And of course it's got customers like us on the other side. And booking.com runs a massive number of experiments. My estimates are, and I'm probably on the low side, they told me it's my estimates it's over 30,000 a year of experiments. And it's a really, really fascinating company. It's also a highly successful company. Their gross profits are in the high 90s percent and they don't really have any assets. They don't really own any accommodation. So it's a super competitive industry too. And so how do they get away with this? And the answer to this is they run a lot of experiments and they created an experimentation culture where almost running experiments is like breathing. You kind of do it every single day. Kurt, you have to think about the numbers here. Even if I'm running a lower number of experiments, I mean, they're running more than 100 new experiments a day. You have to have an organization that can even come up with so many hypotheses.

[Eric Jorgenson]

I mean, you mentioned the number of transactions that booking.com does in a day. How key is that to being able to run experiments? Does that also work for places that just don't have data like that?

[Stefan Tomke]

Yes, it works for places that also have a lot less traffic. The underlying math changes. Sort of what you have to do algorithmically is very different. In fact, if you have very large sample sizes, a lot of traffic, for example, you can really fine tune, you can sort of do very, very small changes and you can kind of pick up whether that change actually causes something to happen. As your sample size shrinks, you're going to have to go for bigger changes. We call it the power of an experiment. You have to power an experiment. Statistical power. I recommend for companies that are sort of smaller that maybe they run experiments that are a little bigger. Now what happens also, and this is something that actually happened at IBM when they started to do this, they realized that they have way too many websites. So yes, they had very little traffic on some of these websites, but they didn't need all the websites. So it actually led to a process of consolidation. They said, listen, we don't really need all these things. So what we'll do is we'll consolidate and we get sort of more traffic on fewer websites, which then allows us to sort of run more experiments.

[Eric Jorgenson]

I wondered if there are companies or industries outside of consumer facing tech or outside of scientific or pharmaceutical companies where experimentation really feels foreign.

[Stefan Tomke]

Well, I mean, the classical companies, I think are sort of in the creative industries, where the assumption is that everything is driven by creatives. Look at entertainment, for example, and look at what Netflix has done. So Netflix kind of flipped that around. And they operate in the creative industry, but they are completely experimentation driven. And I think it was a big wake up call for the entertainment industry because when you go in and you run Netflix, you, you are part of their ecosystem, their experimentation ecosystem. They run a massive number of tests because they want to find out what works and doesn't work. By the way, running the test and getting a result doesn't mean that you have to blindly follow what the result is. Because sometimes there are good strategic reasons why you may not want to implement what the test tells you.

[Eric Jorgenson]

Right. Or there are trade offs to whatever.

[Stefan Tomke]

Benefits or trade offs, for example, or maybe there may be a contractual violation or something like that. But what that test does is it actually adds transparency to the decision. So you cannot pretend that we're doing this because it's good for the customer or something like that, or good for the viewer. It adds clarity that we understand from the test what's good for the viewer. But there may be other reasons why we may not want to do it. And adding that transparency to what you're doing, I think is sort of a big value and allows a company like Netflix to operate really in the creative industry with a testing approach. I don't want to diminish the value of creative talent because creative talent is really important. But that doesn't create certainty in terms of decision making. To me, the creative talent and the intuition is an important part of experimentation because it allows us to create hypotheses. You have to ask yourself, Kurt, where do these hypotheses come from?

[Eric Jorgenson]

Yeah, still from people. It's still people asking questions. Absolutely. Or having ideas.

[Stefan Tomke]

Yeah, absolutely. So what I'm saying is that running all these experimen experiments, they're all hypotheses that came out of product groups and it's the people who come up with these hypotheses. So where do they get the ideas? Well, it's intuition, sometimes it's insights, surprising customer surprises, things that thought that were true. And then they observe something that doesn't quite fit, sort of what they know. It's usability labs. So they still. These companies all run qualitative research and. But they do all the kinds of things that other companies do, but they do it for generating hypotheses which are then rigorously tested versus other organizations that generate the hypothesis and go directly from hypothesis to launch.

[Eric Jorgenson]

Right. Based on whoever is the best speaker. Based on the speaker or makes the best case in a meeting, rather than.

[Stefan Tomke]

Yeah, yeah, there's a word for that in the community. They're called hippos.

[Eric Jorgenson]

Hippos?

[Stefan Tomke]

Yes. Highest paid person's opinion, hippos. And we all know that hippos are very dangerous animals.

[Eric Jorgenson]

Meet Rovo, your AI powered teammate. Rovo connects to your organization's knowledge and.

[Stefan Tomke]

Lives on Atlassian's trusted and secure platform. Connect Rovo to your favorite SaaS apps.

[Eric Jorgenson]

To get the personalized context you need.

[Stefan Tomke]

Unleash AI with AI powered search, chat agents and studio.

[Eric Jorgenson]

Get started with Rovo, your new AI teammate powered by [e-mail].

[Amanda Kirsey]

If you're saving for your future, you may be wondering, what's my next move? If you're not sure, you're not alone. Maybe it's time to talk to a financial advisor at Savant Wealth Management. Right now you can schedule a free consultation. Just go to savantwealth.com podcast. You don't have to go it alone and you shouldn't. Visit savantwealth.com podcast to learn more and schedule your free consultation. Savant is a registered investment advisor. Past performance is no guarantee of future results.

[Eric Jorgenson]

I think a lot of executives are probably also not used to knowing how much experimentation to do. How do you know what to experiment on and how do you know what to let be?

[Stefan Tomke]

Yes, you have to empower people to make that decision. And the reality is right now I think most organizations test too little. So I don't think you should be too worried about testing too much. Okay, yes, there's probably a point at which you test too much because you need an organization that can absorb all that knowledge or all those findings that are generated by all these tests. That's true, and we need to think about that. But I don't think that's the problem in most organizations right now. Right now, they're not doing enough.

[Eric Jorgenson]

If you're bringing this into a company, do you try to do this company wide? Do you try to start with a team or a division and scale it up from there?

[Stefan Tomke]

So there are different ways to organize your experimentation teams. There are three models that I describe in the book. One model is really more a centralized approach. I basically have a center, a group that's responsible for experiments. And they're like a service organization where you can come from a business unit, you can commission experiment, and they'll run it for you and they give you the results.

[Eric Jorgenson]

That's interesting.

[Stefan Tomke]

That's one model. And a lot of companies start out that way because they are kind of a little uncertain how this is all going to work out. They may not believe that the company is ready to do this at large scale.

[Eric Jorgenson]

It probably simplifies training and it lets people dip their toe in without really having to.

[Stefan Tomke]

Exactly. And you have a few experts and they kind of make sure that people don't do foolish things. Then another form is to have it decentralized, completely decentralized. So now we're shifting the autonomy basically to people and allow pretty much anybody to run experiments, and we don't centralize it anymore. And of course, there you have to trust people. You have to know that they're actually capable of doing this. And it's a way, of course, to rapidly scale things. But what happens there is when you start to put all these. You spread all these sort of your experts around, and they're all way sort of through the company, they get very busy, and you kind of lose the focus on building capabilities because you need to always kind of get better and better. And so there's no coordinated approach to this. Everybody kind of does their own thing. So what companies have found is they go from centralized to decentralized, and they want to scale things, but then they realize that they need to have a more coordinated approach, and then they create something which they call a Center of excellence. And the center of Excellence is kind of a hybrid model then, where you have sort of a core group that actually is responsible for developing capabilities, experimentation capabilities, kind of know what tools to use and push the envelope. But at the same time, you take people out of that group and then place them into the different organizational units that are doing this, and they're basically there to help as well. Companies found that that's actually sort of a very good compromise, because on one hand, you kind of empower people to do things on their own. At the same time, you actually have someone who centrally owns this capability as well.

[Eric Jorgenson]

How do you know when it's really working?

[Stefan Tomke]

You know the way you're really working? I think it's a cultural test. And I tell you, here's the test. You sit in a meeting and you're discussing a decision, and you know when it's working, either when someone asks, where's the experiment? Or when someone actually walks into the meeting and says, here is the experiment. When these kinds of discussions are happening every single day without you having to ask for these things, then you know things are kind of working. I call it. It's like running the numbers, right? You wouldn't. When you go into a meeting, you always expect people to do some financial analysis. It's almost a given. So it has to be like that. It has to be like running a financial analysis. It has to be a given that you kind of do a test, you run an experiment, and unless you've done it, we're not going to make a decision.

[Eric Jorgenson]

Say you're an individual contributor. You may be a manager, you may be a frontline worker, but you buy into this like, you see the value of experiments, you want your organization to do more. What do you do to try to bring a culture of experimentation to a place that is still relatively new to it?

[Stefan Tomke]

What you can do as an employee is first of all, raise the awareness around sort of you.

[Eric Jorgenson]

What does that mean?

[Stefan Tomke]

That means basically explaining sort of to people what sort of the value of the experiment, what experiments are. But then also I think at the same time is maybe try to do some of these things in the areas that you control. Yes, I see the difficulty sometimes and I hear this from people saying, okay, I get you, but they're two levels up. I'm not sure that they do. So what can I do? So I always tell them, start small, get going. And then this is often what happens. I've talked to organizations that actually started this way and then got bigger and bigger. They said, we started out and we ran an experiment and we went to the meeting and we told people what the experiment sort of showed us and so forth. And they kind of listened to it and they gradually start to sort of understand the value of it. But you got to get started, don't wait.

[Eric Jorgenson]

What kind of manager is then the successful manager in a company that has a culture of experimentation? Because in the past, maybe it used to be people who had experience, people who had intuition. Now, when you run experiments, what is the type of manager who excels and advances in an organization that has a culture of experimentation?

[Stefan Tomke]

So you can ask the question, if everything is adjudicated by experiments or by test, what's the role of the manager anyway?

[Eric Jorgenson]

Right.

[Stefan Tomke]

I kind of break it down into sort of three different things that they should do. First, role, I think of a manager is to set a grand challenge. What we don't want to do is we don't have an organization that just does experiments willy nilly with no direction. So there needs to be a grand challenge. A grand challenge, for example, could be we want to have the best user experience in the industry. That grand challenge then can be broken down into different pieces which then can be addressed with hypotheses which are then tested so you give them a directionality. There needs to be a program, a systematic program that sort of aims for some bigger goal. So that's the grand challenge. The second thing I think that managers need to do, especially in this kind of environment, they need to place the systems, resources and organizational designs that allow for the large scale experimentation to happen. Things like that don't happen by themselves. You need to invest in tools, you need to make sure that you've got the right organizational design to start out with and maybe then change it when things don't work. So you have to think about that as well. And you need to make sure that all the systems are in place so someone like that employee at Microsoft can just kind of push a button essentially and just launch and run this thing. If it takes employees weeks and weeks to set up an experiment, what are the odds of them doing it at large scale? It's not going to happen. So you got to make it easy as well. And you did empower people to do it. You need to democratize experiments. And the third one is they need to be a role model. They need to live by the same rules. So when we go into a meeting and we propose a course of action and someone says that's really nice, we'll run a test and let you know what happens. We need to then have the humility to say let's do it and let's do it quickly. So we need to live the same way. We need to kind of do the same thing that we ask our employees to do. So that's a different style of leading.

[Eric Jorgenson]

Stefan, thank you so much. Maybe we'll try some experimentation on this show as well.

[Stefan Tomke]

Thank you, Eric. Great to be here.

[Eric Jorgenson]

Stefan Tomke is a professor at Harvard Business School. He's the author of the book Experimentation the Surprising Power of Business Experiments, as well as the HBR article Building a Culture of Experimentation.

[Amanda Kirsey]

HBR on Leadership will be back next Wednesday with another hand picked conversation from Harvard Business Review. If this episode helped, you, share it with your friends and colleagues and follow the show on Apple Podcasts, Spotify or wherever you listen to podcasts. And while you're there, consider leaving us a review. And when you're ready for more podcasts, articles, case studies, books and videos with the world's top business and management experts, find it [e-mail] this episode was produced by Mary Dew and me. Amanda Kersey on Leadership's team includes Maureen Hoch, Rob Eckhart, Tina Toby Mack, Erika Trexler, Ramsey Kabaz, Nicole Smith and Ann Partholomeu. Music is by Coma Media. Thanks for listening.