The Brainy Business Podcast – Episode 540: "The Art of Experimentation to Gain Consumer Insights" with Dr. Henry Stott

Release Date: October 14, 2025
Host: Melina Palmer
Guest: Dr. Henry Stott, Co-Founder of DEC Tech and Behavior Lab

Episode Overview

Behavioral economics reveals that customers don't act or choose the way we expect, making true prediction of consumer behavior tricky. In this episode, Melina Palmer explores advanced experimentation in business decision-making with Dr. Henry Stott, who brings decades of hands-on expertise developing immersive, high-fidelity trials for actual customer prediction. They cover the weaknesses of sanitized test environments, the necessity of contextualized randomized controlled trials, insights from big client projects (including Deliveroo and Lloyds), and how businesses can truly innovate (not just compete on price).

This episode is packed with practical case studies, debunks common testing myths, and provides a window into rigorous yet actionable approaches for designing smarter experiments when forecasting consumer response to new offerings.

Key Discussion Points & Insights

1. Why Traditional Prediction Fails

• Consumers as Unpredictable: Businesses often find their “obvious” decisions flop in the real world, even when backed by surveys or basic tests.

  “They don't do what they say they will do and don't act how we think they ‘should.’” – Melina Palmer [00:33]

• Dr. Stott distinguishes between precedented (data-driven, past-behavior-based) and novel (entirely new product/feature) business problems. The latter is his specialty:

  “Our specialization is more trying to understand how … people will react to this completely new decision environment.” – Dr. Henry Stott [05:09]

2. Immersive Randomized Controlled Trials (RCTs): The DEC Tech Approach

• Case Study – Deliveroo: Launching a subscription service with variable features and pricing (a la Amazon Prime) required pre-launch high-fidelity RCTs because no A/B test was possible.

  “We use our normal approach of an immersive randomized controlled trial in a sort of paid environment … to explore how consumers reaction changed as you changed the proposition.” – Dr. Stott [06:43]

  • Features tested: Delivery window, spend requirements, special promotions for subscribers, pricing points
  • Outcome: Deliveroo Plus was successfully launched with a design (14-day free trial, £11/month, £10 min order) directly shaped by this experimentation.

• Importance: Innovations that can't be A/B tested in market can still be robustly trialed by simulating ‘as real as possible’ buyer journeys online.

  “It's like A/B testing in a lab, but you can A, B, C, D, E, F, G test—covering lots of ground safely before you launch big changes.” – Dr. Stott [25:34]

3. The Danger of Only Competing on Price

• Value Over Price: Many companies fall into the trap of perpetually lowering prices due to lack of real innovation or misunderstanding of customer value drivers.

  “It is important … to not just look at price elasticity … but to look at that in conjunction with what the proposition is … so that it fits snugly into what people want.” – Dr. Stott [09:31]

• Obsessing Over Competitors Can Backfire: Example from Tesco vs. ASDA, where price chasing confused Tesco's own loyal shoppers and damaged price credibility due to loss aversion and erratic pricing.

  “…the chasing of ASDA was deeply undermining … because the majority of people shopping in Tesco were not shopping at ASDA.” – Dr. Stott [13:12]

4. How Context Shapes Decision-Making

• The Order vs. Magnitude of Price Differences: People are more sensitive to who is higher or lower priced than by how much, and they’ll pay more for features meaningful to them.

  “We tend to find that people are much more sensitive to the order of things than they are to the size of the differences.” – Dr. Stott [17:25]

• Experience Enhancements & Waiting Perception: Innovations like Domino’s Pizza Tracker or hospital signage dramatically improve perceived wait times—not because waits are shorter, but because friction and uncertainty are removed.

  “The problem with the delay is not necessarily the length of time, but [the] boredom where there’s nothing going on and you become impatient.” – Rory Sutherland, as recounted by Melina [19:13]

  • Empirical Example: Better signage in UK hospital emergency rooms reduced incidents of violence by easing uncertainty. [20:04]

5. Immersive Testing vs. Conjoint & MaxDiff

• Why Conjoint/MaxDiff Can Fail: These are preference elicitation methods stripped of real decision context, resulting in inaccurate predictions.

  “Conjoint and maxdiff are methods … so divorced from any kind of decision environment … that the results … are heavily distorted.” – Dr. Stott [29:15]

• Why Labs Should Be Realistic: Realistic, large-scale online environments (“immersive labs”) yield closer-to-market behavior and allow observation of drop-off points, understanding, and satisfaction.

  “The best way to get to these things is to immerse customers in an experience … which is as close as possible to the environment that they would naturally encounter.” – Dr. Stott [29:48]

6. Practical Case Study: Lloyds Bank

• Facsimile Journey Testing: Large-scale online RCT explored changes to home insurance renewal, measuring not just conversion and revenue, but satisfaction and comprehension.

  "We were looking to design a home insurance renewal process that worked best for the customers, but also worked best for the bank." – Dr. Stott [23:41]

  • Innovation Example: "Name your price" budgeting feature was tested and discarded—turned out it wasn't popular, sparing Lloyds the costly implementation.
  • Results: Optimized design raised acquisitions from 45% to 75%, a dramatic improvement.

  “Their acquisitions went up from 45% to 75%.” – Melina Palmer [34:43]

7. The Secret Sauce: Test More, Test Weird, Test Fast

• Don't Play It Safe: Throw in oddball, counter-intuitive ideas; sometimes these are the winners.

  “…the best thing about testing is the thing that you kind of throw in where you say this crazy thing … probably wouldn't work, but let's just see… And it performs exceptionally well above what you think should work…” – Melina Palmer [36:46]

• Novelty Sells: In retail and other settings, constant minor novelty or rotating features/experiences can boost engagement and conversion (e.g., lottery tickets themed with emojis or celebrities).

  “Something novel, something fun is quite often one of the outcomes of the experiment.” – Dr. Stott [38:22]

Notable Quotes & Memorable Moments

• On Sanity-Checking Your Experiments:

  “…the conversation around the risk of using sanitized test environments that unintentionally remove friction or complexity, making it too easy for someone to say yes. That kind of clean data may look good in a report, but it won’t predict real behavior under pressure.” – Melina Palmer [40:44]

• The Importance of Realistic Lab Testing:

  “Our clients tend to go on and then do the things that we advise them. And it’s always very gratifying to see how closely their experience aligns when they launch something for real with the advice that they’ve got.” – Dr. Stott [32:21]

• Market Innovation vs. Imitation:

  “Looking to competitors the whole time is, is a mistake.” – Dr. Stott [14:37]

• Small Tweaks, Big Impact:

  “…remarkably innocuous small changes can have profound effects on the economics of your business. …You have to kiss a lot of frogs [though] before you find that subtle change.” – Dr. Stott [34:58]

Timestamps for Important Segments

• Introduction to Experimentation in Business – [00:33]
• Dr. Stott’s Background and Approach – [02:43]
• Precedented vs. Novel Problems – [03:55]
• The Deliveroo Subscription Testing Case Study – [06:07]
• Pricing/Feature Sensitivity & Value Focus – [08:25]
• Fixation on Competitor Pricing: Tesco vs. Asda – [12:07]
• Order > Magnitude in Pricing Sensitivity – [16:13]
• Waiting Time, Experience & Savoring – [19:13]
• Case Study: Lloyds Bank Online Renewal Testing – [23:41]
• Realistic Lab Set-Ups at Scale – [31:05]
• The Limits of Conjoint/MaxDiff Testing – [29:05]
• The Value of Testing Bold & Novel Ideas – [36:46]
• Case: Novelty as a Sales Lever – [38:22]
• Closing Reflections & Takeaways – [40:44]

Conclusion & Key Takeaways

• Real change comes from realistic testing: Simulate the purchase journey as closely as possible to reality before launch.
• Don't just copy competitors— deeply understand your customers’ value drivers.
• Optimize for features, experience, and novelty, not just price point.
• Test boldly and broadly—innovation hides in the “weird” ideas.
• Small design tweaks can have outsized commercial impact, but only if you find them.

Final Reflection:

"Whether you're testing a new offer, onboarding flow, or subscription model, the question isn't just does this work? It's does this still work under real world conditions?" – Melina Palmer [40:44]

Links & Next Actions

See the show notes for connections to DEC Tech, Behavior Lab, relevant past episodes, and behavioral science books.
Find Melina Palmer @thebrainybiz everywhere and Dr. Henry Stott & DEC Tech on LinkedIn.

This summary covers all major topics, actionable insights, memorable quotes, and anchors for further exploration—perfect for listeners (and prospective testers) seeking to take smarter action in their own businesses.