World Bank | The Development Podcast

Episode: Keeping Score: Measuring Impact in Development

Date: September 28, 2024
Host: Samuel Ouzabafi

Overview

This episode explores the crucial question of how impact in development is measured—and why it matters—focusing on the World Bank Group’s new streamlined scorecard. Through real project stories from Mozambique and Jordan and conversations with development leaders at the World Bank and Asian Development Bank, the episode dives deep into the shift from simply tracking dollars spent to measuring lives transformed. The episode emphasizes transparency, accountability, and collaboration across development institutions to maximize real-world impact.

Key Discussion Points & Insights

1. What Does Measuring Impact Mean in Development?

• Personal Stories Illustrate Impact:
  • Mozambique’s Climate-Resilient Classrooms:
    After Cyclone Freddy, schools with retrofitted, storm-resistant classrooms provided continuous, safe education for children.
    • Katerina (Student): “When it rained, we didn’t even sit in the classroom. We put our books in plastic sheets and we ran home. Now the school is excellent.” (00:34, 04:00)
  • Jordan’s Cash Transfer Program for COVID-19 Relief:
    Cash support helped families meet basic needs and offered skills training for better livelihoods.
    • Fadia El Hanawe (Mother): “Before receiving the supplementary support, many things were lacking for my children like paying rent, school expenses, water and electricity. Now, thank God, it covers my needs.” (04:17)
    • Zaher Zayadna (Trainee): “It helped prepare me to find jobs. They gave me a certificate for working in this field so I can rely on myself and improve my family’s living conditions.” (05:07)
    • Fadia, about skills training: “When we receive training at the National Aid Fund, it helps us with finding job opportunities because the support will not be provided for the long term. My dream is to establish my own business in the future.” (06:20)

2. The World Bank Group’s New Scorecard: A Shift in Mindset

• From “Dollars to Lives Transformed”:
  • Ed Mountainfield (VP, Operations, Policy & Country Services):
    “The World Bank Group Scorecard is our yardstick for measuring the results of the development work that we do all over the world ... It’s about shifting from dollars to lives transformed.” (00:48, 07:57, 13:51)
• Fewer, More Focused Metrics:
  • Reduced from over 150 indicators to just 22 critical results indicators.
  • Designed to hone in on outcomes that matter most—such as food security, climate change adaptation, gender equality, and learning poverty. (08:08)
• Radical Transparency & Data Disaggregation:
  • Methodologies are clarified, published, and data is made accessible, allowing anyone to drill down by gender, youth, disability, country, or region.
    • Ed Mountainfield: “It’s much more transparent and systematic ... publish those methodologies, and also to make the data itself much more transparent so you can disaggregate it ... working towards radical transparency as part of this scorecard.” (09:22)
• Tracking Real People, Real Change:
  • Example: Bank’s food insecurity initiatives aim to reach almost 1/3 billion people, with a focus on Africa and South Asia. (10:36)
  • The scorecard enables both accountability and the ability to question and redirect efforts as needed.

3. The Importance of Keeping Score

• Accountability & Motivating Focus:
  • The scorecard is akin to a school report card, helping teams understand where to double down and where good news can be celebrated.
  • Ed Mountainfield: “If you make everything a priority, you kind of make nothing a priority ... really zeroing in on the things that we think are really the most impactful things.” (09:58)
• Communication & Storytelling:
  • New “results narratives” will be included so data is augmented by contextual stories.
    • Ed Mountainfield: “Data is very powerful, but it’s only one way of telling the story ... we’re also going to include in the scorecard what we’re calling results narratives.” (12:43)
• Future Transparency:
  • At upcoming meetings, 20 of 22 indicators will be published on a new interactive website for stakeholders and the public. (12:17)

4. A Broader Development Shift: MDB Alignment & Practical Challenges

• Pan-Development Bank Collaboration:
  • Lu Shen (Director, Results Management, ADB): "It’s about improving people’s lives at the end of the day ... this is at the fundamental basis for everything that we do.” (01:05, 15:19)
• Setting and Missing Targets:
  • Setting targets like SDGs is important, but so is harmonizing how metrics are created and tracked across diverse countries and agencies.
  • Lu Shen: “Ultimately, in order to have any kind of impactful reporting, you need to have the same metrics, people need to measure it in the same way. And everybody has to buy into the same system.” (16:05)
• Project Example – Behavioral Change:
  • Water, sanitation, and hygiene (WASH) project involving handwashing in schools—teaching that impact goes beyond infrastructure to changing daily behaviors as a ripple effect through communities. (17:49)
• Alignment is Key for Credibility:
  • On climate and other global challenges, MDBs (Multilateral Development Banks) are working toward standardized methods to credibly pool and report achievements.
    • Lu Shen: "I think it's extremely important for us to put our heads together to come up with a common methodology … so we can jointly report on what is our collective contributions. It’s not easy ... outcomes are difficult, right? Because we talk about attribution versus contribution all the time.” (19:24)

Notable Quotes & Memorable Moments

• “It’s about shifting from dollars to lives transformed.”
  Ed Mountainfield, 13:51
• “Now the school is excellent. I really like learning. I like to be a teacher when I grow up.”
  Katerina (Mozambique Student), 04:00
• “It’s about improving people’s lives at the end of the day. … This is at the fundamental basis for everything that we do.”
  Lu Shen, 15:19
• “If you make everything a priority, you kind of make nothing a priority.”
  Ed Mountainfield, 09:58
• “Data is very powerful, but it’s only one way of telling the story … we’re also going to include what we’re calling results narratives.”
  Ed Mountainfield, 12:43

Timestamps for Important Segments

• Real-life impact stories from Mozambique & Jordan:
  (02:10–06:42)
• Introduction and explanation of the new World Bank Scorecard:
  (07:56–14:46)
• Discussion with Lu Shen (Asian Development Bank) on impact, targets, and measuring challenge:
  (15:06–20:10)

Conclusion

The episode powerfully unpacks how measuring impact is central to effective development—it's not enough to tally dollars spent; the true score comes from lives improved. With the World Bank’s new, transparent scorecard and initiatives to align global metrics, development agencies are moving toward more accountable, people-focused, and resilient development solutions. Collaboration, clarity, and a relentless focus on meaningful outcomes are key.