Into Africa – The Think Tank Making Impact Evaluation Skills Accessible to Every Tanzanian

Podcast: Into Africa
Host: Katherine Nzucki (Afropolitan segment, Center for Strategic and International Studies)
Guest: Dr. Constantine Manda, Co-founder and Director, Impact Evaluation Lab at the Economic and Social Research Foundation, Tanzania; Assistant Professor, UC Irvine
Date: March 20, 2025

Main Theme / Purpose

This episode explores how the Impact Evaluation (IE) Lab at Tanzania’s Economic and Social Research Foundation (ESRF) is demystifying and democratizing impact evaluation (IE) skills. Dr. Constantine Manda discusses the importance of making these technical skills accessible to every Tanzanian, regardless of educational background or professional status. The conversation highlights practical approaches (from SMS vaccine nudges to radio plays), innovative capacity building for young researchers, and a vision for embedding rigorous, contextually relevant evaluation into Tanzanian policymaking and civil society.

Episode Breakdown

1. Introduction to Impact Evaluation

• Definition and Purpose
  • Dr. Manda defines IE as the science of causally identifying whether a policy or intervention is the true source of observed outcomes, as opposed to other possible factors ([01:21]).
    • Quote: “It's really a way to say, you know, based on the methodologies that we're engaging with, based on the assumptions of these methodologies, how confident are we that we've been able to say for certain that the outcomes that we care about...are realized because of this program…and not anything else?” — Constantine Manda ([01:40])
  • IE is described as especially critical but not limited to developing contexts.

2. Motivation for the Impact Evaluation Lab

• Bridging the Capacity Gap
  • The IE Lab was born from Dr. Manda’s Berkeley fellowship experience. He saw firsthand the gap between the need for locally owned impact evaluation in Tanzania and the scarcity of qualified local researchers able to fulfill that need ([02:50]).
    • Quote: “I was very, very interested in trying to close the gap between the need for impact evaluation in Tanzania and the supply of that impact evaluation from local researchers.” ([03:08])
  • The lab’s mission: make IE training widely accessible, not just for elite scholars in the Global North.

3. Real-World Applications: COVID-19 Vaccine SMS Experiment

• Design and Rationale

  • In the wake of governmental skepticism about COVID-19 mitigation, the IE Lab designed a low-cost, randomized trial leveraging SMS ([04:23]).
    • Empathy-based vs. future-oriented (time preference) messaging were tested against a placebo (unrelated) SMS.
  • Key Experiment Details:
    • Messages ran over six weeks; random assignment ensured validity.
    • The placebo message focused on discouraging child marriage, to ensure health effects were tied to the vaccine-related content—not SMS attention alone ([06:31]).
    • Quote: “We implemented an SMS campaign that highlighted empathy for others... The second treatment...was trying to make Tanzanians think a lot about their future.” ([07:13])

• Outcomes

  • Both messages increased vaccination rates, with empathy messaging performing slightly better—but the difference wasn’t statistically significant ([09:09]).
    • Quote: “So this suggests that Tanzanians care about others more than they care about their future self.” ([09:15])
  • Katherine draws connections to Ubuntu and communal values ([09:24]).

4. Building Capacity Through Research and Trainings

• Youth Empowerment & Policy Uptake

  • A strong legacy of the Lab: supporting young Tanzanian researchers, many now entering top global doctoral programs ([05:46]).
  • Training for Policy Makers
    • High-level officials: from one-day to week-long custom introductions; core aim is exposure, not expertise ([09:53]).
      • “[We] equip them with the abilities to then consume research so that...they can adjudicate whether this is a research project that the government should put resources towards...” ([10:32])
    • Regular community and student trainings run longer and focus more on methodology ([13:51]).

• Why Focus on Bias?

  • Training prioritizes understanding and detecting statistical bias, given policymakers’ need to cut through informational noise ([12:06]).

5. Making Evaluation Mainstream

• Universal Exposure as a Development Imperative
  • Tanzania faces limited resources and abundant development ideas—the IE approach cuts waste and speeds efficient progress ([15:33]).
    • Quote: “Knowledge on what works and what doesn't work is going to save enough money and resources...such that we would be better able to inform the government and the private sector on what works for development.” ([15:51])

6. Creative Communication and Community Involvement

• Radio Dramas Against Misinformation

  • The Lab partnered with media, researchers, and the health ministry to run evidence-based radio plays that counter public health misinformation ([17:13]).
    • Quote: “…an amazing infrastructure to deliver really rigorous information from medical professionals that is informed by rigorous research…” ([17:36])

• Feedback Loops: The Drama Group Method

  • After a Dar es Salaam meal voucher study, results were dramatized by a local university troupe and played for participating communities, closing the research feedback loop ([19:36]).
    • Quote: “…we didn't want just this benefits to accrue to just the people who received the vouchers. We wanted to explain the results of this project and that it could be applicable across all other households.” ([21:54])

7. Specific Projects in Focus

• Meal Voucher Behavioral Study
  • Vouchers for healthy food (eggs, dairy) given to households in a poor Dar es Salaam neighborhood — led to sustained positive behavior changes ([19:36–23:10]).
    • Unique twist: research findings communicated via a community play ([21:54]).
  • Katherine’s Note: “Very few people have time to sit down and read a 10,000 word policy brief.” ([23:53])

8. Ethics, Diversity, and Decolonizing Research

• Addressing Extractive Habits of Academic Research
  • Dr. Manda underscores the progress and limits in moving impact evaluation away from extractive models. Diversity at every level—especially among researchers—builds empathy and ethical rigor ([24:03]).
    • Quote: “Diversity is really, really the key in relaxing this tension…you are going to allow for the ability of people to have a much greater empathy for human subjects.” ([24:23])
  • Discusses a new initiative connecting UC Irvine and HBCUs to widen the pipeline for Black scholars studying both Africa and its diaspora ([25:20]).

Notable Quotes & Memorable Moments

• On the “Why” of Evaluation:

  • “As somebody who grew up in Tanzania, seeing many wasteful programs being implemented, it's really frustrating. And this is really why I've become very passionate about program evaluation.” – Constantine Manda ([16:38])

• On Impactful Capacity Building:

  • “A few of them have succeeded in getting really amazing admissions to doctoral programs…really the most reputable institutions around the world.” ([05:46])

• On Sustainability and Community Ethics:

  • “We publish at fancy top journals, but there's very little feedback. Sending them a policy brief or inviting a few of them to a workshop is not sufficient.” ([21:46])

• On Diversity and the Future of Research:

  • “If we get as many different people who reflect the amazing diversity of the world, we'll get more interesting questions and hopefully learn more about ourselves as a result.” ([26:38])

Timestamps for Core Segments

• 00:01-01:16 – Introduction and overview
• 01:21-02:45 – Defining impact evaluation
• 02:50-04:08 – Motivation and founding story of IE Lab
• 04:23-09:00 – COVID-19 SMS experiment: goals, design, outcomes
• 09:37-16:57 – Policy trainings, methods taught, focus on bias, universal exposure
• 17:13-19:36 – Radio play research, community impact
• 19:36-23:10 – Meal voucher randomized trial & community feedback innovation
• 23:10-26:52 – Ethics in research, US-Tanzania academic partnerships
• 27:37-end – Recognition of young Tanzanian scholars and wrap-up

Summary Takeaways

• The Impact Evaluation Lab at ESRF is shifting the paradigm of who designs, implements, and consumes evidence in Tanzanian development.
• Through creative outreach—from SMS to radio plays—the Lab is embedding evaluation skills and skepticism into government and society, aiming to optimize limited resources.
• Capacity building for the next generation of Tanzanian researchers is at the Lab’s heart, with tangible results in global scholarly achievement.
• Manda and his team prioritize ethical, inclusive research—emphasizing feedback, community involvement, and diversity.
• The future, per Manda, lies not just in better methods, but in widening access so all Tanzanians—not just elites—can understand, critique, and benefit from rigorous evaluation.

For further reading and links to the publications discussed, check the show notes.