The CGD Podcast: How Will We Measure the SDGs? – Casey Dunning

Date: March 22, 2016
Host: Rajesh Merchandani (Center for Global Development)
Guest: Casey Dunning (Senior Policy Analyst, CGD)
Main Theme:
A critical discussion of the adoption, complexity, and challenges surrounding the 230 indicators chosen to measure the Sustainable Development Goals (SDGs)—exploring their practicality, the dilemmas they introduce, and the implications for global development measurement.

Episode Overview

This episode dives into the final piece of the SDG puzzle: the 230 indicators agreed upon for measuring progress towards the 17 Sustainable Development Goals and their 169 targets. Host Rajesh Merchandani and guest Casey Dunning unpack the logic, challenges, and potential pitfalls of this ambitious framework, reflecting on its implications for global policy and data collection.

Key Discussion Points & Insights

The Numbers behind the SDGs

• 17 goals, 169 targets, 230+ indicators:
  • The SDGs consist of an expansive structure built for comprehensively addressing global challenges.
• Actual complexity is even higher:
  • While officially there are 230 indicators, Dunning notes, "if you look at the list, there's actually 241 indicators attached to targets. So some of the indicators are playing double duty." (Casey Dunning, 01:32)

Why Indicators Matter

• Indicators’ role:
  • They provide the measurement framework necessary to hold countries accountable and move beyond aspirational rhetoric.
  • "If you don't have something to measure the goals and targets, then there's simply high soaring rhetoric with no actual teeth.” (Casey Dunning, 01:02)

Variability in Indicators: From Clear to Confounding

• Straightforward indicators:
  • Some are simple and already tracked globally, e.g.,
    • “3.2.1 under 5 mortality rate”
    • "That one, through some research we have shown every single country except Liechtenstein has a data point for that indicator for every year looking back to 2000.” (Casey Dunning, 04:38)
• Problematic, vague, or complex indicators:
  • Others are convoluted and hard to measure, such as those relating to curricula or perceptions:
    • E.g., “Extent to which 1 global citizenship education and 2 education for sustainable development, including gender equality and human rights, are mainstreamed at all levels…” (Casey Dunning & CGD Colleagues, 03:32–03:56)
  • Dunning’s critique:
    • "They defy logic. You can't even define them in a single country context, much less apply it to a global context." (Casey Dunning, 08:08)
    • “How you would measure it, far from clear. …The prioritization of the indicators is lacking because they're all prioritized.” (Casey Dunning, 09:41)

Measurement Challenges: Universal Yet Impractical?

• Universal applicability is idealized but problematic:

  • Some indicators make little sense for certain countries (e.g., measuring extreme poverty in high-income nations like Belgium or Luxembourg).
  • Raises questions on relevance and efficient resource use.
  • “There are finite resources and time and personnel. …If you go with the latter logic, you then open up the whole agenda for a country to say, actually that indicator is not nationally appropriate… And you can use that same logic to apply to sexual and reproductive health and rights, or governance or democracy.” (Casey Dunning, 05:12–07:37)

• Data Gaps Remain:

  • Even for high-profile indicators:
    • “72 of the 193 UN member states have not reported a single data point on [extreme poverty] since 2000.” (Casey Dunning, 05:12)

Political vs. Technocratic Process

• How indicators were selected:
  • Intended to be technocratic, not political.
    • “One of the hallmarks of the indicator selection process...was meant to be a highly technocratic process, not a political one.” (Casey Dunning, 11:07)
• Consequences:
  • Some indicators are not ready or practical to implement.
  • “The group at the UN chose the path of having indicators that covered the full agenda, no matter if those indicators weren't ready for primetime.” (Casey Dunning, 11:07)

National Implementation: The Capacity Gap

• Developing countries face real constraints:
  • Not all have the capacity or resources to measure complex indicators.
• Case Study: Indonesia's feedback:
  • For one indicator (Labor Rights Compliance):
    • “We would absolutely love to measure and report this data. We don't know what it means, how to do it and don't have the capacities to do that.” (Casey Dunning, 13:25)
  • For another (Female genital mutilation):
    • “We can't report on this indicator because it's not nationally appropriate.”
  • Underscores practical dilemmas with universal measurement.

Possible Solutions: Prioritize and Tier

• Focusing on what can be reliably measured:
  • “Focus on a subset of indicators for which we have full country coverage and reliable reporting, are methodologically sound, and have regular updates…” (Casey Dunning, 15:15)
• UN’s tier system:
  • Tier I: Methodologically sound, data available
  • Tier II: Methodology set, data not universally available
  • Tier III: Methodology not established
  • "Helpfully, the interagency group on the indicators at the UN has classified these 230 indicators into three tiers." (Casey Dunning, 15:15)
• Risk of losing comprehensiveness:
  • Prioritizing only Tier I indicators risks reducing the holistic ambition of the SDGs to a “menu of options”—potentially neglecting difficult or controversial areas. (Casey Dunning, 16:30)

Notable Quotes & Memorable Moments

• On the challenge and ambiguity of some indicators:

  • "They defy logic. You can't even define them in a single country context, much less apply it to a global context."
    —Casey Dunning (08:08)

• On universality vs. practicality:

  • "There are finite resources and time and personnel... And so do you want people to spend time on this issue that is not at all relevant to them as policymakers?"
    —Casey Dunning (05:12)

• Indonesia’s candor:

  • “We would absolutely love to measure and report this data. We don't know what it means, how to do it and don't have the capacities to do that.”
    —Casey Dunning (13:25)

• On the potential downside of too many overlapping indicators:

  • “It makes me wonder, kind of, in having so many indicators, if we're not kind of shooting ourselves in the foot a little bit.”
    —Casey Dunning (09:41)

• On the tiered approach:

  • “The tier one indicators are the ones that would fit this description, the Tier 2 and Tier 3...still a lot of work to do to report on the indicator.”
    —Casey Dunning (16:18)

• Rajesh’s closing summary:

  • “Casey Dunning, thanks for making my brain hurt. Sorry, this is a conundrum."
    —Rajesh Merchandani (16:58)

Key Timestamps

• [00:06] – Introduction of the numbers: 17 goals, 169 targets, 230 indicators.
• [01:32] – The reality: more indicators than officially stated.
• [02:20] – What indicators are designed to do.
• [03:00–03:56] – Colleagues reading various indicators: from straightforward to convoluted.
• [04:38] – The exemplary under-5 mortality rate: how it’s universally measured.
• [05:12] – Data gaps in “proportion of population living in extreme poverty.”
• [08:08] – The critique of vague or impractical indicators.
• [09:41] – The challenge of prioritization and the risk of making the SDGs a “menu of options.”
• [11:07] – The technocratic nature of indicator selection.
• [13:25] – Indonesia’s real-world feedback on capacity constraints.
• [15:15] – Suggestion: start with the most measurable indicators (Tier I).
• [16:58] – Closing thoughts and appreciation.

Tone & Style

The conversation is analytical, occasionally wry, and constructively critical. Dunning is especially careful and diplomatic, even as she raises tough issues ("You've been so polite for the first few minutes… I think you've been champing at the bit to say that...").

Summary Takeaway

The SDG indicators, while a major achievement in scope and ambition, face massive challenges in practical measurement and relevance, particularly for countries with limited resources—threatening both the feasibility and universality of the global agenda. A focused, realistic approach—perhaps starting with the most actionable indicators—might be necessary to turn global aspirations into meaningful progress.