← The CGD Podcast
The CGD Podcast
CGD Podcast: Local Data for International Impact with Rakesh Rajani and Halsey Rogers
00:52:34
CGD's Eeshani Kandpal speaks with Twaweza founder Rakesh Rajani and the World Bank's Halsey Rogers about how Twaweza's learning assessments influenced the 2018 World Development Report on education. Together they shed light on how partnerships between...
Loading summary
Transcript55 lines
- [00:01]
A
Covid climate and conflicts, the United nations.
- [00:04]
B
The World bank, private actors, locally rooted.
- [00:06]
A
Institutions, locally rooted experts.
- [00:08]
B
What we're really talking about is a very complex ecosystem.
- [00:13]
A
How do we find the interlinks to.
- [00:15]
B
Innovate and come up with the best solutions to actually rethink what we do.
- [00:20]
C
As a broader development community?
- [00:26]
A
You're listening to the CGD podcast where we explore smarter policies for a better world.
- [00:34]
B
Hello and welcome to the CGD podcast. My name is Eshani Khanpal, I'm a senior fellow here at cgd. At CGD we spend a lot of time thinking about how to translate research into policy impact. At the end of the day, the goal of development economics is to make policy decisions that have been informed by the best evidence that is out there. But who gets to decide what the evidence is is being taken into account? What is the evidence that is informing our policy decisions is a question that has at times been quite contentious and has in fact led to a large and increasingly influential movement in development economics that is about the so called decolonization of research and of policymaking. Today we're going to hear from two people who have worked in opposite ends of the research into policy process. First we're going to hear from Rakesh Rajani, who is a Tanzanian civil society leader. Rakesh has worked and had influence in a number of roles over the course of a really illustrious career. In 2001 he founded Haki Olimo, which is a nonprofit based in Tanzania that advocates for young people through education. He then went on to found Tuaweza, which is going to be a focus of a lot of our conversation today. Tuaweza, also based in Tanzania, is a nonprofit that advocates for better education service delivery in East Africa, in Tanzania and Kenya and Uganda and it promotes access to information to citizen agency, all with that aim of improving education service delivery. A striking policy impact of Tuawesa's work has been the uptake of its major data collection effort which is called Uwezo, which is a large scale survey of basic literacy and numeracy in Kenya, Uganda and Tanzania, which went on to anchor some of the main findings of The World Bank's 2018 World Development Report, which was on education, which in turn has influenced the strategy of the World Bank's education global practice and how it does its operations in its client countries. After we hear from Rakesh about his work with Toahuza and about how Uezo came to be this citizen driven and citizen led effort to improve education service delivery, we'll hear from Halsey Rogers, who is a lead economist with the World Bank's Education Global Practice in the Latin America region and was one of the co directors of that 2018 World Development Report, which is formally titled Learning to Realize Education's Promise. He helped lead many of the World Bank's initiatives in education that advanced the vision laid out in that World Development Report, including the Global Learning Poverty Estimates and the Global Education Policy Dashboard. And all of this is sort of seeded in the work done by Rakesh and Rukmini Banerjee and her nonprofit called Prathan, which is based in India. So we'll hear from Rakesh about how the data came into being and how they became influential locally. Then we'll go to Halsey, who'll tell us about why they chose to anchor their findings in these locally produced data and how intentional of a decision that was and how that in turn led to the influence of this report. So let's start with the conversation with Rakesh. Rakesh, thank you so much for joining us today. It's a real pleasure to have you on.
- [04:04]
A
Thanks for having me on.
- [04:06]
B
I wonder if you would agree that a thread that connects your work through your various positions over time has been sort of that pivotal role that is played by data in improving the quality of governance and of policymaking.
- [04:19]
A
I am a big fan of data because I think data can help illuminate in ways that can get us beyond our kind of ideological or political preferences or differences. It can help people come together and focus the mind on what matters. Data can help reveal new things that we were not aware of. Often, if you want to kind of get beyond our assumptions, beyond our priors, beyond our preferences, we have a kind of democratic influence, especially where there are power imbalances such as between citizens and state or gender imbalances. So imbalances having to do with class and wealth and caste. It can be a way of having a common conversation rather than talking past each other in terms of policy making and governance. I think it can be a very useful way also focusing on outcomes, because a lot of the time conversations naturally get focused on activities rather than outcomes. And so it's for the longest time we thought that the big challenge in education, for example, was to get kids into school because so many were out of school. And we just assumed that once kids get into school, they will learn very difficult to think about outcomes that are less tangible and that are kind of more long term. So we look for buildings and desks and books and the most tangible elements. And data can very helpfully allow that whole polity to focus on outcomes and in that sense bring a kind of healthy accountability as to what is our collective purpose, what are we trying to achieve, and what progress are we making towards that purpose.
- [05:57]
B
Tell me a bit more about that last example you gave. You know, we focus on chairs and textbooks rather than what's happening in the classroom.
- [06:06]
A
So I think one of the most fantastic things that has happened over the last two decades across much of the global south is truly the expansion of primary and now secondary schooling so that all boys and girls can go to school. This was a serious problem. If you looked at data and even just lived experience 20, 30 years ago, there are millions and millions of children who are simply not, and in many contexts disproportionately so, women and girls. So that progress to truly universalized schooling is amazing and something that should be celebrated. And what we began to see is that it was necessary but not sufficient because these children who are in school were not, in effect, learned. Now, we knew this anecdotally, we knew this through small scale surveys, but it was very hard to convince policymakers who are basking in their achievement, understandably so, to be able to say, yes, we've made this great progress in terms of getting kids into school, but we need to look further in terms of what is actually happening. So this is where these large citizen surveys first, pioneered by Pratham in India, what I call assa, the annual State of Education report, which was a brilliant exercise using a very simple, accessible, democratic tool to measure not perceptions, but actual learning levels. And you could do it at scale, so you could disaggregate the data down to the district level, for example. And what it showed in ways that were truly revealing, it was that large numbers, in many cases majorities of children in school were not learning, were not able to read and count at the second grade level, even though they might be in fourth grade or fifth grade or sixth grade. So it kind of undermined the entire learning exercise. It demotivated the children, but it also demotivated teachers because teachers knew that they were going through the motions, but it wasn't working. And yet the system and policymakers were not able to confront this. So the work that Asser did in India, which we then adapted and replicated Tanzania, Kenya and Uganda through something called uesro, showed that, you know, there are these huge gaps. Initially there was a lot of resistance, but relatively soon, over time, not only the policymakers, but other elites, community leaders, ordinary parents, in some cases the children themselves, and the kind of whole society came together to one, understand that schooling isn't the same as learning and to realize that what really matters is not schooling inputs, but learning outcomes. It has had a huge impact on policy and practice and framing in India, in Kenya, for example, and across the world among funders and major institutions like the World Bank. The World bank report on education a few years ago. And the idea that foundational literacy and numeracy FLN is core to getting learning right, confidence right jobs is now kind of well established in ways that it certainly wasn't even 15 years ago. And this really is the impact of these citizen surveys. This was, I think, a wonderful example of kind of south south partnership where we worked directly with Pratham in India. I led a team of people from Kenya, Uganda and Tanzania to visit India. We went to Delhi, we went to Gujarat, and we went right to the ground. We saw what this was in practice. How did it work? Very concretely, how was it organized? We looked at the relationships between the authorities and the organization as it did it. You can imagine the logistical issues are very complex when you're doing a large scale survey, kind of try to understand them. And it was a powerful experience where we, after doing this two week visit, we, as the East African group sat down and asked ourselves, is this relevant for us? Will it help us in East Africa? And there was kind of resounding yes. And as a result, Oweso was set up and we did it an even larger scale than in India, like in just those three countries for a while. At the beginning, we were surveying 200,000 households every year. Wow. So second to the census, it was the largest survey of its kind anywhere in the world. And it's profound, profound in terms of its impact in kind of reframing the debate and understanding.
- [10:34]
B
So you talked about some initial resistance to later on, you know, The World Bank's 2018 World Development Report in education really centered around this idea, which is that schooling isn't learning. Tell us about where that initial resistance came from.
- [10:50]
A
Look, I think I can understand these are very political issues. So the progress that governments had made in education was something they were very proud of. And this narrative punctured that story. Now, if I'm being candid, if I could rewind the clock, I would have probably approached it differently because the approach we took with this data, and this is a kind of more general lesson I've picked up as well, is that while we shared it in the public with the goal of creating debate and understanding, we had a bit of a 10 year to the kind of emotional lives and the political economy of government as to what this sort of data and making it public would have. In addition to creating great debate and illuminating the story, it also created a sort of humiliation for people in government. It put them on the defensive, it made them feel unprepared, it caught them a little bit off guard. And had we taken the time to exercise some empathy and curiosity, we might have realized that in fact the very champions of the kind of information we were sharing would be undermined by this debate happening in public. The impact it had on government is that it put them in a defensive crouch. And the reformers inside government had less space to operate to make progressive changes, because now the entire government was under public assault. So they had to kind of almost declare their allegiance to the government. And that meant they had to discredit the data, even though they knew better. Now this is tricky because on one hand you don't want to suppress evidence, you don't want to sugarcoat it. On the other hand, I do think if your goal is to actually change policy and practice, that it's important to be mindful of the political economy, of how change happens inside government. What sorts of incentives and motivations and power structures operate inside government, and what will it take that you often need a coalition of people inside government and outside who are able to thread this careful path, make some judgment while they navigate this path.
- [12:58]
B
One thing that it brings to mind is work. For example, by Martina Bjorkman Nyquist and Jakob Svensson on citizen scorecards and how effective they can be at improving quality health service delivery. In some work I've done in Nigeria, for instance, we tried to replicate that approach at lighter touch version of having citizen accountability in a smaller pilot. We often find around the world these sorts of pilots can have impact and yet it's when you try to scale up that you hit a bit of a bottleneck. And I wonder if what you've just described is to some extent that bottleneck to this idea of data sometimes puncturing government narratives. It almost makes data and ev seem all powerful. And yet I wonder if that's the case, if data and evidence aren't just as susceptible to delete capture why we view them as unimpeachable when we know so well that they're not.
- [13:53]
A
Yeah, yeah, I think data is not as powerful on its own to actually drive change. But data is very powerful in terms of media headlines and causing, as far as government is concerned, putting it in hot water, right? But in many, many places of the world, governments are just very reluctant to share data or often will go and cook the data. So then data gets released, but nobody actually trusts its credibility, which is in some ways worse. So that's the first point that I think it can, you know, whoever releases the data can get into trouble. So often they don't. And there'll be often a lot of self censorship on their part. Now we should also know that by the time we've got to a situation where a government agency is reluctant to release data, it should point us to the fact that there is a deeper underlying problem. And that deeper underlying problem is the kind of basic institutions of how the rules of the game of how this.
- [14:53]
C
Work have broken down. Right.
- [14:54]
A
So it's almost like one needs to, instead of being fixated on is this report going to be released or not, there should be others who try to do the quiet behind the scenes work that tries to say, look, what are we trying to achieve here? What is our common purpose and what does it take to get there? So there is a way in which I think the hard work that needs to be done is to create, as I say, working relationships across difference and across oppositions.
- [15:26]
B
You did remind me a little bit of the great BBC show that I grew up on, yes Minister, you know, which talks exactly about. It's sort of a tongue in cheek way, of course, but exactly about that. The interactions between the bureaucrats and the politicians.
- [15:39]
A
We also have to somehow appreciate the politicians because in that show we poke fun at them. But the real life politicians that I know, and not all of them, but they are having to balance many things, right? They have to win votes, they have their constituencies, they have to deal with, they, they have to do their job and raise the money for the next election. They, I know ministers in government who, you know, their phone is ringing off the hook because a member of the constituency says the water pipe is broken.
- [16:09]
B
Tell us a little bit about how starting to always shaped you and what you did next, where you went from there next.
- [16:18]
A
I was lucky enough to go to university in the U.S. after I did my my first two degrees, I went back home very much feeling a kind of existential crisis. Back home in Tanzania around the HIV AIDS pandemic, there were communities where the prevalence rate was something like 40% and at that time there was no cure. You know, basically when I went back, I worked on HIV and worked on children and it became very clear that if we wanted to have impact at scale on these matters of children's rights and rights around hiv, AIDS and well being we had to think institutionally rather than in projects. And the most important institution by far was education. There are schools everywhere. They had reached, they had mandated, and so on. So my work then focused on education through an organization called Haki Elimu, Tanzania's leading education rights organization, which is still thriving today. And our focus there was very much looking at institutional governance issues rather than just the technical aspects of how do you improve pedagogy or examinations or teacher training or what have you. And doing that work, again, made it very clear that at core, education reform was not about education, it was about governance around the political economy, of policy making, and particularly practice and implementation. And not only are we doing work in East Africa, we were also helping shape global narratives. Right. If you look at that World Development Report on education we referenced a few minutes ago, its first reference is to OWEZO and the Open Government Partnership, which was launched, developed and designed with the Obama White House was very much an idea that emanated from East Africa and that we work together with leaders across the world and the White House to develop and shape it. So here's a global initiative that is not kind of received from up above, but co created together. And I resist and resent the idea that, you know, you have funders and you have implementing organization, because if the idea of an implementing organization suggests you're not doing the thinking, you're just doing the implementing. This was a way of showing that we can do the thinking and the implementation.
- [18:40]
B
You know, development economists, and certainly a card carrying member of that club, we are guilty absolutely of what you've described. Right. You know, we collect a lot of data, we fly into remote communities, we survey them, we use the data to write our papers, maybe use the data to inform policymaking. Although it seems to me that sometimes we use the data to justify policy decisions that have already been made. But in any case, there's a lot of data that's collected and it never gets reported back to the community from which we're collecting the data. So it seems like an inherently extractive process to me and not a just one. So I don't wonder if this is a just process and what we can do along the lines of decolonizing development to maybe make it a more participatory and inclusive process.
- [19:25]
A
I think the issue is what role does that evidence play and who gets to make decisions. Right. Because the core issue is not only figuring out what is the right thing to do, but how do you get a system to adopt and adapt it and own it, and then as it Rolls it out, how to make that work, right? And a lot of the time, the kind of researchers who push this, all the funders behind that don't have this other part which is crucial, right? So it's evidence plus contextual understanding plus deep rooted, respectful and trustful relationships, plus being able to read the moment. You know, you have a number of these plus plus plus that are absolutely crucial to success. But if those are missing, then even a good idea or a good piece of evidence doesn't go far. So what I would put is a slight twist to your question and to say that I think what is really important is to figure out in the everyday practice of government, rather than just when a study is done, how do you create mechanisms and platforms where citizens can get the information they need, when and how they want it in ways that are useful to them? How does it get just built in as a matter of course, that becomes a new norm of how systems is really more important. Dan Hunnig has done really great work on this and his new book that just came out that I highly recommend called Mission Driven Bureaucrats, I think is worth reading. And his whole concept of relational accountability, I think is also very important because it may. I mean, at core it just says, let's see each other as human and understand not only incentives, which of course economists love, but also motivations, right? We can have multiple motivations, we can be motivated in different directions. And motivation, I think, is a more complex term because it integrates incentives and interests and desires and hopes. So I think being able to understand it. One of the things I wish I knew better 20 years ago, because I certainly didn't then, was to. And I wish I was more curious, intellectually curious and emotionally empathetic to understanding what it means to be a politician or to be a senior civil servant and to treat them as human beings, even the ones who are doing things that I really did not like or disagreed with, but if I saw that it would just open up ways of engaging with that person and understanding that person, that would then create avenues for change and partnership in ways that I think I just did not do. So the last thing I would say, this kind of sounds corny, but I think is really important is around the quality of attention, right? In the very busy lives that we live with social media and thousand emails and travel and Internet, like our lives are full of stuff. The casualty there is the quality of attention. And so we are only fleetingly present to each other, even when we are in the same room or on a call, because there's lots of other things going on in your mind. And I think when we show up with people, including people who are different than us, with a kind of attention, a kind of focus, a kind of openness and grace, it can be transformative because it can allow us to get to new understandings and imagine new possibilities of collaboration and change, rather than just putting you in the box that you are and you putting me in the box that I am. And so for me, the quality of attention is, is crucial to being able to do the deep, deep work that is needed in development.
- [23:15]
B
I agree with you wholeheartedly that the greatest gift we have to give is our presence, is our attention. And it is true that there are many competing demands on those two things. And so to have that sort of intentionality at work, to show up with openness, with attention, with humility, that might be a transformative experience for both sides of that conversation. Going back to something that you mentioned at the very beginning, which is this sort of defensiveness that you first sort of accidentally engendered among the bureaucrats in with oso, I wonder if we should be thinking about relational accountability between sort of the funders and the implementers, and to think about how sometimes the funders have the incentives to do something a certain way. But the intrinsic motivation that you described is often with the implementers, much more so than with the funders.
- [24:11]
A
I think that's really important, and I've had a chance to reflect on it because I've been working in civil society and with government for 25 years, and as a funder for nine years, in some ways, I've occupied all sides of this. I empathize with funders because I think a lot of the time when they make decisions or insist on conditions that are not as helpful, they are coming at it from a position of anxiety. They're giving away money and they don't have certainty for impact, and they're worried, they're anxious that will this make a difference? And how do I somehow provide protections? A lot of these conditions and strings and short term and all the sorts of things that are really antithetical to deep change. So we have to work with them to allay their sense of anxiety. And that requires patience, that requires conversation, that often requires people, other funders who've been around the block and had experience, maybe speaking to them or speaking to their bosses, because anxiety is a very powerful emotion. But as you say, where there is trust and experience, you can take risks, that results are not predictable and certain and linear. So that being the case, what your best assurance comes from are the people that are being entrusted with this money, are they thinking and engaging in the right way? Do they bring the right values? Do they bring the right focus? And do they bring the right kind of mindset and practices and muscles to be able to, for example, pivot when things are not working, Knowing which questions to ask when something happens, like there's a regime change which will change everything.
- [25:58]
B
As someone who has done so many things, played so many different roles at so many different levels, if you could wave a magic wand and implement any one policy anywhere in the world, what would it be, where and why?
- [26:12]
A
I think a lot of the world is in trouble because we believe things strongly and then we simply stick to them and are not open to considering that we might be wrong or that there might be some truth or value on the other side. And it kind of fosters groupthink, it kind of deepens polarization and it gets us to take joy in tribal affirmation rather than the achievement of something bigger and powerful and public purpose. So that would be my magic wand that everybody, like researchers, think tankers, ministers, civil servants, NGOs, stop being fundamentalists about whatever they are in. In terms of very concretely, I'll go back to the example of education because I think it is so foundational to all the other progress we seek in the world, which is I really do think that while we have come to appreciate the centrality of learning outcomes as opposed to inputs, in practice, too much of our money, too much of our infrastructure, too much of the attention and the human mind is not going to focusing on the kind of outcomes that are needed.
- [27:26]
B
Thank you so much for your time. This was a real pleasure.
- [27:29]
A
Thank you, Shani, for making the time and giving me this opportunity.
- [27:34]
B
Hearing Rakesh talk about how the UESO and Pratham data influenced the World Development Report, which in turn went on to shift the paradigm, in his words, as he said in Global Education Policy, to really center learning as opposed to school infrastructure. Makes me wonder what it is about Uwezo and Pratham that was so notable or so striking to the team that worked on that World Development Report. So now let's turn to the conversation with Halsey Rogers where we'll ask him about why the World development report of 2018 made the choice to center to really start with locally produced data as opposed to data produced by the World bank, by the un, which is the norm really for World Development Reports, and whether he thinks that that was part of the enduring impact of this particular report. Halsey, thank you so much for joining us today.
- [28:34]
C
It's my great pleasure. Ishani, thank you for the invitation.
- [28:38]
B
Before we get into the story of how this WDR was different, tell us a little bit more about the global education policy landscape going into that report.
- [28:48]
C
So where were we when we started on this report, which we actually started working in 2016? Now the narrative in global education policy I think was quite inconsistent. First we had this internal inconsistency within these education related policy discussions. I mean on the one hand, by 2016 we had really made progress in putting learning on the agenda. So the whole, as we'll discuss, the whole thrust of the WDR 2018 is to explore the consequences of the idea that schooling is not learning. So at the World bank we had launched our 10 year education strategy called Learning for All back in 2011. USAID, DFID, the former FCDO, the UK agency, also had strategies that they came out with emphasizing learning at around the same time. So that was great. Some of the key players were focusing in on learning and realizing that just getting kids into school was not enough. We had more and more evidence the kids weren't learning. We also, I think in a huge factory, we got learning embedded in the Sustainable Development Goals. That was a big achievement. I think many participants contributed to that. And it's not just the global actors. It's very much also groups like Pratam in India, like Twyweza in East Africa, that were highlighting the learning crisis really effectively in their countries in terms that people could understand. So that helped in making sure that we had goal in SDG 4 that was focused in on making sure all children learn. But too often this seemed like a rhetorical commitment to this goal without enough following up on it, without people thinking through, okay, why aren't children learning? Disagreement, learning mattered. But not enough discussion of the actual barriers, not enough discussion of what it would take to lower them, not enough discussion of effective strategies. So that was this internal inconsistency. So second, we had this external inconsistency between this professed goal and then the practice on the ground. In school systems at the center you had a lack of measurement and low awareness of low learning. So it's kind of an invisible problem. You saw children in school, nobody was measuring, especially in primary education, which is when kids get all the most important foundations, reading, numeracy, whatever, number two, when you look at what drives learning all those things, the school based factors were breaking down. So we know what's important. You need children who are in school, prepared to learn, you need teachers who are also prepared, who are well trained, who are motivated. You need materials, infrastructure, but you need them to be used for learning. And then you need school management, sort of effective school principals, school directors, et cetera, that can bring all this together. All of those were breaking down in ways that we went into in the report. And then finally, this broader education, the broader system, the political system, wasn't sending the signal to all these participants at the school level that learning mattered. So that really is what we were seeing. And that's why we thought it was so crucial to write a report that would focus in on learning, on what the barriers were, and on how we can tackle them.
- [31:57]
B
Great, thank you. I want to take a step back and go back to this insight that schooling is in learning. What strikes me about this is in some sense, it seems obvious that really what we care about are learning outcomes and not whether there are desks in school and textbooks. Obviously, those things are important parts of the production function. But the outcome seems so clear and so easily measurable that it has always struck me, I guess, as a bit strange that it really only occurred to us in the early 2010s, if I understood your timeline correctly.
- [32:30]
C
Yeah, no, great point. We sort of knew that for a long time, but there were just so few assessments other than sometimes as part of randomized evaluation, you do an assessment. You notice even in the baseline, you had very low levels of learning in the typical school. That was shocking, but often it wasn't representative. And so we really needed kind of a revolution in getting assessments into schools, assessing representative samples. Your second point is an important one, too, about the inputs about looking at whether schools have desks, whether there are schools, et cetera. This is important, but for too long, this was really the measure that many donors, many governments would use to say, okay, are we making progress? And they would count the number of schools built. They would count the number of schools with desks, although often even then, you wouldn't really have a random sample. You wouldn't have people looking at the condition in a typical school, especially the more remote schools, et cetera. There. That was just as. There's a bit of a mystical belief that number of years in school would translate to a certain amount of learning. There's probably also a little bit of a mystical belief that if you build it, if you provide it, they will learn. Right? And that's been a parallel recognition because nobody's really focusing on the bottom line. Are children learning? Are the children in the remote areas more Disadvantaged areas of those learning the tests they tend to have in mind are the school living exams, perhaps at the end of lower secondary, in some cases the university entrance exam. These get say widely covered in the media. People have an idea in some systems about this many systems. The problem is you're missing a lot of the children who have already dropped out of school because they haven't been able to learn anything. Or you're really looking at the elite. In many cases you're looking at the top of the distribution in places like India for example, where obviously the top of the distribution does extremely well by global standards. But we know at the same time that the median child and particularly those at bottom are really struggling. They're not acqu the basic learning they should be having.
- [34:35]
B
So Halsey, a couple of times now you referred to the implicitly a little bit the challenges of staffing remote public facilities. I think it bears emphasizing a bit. In many low and middle income countries, public services for education, workforces for education for health are centrally recruited. And then teachers or health workers, bureaucrats of all sorts of are assigned and there it becomes very difficult sometimes to get particularly the best teachers or the best health workers to go to the most vulnerable or marginalized facilities which tend to be in the remote areas. My mother worked in the Indian civil service in one of the early batches of the Indian civil service. And I do remember hearing stories in the mid-60s of going on field visits on horseback. So she had to learn how to ride a horse while wearing a sari, which is. Which can be challenging and you know, staying overnight in tents. So none of this is this downplay how difficult it can be to staff those remote locations. It is really important to do that. One thing that struck me about even the way you open the 2018 WDR is that you rely on these surveys that you just mentioned, Assar and Owezo. When I look at the introduction of many other WDRs, they rely on internally produced data. And yet you made this decision to rely on locally shaped surveys like Assar and Owezo. Tell us about how these surveys influenced your thinking going into that landscape that you just described and in that process of shaping the narrative for the 2018 WDR, was this decision to rely on locally shaped data an intentional one and an intentional departure from the norm for WDRs and why?
- [36:17]
C
Yeah, great question. Let me step back and just say a little bit about WDRs and sort of data. Really what we want to do is use the best evidence possible on a problem. As I said, it's often an invisible problem and we want it to make the problem visible. And we saw from past WDRs the kind of research or data just establishing very basic facts or basic metrics, not fancy econometrics. It's really the most powerful for changing people's worldviews. And that's what WDR tries to do. It tries to change the way that the broad development community thinks about a problem. And some examples from past WDRs, as I was thinking about this one, is the data in the 1993 WDR on huge disparities in health outcomes across countries like measured in terms of DALYs that famously had a huge influence on Bill and Melinda Gates as they were starting up their foundation and sort of helped convince them that this was a place they should be putting a lot of their efforts. Or you look at the 1997 WDR where they had survey based indicators of bribes being paid for firms. That was really the first time we had seen that. That was at a time when we were shifting the narrative in the bank. Jim Wolfenson was shifting the narrative to put a lot more focus on corruption as a cancer for development. So we wanted to really get these basic facts that people would remember because people are never going to remember a whole report. So where there was good data, we wanted to use it. And as I said before, part of the problem is just governments weren't measuring learning and basic education so we couldn't really use government data. But we were aware of the great work that was being done through the Asser and WESO initiatives by Pratam and by Torweza in India and Pakistan, East Africa. And so we thought it would be great to open the WDR with data from them. And these are very memorable stats because they're just so straightforward, easy to understand. We presented this WDR in some 60 countries around the world. Always open with that. And just for listeners, I mean the if I can just read the first three sentences so they get an idea. And so what we said was schooling is not the same as learning. In Kenya, Tanzania and Uganda, when grade three students were asked recently to read a sentence such as the name of the dog is puppy, 3/4 did not understand what it said. In rural India, just under three quarters of students in grade three could not solve a two digit subtraction such as 46 minus 17 and by grade five, half could still not do. So compare this to the way we normally express learning data which is like achievement of a certain percentage of curricular expectations or even PISA scores. The OECD PISA assessments, which get a lot of attention, but if you tell them, okay, here's the score relative to the average of 500 across OECD countries, that's not going to mean anything to people. But these examples from wesl, from OSA were just so easily understandable. We didn't just use these citizen led assessments to dramatize the problem. They also helped inspire the strategy we laid out for improving. We had a three part strategy for promoting learning for all. So first was assess learning to make it a serious goal so that it wouldn't be invisible anymore. Make sure you're measuring. Second, act on evidence to make schools work for learning. And that's in areas like early childhood development, effective teacher training, effective use of ed tech. But then third, what we called align actors to make the whole system work for learning. And how do you do that? Well, a big part of it is by building coalitions for learning that go beyond just the education sector. And part of that we argued, is that you have to have clear information, clear metrics that people can understand. Again, if you try to rally support society wide by saying, okay, our PISA score is 372, let's try to get it to 395, that's not going to rally support. People don't understand. But if you tell them facts like the ones I just said, right, that half of grade five students can't do a two digit subtraction correctly, everybody understands that's a problem. Right? And so just seeing the power of what Pratom, Tway Hueza, others had done really influenced our thinking on how we can tackle the learning crisis.
- [40:45]
B
Perhaps one distinction is that with the PISA assessments, for instance, which are done in OECD countries, as you said, there is a well functioning accountability system. There are government investments in the right place in the education system. Although you might wonder if that's the case here in the United States and to what extent. But I think broadly there just isn't the need to convey the data as broad a section of the population. Perhaps there the intervention is really focused on the policymaker and policymakers will understand. You know, even finance ministers can be helped to understand a PISA score. But really, if you want community support, it is important to be able to present the data in a way that is relevant and salient to that community. And maybe this is one way in which local shaping is important because you understand your audience better and therefore you understand how to package the results better.
- [41:39]
C
No, that's absolutely true. And I know Pratham in India better. Meenith Banerjee Creative Pratham I've been just very inspired over the years watching what she does in both working with communities to get this data, but then going back into the communities to let them know what they've learned from them and then also what can be done about and really working with them in participatory ways to figure out the most effective methods to try to tackle this learning crisis and meet the other needs that they've identified in the education sector.
- [42:12]
B
Now Hossi going Back to the 2018 WDR, Rakesh described it as a paradigm shifter in the sense of Thomas Kuhn's coinage and use of that phrase. He is right that the WDR centered the learning crisis narrative in both the World Bank's education operations also went beyond the World Bank, I think quite broadly in education policy and research to really center that learning crisis. Looking back, what are three factors that you think were key to what I would call this exceptional impact on policy?
- [42:42]
C
So I think there are three main keys. So one is we had a focused storyline and the right team to tell that story. Second, we had an explosion of high quality empirical research on the problem and on the solutions that we were able to Dr. And then third, we developed new metrics that would help motivate progress after the wdr and we were able to track the institutional support for using those metrics. We didn't mince words. We said there is a learning crisis. And at the time that was not an uncontroversial decision to use that term. Interesting. One of the impacts we've seen is a great rise in the use of the term learning crisis, both in the academic literature, in the media, et cetera. Since then we've been able to document that, but we called it learning crisis. We kept it focused on foundational learning and then we said, okay, let's really delve deep into that problem rather than trying to cover everything. Now this storyline was inspired substantially by the great insights from local research and experience that I mentioned before. This research from often local researchers, in particular national context, made the problems that were faced by children in low and middle income countries much more concrete and really forced the global community to pay attention to what was going on sort of in the classrooms and schools, not just engaging in high level debates about how much of a budget should go to education. Right. An example of that that I thought was thinking of is the Probe Report. So the public report on basic education in India. This is a report that came out in 1999 that influenced me very early when I was just starting work in this line of education research because it really drove home the actual conditions in school, the cases where teachers were away for weeks at a time and didn't show up, or the cases where politicians showed up only in election cycles, otherwise didn't seem to care about what was going on in schools, et cetera. So that local research is really crucial. So we had a focused storyline, inspired, and one that was very important for policy. And then we had the right team to tell this story, including people with a great deal of experience already. I'll mention just your, for example, your former CGD colleague David Evans, one of our great WDR team members, but we had numerous others. All of them had worked in depth with countries, many of them coming from the regional departments of the World Bank. So that's first, this focus storyline. Secondly, there was, as I mentioned, there was this explosion of empirical research over the previous decade or decade and a half that we can draw on. And this is crucial because otherwise we'd just be kind of waving our arms and saying, learning is a problem. What do we do about it? Which too much of the discussion had been, some of this research is homegrown, I'd say, like, as I said, Asser Ozo, as you mentioned, are very powerful examples and I think great credit to Rukmini Banerjee, to Rakesh Rajani, their colleagues, for all the wonderful work they've done over many years. I think the emphasis on political economy of reform, obviously you can't do that without really good local researchers. There's no way coming in from the outside we can do good research on that. And then third, I mentioned new metrics developed for within our WDR or as a response to wdr. And what these did is they helped build support and continuity and sustainability within and outside the institution to translate what we were saying into a sustained focus on this, especially outside the education sector.
- [46:08]
B
Halsey we end our podcast here at CGD with a couple of questions. Typically, one of them is you had a magic wand that you could wave to implement any policy anywhere in the world. What would it be and why the.
- [46:23]
C
Change I'd like to see this is not exactly a policy would be to have what lan Bridget has called a social movement for learning where everybody recognizes how crucial it is for all children to learn that everything else in society hinges from that. So that's really a sea change I'd like to see. But in terms of specific policies, if you want to get more specific, I think what I would want to see is that all teachers in struggling Low income, middle income systems receive practical teacher training and support through a method that actually works, like teaching at the right level, like structured pedagogy. We have evidence now that these approaches are often orders of magnitude more cost effective and more effective than what systems typically do, just to increase the chances that all students can start their education with a firm foundation for future learning. Life in the real world. That's not going to do it by itself, but that would be a big step in the right direction.
- [47:20]
B
The other question we ask is, is there a memorable story along, sort of related to the conversation we've had that you'd like to share?
- [47:27]
C
I mean, one thing that inspires it, this is unrelated to the WDR itself, but in the wake of it, when we talk about the institutional takeup, actually this is kind of a fun personal memory is I Remember at the HD Week in 2019, so when all of our human development staff came together for a week from all around the world, which is a very inspiring event in general because you meet these dynamic, amazing, enthusiastic, smart colleagues from all over the world. But we have a talent show as part of that. And what's great fun is that we did, and this shows how much this has been institutionalized as we have a tradition of a friend of mine and I, Tim Johnston, have a tradition of kind of writing parody songs for these talent shows. And we did one. We took the opening number from Hamilton and rewrote it as Building Human Capital. And it's amazing, actually one of my favorite moments of the World bank because we were able to sing about all these key issues on human development, Human Capital, in a way that we hoped would be both funny and inspiring. And personally it was exciting because one of the other singers was my then 17 year old daughter. It was wonderful to be up there with her. But what was great was that everybody knew what we were talking about because this agenda was now so well understood so quickly. And part of that grew out of the wdr, the narrative we were using for that.
- [48:53]
B
I was in the audience for that, actually. I remember that performance well. It was fantastic.
- [48:57]
C
Thank you.
- [48:58]
B
That's actually a great way to end this because perhaps the bar should be that research report messages should be translatable into music. They should be so straightforward, we should be able to sing about them. All right, great. Thank you so much, Khalsi.
- [49:12]
C
No, thank you so much, Ishani. This was great.
- [49:15]
B
So I found this conversation really illuminating of this one example, but a really powerful example of how locally rooted data and research have the potential to make an outsized impact on policy. Both Rakesh and Halsey highlighted how important it is to anchor in local context how important it is to frame results in a way that is comprehensible to the population, not just to policymakers. Rakesh also highlighted how important it is to approach the relationships with partners in the field, whether they're researchers or policymakers, with openness, with humility, and with the intention to build trust. Underlying the conversation that we have today is a lot of research on education and education systems, on improving learning outcomes for children, and on global education policy policy reform. If this is a topic of interest to you, you might want to check out the work done by our amazing education program here at cgd. Halsey referred to one of our great former colleagues, Dave Evans, who is now at the Inter American Development bank but has worked extensively on education systems around the world. Other colleagues whose work I wanted to highlight include Justin Sandifer, who has looked at improving schooling quality in a number of different settings. More recently, Binyamin Bedasso and Justin Sandifer have actually looked at the extent to which the World Bank's operational work, its lending work, actually implements that vision laid out in the 2018 WDR. Binyam, Badasso and other work has looked at whether school feeding can be an important component of improving children's learning outcomes. If children are hungry, they can't pay attention to what's being taught to them. And in another complementary set of interventions that Halsey touched upon today, Pamela Jakila, who is a former CGD colleague now a professor at Williams College, has looked at, with Dave Evans and other co authors, the importance of early childhood development interventions. And last but not the least, our new president, Rachel Gunnister is a great authority on education systems and has worked extensively on global policy reform. So if education related research is your thing, go check these out. I hope you enjoyed listening to this conversation and you found it illuminating to some extent as well. Until next time.
- [51:55]
A
Thanks for listening to the CGD podcast. You can learn more about the topics discussed on our website cgdev.org that's cgdev.org See you next time.
- [52:11]
C
Sam.