
India is a great example of what not to do, suggests Professor Karthik Muralidharan of the University of California, San Diego, a leading researcher on what works – and what does not work – in education in developing countries. Seems...
Loading summary
A
Foreign.
B
Welcome to the latest edition of the CGD podcast with me, Rajesh Merchandani. Now, it has been shown that education is one of the most sure fire routes out of poverty. And for a long time the wisdom has been that if you spend more on education in developing countries, then people there will get a real boost and it will help their countries grow and develop. Except more and more evidence seems to suggest that children in developing countries are going into primary education for years and coming out without the ability to read and do basic arithmetic. Why is that? Well, that's something that the RISE Project aims to understand. RISE stands for Research on Improving Standards of Education. CGD is a major part of this project, and my guest today is at the forefront of that research. He's been taking part in a major conference here at CGG to launch the RISE Project, and he's currently an associate professor of economics at the University of California at San Diego. Dr. Karthik Moralindaran, thank you for joining us on the podcast.
A
My pleasure.
B
So this wisdom, this intuition, you can sort of understand it. You spend more on education and kids learn more. Why is that wrong?
A
The default of putting more resources into schools, more buildings, more teachers, more resources, does not seem to be translating much into outcomes. And that's because the two key missing pieces are pedagogy and governance.
B
So what do you mean by those?
A
So what do I mean by those? So let's think about what are the things that broadly go into generating education or learning? Learning is a complex process. It's way more complex than most other things that we try to do. And it involves household inputs. So how much do parents demand education? Are they sending their kids to school? How much time are they spending with their kids? It depends on what the school has in terms of facilities, inputs. You have teachers, buildings, desks, materials. And then it depends on pedagogy, which is what are you actually doing to meet with those resources? How are those resources being combined into giving you learning outcomes? And then the last part is governance, which is how are these systems being held accountable and rewarded for performance? And so in general, what seems to be happening in bureaucracies is bureaucracies are good at doing what you measure. So you kind of measure the buildings, you measure the teachers, you, you measure the staff, you measure how many kids are enrolled, and you see progress. But learning outcomes cannot be measured. And so nobody's bothered learning outcomes, that.
B
Is how much children are learning, what can they do when they leave school?
A
Exactly. Right.
B
That sort of seems counterintuitive. Though, doesn't it? Someone who's not a trained economist, not a trained development expert, that is me then I come from the world of journalism. You'd think it's kind of counterintuitive that all this money has been spent on education, but what we're not doing is actually figuring out whether children are learning, right?
A
And I think. So there's two answers to that, right? I mean, so the first part is just. So let me give you a specific example, let me give you a specific example of why this cognitive disconnect happens, right? So if you go to a school in Tanzania, the first thing you would see as a journalist going from the US or the UK is you see the school and say, oh my God, there's 11 kids sharing one textbook. So the lack of textbooks must be a constraint. How can you possibly expect that kids are going to learn if they don't have textbooks? And so it's perfectly reasonable to think that if you had a program that provided free textbooks to kids that this should improve learning outcomes. And this is exactly what we did. Now we have multiple randomized evaluations. And I can talk about methodology if you want, but without getting into the nitty gritty, part of what's kind of been really interesting in education is we built up a body of high quality evidence in the past 10 or 15 years through these randomized evaluations. And each of them, each individual study, doesn't quite give you the whole picture, but put together you're kind of building an edifice of knowledge on much more sound foundations. And what we're now seeing in many studies is that simply providing these resources, the books, the materials are not translating the outcomes. And there's many possible explanations why this might be happening. And I'll just give you two of them. So in one case you have the textbooks that are delivered to the school, but you found that the textbooks were not actually given to the kids. They were kept locked up in these cupboards. So that when the donor, if they were to come and see, we gave these books to the school. Where are the books? You see the books nicely presented in the cupboard. I think what's even more worrisome and deeper than that is a study with where pre textbooks were given out in Kenya. The kids got them, the kids had the books. There was no substitution or offset of any other source of resources, but still you had exactly zero impact from textbook. And so this was a study done by Michael Kramer at Harvard and Paul Bleri at the University of Minnesota. And what they then realized was if you looked at the data based on how much the students knew before they got the books, you found that the top 20% of students benefited a lot from. From these free textbooks, but the remaining 80% did not. And so the average effect was close to zero. And so what's going on is basically that to benefit from the textbook, you need to be able to read. And so for kids who could not read, the book was not making any difference. And so I think where the research is pointing to increasingly is that you need to identify what the binding constraint is for improving outcomes in the setting in which you are.
B
Can there be like a kind of, you know, a general theory of education in that sense? The examples you're giving seem so particular. In one case, the books weren't given to the students. In other cases, the majority of students couldn't use the books because they couldn't read. So can there actually be that kind of, you know, universal law that tells you what works?
A
Right? So I think. So let's. I think it's almost more useful to focus on what does not work? Right? So there's. There's two. Let me take a batch. There's two broad classes of studies and evidence that I think have been particularly striking in the past 10 or 15 years. One is research that looks at the biggest categories of spending and then tries to identify whether this spending makes a difference. And I'll give you two stark examples of two very large categories of spending that have no impact. So one of these is the books and the materials that I talked about. And we now got at least five or six different studies that show that simply providing this does not seem to be enough. Now, it may be necessary, but not sufficient. So this is not to say that the books don't matter, but simply providing the books by itself is not going to get you the outcomes unless you also reform pedagogy and. Or reform governance. Now, the second study, which is even more striking is teacher salaries. So if you look at reports many education activists around the world, I mean, we talk about poor teacher salaries, poor teacher motivation, how you need to increase teacher salaries to get them more motivated. But I think what the research is so clear on is that simply increasing teacher pay has absolutely no impact. So the government of Indonesia passed this massive teacher reform in 2005, doubling teacher pay across the board. One of the most expensive education reforms, this kind of increased their education budget by 30, 40%. They were doubling teacher salary. And we went in there, did a very careful randomized evaluation over three years, and we find exactly zero impact of this. Now, it may have affected in the long term the quality of teachers who enter teaching because it increased the pay. But for the thousands and hundreds of thousands of teachers in the system, doubling their pay had absolutely no impact. Why? Because you were getting that pay increase in a way that was completely disconnected from performance. Whereas in India we did a different experiment with teacher performance pay. So again, we raised compensation, but in this case it was linked to how much better your students actually did. And what we found is that the average bonuses paid under the performance Pay program were just 3% of salary. But that gave you massive impacts over two years and five years compared to this policy in Indonesia that doubled teacher pay and got you exactly zero impact. So I think the one big lesson from all of this is that when you think about personnel policy, who are the frontline service delivery workers in education, health? Everybody talks about low salaries as a constraint, but what I'm very clear about is that it's not the level of the salary that matters, it's the structure. Does your pay depend on what you do?
B
And trying to create an incentive?
A
It doesn't have to be bonuses or incentives, but yes, broadly, there needs to be to be something in your professional life that depends on what you do. Now, it could be performance bonus, it could be promotions based on performance, it could be better postings, it could be. So there's multiple ways of doing this. But the point is, if you look at the status quo of public personnel management in most developing countries, there's really nothing there. So you could have been there for 25 years and it's almost like over time, you're the sucker if you're working hard because there's nothing in the system that actually rewards you. So one of my first and probably best known studies in this area was the study we did almost 10 years ago on teacher and doctor absenteeism in the public sector. And one of the most perverse findings we had was not just that the level of absenteeism is high, but the teachers with the highest job satisfaction were the teachers with the highest absence. Which seems completely counterintuitive. You think, you know, you love your job, you show up.
B
What was the explanation is you talk.
A
To the teachers and you find out that if you get your salary without having to show up, you really love your job. Right? So, but for the teachers, a good way.
B
Thank you.
A
But for the teachers who have intrinsic motivation, who show up, who work hard but don't get rewarded by the system, they're the ones who are rather disgruntled because they feel like, wait, I'm doing all of this work and there's nothing in the system that recognizes. So I think what's important about performance pay, and I think part of the reason this has been so controversial in the US is if you frame performance pay as part of a teacher accountability system, then you've lost half the because this is then an adversarial conversation between administrators and teachers. But if the way you frame and think about performance pay is that it's recognition and rewarding of outstanding teaching and that it's actually unfair and demotivating to have high performing teachers who are not objectively recognized in some meaningful way, I think you can't have built a system for performance. So coming back to what does, what do we mean by systems education systems? I think I'll come back to that, flesh it out in more detail. But I think one critical part of that is personnel policy. How do you select, how do you train, how do you keep motivated? And how do you kind of make sure that you're continuously maintaining some motivation and rewards for high performing teachers over the course of their life? So those are kind of two examples of two big classes of spending that we found no impact on. So just more books, more materials, no impact.
B
Why?
A
Because, and I'll come back to that in just a second. And the second thing is just providing more teacher salaries. Now this sounds very depressing, right? But on the other hand, there's also good news. There's good news of classes of interventions that are found to have large effects at relatively modest cost. So one very famous example is Michael Kramer and Ted Miguel study of deworming in Kenya. And the point here was that sometimes the most effective solutions are, are outside the box because they're not in your silo of an education ministry. It could be a health intervention, as happened in that case, but perhaps kind of more directly relevant to teaching. We now have evidence from five or six different randomized controlled trials that providing supplemental instruction to children at the level of learning at which they are is remarkably effective. And this can be delivered by local volunteers who are often not very highly paid or not very well trained. They're often young women with a high school degree who are from the same village who spend an hour or a couple of hours a day as volunteers in supplemental instruction sessions. But the key is this. Why does this work? It works because they throw aside the textbook and they focus on where the child is. So I think one of the most interesting lessons we've learned in education research in the past 10 years, which most people don't relate to, is the fact that this comes back to the textbook example. Why the textbooks didn't work is the fact that historically, education systems in developing countries have only enrolled 20, 30, 40% of the population. So it was never designed to cater to the 50% of their education system that was first generation learners whose parents had not been to school and who had no support structures at home. So the textbooks of the curricula are typically written by people in the top 5% of the distribution who kind of assumes a certain rate of progress and growth that is commonplace in the worlds that they inhabit. But that is completely unrealistic when you have this mass first generation set of learners coming into school. And if you just go into a classroom. I talked about pedagogy and governance as the two binding constraints. So one part of the problem of course is governance is absenteeism, accountability. Right. So that's certainly a big part of the problem. But now let's look at the other side of what happens when you have a motivated teacher. So this is a very committed teacher who comes in, who's sincerely teaching. And if you go sit in a class of a sincere motivated teacher, how does this teacher define his or her goodness? His or her goodness is defined as, did I complete the textbook? Did I complete the syllabus? So you have a textbook that you're expected to complete. The good teacher has mapped the textbook into weekly lesson plans. You go to class and you conduct your lesson plan. Now if a kid is confused and actually raises his hand and saying, I don't understand, the teacher might repeat that once or twice. But beyond the point, the teacher has to stick with the program and go on with the textbook and the syllabus.
B
There's plenty of examples of most basic teachers who spend extra time giving extra cheap teaching to kids who are behind, of course.
A
But at some point, the sheer disparities of what you see in the schooling system and, and that's where the motivation also comes in. So you could certainly have a few very motivated teachers who do that. But I'm talking about the average teacher who kind of is not a shirker, but is not going over and beyond. Right. So the average typical public sector worker goes in, punches the clock, I'm in the class, I'm doing my job, I'm doing my textbook and I'm going, so the history of.
B
Even if they're well meaning and motivated.
A
Exactly. But beyond the point, it means like I've done my job. And, and on they Go. And I think the other important point to keep in mind is the history of many education systems has always in developing countries has not been about how do I educate the entire population. It's always been about how do I identify who's smart and channel these people into positions of leadership. So one of the schizophrenias of the Indian education system is at one level you produce graduates of the IITs and the IIMs.
B
I mean, who go on giant institutions.
A
Which are the top institutions who go on to becoming the world's very best at what they do. The CEO of Microsoft came from that system. The chancellor and the executive vice chancellor of my university are both from India. They both came from the IITs. And so how, how do you explain that on one side and on the other side you have 60% of kids in school who don't read at a second grade level.
B
And the point is the system is.
A
Set up and the point is when you have a billion person distribution, the very best of that billion person distribution is going to be fantastic. And you set up an education system that is much better at filtering who's smart than one that is able to teach the kids who actually are not that smart.
B
So all the things you've been saying are fascinating. And you've talked about spending amounts of money that don't work on say teacher salaries or whatever, kids not getting books or curricula that are designed in the wrong way. These are all the things that don't work, which is what you said it's better to look at. But is it possible for you at the forefront of this research to even start to glimpse a model of what does work? This is what people want.
A
Exactly. And I think what does work, what.
B
Are the elements of that model that you know at the moment?
A
Right. So I think when I think about developing country education systems, the two big pieces of classes of interventions that I think we have evidence that things do work are what I said, pedagogy and governance. So if you think about just inputs, business as usual, not working.
B
I'm a developing country government and I come to you and say I want to get my children to learn more.
A
Exactly.
B
So instead of telling me, okay, focus on pedagogy and government, can you say, okay, 1, 2, 3, 4, 5, these are the five things you should do.
A
Right. So I think where we are on the pedagogy side is what I think I'm very comfortable saying, is that you absolutely need to make sure that every kid is functionally literate and numerate by second grade because if they don't make it by second grade, you've lost those kids forever. So one of the most striking pieces of data I have from five years of research in the state of Andhra Pradesh, where you followed 40,000 kids, the same kids year after year after year, is you can look at these learning triggers, trajectories over time, and plot this by percentile. And what you see is that it's Only the top 10% of the kids who are keeping pace with the textbook. The middle 70% are kind of struggling behind and the bottom 10 or 20% are learning nothing even though they've spent four years in school. And after second grade, if you've lost them, it's completely flat.
B
So one, make sure that children are literate by second grade. What's number two?
A
Right? So I think, you know, if you think about just good management practice, it starts by saying identify the goals, make those goals very clear to your system, measure and hold people accountable for delivering on those outcomes, right? So then that gets me to governance. So the system needs, I think, an education system that tries to do too much basically does nothing, right? So the first key piece of advice. So let me give you an example, right? Governments, sensibly, I think, talk about education policy goals around access, equity and quality. But the problem is once you translate these concepts into bureaucratic practice, access is defined as I need to build more schools. Equity is defined as I need to create special scholarships and schemes for marginalized and underprivileged communities. And quality is then defined as more inputs and more training. And so my single point of advice to any government would be to say.
B
We'Ve gone down from five to one point. One point, okay, even better. Policymakers will love this one point of advice.
A
It's one point is single minded focus on learning outcomes and functional literacy and numeracy by grade two. And that solves all of these problems. Why? Because today the access problem is not so much about do I have a school to go to? Because people keep worrying about dropouts, right? Oh, I need to build more schools because kids are dropping out. But what is so clear is that the dropouts are happening not because of child labor, not because there's economic needs to do that. The dropouts, and I speak mostly from India, which is where my experience is, but I think this is true in many parts of the world. There's no parent who doesn't want his kid to get educated, right? I mean people recognize just viscerally that given the poverty, they're in that education. So why they come out, they're dropping out not because they don't want education. They're dropping out because they're learning nothing. Okay? So if you spend. So Lanta did this once four years ago, and I, I do this with policymakers in India that in the middle of a session of 100 people, I just switch to talking in a different language. So I will talk in Tamil or Chinese. And you kind of do this for two minutes and you just see the stupefied silence in the room and maybe 10% of people who understand it. And then you say, you tell them that you've just spent two minutes in the shoes of a kid in an Indian classroom because you're sitting there with material that's just kind of above your head. You're just not getting it, right? So after two, three, four years, this is like the most, it's emotionally disruptive, right? I mean, you're sitting there not learning anything. So you drop out because it's miserable, right? So you don't drop out because you don't want to get educated. You drop out because it's miserable. Now people who are in the field get this. And so you kind of talk about child friendly schooling and all of that stuff. But I think if you wanted to talk to policymakers, here's one outcome, right? It is if you get that functional literacy and numeracy to the point where the kid can pick up the book and read something on his or her own, that would be the single biggest win for developing country education in the next 10 years. Because the access problem is you have dropouts because kids are not learning equity. Why is this all about learning outcomes? Because when I follow those learning trajectories over time, of course the inequality is going up. Of course it's the kids with the first generation learners and more socioeconomically disadvantaged categories who are falling further behind. And then quality, because bureaucracies define quality as more inputs, right? Meaning you kind of tick the quality box by saying, oh, I've upgraded infrastructure, oh, I've created more teacher training, oh, I've hired more teacher trainer trained teachers. Oh, I have increased teacher salaries, right? So you can tick those bureaucratic boxes. But again, if you just say, all I care about is quality learning outcomes. So as a government, if I were advising the education Minister of India, I would just say, here's the thing. In the next five years, I want to go back to the voters and saying that we have delivered universal functional literacy and numeracy by grade three. This is the single minded goal that my primary education system is working towards. I'm going to have independent sample based measurement and I'm going to reward and penalize my district level officers on the basis of delivering that. And I'm going to let them figure out the details of how they're going to then do this at their district level. So clarity, clarity of goals, orient the organization towards that purpose, provide the resources, provide the autonomy and hope.
B
I have a feeling we could talk about this all day, but we're going to have to leave it there. Fascinating. And I hope policymakers are listening to that singular piece of advice that you gave them. Be focused on outcomes and make sure that your children are literate by the second grade. You've boiled it all down so well for us. Dr. Karthik Muralindaran, it's been fantastic to have you on the CGT podcast. I can see why you are a leader in your field. Thank you you for joining us.
A
You're welcome. My pleasure.
B
You can learn much more about the Rise project that this is all part of on CGD's website, which is www.cgdev.org. as ever, join me, Rajesh Merchandani, for another edition of the CGD Podcast from the center for Global Development.
The CGD Podcast – Center for Global Development
Host: Rajesh Merchandani
Guest: Dr. Karthik Muralidharan, Associate Professor of Economics, UC San Diego
Date: August 18, 2015
This episode explores the chronic disconnect between increased spending on education in developing countries and actual improvements in learning outcomes. Dr. Karthik Muralidharan, a leading researcher in the field and key contributor to the RISE Project (Research on Improving Standards of Education), discusses why simply investing in more resources—like buildings, books, or teacher salaries—often fails to deliver better education. The conversation dives into what policies and practices do make a difference, centering on the importance of pedagogy (how children are taught) and governance (accountability and incentives in education systems).
On resource spending:
“Simply providing the books by itself is not going to get you the outcomes unless you also reform pedagogy and/or reform governance.”
(Muralidharan, 05:00)
On the real impact of performance pay:
“The average bonuses paid under the performance pay program were just 3% of salary. But that gave you massive impacts over two years and five years, compared to [doubling] teacher pay and [getting] exactly zero impact.”
(Muralidharan, 07:57)
On system incentives:
“You're the sucker if you're working hard because there's nothing in the system that actually rewards you.”
(Muralidharan, 08:49)
On the historic role of education systems:
"The history of many education systems in developing countries has not been about how do I educate the entire population. It's always been about how do I identify who's smart and channel these people into positions of leadership."
(Muralidharan, 14:05)
A vivid analogy on learning frustration:
“In the middle of a session of 100 people, I just switch to talking in a different language... and then you say, 'You've just spent two minutes in the shoes of a kid in an Indian classroom.'”
(Muralidharan, 18:10)
Dr. Muralidharan provides robust evidence and clear recommendations rooted in research: education policy in developing countries must focus on learning outcomes, tailoring pedagogy to students’ needs, and revamping governance to reward effective teaching. The single most important reform is to ensure all children can read and do arithmetic by the end of grade two or three—a goal both ambitious and transformative.
For more information on the RISE Project, visit cgdev.org.