Santi Ruiz (4:33)

That decentralization you're talking about within usaid. Will you explain that a little bit more to me? What's the nature of that decentralization?

Dean Carlin (4:40)

Sure. Directives come down from Congress, earmarks and directives that are both about sector so so much, you know, a billion dollars to spend on the following sector, you know, primary school education to Move the needle on children's learning outcomes, Food security, pepfar, one of the biggest earmarks that come down to spend money specifically on prevention and treatment on specific diseases. And. And then there's also directives that come down about countries and how to allocate across countries. This is actually, for what it's worth, those are two conversations that I have very little engagement on because some of that comes from Congress and is a very complicated, intertwined set of constraints that are then adhered to and allocated to the different countries. And then what ends up happening is this is the decentralized part. You might be a Foreign Service officer, you're working in a country, and your focus is education, and your budget for that year you're given from that earmark for education until, you know, go spend $80 million on a new award in education. And so you then are working to figure out, okay, how should we spend that? And there might be some technical support back from the headquarters about what to do, but ultimately you're responsible for making those decisions. And that was part of our role, was to help provide more support to those Foreign Service officers, to help guide them towards the kinds of programs that had more evidence of effectiveness.

Santi Ruiz (5:57)

You were there for, I guess, a little over two years. Is that right?

Dean Carlin (6:01)

That's right. About two and a half. Yeah.

Santi Ruiz (6:03)

Got it. So in that time, you had a couple different budgets come from Congress. I think it would be helpful if you would talk a little bit more about the way that these earmarks come from Congress, because there's, I think, a popular perception that USAID decides what it wants to fund, and it does at the kind of object level. But these big categories of humanitarian aid or health or governance, these are all decided in Congress. And often it's specific congressmen or congresswomen who have particular pet projects that they really want, want to get funded. Is that right?

Dean Carlin (6:34)

That's right. And the number that I heard is that something in the ballpark of 150, 170% of USAID funds were your market. That might sound horrible, but it's not, because it is. How is that it's completely legitimate to double dip in a sense that you have two different demands. You must spend money on these two things. And if the same dollar can satisfy both. And that was completely legitimate, There was no, like, hiding of that fact. I mean, it's all public record, obviously, and it all comes from congressional acts that are creating these earmarked. There's nothing hidden underneath the hood on. On this.

Santi Ruiz (7:04)

Will you give me a couple examples of what that kind of double earmarking would be in practice, what kinds of goals could you satisfy with.

Dean Carlin (7:11)

With both.

Santi Ruiz (7:12)

The same. Yeah, with the same dollar.

Dean Carlin (7:13)

So like there was an earmark for DIV to do research, div development, innovation, ventures. And there's an earmark for that, and then there's an earmark for education. And so if DIV is going to fund an evaluation of something in the education space, that's a possibility that that then can satisfy a dual earmark requirement. I don't know if that actual example took place, just to be clear, but that's the kind of thing that would happen. One is a process, basically, that's saying here's an earmark to do really careful, highly rigorous evaluations of interventions so that we can learn more about what works and what doesn't. And here's money that has to be spent on education. And so if that example is right and you know, empirically, I'm not saying that it's. I don't know the details of exactly how that earmark got spent, but that would be an example of a double dip on an earmark.

Santi Ruiz (8:01)

Let me make sure I'm modeling your role as chief economist correctly. Congress earmarks a bunch of money. This much for humanitarian, purely humanitarian aid, this much for governance, this much for education. And then within those categories, basically, that's where you came in. You wanted to try and help USAID optimize the funding that's going to education. If you're trying to spend $2 billion on education, let's be as effective with that money for that goal as possible. That's right.

Dean Carlin (8:29)

That's exactly right. And then we had two teams. We called it evidence use and evidence generation. And it was exactly what it sounds like. So if there was an earmark for a billion dollars on education, we worked carefully to do systematic analysis of what is the best evidence out there right now for what works and what doesn't for education, for these specific goals, which were primary school learning, primary school learning outcomes. And then how can we help do the analysis necessary to map that evidence to the kinds of things that USAID funds. What are the kinds of questions that need to be figured out? Because it is not a cookie cutter. You don't come from the top, do a systematic review, say, here's the intervention. Boom. It's a simple thing now. Just roll it out everywhere. So we had to do work with the miss with people who know the local area to understand what is the local context that you're facing there and how do you adapt this program in the context of a procurement so that you can hire people to use that evidence, but then figure out how to appropriately adapt it and contextualize it into that country. So that would be the evidence use team and effort. Other things. If there was something new and innovative that USAID was funding, then we were huge advocates of, okay, great, let's contribute to the global public good of knowledge so that we ourselves can learn more in the future about what to do, others can learn from us. And so let's do good. Careful evaluations and all also does serve the purpose of accountability, of being able to demonstrate what good came of it. But I always was a little bit careful about. I've never been a fan of doing really rigorous evaluations just for the sake of accountability. It could discourage innovation, it could discourage risk taking and trying new things, because then if you fail, it's seen as a failure rather than as a win for learning that a good idea that people thought was reasonable didn't turn out to work. And let's learn from that. And it also probably leads to an overspending on research rather than doing programs. So if you're doing something just for accountability purposes, you're better off with audits. You're better off with just did you actually deliver the program that you said you would deliver or not? So our evidence generation team was trying to identify where are their knowledge gaps, where we as an agency can be a leader in helping to produce more knowledge about what works and what doesn't. But one extra thing to mention is that one of the new processes that was put in place, it was actually a revamped old process, was that awards that were over $100 million did go through the front office of USAID for approval. And we added a process where they basically stopped off in my office, where we were able to then provide guidance and opinion on the cost effectiveness of what was being proposed that would then be factored in into the decision on whether to proceed or not. And this was a revamp of an old process that had been put in place. And ironically, when I first read Project 2025 and was trying to understand, because we saw that as a blueprint for what was to come, and I was reading through to understand what types of changes to expect. And that was actually one of the changes, was actually that process. And I remember thinking to myself, well, okay, we just did that. Hopefully this change in the process that they had in mind when they wrote that was what we actually did put in place. But I thought it was a healthy process. That had an impact not just on that one award, but it also helped set a stage and set as an example for smaller awards of this is how to be more evidence based in what you're doing.

Santi Ruiz (11:34)

That was one thing that struck me when preparing for this interview was you talked about how it's useful for USAID to run more research on its own programs, on its own effectiveness, especially where there's knowledge gaps. But also a big value add would just be for USAID to take into account more of the research that's already been done on global development and humanitarian aid. The ideal wouldn't be for USAID to do really rigorous research on every single thing it does. The bar is kind of in the floor and you can get a lot better just by incorporating things that other people have learned from other programs.

Dean Carlin (12:08)

That's absolutely right. And look, the reality is, I can say this as a researcher, that to no surprise, it's more bureaucratic to work with the government as a funder of research and programs than it is to work with foundations and nimble NGOs. So as a researcher, if I want to produce an evaluation of a particular program and you give me a choice of who the funder should be, the only reason why I would choose a government funder is if it had a faster on ramp to policy by being inside. But that's not ideal at all in the sense that the people who are setting policy should not be putting more weight on evidence that they paid for versus somebody else. In fact, one of the slogans that I often used at usaid, and this exact point is evidence doesn't care who pays for it. And we shouldn't be as an agency putting more weight on the things that we evaluated versus things that others evaluated without us. And that we can learn from, mimic, replicate, scale. Another way of saying the same thing too is that we put too much weight on individual studies. And the we here is everyone. I mean this as an academic, I mean this as a policymaker. We all put too much weight on individual studies in a horrible, sometimes hierarchical kind of way. The first to publish on something gets more accolades than the second and third and fourth. And that's actually not healthy when it comes to policy and recognizing that it's the preponderance of evidence and the plethora of evidence and understanding more about the context in which things work and don't work. And it's that collection that is what we should be using to set policy. And if we put too much weight on our own evidence, then we end up Putting too much weight on individual studies that we happen to do. And that's not healthy either. And that was one of the big things that we kept trying to push and say internally at USAID and some of the culture change work we were doing. We have this one slide that used repeatedly that just showed the plethora of evidence that is out there in the world as compared to 20 years ago and how that world has changed with just a lot more studies that are now usable and you can do aggregation of that evidence and form much better policies.

Santi Ruiz (14:04)

I do just want to hear a bit about that culture change piece that you talked about. Obviously you had political support that not everybody going to the government in an innovative role has. And USAID at least somewhat cares about effectiveness and Congress cares about the effectiveness of the dollars spent. On the other hand, USAID is a big bureaucratic entity. There are all kinds of cross pressures against being super effective. I'd be curious to hear what kinds of roadblocks did you run into internally in trying to do that culture change?

Dean Carlin (14:32)

We did. We also had a lot of support. And, you know, one of the things I would say is we did have political cover in the sense that the political appointees. I was not a political appointee, but the ones who were were huge fans of what we were doing. But it's also the case that political appointees under Republicans have also been huge fans of what we're doing. The disagreements are more about what to do and what causes to choose. But the basic idea of being effective with your dollars, to push whatever your policy agenda is, is something that cuts across both sides. So in the days leading up to the inauguration, we were expecting to continue the work we were doing. And we were expecting that the purpose of being more cost effective was something that some of the people that we thought maybe were coming in were actually huge advocates and did make progress under in pushing USAID in that direction. And we saw ourselves as able to help further that goal. Obviously, that's not the way it played out, but there isn't really anything political about being more cost effective.

Santi Ruiz (15:27)

I just want to talk about the couple of years you spent in the Biden administration as a civil servant. And USAID is full of all kinds of people with all kinds of incentives, some folks who are fully on board and supportive. I'm curious, what kinds of challenges do you have in trying to change the culture to be more focused on evidence and effectiveness?

Dean Carlin (15:44)

A couple thoughts. One is that there was actually a fairly large contingent of people who just welcomed us, who were eager who understood the space that we were coming from and the kinds of things that we wanted, and greeted us with open arms. And there's no way we would have accomplished what we accomplished without that. We had a bean counter ourselves of moving about $1.7 billion towards more effective programs or with strong evaluations. And that was our kind of internal office bean counter from the office of the chief economist. That would have been zero had it not been that there was some culture that didn't need changing, that there was some individuals that were already eager to do that and just didn't have the path for doing it. That path was created. A second thing that I would say that was very useful was I think a lot of times people see economists as people who are going to come in and be in a sense, negative and a bit dismal. The dismal science, so to speak. The reason I got into economics in the first place was for a positive reason. We tried showing this as often as possible that with an economic lens we can help people achieve their goals better, period. And one of the things that I would and my team would say repeatedly to people is we're not here to actually make the difficult choices. We're not here to say whether health versus education versus food security is the better use of money. We're here to accept your goal as it is and help you achieve more of it for your dollar spent. And so in that sense, we tried always sending a very disarming message that we are there simply to help people achieve their go and help illuminate the trade offs that naturally exist. Now. One of the things that you would see happen within usaid, and I think this is true of other contexts as well, is when you have a consensus type organization and you have say, 10 people sitting around a room trying to decide how to spend money towards a common goal. If you don't really crystallize what are the trade offs between the various ideas that are being put forward, you end up seeing a consensus built around. Everybody gets a piece of the piece of all to the same common goal. And so our way of kind of trying to shift the culture a bit is to take those moments and say, wait A second, all 10 of those ideas might be good ideas relative to doing nothing, but they can't all be good relative to each other. We all share a common goal. Great. We're here to embrace that and help you achieve more of it. And let's be clear about what the trade offs are between these different programs that you're about to fund. We're thinking about funding and let's identify the ones that are actually getting you the most bang for your buck.

Santi Ruiz (18:09)

Can you give me an example of what those trade offs might be in a given sector where we all care about health outcomes or humanitarian aid? What kinds of trade offs might you have there?

Dean Carlin (18:18)

Sure. So let's take social protection. So something that's on what we would call the humanitarian nexus development space. So it might be working in a refugee area, not dealing with the immediate crisis, but it's dealing with the crisis one year, two year, five years, ten years later, and trying to help bring the refugees into a more stable environment. Economic activities, et cetera. In a program like that, sometimes you would see some cash provided or some food provided to households and they would all have the common goal of helping to build sustainable livelihood for households so that they can then be more integrated into the local economy. I'm talking about this kind of hypothetical scenario, a place where there's no returning to their home country. So it really is about long term assimilation. How do you deal with these issues? There might be a program that was providing some efforts on water. There might be efforts that were providing financial instruments and savings vehicles. There might be programs that were doing support for vocational education. So it'd be a myriad of things. Right? But all on this focused program, focused idea, focused goal of income and generating activity for the household so that they can be more stable in the long run. A lot of times, evaluations of those kinds of programs that are doing 10 different things do not actually lead to observable impact. Five years, 10 years, actually, I don't know of any that lasted 10 years, the evaluations, but the even shorter run, those did not look good in terms of a generating impact. But a more focused approach has gone through evaluations. And that's a good example of one where reducing doesn't always mean reduce just to one thing. But in some cases like that, there is this default option of saying, well, how much do we get actually from a cash transfer? Let's start with that as just a base case. And say, what does a cash transfer generate?

Santi Ruiz (19:57)

And just to clarify, for folks who don't follow development economics, the cash transfer is just, what if we gave people money?

Dean Carlin (20:03)

Sometimes it is just that. But sometimes it's a little bit more like thinking strategically about maybe we should do it as a lump sum so that it goes into investments, maybe we should do it with a planning exercise to make those investments. And so sometimes it's a little bit more. But let's just call it Cash plus or cash with a little plus then you know, different variations of that nature. There's a different model, maybe call it cash plus plus, called the graduation model, that has gone through probably about 30 different randomized trials now showing pretty striking large impacts on long run income generating activity for households.

Santi Ruiz (20:35)

What is that plus plus?

Dean Carlin (20:36)

At its core it is a cash transfer, usually along with some training about income generating activity, ideally one that is exporting in some way, a product producing and exporting and could be exporting even like a local export to the capital and access to some form of savings. So in some cases that's an informal savings group with a community that comes together and saves together. In some cases it's local money and that's the kind of the core. So it's a much simpler program and it's easier to do it at scale. And that has generated considerable and measured repeatedly positive impacts, but not always. There's a lot more that needs to be learned about how to do that, how to do it more effectively, et cetera.

Santi Ruiz (21:16)

But if I'm understanding you correctly, one of your kind of recurring refrains is if we're not sure that all these other ideas have an impact, let's just benchmark against some cash transfer model. Would a cash transfer model just give us likely more bang for our buck than this panoply of other programs that we're trying to run?

Dean Carlin (21:33)

In a lot of cases I think cash is the right benchmark and there are other cases that means a different something else should be the benchmark. But the idea of having a benchmark I think is a great approach in general, like you should always be able to beat X and X might be different in different contexts. So go back to education. What's your benchmark for improving learning outcomes for primary school? Actually, cash transfers is not the right benchmark for that. Cash transfers will help move the needle on some educational outcomes, but the evidence that they will single handedly move the needle on learning outcomes is not that strong. On the other hand, a couple different programs, one called teaching at the right Level, another called structured pedagogy, have proven repeatedly to generate very strong impacts and at a fairly modest cost and so on that space, those should be the benchmarks. And now if you want to innovate, great, innovate. But your goal is to beat those. And if you can beat them consistently, then you become the benchmark. And that's a great process for long run. And that actually, for what it's worth, is very much part of our thinking about what the future of Foreign age should look like is to be structured around that kind of benchmark.

Santi Ruiz (22:34)

So let's go back to those roundtables where you're trying to figure out what the right intervention or set of interventions is for. In your example, a group of refugees in a foreign country, you were often in the position of saying, all these interventions can't be equally good relative to each other. We should slim down, we should gore somebody's ox. I don't want to make you name names, but what were those conversations like where you said, look, if we're all pulling in the same direction, we're going to have to throw out the three worst ideas here relative to. Relative to each other.

Dean Carlin (23:03)

So I think one of the challenges that we face, I think of it as a bit of the psychology of ethics issue. There's probably a word for what I'm just about to say and describe. But one of the objections we would often get was about the scale of a program for an individual. And so someone would argue that this won't work unless you do this one extra thing. That extra thing might be providing water to the household along with a cash transfer, income generativity and financial support and financial inclusion and bank accounts, but you also have to do water. Or another would be that you have to also provide consumption and food up to a certain level. And these are things that individually might actually be a good thing relative to nothing, or maybe even relative to other water approaches or cash transfers. But let's just take the cash transfer as an example. The basic question about whether to satisfy the household's food needs or provide half of what's needed. Ultimately, if that's all you're thinking about, the trade off between full and half, you immediately jump to this idea that like, well, no, we have to go full. That's really what's needed to help this household. But if you think about the fact that if you go to half, you can help more people, now there's an actual trade off. There's 10,000 people that are going to receive nothing because you're giving more to the people that are in your program. The same is true for nutritional supplements. The basic idea that like, should you provide 2,000 calories a day or a thousand calories a day and more people. It's a very difficult conversation on the psychology of ethics because there's this idea of the people like are in a program or kind of sacrosanct and you must do everything you can. But that actually ignores the ethics of all the people who are not being reached at all. And making that trade off salient is real. I would find myself in conversations where that's exactly the way I would try to put it. And I would say, okay, wait, we have 2 million people that are eligible for this program. In this context, our program is only going to reach 250,000. That's the reality. So now let's really talk about how many people were willing to leave untouched and unhelped whatsoever. And that was, at least to me, the right way to frame this question in terms of do you go very intense for fewer people or broader support for more people, and did that help.

Santi Ruiz (25:13)

These roundtables reach consensus or at least have a better sense of what things are trading off against each other?

Dean Carlin (25:20)

Definitely saw a movement for some. I wouldn't say it was uniform. And these are difficult conversations, but there was a lot of appetite for this recognition that we are, at the end of the day, as big as USAID was. It was still small relative to the problems being approached. And there were a lot of people in any given issue, any given crisis, that were being left unhelped. And so the minute you are able to help people focus more on those big numbers, as daunting as they are, I would see more openness to this recognition that, okay, let's look at the evidence to see how do we figure out how to do the most good we can with the resources we have. We must recognize these trade offs that we're inherently facing, whether we like it or not.

Santi Ruiz (25:58)

There's a question I wanted to ask you about trade offs in the USAID process. So not necessarily among the different options of programs you might run for a given population. Back in 2023, you talked to Dylan Matthews at Vox, and it's a great interview. And you talked about the challenge of pushing folks on cost effectiveness to measure cost effectiveness when that's another step in a big, complicated bureaucratic process of getting eight out the door. And you said it's not just that you're adding this new step about can you measure cost effectiveness? You said, quote, there are also bandwidth issues. There's a lot of competing demands. Some of these demands relate to important issues on gender, environment, fairness in the procurement process. These add steps to the process that need to be adhered to. What you end up with is a lot of overworked people. And then you're saying, here's one more thing to do that is measure cost effectiveness. I'm curious, looking back at your work at usaid, what do you think of those additional steps in the process on the kinds of contractors you're hiring on fairness in the procurement process.

Dean Carlin (26:57)

I think, given that we're going to be facing a new environment, that there probably are some steps in the process that hopefully when things are put back in place in some form, someone can be thinking more carefully about. In some sense, it's easier when you start with a blank slate to put in a cleaner process that avoid some of these hiccups. Having said that, it's also going to be fewer people to dole out less money, but the ratios are not there. There's definitely a challenge that we're going to be facing as a country to push out money in an effective way with many, many fewer people for the oversight and whatnot. Our goal in that I don't think would be accurate to say that we achieved this yet. But I can tell you what my goal was, was to make it so that the adding of cost effectiveness was actually a negative cost addition to the process, that as long as we were able to do it in a way that successfully recognized the need to contextualize what you were doing to a country, that it wasn't a cookie cutter solution from up top saying thou shalt do this, and here's your cookie cutter answer, and now just power it out. But that the work to contextualize in a country was able to be done in a way that actually simplified the process for whoever's responsible for putting together the procurement docs and deciding what to put in them. And so I do stand by that belief that if it's done well, that we can actually make this a negative cost process change. But we didn't get there yet. I would not say we achieved that in what we were doing, but that was definitely the goal, was to get to that point of simplifying things.

Santi Ruiz (28:22)

I just want to push on here a little bit. Would you be supportive of a USAID procurement process and a contracting process that stripped out a bunch of these requirements about gender, the environment, or fairness in contracting? Would that make USAID a more effective institution?

Dean Carlin (28:39)

I think there's no doubt that some of them. There's some of those types of things that did serve an important purpose for some areas and not others. And the tricky thing is, how do you set up a process to decide when to do it, when not, right? So, yes, there's definitely cases that you would see an environmental review of something being done that really had absolutely nothing to do with the environment. And so it was just a cog in the process. But somehow you have to have a process for deciding the process I don't know enough about what the legislation was that was put in place on each of these to say, was there a better way of deciding when to do them, when to not do them? What I would say on gender was that one of the things that we tried pushing on through my office, there's a fair amount of evidence in different contexts that says the way of dealing with a gender issue, a gender inequity, is not to just take the same old program and say, oh, we're now going to do this for women, and that you need to understand something more about the local context and what are the constraints that are leading to this stylized fact of women having either lower access, lower freedom, lower access to institutions, or something that is leading someone to want a program that has that focus, or it's under the premise that a program directed towards women will have a bigger impact on children. One of those two things is usually driving that approach. But if all you do is take programs and say, add a gender component, you end up with a lot of false attribution, false labeling of things, and you don't end up being effective at even the very thing that the person doing that cares to do. And so that's a bit unfortunate. That is not something that I was involved in in a direct way in sense. It was never put on the table like, hey, let's think about redoing how we introduce gender and think through gender in our procurement process. The area that I got involved in that was more if our goal is to improve some of the injustices or increase the impact through thinking about not just agriculture, but agriculture for women, then that's where our office did get heavily involved in saying, okay, great, well, what's the evidence say on different ways of doing that where we can see interventions which not only increase income for the household, but increase the power for the women to control that income in the household and thus spend more money on the children's health and education. And so that was an example of how we would get involved. That was not changing the procurement rules for how to incorporate gender, but just saying, what does the evidence say about how to achieve those goals better?

Santi Ruiz (31:01)

Couple more questions on problems with the USAID paradigm and then I promise we're going to get to timeline and forward looking. But I just want to stick here a little longer. In a different conversation with Dylan Matthews, he says, this is not your quote, this is him. USAID relies heavily on a small number of well connected contractors to deliver most aid, while other groups are often deterred from even applying by the process's complexity. And he goes on to say that the use of rigorous evaluation methods like randomized controlled trials are the exception, not the norm. We talked to Kyle Newkirk maybe a year ago, who ran USAID procurement in Afghanistan in the late 2000s, so more than a decade before you showed up. But he talked a lot about those well connected contractors that in practice in Afghanistan, which was a very difficult environment for USAID to operate in, you had a small set of contractors that took most of the contracts and subcontracted to groups in Afghanistan. And often there was very little or no oversight from usaid, either because it was hard to get out to those locations in a war torn environment or because the accountability wasn't built there. I'm curious, in your time at usaid, did you talk to people about lessons learned from USAID operating in Afghanistan? Because that is kind of classically understood as a real low point for USAID as an institution in terms of its effectiveness.

Dean Carlin (32:20)

For it's worth. No. I mean only to the following extent. The lessons learned there, as I understand it, wasn't so much about the choice on what to fund from an intervention perspective, but it was a procurement. It was about the local politics and engagement with the government, or lack thereof, and dealing with the challenge of doing work in a context like that, where there's more risk of fraud and issues of that nature. Our emphasis in the work we were doing was about the design of programs to say what are you actually going to try to fund and dealing with whether there's fraud in the execution? Things of this nature would fall more under inspector general and other kinds of units. And that's not an area that we engaged in when we would do evaluation. This actually gets to a key difference between kind of impact evaluations and accountability. It's actually one of the areas where we see a lot of loosey goosey language in the general media and reporting and Twitter and things of this nature.

Santi Ruiz (33:11)

It would be very helpful actually if you just define those two terms for us.

Dean Carlin (33:15)

So my office and what we were focused on was an impact evaluation. What changed in the world because of this intervention and by change in the world, we really are making a causal statement what changed that wouldn't have otherwise changed? And that's talking about setting up things like randomized controlled trials to find out what was the impact of this program. And that's something that is done. And yes, it does provide some accountability, but it really should be done to look for it in order to know does this help achieve the goals we have in mind, and if so, let's learn that and let's replicate it, let's scale it, let's do it again, let's have others learn from us. And that's what an impact evaluation is supposed to be doing for an accountability purpose. If you're concerned about there being fraud, if you're going to deliver books to schools or medicine to health clinics or cash to people, and you want to know about accountability, then you need to just audit that process and see, did the books get to the schools, the medicine to the people, the cash to the people. You don't need to ask, well, did the medicine work? Did it solve the disease? Right. That's something that there's been studies already. There's a reason that medicine was being prescribed and it would be a complete waste of money and it would be a mess to sit there and say, no, no, we need to know whether this tuberculosis drug worked in this particular context. But if it's the prescribed drug, then you just prescribe the drug and you do accountability exercises to make sure that the drugs are getting into the right hands and there isn't some theft or something like that along the way. Where there's a lot of times confusion in the social science space, I think it's a very intuitive thing. We understand, like if you're going to provide a medicine, a vaccine, we understand the idea that you don't need once it's proven to be an effective drug, an effective vaccine, that you don't run randomized trials over and over and over again for decades and decades to continuously learn what you already know, which is that the drug is effective. I think there's a little bit of a confusion that often takes place in the social science space, in the economic intervention, sort of education interventions, that somehow forget about that fact that once we know that providing a certain program generates a certain positive impact, then we no longer need to track continuously to find out what happens to food security, what happens to malnutrition. Instead, we just need to do accountability to make sure that the program is being delivered as it was designed to be delivered, delivered and previously tested and shown to work.

Santi Ruiz (35:32)

That makes a lot of sense. Maybe this is a dumb question. You'll tell me if it is. There are all these criticisms, not from the impact side, but just from the waste, fraud and abuse corruption perspective of USAIDS working with a couple big contractors. They're criticisms that really don't touch on impact. They're just criticisms. It shouldn't be set up that way. Is there, in your view A relationship between the fact that USAID works largely through these big nonprofit or for profit development organizations like Chemonix and impact. Is there any causal tie there? Would USAID dollars be more effective if there was a larger base of contractors that it worked through? Do we know?

Dean Carlin (36:12)

I don't think we know. And I would say that I think there's a certain element where I've always been of the ilk, that there's probably a few different operating models that can deliver the same basic intervention. And we need to focus on is what actually are we doing on the ground, what is it that we want people to receive or hear or do that are the recipients of the programs? And then think backwards from there, okay, fine, who's the right implementer for this? And if there's an implementer that is just much more expensive for delivering the same product to households or communities, then no, let's find someone who's more cost effective at doing that. So there was always this element when we were trying to push for more cost effective programming. It is helpful to break that off into two things. It's the idea, the intervention itself and what benefits it accrues. And then there's the cost for delivering that. And sometimes the improvement is not about the intervention, but it's about the delivery model. And then maybe that is what you're saying. It's like, okay, now these players, ultimately they were too few and too large and they had a kind of a grab on the market that made it so that they were able to charge too much money to deliver something that others were equally able to do at lower cost. So if that's the case, that says, hey, you know what, we should reform our procurement process. Because the reason you would see some of that happen is simply put, they were really good at complying with USAID requirements. That came out USAID from Congress. And you know, you have to do all these different things and you had an overworked workforce that had to comply with all these different requirements. If you had a bid between two groups, one of which is repeatedly delivered on the paperwork that you will need in order to get a good performance evaluation, and a new group that doesn't have that track record, who are you going to choose? So that's basically how we ended up where we are. My understanding on the history of that, for what it's worth, does come from this might be an overly simplistic statement, but comes from a push from Republicans in the 80s, from Jesse Helms, to outsource USAID efforts to contractors like that. So this is not a left leaning thing that came out, you know, from the left to say let's do this. And I wouldn't say it was right leaning either. It was just a decision made decades ago. And you combine that with the bureaucratic requirements of working with USAID and you ended up with a few very skilled firms and nonprofits at dealing with usaid.

Santi Ruiz (38:26)

I'm just a layman here as well, but it's definitely my impression that at various points in American history, different partisans, different kinds of folks are calling for insourcing or for outsourcing. And at a given time, if you were going to kind of flip a coin, it's not obvious which group is going to say we should bring that inside or we should push that out. But definitely I think you're right that the NGO cluster around USAID I think does spring up out of a Republican push in the 80s. We talked to John Kamensky recently who was on Al Gore's, you could call it a predecessor to Doge in the 90s.

Dean Carlin (38:59)

I listened to this.

Santi Ruiz (39:00)

Yes, and I'm glad to hear it. I pushed him a little bit on there was an equivalent outsourcing of other functions elsewhere in the federal government as a result of that push to cut the federal workforce in the middle 90s. Couple more questions here and then I want to pivot to go back to something you mentioned a little earlier. You talked about a slide that you guys had internally where you showed what we've learned in the field of development economics over the past 20 years. Will you narrate that slide for me and for our listeners? What have we learned?

Dean Carlin (39:28)

Let me do two slides for you. Sure.

Santi Ruiz (39:31)

I'll take as many as you want.

Dean Carlin (39:32)

The slide that I was picturing my head when I said that was a slide that was a count of randomized control trials in development that shows a fairly exponential growth that took place. The movement really started in the mid to the late 1990s but really took off in the 2000 knots. But then even just in the past 10 years, it's seen considerable increase as well. So that now there's about something in the order of 4 to 5,000 randomized control trials that have been evaluating various programs that are get the kinds of things that USA funds. That graph is literally just a graph of the count. That's very great. That doesn't tell you the substance of what was learned. Here's an example of substance, which is uncast transfers is probably the most studied intervention out there. Last I am aware of, we have a meta analysis that we counted 115 studies that have been done. And so that's where you start really getting into having a preponderance of evidence to be able to say something concrete. Now, of course you get different results in different places and there's some variation and they have different focus and different targeting and different ways of doing it. And that's part of what a good systematic analysis can help try to do and tease out what can we be saying not just about the effect of cash, but also how to do it and what to expect differently depending on how it's done. But with 115, we can just say so much more than we could have 15 years ago when we saw the first few come out. And then you just have something like, oh, that's interesting. It's. But it's a couple studies and how do you form policy around that?

Santi Ruiz (41:01)

What else do we know? I mean, I'm greedy here. What else have we learned about development that USAID operators of the year 2000 would not be able to act upon?

Dean Carlin (41:11)

I think another area that we're seeing a fair amount of exciting innovation about in the evidence process is on more integration of evidence into the actual operations of local government. Think about it as breaking the development process into two steps. One is choosing good ideas, choosing interventions, and the other is implementing them. Well, one of the things that I think is historically underdone is the study of the implementation side. Some of it is the same toolkit that you can use for both. I think the challenges that we face. This is an area that I was hoping USAID could make inroads on was if you're going to run an intervention that is on some new topic, a new intervention from an academic perspective, that might be something that is of high reward. But if you're now going to be doing much more granular work to say, well, that was a interesting program that created these groups to do something and now let's do some further knock on research to find out whether those groups should be made of four, six, eight or 10 people. That's a lot less interesting to an academic to do that research. Although there were some interesting questions you get at from that. But it's going to have a lower reward, but yet at the same time be incredibly, incredibly important. And now imagine also it's the direct marketing equivalent of the color of the envelope, right? You might run tests if this were old style direct marketing, whether the envelope blue or red, and you might find that, okay, blue worked better for us, great. So that's not interesting to an academic, but at the end of the day, if you run 50 of these on a myriad of topics about how to implement better, you end up with a collection that is moving the needle on how to achieve more impact per dollar. That collection, I would like to think is actually useful and actually can be put together in a way that is not just important knowledge for policy, but also does help us learn more about the development process and what the bottlenecks are for implementing good programs and seeing more growth and development in countries. And so that's something that as we are seeing more and more platforms, digital platforms, digital data being used, that kind of work is now more and more possible to do as compared to where we were 10 years ago, 20 years ago, where most of the research that was being done was more at, let's call it the intervention level. Does this intervention work as is? And so that's a transition that I think we're starting to see more and more of that I think is exciting and healthy. And it's also, I think, a path to seeing how foreign aid can actually really help in individual contexts, work with local governments and move the needle on helping that government be more efficient with its own resources as a, as a model of foreign aid.

Santi Ruiz (43:47)

There's a argument I've seen quite a bit recently over the past year on this point. The idea basically being that of the money that we spend on foreign aid and development, we're underinvesting in governance, and that actually if we care about all these other things, including long run economic growth and humanitarian outcomes, and you can run down the list, we should be spending a lot more of our time and our focus and our money relative to all these other ways that we could intervene on supporting local governance. What do you make of that claim?

Dean Carlin (44:20)

I agree with it, actually. But I would say that there's a big difference between recognizing the problem and seeing what's the tool to address that problem. And so it's one thing to say politics matters, institutions matter. There's lots of evidence to support that, including the recent Nobel Prize. It's another beast to say this intervention will move the needle in improving institutions and improving governance. I think there's little, I don't say little disagreement, of course, everybody, there's always something to disagree with. But there's a lot of consensus and understanding of the importance of the governance and politics and institutions. And the challenge then goes back to the very pragmatic question of what do we do about this? What actually is working to improve this? What is resilient to the Political process, what is not. And the minute you get into those kinds of questions, it's almost like the other end of the spectrum from a cash transfer. A cash transfer has that kind of universality to it. Not to say you're going to get the same impact everywhere, but it's a little bit easier to get your head around what those paths are and think about measurement and think about design of a program. For instance, you know, you have fewer parameters to decide in the cash transfer. Who are you going to give it to, how much are you going to give, how often are you going to give it? Things like this. When you want to think about efforts to improve governance, improve institutions, you have obviously bespoke thinking that needs to happen in every single place.

Santi Ruiz (45:34)

As you point out, it's something of a meme to say institutions matter and to leave it at that. But the devil is in all of those details.

Dean Carlin (45:43)

In my younger years, I feel old saying that, but I do feel like I can say that now my age. I used to do a lot of work on financial inclusion, and financial literacy was always my go to example for that. On a household level, it's really easy to show a correlation that people who are more financially literate make better financial decisions, more wealth, et cetera, et cetera. It's much harder to say, how do you move the needle on financial literacy in a way that actually helps people make better decisions and then absorb shocks better, build investment, better save better, et cetera. And so it's really easy to show that the correlation is there. It's much harder to say this program here will actually move the needle in a way that changes outcomes and changes lives in the way we want. And that same exact problem is true and actually much more complicated when thinking about governance and institutions.

Santi Ruiz (46:26)

Let's talk about USAID as it stands today. You left USAID when it became clear to you, and you're telling that a lot of the work that you were doing was just not of interest to the people now running USAID or dismantling USAID to start off, how did the agency end up so disconnected from a political base of support? I mean, obviously today there are still plenty of people who support USAID and would like it to be reinstated and built back better, but it was at least vulnerable enough to be tipped over by DOGE in a matter of weeks. How did that happen? Do you have a sense of how support for USA deteriorated to that point?

Dean Carlin (47:06)

I don't know that I would agree with the premise. I'm not sure that public support of Foreign aid had actually changed. I'd be curious to see that, but that's not my impression. I think more would happen was that foreign aid has always been misunderstood. There's some public opinion polls that are fairly well done, as far as I know, that show people thought 25% of the US budget was spent on foreign aid. And when said, what do you think it should be? They said 10%. And the right answer is about 0.6%. You could say, fine, people are bad at statistics. But those numbers are pretty dauntingly off in terms of the perception. I don't know that that's changed. I heard numbers like that, you know, years ago as well. So I don't know what that trend looks like, but I think there was just a certain vulnerability to an EFF that doesn't in a sense, create visible impact to people's lives in America on a daily basis, the same way that Social Security does and Medicaid and Medicare and things of this nature and roads where you actually see in your daily life the consequence of government's actions in these ways. And foreign aid just doesn't have that luxury. And so it's vulnerable. I think it's always been vulnerable in that way and has always then had some bipartisan support because of the understanding of the bigger picture in the long run and the soft power that's gained from it. And also just the recognition that we as a nation are a nation that is built on the idea that generosity and being good to others and the Golden Rule is actually a good thing. It's fundamental to Christian values, Jewish values, Muslim values, and the idea that our government is a representation of our people is, I think, a noble and not dying concept. And foreign aid is basically a manifestation of that generosity in that moral sense. And so that was always there. But it required the intermediaries of Congress to step in and say, okay, let's go spend this money on foreign aid. And it also does have some clear benefits when it comes to things like soft power. So there is that kind of more practical step. It's not entirely about the Golden Rule. I could be wrong, but I don't think that changed. What changed was that you ended up with an administration that just did not share those values.

Santi Ruiz (49:13)

I've got a lot of questions for you here, but I guess one is you've talked about. You didn't put it in these terms, but I'll put it in these terms, and if you disagree, you push back. But there's an issue in foreign aid where Congress picks a bunch of different priorities that it actually cares about. And those priorities are not a rank ordering of what Congress cares about. It's a combination of a bunch of different interests and pressures in Congress generate this list of things that USAID is going to fund. You could say doing it that way is necessary to build a broad base of support to build buy in from a bunch of different political interests for the work of foreign aid. On the other hand, I think there's a good argument to be made that that creates a list that's not the things that are most important to fund. It's kind of this emergent list. And clearly that kind of congressional buy in wasn't enough to protect US aid and the product of foreign aid generally from doge or from other political pressures. So given that whole mix, how should people who care about foreign aid be reasoning about building a version of USA in the future that's maybe more effective and less vulnerable at the same time?

Dean Carlin (50:18)

Fair question. Look, I have thoughts, but by no means do I think of myself as the most knowledgeable person to be able to say here's the answer on the way forward on that. The first thing I would just say is just going backwards. One reality is even if Congress did object, they didn't really have a mechanism in place to actually object. They can cajole the power of the purse the next round, but we're going to probably be facing the Impoundment act. And I'm not a lawyer to weigh in on how that's going to play out. But my understanding is that this is effectively dubbed a constitutional crisis to see can the executive branch impound money that Congress spends? We'll see how this plays out. Aside from taking that to court in some way, all the Congress could do is complain. And so you still had this vulnerability and you still had this desire to show we're doing something big and bold and radical. And so what's the most vulnerable unit out there? The one that doesn't actually immediately touch American lives? And so seeing that as vulnerable, how do you protect that in the future? I don't know that that's actually truly possible to truly protect. What I would say is I would like what comes back to have two things done that I think will help it, but it doesn't make it in the end. One is to be more evidence based because then attacks on being ineffective are less strong. But the reality is some of the attacks on its, quote, effectiveness and the examples used had nothing to do with actually poorly chosen interventions. There was a slipperiness of language of saying there's something that they don't like as a purpose and calling that fraud and waste just because they didn't like its purpose. Which is very different than saying we actually agreed on the purpose of something, but then you implemented it in such a bad way that there was fraud and waste. And there was really no examples given of that second part. It was just, hey, we don't like things that support lgbtq, we don't like things that support climate. And so we're going to just call that fraud and waste. Evidence was never what was being really used. At least the way I use the term evidence in sense of like, is there evidence that there was ineffective achievement of goals that we shared in a bipartisan way? So I don't know that being more evidence based will actually protect, given that that wasn't the way it was really genuinely taken down. But that is, I think, something that would at least provide some support. And the second is some boundaries, the things that are more controversial. I think we need to find a way, whether it's the left or the right that wants it to put it in a separate bucket, somehow let the team that wins the election turn that off, turn that on as they wish. But, you know, there is a core set of activities which actually do have bipartisan support. How do we structure a foreign aid that just is focused on that and let the other stuff sit somewhere else on the side where you can turn it on and off again and doesn't actually adulterate the core part that has bipartisan support, because there's a lot of things that were being done that sure could be done better with a focus on various process changes could improve there, but that did actually have genuine bipartisan support and were shut down. And so that's the key question from a political, From a logistical process. Can we set up a future process that somehow partitions those to be on their own so that they don't have that vulnerability?

Santi Ruiz (53:30)

Want to ask you a little bit more about how you would implement something like that, and you and I talked about this maybe a week ago. My counterexample is pepfar, which had a broad base of bipartisan support from both sides, got long term reauthorizations from Congress, I think precisely because of the dynamic you're talking about. It was a focused, specific intervention that folks all over the political spectrum could get behind and save lives. But over time, I think one dynamic you often see in government programs is if something has a big base of support and if something is effective and broadly supported, the incentive to Stuff your particular pet issues in there, even if they are more partisan increases precisely because this thing works for the same reason that must pass bills in Congress. Get stuffed with everybody's little thing. This is just like a fundamental political dynamic. So is your idea of finding ways to carve out the things that are truly bipartisan in usaid? Does it have a timer attached? Basically that on a long enough timeline, it becomes politicized or partisan in the same way everything else does.

Dean Carlin (54:33)

The theorems in economics about the nature of a repeated game. You can get many different equilibriums in the long run with a repeated game. I'd like to think there's a world in which that is the answer and that the nature of the repeated game helps solve. But we have seen an erosion of other things. That's the same argument for where we saw the line on filibusters regarding judges and we saw that kind of go away. Each one makes a little move in some direction and then all of a sudden you change the equilibrium. We always have that kind of risk. But that would be the goal is how can you establish something where that doesn't happen? And the reason it doesn't happen is because they recognize that if we do that, we might actually kill the whole thing. And we don't want that. It might be that what's happened now is helpful in an unintended way to build those kinds of equilibrium in the future that keep things focused on the bipartisan aspect. And I mean this on both sides in terms of whatever's being done, whether it's the left or the right that wants to do something that they know the other side will object to, that they hold back and they say, wait a second, maybe we shouldn't do that, because then we do rest. The whole thing gets blown up in.

Santi Ruiz (55:35)

A world in which a couple years from now, let's imagine you're back at USAID with an expanded portfolio and you have broader latitude to organize our foreign aid apparatus more around impact and effectiveness. What other things might we want to do beyond measuring programs a little bit better, beyond making these consensus decisions with trade offs in mind? What would the upshot be if we really wanted to focus on effectiveness? Will we do fewer things, simpler and bigger, for instance?

Dean Carlin (56:05)

Two things. I think we would do fewer things simpler and bigger. But I also think we need to recognize that even at our biggest, we were tiny compared to local government or compared to the World Bank. But let's skip over the multilaterals. Just go straight to the local government. We were tiny in any given country in terms of the resources we were putting in compared to their budget, in the long run, if we can do more to use our money as a leverage point to help them be more effective with their money, that's the biggest win that we need to go for. That starts looking a lot like things like Mark Green was putting in place under Trump one under the journey to self reliance. The reality is sometimes that's done in the context of oh yeah, let's do that for five years, 10 years, and then we can like stop giving aid to that country. That was the way MCC talked about their country selection initially, and then eventually they stopped doing that because they realized that that just was never happening. And I think that's actually okay because the reality is, as much as we might help make some changes, even if we succeed in working with the poorest country in the world and helping them use their resources better and receive huge positive return on investment, society is better off. They're still going to be poor, we're still going to be rich, they're still maybe even going to be the poorest. Because if we do that in the 10 poorest countries and they all move up, then sure, there's a world in which maybe the 11th becomes the poorest and then we can work there. But the point is there's still going to be that inequity, there's still going to be problems. And I don't think that getting off of aid is necessarily the objective, but if we got to a point in time where that was clearly the right answer, that's a huge win. If we've done that by helping to prove the institutions and the governance of that country so that it is rolling out better policies, it is helping their own people better, they're collecting their own tax revenue, et cetera, that's a huge win. If we can have an eye on that, then that's a huge win for foreign aid in general, not just from the United States, but as a path for other donors as well.

Santi Ruiz (57:57)

Dean, a moment ago you talked about the value of USAID for soft power. And basically we can kind of bucket that in value of USAID for advancing American interests at the national level. I guess I'm curious for your thoughts, both broadly on how did you think about this trade off between soft power American national interests and pure humanitarian interests at usaid, and also from a development economics perspective or from an economist perspective generally, how are we supposed to be measuring the impact of soft power? I think that's a term that's not now much In Vogue in D.C. there's.

Dean Carlin (58:31)

No one answer to how to measure soft power. It's described as the influence that we gain in the world in terms of geopolitics, in terms of everything from treaties and United nations to just even access to markets, trade policy, labor policy, cooperation, of military cooperation for everything from military bases to flyover rights, depending on where the country is and things of this nature. And the basic idea of soft power manifests itself in all those different ways. And so it's a little bit like a more extreme version of the challenge of measuring the impact of cash transfers. You want to measure the impact of a pill that is intended to deal with disease. You measure the disease and you have a direct measure. You want to measure the impact of a school intervention that's intended to improve school learning. You can measure school learning. You want to measure impact of cash. Now have to measure a lot of different things because you don't know how people are going to use the cash. But it's still actually a little bit of a constrained problem. You have a sensible list of things. Soft power is now even further down the spectrum of the challenge that you don't know exactly how is that aid that is going to that country, that is helping build our partnership, build a friendship, so to speak, with a country, both with its people and its leaders, how is that going to manifest itself in the future? That becomes that much harder to do now? Having said that, there's certainly literature that academics study all the time that they do manage to document different ways in different contexts, everything from attitudes from people about America to votes at the United nations that follow aid, and things of this nature. So there's definitely things that can be done, but it's not like there's one core set. And that's part of what makes it a bit of a challenge.

Santi Ruiz (60:07)

I'll just put my cards on the table here and say I have been pretty skeptical of the defenses of USAID in the aftermath of the Dojing that rely on this idea that USAID is a really valuable tool for American soft power, for maintaining American hegemony, et cetera, both because it's hard to measure and because I care a lot about the humanitarian impacts of usaid, that it'd be a much easier defense of USAID just to say it does this excellent humanitarian work and that's valuable and we should defend it, rather than to try and draw a connection that may just be harder to substantiate about what's the national security effect of usaid, Although I'm sure there Are some.

Dean Carlin (60:42)

I think we're on this. The concern I have too is that when you have such a wide set of things you can look at, then it's not so hard to imagine a world in which, first of all, bias from a researcher might actually lead to selection of what outcomes, selection of the context and evidence of no impact here and yes impact there. How do you net this out? It's not a well defined enough concept to be able to say, well, it worked in 20% of the time and it did not in these. And the net average, like an average over what? Even though there's good case studies that show various paths where it has mattered, there's case studies that show it doesn't. I do get nervous about an entire system that's built around that because it turns foreign aid into too much of a transactional process and sort of a relationship, one and one that is also just built on, as you said, the golden rule of just, hey, there's people in this country that we can actually help. And sure, there's this hope that it'll help further our national interests. But you know what? At the end of the day too, they are in a really bad spot. They're suffering from a drought and a famine and we can provide support and save some lives, or we can do longer term development and save tomorrow's lives. We ought to do that. The fact that our country at the end of the day represents us as people and we as people do care, at least enough of us do care about others, that is a good thing for our country to do. I actually completely agree that that is a reasonable way to look. But yet the conversation does often come back to this question of soft power. And so I do think it helps to have some understanding of what those paths are and what evidence is that's out there. But I get very worried about it. An aid that turns it into a transactional process. Because as an example, I was on a conversation with someone the other day, we were talking about a bipartisan effort that we're trying to corral and we're talking about scheduling another call. And the question was, what about Monday? And the answer was, oh, I can't do Monday, it's my partner's birthday. And it made for a perfect analogy to the benefits of a relationship versus transactional. Because the question was like, okay, so do you have an explicit agreement of no phone calls on that day? That's what you would need if you needed transactional. But why were we not having phone calls? Because there's a relationship There where sometimes you do things for the other side that are not absolutely quid pro quo, itemized in advanced and detailed as to what you're going to do. And that's the nature of having a good relationship, is just doing things sometimes for the other side just because you want to. And the problem with transactional is you get exactly what you contract on, nothing more, nothing less. There's too many unknowns here when we're dealing with country level interactions and engagements between countries. So it needs to be about relationships. And that means supporting even if there isn't a contract in place that itemizes the exact quid pro quo that we are getting for something.

Santi Ruiz (63:22)

I want to leave some time here to talk a little bit about Doge and what you observed in the administration change and kind of use it as a verb now, the doging of usaid. Because I think plenty of observers, myself included to an extent, looked at this in the beginning and thought, oh great, it's high time that a lot of these institutions were cleaned up and someone took a hard look at how we spend money. There's I'm curious, any of your thoughts there on perception versus reality?

Dean Carlin (63:50)

The first distinction is that there was not really any looking at any of the impact of anything that was never in the cards. There was a 90 day review that was supposed to be done. The data that were collected to do that didn't have anything to do with. There was no questions asked, there was no data being collected, there was nothing whatsoever being looked at that had anything to do with what's this award actually accomplishing what it set out to accomplish. So there was no process in which they made those kinds of evaluations on what's actually working, what's not. You can see this very clearly when you just think about what their bean counter was at Doge was expending that they cut right. And it's kind of like me saying, oh, I'm going to do something beneficial for my household by stopping all expenditures on food. Like, well, no, but we were getting something for that food. Maybe we could have bought more cheaply. Maybe we could have switched grocery stores and made a change there that got us the same food for less money. And that would be a positive change. And then we can say, great, we found a way of reducing our food expenditure but keeping the same quantity of food purchased and the quality of food. And then that would be a savings. But you can't cut out all your food consumption, food expenditures and then call that a savings and then not have anything to eat that's just bad math, bad economics. But that's exactly what they were doing throughout the entire government. That bean counter never once said benefits foregone. It was always just lowered spending. Surely some of that probably did actually have a net loss and hence a net gain. Maybe it was 100 million spent on something that only created 10 million of benefits to Americans and so that's a 90 million gain. But that's not what was recorded. It was recorded as the 100 million. And the point is they never once looked at what the benefits are that were being generated from the spending. And now you know that from the inside. And seeing what was actually being asked of within USAID had nothing to do with what is actually being accomplished by any of the money that was being spent. It was never even asked. So there's no way that it was being looked at where it wasn't even being asked for.

Santi Ruiz (65:50)

How do you think about risky bets in a place like usaid? Because I think the economists equilibrium the best practice here would be is something like we take a lot of bets, including some things on high risk, potentially high reward interventions and we're willing to spend money that will be quote unquote wasted at the end of the day where the outcome will not have an impact because we expect that some of those shots will land, they'll be on target and we'll get a lot of bang for our buck. Obviously I think most listeners of this podcast will recognize that approach can be pretty hard to defend politically because the failures, the misses are a lot more salient and visible and available for criticism and the successes people say Great, thank you. Moving on. How should a future USAID think about this sort of thing in a world where you have political pressure or people want to find things to fault you for and you want to show impact and effectiveness.

Dean Carlin (66:47)

This is a great question, one that I have some thoughts. I don't know if this is right or not, but one is that there's an important distinction to make. You can have a portfolio of evidence generation of which some things work and some things don't that collectively can be shown to move the needle in huge ways that are contributing towards knowledge and scaling of effective programs. So USAID actually had something like this. It was called Development Innovations Ventures Div and was in an earmark from Congress. And as well as it, it was actually so good that they actually raised money from the effective alchemist community who sent money into USAID to further augment the pot of money that they had. The distinction and why this was strong is A lot of it was not evaluating USAID interventions. It was just funding a collection, a portfolio of evidence generation about what works and what doesn't, but implemented by other parties. And so in a sense, there the failures aren't as devastating because then you're showing a failure of some other party that it wasn't USAID money paying for an intervention. And so that's part of why that was, I think, in a sense, a strong model for how USAID can take on some risks, do some evidence generation that is immune to the issue you just described. If you're going to now do programs of USAID where you're doing evaluations of USAID money, then, yeah, I think the issue you're describing is very real. You know, my kind of maybe overly simplistic view on this is that a lot of what USAID should be doing should not be getting a highly rigorous impact evaluation, that it should just be rolling out. It's simple and at scale things that have already been shown elsewhere. And let the innovation take place pre USA by other players, maybe div funding the evaluation, but the intervention getting funded elsewhere. Let smaller and more nimble nonprofits be the innovators and the documenters of what works and what doesn't. And then USAID can adopt the things that are more effective and then be a bit more immune to this kind of issue, because this is a very real issue. And I saw this the very first time I did any sort of briefing with Congress when I was chief economist, that the question came at me, why doesn't USA show us more failures? And I remember thinking to myself, well, gosh, like, are you willing to promise that when they show the failure that you won't then punish them for the failure, that you'll reward them for documenting the failure and learning from the failure and not doing it again? And that's a very difficult nut to crack. So, yeah, there is a world that is not first pass that says, let USAID do the things that have strong evidence already. And when it comes to actual innovation, which is where we do need to take risks that things won't work, let that be done in a way that may be supported by USAID but partitioned away.

Santi Ruiz (69:16)

I'm looking at a chart here of USAID program funding in fiscal year 2022. The basic idea is the three big buckets are humanitarian, health and governance, all on the order of 10 $12 billion. And then you've got a couple billion for administrative, a billion for agriculture, almost a billion for education. And then way down at the Bottom half a billion dollars for what's called economic growth. What's in that bucket that USAID funds, and should that piece of the pie chart be larger?

Dean Carlin (69:47)

So I do think that should be larger, but it depends on how you define it. And I don't say that just because I'm an economist. It might come across that way. But that goes back to the comment earlier about things that we can do to help improve the governance of local governments and how they're using their resources. So if there's, for instance, funding under that, under economic growth, the kinds of things that might be funded under that would be efforts to work with local government to improve their ability to collect taxes and to set up efficient regulations for the banking industry so the banking industry can grow and provide access to credit to people and things like this and access to savings. So these are things that can help move the needle on macroeconomic outcomes. And then with that, you have more resources that helps health, that helps education, and you have these downstream impacts of across the space. And so that is an area that received much less. As you just pointed out, the budget on that was tiny. The earmark was tiny. It did not have quite the same heartstring, perhaps, as the source of that. But the logical link is huge and strong, that if you strengthen the local government's financial stability, then the benefits very much accrue to the Ministry of Health, the Ministry of Education, Ministry of Social Protection, jobs, et cetera. And so fighting your way out of poverty through growth is unambiguously good. You can look at many countries around the world that have grown economically and through that reduce poverty. The question is, it's one thing to say that growth will alleviate poverty. It's another thing to say here's aid money that will trigger growth. That's the tricky part. If we knew how to do that, we would have done it long ago in a snap. And that's not necessarily an easy thing to do. But that as a goal is quite admirable. And it's an area where I would say we need more evidence to understand how to make those links happen.

Santi Ruiz (71:29)

Well, Dean, this has been a real pleasure. Last question for you. I want the Dean Carlin Vision for the Future of usaid. Let's say it's clean slate and it's in a couple years and you have wide latitude to go in, give me the bullet points. How is USAID being reorganized in the Carlin regime?

Dean Carlin (71:49)

First of all, I would say that it needs to have at the high level of recognition that the Golden Rule is An important principle that guides our thinking of foreign aid and that we want to do unto others as we have them do unto you. Being generous as a people is something that we pride ourselves in, and our nation represents us as people, and so we shouldn't be in any way shy to use foreign aid to further that aspiration of being a generous nation, the actual way of delivering aid, I would say three things. Simpler. Let's focus on the evidence of what works and what doesn't. But let's also recognize the boundaries of that evidence and how to contextualize that evidence. And it's not to say you never want a cookie cutter intervention. You just do everywhere. So there is a strong element that needs to be understood about what it means to be simpler and how to. How to identify what that means in specific countries and contexts. The second is about leverage. It's about leveraging local government and working more to recognize that as big as we may be or we were and could be in the future, that we're still going to be tiny relative to local government. And if we can do more to improve how local government is using their resources, we've won. The third is about finding common ground. There's a lot of common ground. It's one of the reasons why I started working on a bit of a consortium with Republicans and Democrats and me basically best described as pretending to be nonpartisan because the things I care about are genuinely nonpartisan. So in a sense, my politics I think of as irrelevant. The goal is to take the aspirations that foreign aid has about improving health, education, economic outcomes, food security, agricultural productivity, jobs, trade, whatever the case is, and how do we use the evidence that's out there to rev the needle as much as we can towards those goals? There's a lot of topics like that that have common ground. How do we set up a foreign aid system that stays true to the common ground? And that's, I don't think that hard. I'd like to think it's not that hard. That's what I think would be great to see happen.

Santi Ruiz (73:39)

Well, that's an encouraging note to end on. Dean Carlin, thank you for joining.

Dean Carlin (73:42)

Thank.

Santi Ruiz (73:49)

You, Sam.