Loading summary

Transcript48 lines

[00:01]
Tamina Mah Dawn
Health, education, climate, the United nations, the World bank, private actors, locally rooted institutions, locally rooted experts. What we're really talking about is a very complex ecosystem. How do we find the interlinks to innovate and come up with the best solutions to actually rethink what we do as a broader development community? You're listening to the CGD Podcast where.
[00:29]
Han Shang Chia
We explore smart policies for a better world.
[00:32]
Tamina Mah Dawn
Each episode highlights an important development topic hosted by a CGD researcher and featuring experts from around the globe. Tune in to learn how independent research.
[00:43]
Han Shang Chia
Can bring global prosperity.
[00:47]
Marcus Goldstein
Welcome to the CGD Podcast. I'm Marcus Goldstein, Vice President and Senior Fellow at the center for Global Development. Joining me is Han Shang chia, one of CGD's fellows and the director of our new AI initiative, and Tamina Mah Dawn, CEO of the agency Fund. CGD and the Agency Fund have been partners in an accelerator to support and study the use of AI in development service delivery, as well as on a broader effort to define new ways of evaluating AI powered interventions. Today we're talking about AI and its rapidly growing footprint in the world of development. Everything from nutrition coaching to advice to farmers to tutoring children. We hosted a recent CGD event looking specifically at how, and more importantly, why, to evaluate these types of interventions. And today we're going to have a broader discussion on AI development. What promising avenues we're seeing, how we should be thinking about safety and alignment and what needs to be done to nudge the sector in a better direction. There's a lot of interest in AI, like genuine interest. How should we think about this in the development space? Where's it being used? What does the picture look like? What and who are driving adoption?
[02:08]
Tamina Mah Dawn
Thanks so much Markus for hosting us. I would start by saying that civil society or sort of nonprofits and and tech companies are probably at the leading edge and they're doing a lot of experimentation. I've seen less engagement and direct delivery of AI products by government, although that could change. We're seeing adoption in the agriculture sector, getting advising to farmers, getting them weather information and more supportive insights to help them manage their farming businesses. We've definitely seen a lot in health and many donors go to health first because it does have a digital backbone in many cases. And so we're seeing everything from chatbots that support moms who are expecting a newborn, to nutrition coaches for people who may be undernourished and need advice on how to supplement their diets, to advice on how to adhere to TB and bots that kind of coach you on maintaining your adherence to medication regimes. We've also seen a bit of an explosion in livelihoods. The reality of no jobs for most of the young people entering the labor market today is, is one that AI threatens to exacerbate. Right. Everybody's afraid that AI is going to knock out jobs. But also some people think AI may play a role in it may help coach people toward creative livelihoods that they themselves develop through entrepreneurship. It's still yet to be seen whether those approaches are going to be effective. I would say it's been a mixed bag. Who's involved in AI is a pretty lopsided picture. Most of the foundation models and even open source and fine tuned models are from the global North. They are in English language or other languages that you find on the Internet. And what that means is that there are a lot of people not involved, not driving adoption, not getting access. And I think a lot of funders are trying to address this by making sure that AI is in the languages and reflective of the cultures of the majority of people on the planet.
[04:06]
Han Shang Chia
Just a bit on what Tamina is sharing. I think at CGD we are hoping to expand the conversation even further beyond hey, look how powerful these technologies are. And we should extend their access across the world to does the powerful technologies and the increasing access actually improve development outcomes? Because we know that powerful technology and access alone does not necessarily lead to improvements in infant mortality or increases in agriculture yields. And so I think that's where our collaboration with the agency fund that is supporting nonprofits make that final step. Deploy interventions powered by generative AI that hopefully moves those outcomes we care about is so critical.
[04:49]
Marcus Goldstein
Tavina, what is exciting you the most in the intervention space, we've seen the.
[04:56]
Tamina Mah Dawn
Biggest effect sizes, I believe in health and education. So in health, at least at the level of user engagement with health information services, when you make it conversational, people are more likely to listen, to ask questions, to participate, to engage. And so with maternal health bots that are coaching these new moms through safe delivery and neonate survival, you sometimes see two, three times the engagement of women. They're taking agency, they're asking questions, they're asking for help, they're getting referrals, and that relies on having a functional health system around you. An AI bot where there's no affordance to access care is pretty useless. But where you have those affordances, we're starting to see people really getting engaged. And there actually was an RCT during COVID Susan Athey is a co author suggesting that in Ghana, access to Covid information that was conversational in two way was much more attractive to users than these like information blasts that are static, the sorts that people are used to. It's too early to say if this is going to avert health, morbidity and mortality. I think these AI conversation bots are giving people a sense of connection to the health system and awareness and engagement around their health. I think similarly in education we've seen these kind of almost implausible effects of AI tutors who are coaching kids. I guess my spicy take is that what I'm actually most excited about are AI products that are built for people, not programs. So I don't love the idea of here is my education program. How do I ram AI into it to make it work better? But rather, here is a teacher who has to manage a classroom of 70 kids. She has to teach at multiple levels. She is pretty hassled. How do I improve her experience in her work? Both supporting her emotionally through a coaching bot or a supportive bot, a conversation that helps her, but also through providing maybe cutting edge techniques that we're learning are more effective for managing a classroom or for improving numeracy. And so I really advocate a deep focus on the user and solving their pain points as opposed to coming in with a central planner's perspective and asking how we get to numeracy. Straight off the bat, you raised this.
[07:14]
Marcus Goldstein
Issue that if there's a functioning health system, then this is good for connecting. What if there's not a well functioning health system? We know absenteeism is a pretty significant issue. I did some work back in the day which showed that absenteeism was often for training and there just wasn't a second nurse at this maternal health clinic. Right. So it's not just not showing up for work, it's a thinly stretched system, stretched even thinner when you try and do capacity. So absenteeism with a system that has gaps, is it possible that we'd see AI help fill some of those gaps too early?
[07:55]
Tamina Mah Dawn
I'll jump in, but I'd love to hear what Hans has to say. I do think we can fill those gaps to some extent, but we do have to identify the pain points. So if a nurse is absent, why is she absent? And how do we support her being present at the facility? I do think that bots for frontline workers should be a government's priority or even development bank's priority, because that is where the government is vesting massive resources. So much money goes into hiring and paying those nurses, not the community health workers, sadly. But where they're salaried, they're also an investment that the government's making. We need to help those people do their jobs and leverage that investment by providing supportive co pilots. But you raise an important question, which is you see AI companies and CHIP companies telling global self governments that they need to build data centers and they need electricity to build those data centers. Is that ethical when the local clinics and hospitals don't have electricity for safe delivery of a baby? We do have to talk about where our priorities lie. And is it electrification for AI chatbots or is it electrification to provide light in clinics where women are delivering babies? So I think there is a bit of a moral dilemma there.
[09:06]
Marcus Goldstein
I do wonder if there's an imperative to build a data center and there's private capital, we might actually get a positive external.
[09:16]
Han Shang Chia
Right.
[09:16]
Marcus Goldstein
We might get electricity for firms and for clinics and schools. Hans, over to you. What are you most excited about?
[09:25]
Han Shang Chia
I'm most excited by AI's potential to rapidly increase the scale of intervention delivery and to drive down the cost of traditional methods of delivering interventions. So for example, ag extension work, you know, having an extension officer go out in the hot sun, go farmer to farmer, can be quite costly, can be very, very time consuming. And so one of the questions that is most exciting for us is can a digital extension worker, delivered over a voice call or delivered over WhatsApp or SMS chat, allow the delivery of information to farmers much more inexpensively? And are we able to cover just a much wider geography at a fraction of the cost? I think that's incredibly exciting. And then you have maybe the same pool of human extension workers on the back end reading and tracking the bot's responses to the farmers and correcting when there may be errors. And so I think that's an incredible multiplier of your existing workforce that I'm quite excited by. But to come back to your point about what if there just isn't existing infrastructure? And if we were to stick with the pharma example, what if there were no agro dealers in the area that could supply the necessary inputs, fertilizers, seeds, et cetera. AI is not going to fix that. And I think this comes back to the idea that we often study in development economics, that there might be multiple constraints to addressing a development challenge. AI could address one of those constraints, but that's why we should not be thinking of AI as a silver bullet, but really evaluating if ultimately we are achieving the outcomes, because there might be other constraints that the AI is just Unable to help a user overcome.
[11:18]
Tamina Mah Dawn
I have heard people say that we need to find ways to connect AI to the physical world and that probably means AI along the supply chain. So yes, deliver support to people who are end users of government services. Yes, deliver it to the frontline workers, empower them to deliver in person services better, but also connect it to the pharmaceutical supply chain and the vaccine cold chain and connect it to the network of agro dealers or pharmacies or other physical entities in the world that deliver goods and services. And yes, that takes a systems approach. And I think, Markus, that's sort of what you were alluding to. We do need to think of where this inserts along the supply chain. Some of the cases people are not as excited about enabling the querying of complex logistics and transport data to understand where rapid diagnostic tests or vaccines are in a country. But that probably is as important an intervention and we've certainly seen that in the private sector that there are big unicorns doing AI logistics for warehousing and transport and getting goods to retail. And we probably need to do the same thing in the development sector.
[12:27]
Han Shang Chia
Yeah, absolutely. I think that's where we have to go. This combination of AI interventions plus a platform play. And what I mean here is you can imagine a digital agronomist providing advice to farmers saying, hey, I've detected a pest. Based off the photo that you have uploaded, I recommend pesticide abc and here are six agro dealers in your vicinity that I think has stocks and supplies for the pesticide. And so this combination of connecting you to a marketplace, connecting you to supply chains, I think is absolutely where we have to go next.
[13:06]
Marcus Goldstein
I was reading an article recently about why AI isn't going to be a super accelerator for growth and one of the arguments was it's just going to make bottlenecks more obvious, right? Or more painful, to your point. But here I'm hearing a more optimistic argument which is we're going to find these bottlenecks and pinpoints and then we gotta bring the sort of next gen solutions to start dealing with them. There may be limits, I think ultimately with physical infrastructure, like if there's no road to deliver those pesticide hunts, then at the end of the day all the AI in the world, unless we get at building roads, isn't going to help us. I'm intrigued to see what the second generation problems are going to be. And to your point, the systems approach is I think going to be critical there.
[13:57]
Tamina Mah Dawn
Marcus, what are you most excited about?
[14:00]
Marcus Goldstein
I. I think you both captured things that I'm really excited about the empowerment aspect that you alluded to. Tamina personally excites me and I think that could be a game changer for a lot of folks. I worked with some psychologists on this business training which was helping to develop personal initiative or helping people tap into their personal initiative. And the program's been scaling since I first worked on it pretty significantly. There's the day to day constraint of tapping into that which I myself experience. And a lot of times it's just asking you a question, you know, hey, Markus, what are you afraid of today and what are you going to do about it? To me, like bringing that kind of approach to helping people develop their agency, that's like super exciting.
[14:54]
Han Shang Chia
I think that is the question of our time, which is, does the use of generative AI enhance or remove our agency? Does it enable us to introspect, to be reflective and unlock our potential? Or does it substitute away from our own self directed learning and critical thinking and introspection and then just hand us the answers? We're already seeing some worrying signs in the education space. There was a study that was run in Turkey that provided an AI chatbot to students with no guardrails. And one you saw was that students used it as a crutch and they did not write self directed learning. Students just wanted the answer. They did not challenge themselves and they performed worse than students who were provided a chatbot with guardrails that didn't give the answer away.
[15:49]
Marcus Goldstein
So, Hans, you opened this question about guardrails and thinking about how we design AI to make the world a better place instead of a worse place. There's lots of discussion on this in the Global North. There is a fair amount in the global South. How is the question on what guardrails should look like and what's important to think about when we talk about development? So what's the development cut on this question?
[16:23]
Han Shang Chia
Let me give a very brief history of this question, of how do we get generative AI models to align to human interests? And the conversation so far has been dominated by very important conversations about safety. So you want an AI that doesn't encourage the user to engage in self harm. You want the AI to not tell you incorrect answers during medical emergency or a sensitive period of your life. But to me that's table stakes. And when I say table stakes here, I don't mean that it's necessarily easy to execute. There are a lot of incredible researchers who are working very hard to get AI to be more safe. But it's table stakes to Me, because you can create a perfectly safe AI and yet it does not generate the development outcomes that we care about. So you kind of have an AI that very rarely tells you to engage in self harm, but doesn't necessarily exhibit pedagogical best practices when it's tutoring the user that still gives the answer away. So to me, these traits that we want in development best teaching practices, like exhibiting concise and empathetic agronomic advice to farmers, these traits are additional and built on top of a base layer of safety. And that's where I think we need to take a further step beyond the conversation of safety.
[17:57]
Tamina Mah Dawn
What I would add is a shout out to some of the people in the Global south who have really been pioneers in thinking about safety in context. TATL and Tarunima is an organization in India that's doing academic networks that work in this space. Part of it is understanding that what is appropriate advice in one culture, for example, I've had a fight with my husband. How do I resolve this assertion? May be appropriate in some cultures. In other cultures there's a need for stepping back and not always asserting ourselves. And we, you know, I myself as a Western feminist would find that odious in my own life. But I live in an environment which gives me that affordance to feel that it's odious to not assert myself. But you know, for people who don't live in my society, I need to provide advising and coaching that is safe for them. So I think there's a component of safety which is around culture, social norms, also language, how we use language, when it's appropriate to use I versus we. So much of our Western global North culture is baked into these models at the most foundational layers. And it may take a lot of work to develop different kinds of technologies, other foundation models that don't come out of the global north that better suit the diversity of global south contexts that exist. I guess the other ethical issues I would raise are ensuring that the voices of people who are engaged in social service delivery and in the use of social services is brought into these models. And what do I mean by that? I mean that the language that we often use to train models and even fine tune models often comes from the written word. We're getting more and more data now with philanthropic investment of spoken word, more and more language models that are spoken speech to text, text to speech and speech to speech in local languages. But beyond capturing vernacular for every sector, we need to understand the kinds of questions people are asking. What kinds of help do they want the example that I've gotten from Rikkin at Digital Green is that an understanding of rust in the context of wheat versus rust in the context of other plants versus rust in the context of a rusty nail. Different models deal with that differently and it depends what they've been exposed to. What kinds of language? If I'm a farmer, a smallholder, I probably need my vernacular, my sector specific language represented in these models. This is less about ethics, but it's more about inclusion and making sure that we, we capture the voices and the needs and the asks of people who we want to benefit with this set of tools.
[20:35]
Han Shang Chia
I guess an open question here is who is going to be responsible for building these products? Should the responsibility lie with the major tech companies at the foundational model level? What is their relationship to the nonprofits or to the governments that will be the the ones discovering these six definitions of rust that Temina just brought up? And at the end of the day, should we be developing products that live in two separate worlds? For example, the agriculture specific product that has that use, case in this particular region in India where they refer to rust in this very specific way, or should that information be fed back to the tech companies such that at the foundational layer level, this unique cultural and contextual insight is then incorporated into the foundational product? I think that's an open question. And that relationship between the local actors and the major tech companies is yet to be defined.
[21:40]
Marcus Goldstein
I was reading an article this morning about how Google DeepMind is making progress on treatment for cancer. Right. Like, it sounded like somewhat fundamental ways. I don't see anything about malaria or other tropical diseases. I'm in agreement with everything you've both said, but it's not free. And I do think the resource allocation is skewed to where the profits are. And so I'm worried that this doesn't happen because of how resources are being allocated. And I'm not sure that concessional financing, philanthropic financing is going to move fast enough to do this before it's either too late or just sort of kind of add on. Because I think you're both raising sort of fundamental questions and issues. And if it's later, it's just going to be sort of a layer that's put on top of whatever we're using. And these issues are, I think, a little bit more core reactions. Tell me I shouldn't be so sad.
[22:44]
Tamina Mah Dawn
No, I think you're right. It took us a long time to tax carbon, right? And we're still not taxing it at the level we need to. What if any company that has taken more than $100 million in investment has to tax every hit to its API? So an API is an application programming interface. It's the portal between a set of software that a company might offer, like an AI model, and the user who in our case might be a nonprofit building an AI powered chat product. And what if we had an API tax and we started charging companies that are generating wealth and generating revenues, some tax, a public contribution that goes into building products for the rest of the world. You don't, as a company need to build the product, but you contribute through your revenue to somebody building that product. I do want to give a little plug for Google. They do have a TB cough detector that they've built a model that was trained on human sounds related to health, cough, sniffles, et cetera. And my understanding is that others have done this as well. And there is application to neglected diseases, to tropical diseases. As you say, it requires investment. Development is never free. Right? Development is an investment we make in ourselves. And yes, somebody has to pay for it. It's important to get it in the right path. Now, we didn't do it with electricity, with gasoline, until it was probably too late for the environment.
[24:07]
Han Shang Chia
On the flip side to the tax, you could also have a credit where the major tech companies, you know, if you're contributing a data set in a minority language, for example, or you're using their models for socially beneficial purposes, for every unit to be giving a credit back to the user, whether it's a tax or whether it's a per unit usage credit. These are novel approaches. These are newer than the traditional corporate social responsibility investments that are a tiny, tiny fraction of all the private investment that's going into this space.
[24:41]
Marcus Goldstein
So you both have been involved in this accelerator. Tell us a little bit more about the basic idea, and then I'm excited to hear about some of the experiences you've seen.
[24:53]
Tamina Mah Dawn
So CGD, the agency fund, worked in partnership with OpenAI, providing credits to help nonprofits think about how to incorporate AI into their service delivery. We worked in three domains with eight nonprofits. Healthcare, specifically maternal and child health, agriculture, and specifically smallholder agriculture and education. And I would say our interest from the agency Fund's perspective in going into this accelerator was not necessarily to come out with a bunch of success stories. Our interest is in to understand the struggle that nonprofits face in adopting this technology and bending it toward development outcomes, and really even in being able to evaluate and test and learn. And so we wanted to understand the frictions. What is it going to take for the social sector to mold this new technology to its purpose? And I think we've seen a lot of interesting and in some cases painful frictions arise. Obviously the scarcity of data for fine tuning or even for evaluation of models has been an issue.
[26:04]
Han Shang Chia
I think one of the investment thesis for the accelerator was that I think all seven or eight of the nonprofits draw from some existing evidence base that suggests that their intervention prior to AI had some demonstrated literature on effectiveness, that their pre AI version, whether it was a human delivering maternal health coaching to pregnant mothers, or whether it was an AI tutor delivering a phone call math lesson, they all had some indication that their programs were effective prior to AI. And one of the big questions was can AI be used to help potentially improve effectiveness, drive down cost, expand scale? I think even within the cohort you saw a handful that were digitally native. So prior to AI they were already delivering maybe a chatbot just without an AI component. And then we had more traditional analog first organizations who never had a digital engagement of their user, for example. And I think having a mix of both digitally native and analog nonprofits has been very instructive on what it would take to get the development sector at large more on board with AI applications.
[27:21]
Marcus Goldstein
And any lessons there?
[27:23]
Tamina Mah Dawn
Definitely a lot of lessons. I think a First is some NGOs are jumping straight into evaluating the impacts of an AI product or service without getting it to stability first and without understanding whether people are using it first. And so there is this rush to generate evidence about AI's effectiveness. The reality is you really need many iterative cycles of product development before you make that investment. Another thing we've seen is there are very well established metrics and benchmarks and strategies for evaluating AI model performance, and there's even tooling for it. But the tooling requires expertise in software development. And we're seeing that a lot of organizations are vibe checking and vibe evaluating the model based products they're building rather than actually using the tooling that exists. So there are things like Deep Eval and Langfuse for model observability, but also for model evaluation, testing whether the model is behaving as you expect it to. We would like to see more nonprofit adoption of these tools. We'd like to see adoption of a user funnel. If you're now using a digital product with AI and you're delivering it to a user, can you show me the stages of that product as the user moves through the experience from onboarding to first use to engagement and retention. And can you show me how many people convert to each stage as they move through the funnel? We're just not seeing enough inspection of how a product's getting adopted, how many people are adopting it, who's adopting it, and how do we refine the product to increase meaningful adoption.
[28:58]
Marcus Goldstein
It's kind of like this blind faith in technology. So if I'm an analog organization, I know how to do a program. I know if people aren't showing up to my health core clinic, something's going on and I'm going to go find out what it is. But now I bring out a bot and of course, like somehow I think it's just going to be great and everyone's going to use it and I'm going to assume that.
[29:24]
Han Shang Chia
So it's this switch to this over.
[29:26]
Marcus Goldstein
Optimistic sense that the technology will just be fine.
[29:32]
Tamina Mah Dawn
I think so. And the irony in that is that we have so much more visibility with the digital product. You can actually see whether people are using it. You can actually see what the model is doing. We can record the inputs and outputs. And so it's ironic that in this context, where there is something digital that is less physical, it's much easier to monitor and trace. People don't have the tooling to do that and they really need it.
[29:56]
Marcus Goldstein
Yeah. Oh, gosh, yeah. Leaving data on the table, ah, it's a cardinal sin.
[30:02]
Han Shang Chia
One of the things that worries me about a rush towards impact evaluation without tracking how the model is performing or without tracking how users are engaging, is that at the end of the day, there is no1 singular AI. These products move and change every day. There are knobs and buttons that you can press and twist to make the AI perform differently and to increase engagement or reduce engagement. And when we don't track this earlier stage performance, the model or an engagement, what are we really doing an impact evaluation on at the end of the day? One day we may find that an impact evaluation boosts crop yields or boosts learning, and then the next day, after we've changed the knob or an AI company has tweaked the dials a little bit, you find that the AI produces a different impact. I think we've all seen enough impact evaluations making claims about what AI can and can't do. And the reality is that there's very little specification on what exactly the AI system is under the hood and how it's been developed and designed. And so I think we need much more evaluation and reporting of what the product is actually doing and how users are actually Engaging or dropping off?
[31:22]
Marcus Goldstein
We're getting close to the end of our time, so I wanted to ask one of the CGD standard podcast questions. You both have worked in a pretty diverse set of organizations and experiences, so I was wondering, can you share a memorable story from your work?
[31:40]
Tamina Mah Dawn
I go back to some work we did with Breck when I was at the center for Effective Global Action. We had Breck. Brac is one of the world's largest NGOs. They're a non governmental organization established by Bangladeshis, headquartered in brac, but now with programs around the world. And they really focus on empowerment, which is, I think, in great alignment with our work at the Agency Fund. We had BRAC as a partner for research and BRAC researchers would come to the US and teach us and learn from us. And then we would go to Bangladesh and some of the countries in Africa where they were working. And I remember visiting a woman who was part of what would become the graduation program. This was maybe around 2009, maybe 2010. And sitting in the small, not even a room. It was a little space where a woman lived with her three kids. It was tiny, but she was an entrepreneur and she had been endowed with coaching from someone at BRAC who cared about her enough to invest in her ability to navigate. She had become a trader in fabrics and their bed was made out of a big pile of fabrics that she slowly, you know, that was her inventory, that was her startup grant, and they were selling these at market. And it was not an easy life at all. But this woman had been afforded a sense of dignity from the investment that was made in her in coaching and supporting and listening. And I don't want to say that AI replaces that. What I hope is that AI supports the coach in the graduation program to be a little bit more responsive and maybe nurturing or maybe inspires the coach in that program to think a little bit more with empathy for that woman's situation. Whatever it is, I hope that AI helps us make those investments in each other. Because that sense of dignity of a woman who has been given sort of a second chance at life, to me that's quite compelling. And I don't know if this woman had removed herself from extreme poverty yet. This was where I was doing some qualitative work, but that's a memory that sticks with me.
[33:53]
Marcus Goldstein
You're giving me flashbacks to visiting the adolescent girls in Uganda that were part of BRAX program. Like one of the roughest sets of questions I've ever gotten in a meeting was from those adolescent girls they were, yeah, I'm with you, Ans.
[34:11]
Han Shang Chia
So my story is from 2018, 2017, 2018, when I was an implementer of a cash transfer program and I was responding to a hurricane in the Caribbean. One of the big pain points that I experienced was not distributing the cash transfer itself, but it lay in targeting, finding and enrolling households that had been affected by a hurricane that had their homes damaged and lived below a particular poverty line. And that process of targeting and enrolling households was incredibly time consuming, very slow, and I felt like I was letting down the families that we were trying to distribute cash transfers to. It took us something like six months to identify, enroll, and pay just 5,000 households. And so after that experience and prior to the world of generative AI that we now live in, when predictive AI was just making its way into the development sector, I reached out to Google and reached out to professors at UC Berkeley to ask, hey, surely the machine learning techniques are getting advanced enough such that we can use satellite data and other big data sources to identify, based on satellite imagery, which households had their roofs blown off by a hurricane. And beyond that, whether an urban setting, you know, whether there are mud roads or tar roads, whether there is urban planning, whether roofs are made of grass or made of tin, surely all of those factors could form a composite index of wealth and poverty. And surely a machine could help us identify regions that were poorer than others. And over the next few years, we partnered to be able to use machine learning techniques to predict not just damage, but household wealth and poverty. And in 2021, 2022, during the COVID pandemic, I worked with a number of governments, including government of Togo and the government of DRC, to be able to deliver cash transfers to 200,000 households in weeks. So we went from in 2017, taking six months to identify, enroll, and pay just 5,000 households to several hundred thousand at weeks. And that's what motivates me with this new crop of AI technologies, to figure out how we can just go to scale at a fraction of a cost, and using and applying these technologies to proven interventions like cash transfers. And so that's my favorite memory of AI and development.
[36:39]
Marcus Goldstein
Thank you both for a really interesting discussion. I've learned a lot about thinking about using AI to help us boost programs to scale and increase speed, and also to think about the deep potential for possibly helping people tap into their agency and increasing it. And I've heard cautionary notes about building mechanisms now to make sure that we don't leave parts of the world behind and exclude them from what could be a really powerful technology. You can learn more about CGD's work on AI and development as well as the other things we work on on our website.
[37:20]
Tamina Mah Dawn
Thank you Markus. This is a lot of fun.
[37:22]
Han Shang Chia
Thank you Marcus. Thanks for listening to the CGD podcast.
[37:31]
Tamina Mah Dawn
You can learn more about the topics discussed on our website at cgdev.org@cgdev.org work.
[37:40]
Han Shang Chia
See you next time.

Summary6 min read

The CGD Podcast: Where AI Meets Development

Guests: Temina Madon (Agency Fund CEO), Han Sheng Chia (CGD Fellow & AI Initiative Director)
Host: Marcus Goldstein (CGD Vice President and Senior Fellow)
Date: October 30, 2025

Episode Overview

This episode explores the integration of artificial intelligence (AI) into global development, focusing on its practical applications, ethical implications, and the unique challenges faced when adapting AI tools to diverse contexts in the Global South. The guests discuss current trends, promising interventions, evaluation hurdles, inclusion, and the future role of local actors in shaping AI for development outcomes.

Key Discussion Points and Insights

1. The Current Landscape of AI in Development

[02:08] Temina Madon:

Civil Society Leadership: Nonprofits and tech companies are at the forefront, leading much of the AI experimentation; government engagement remains limited but may grow over time.
Sector Adoption:
- Agriculture: AI delivering support to farmers via weather information and farm management.
- Health: Digital chatbots providing health advice (e.g., maternal health, nutrition, TB adherence) due to a pre-existing digital backbone.
- Livelihoods: AI's impact on job creation is mixed—while it threatens some jobs, it could support creative livelihoods through entrepreneurship.
Accessibility & Inclusion:
- "Most of the foundation models and even open source and fine-tuned models are from the global North… in English or other major languages."
- This creates a lopsided landscape that funders are attempting to address via localization and inclusivity.

2. Rethinking Impact: Technology vs. Outcomes

[04:06] Han Shang Chia:

CGD’s focus expands from access to improvement in actual development outcomes.
Quote:

"Powerful technology and access alone does not necessarily lead to improvements in infant mortality or increases in agriculture yields." [04:18]

3. Where AI Is Making a Difference

[04:55] Temina Madon:

Health & Education Impact:
- Increases in user engagement, especially with AI health chatbots ("two, three times the engagement of women").
- AI tutors in education showing "almost implausible" effects, though long-term impact still to be determined.
- Quote:
  
  "I really advocate a deep focus on the user and solving their pain points as opposed to coming in with a central planner's perspective." [06:58]
Supporting the Front Lines:
- Bots and AI co-pilots for health professionals could help alleviate absenteeism and support stretched public systems.
Moral Tradeoffs:
- Quote:
  
  "Is it electrification for AI chatbots or is it electrification to provide light in clinics where women are delivering babies?" [08:36]

4. Scaling, Cost Reduction, and Real Constraints

[09:24] Han Shang Chia:

AI can massively scale interventions—e.g., digital ag extension services to reach wider populations at lower cost.
But "AI is not a silver bullet"—it can't solve infrastructure deficits or supply chain problems alone.

[11:17] Temina Madon:

Urges a systems approach: AI needs to connect to physical supply chains, logistics, and the broader ecosystem to be effective.

[12:27] Han Shang Chia:

Envisions platform-based solutions, e.g., a "digital agronomist" that not only provides advice but also connects farmers with local suppliers, thus bridging digital and physical worlds.

5. Bottlenecks, Systemic Challenges, and Optimism

[13:06] Marcus Goldstein:

Technology might spotlight bottlenecks (e.g., lack of roads or supplies) but could also drive innovation in tackling these pain points.

6. Agency, Empowerment, and Education

[14:00] Marcus Goldstein:

AI-informed coaching to foster personal initiative and agency excites many in the sector.

[14:54] Han Shang Chia:

Raises fundamental questions—does generative AI enhance or diminish user agency?
- Evidence suggests unguarded AI tools can harm self-directed learning.
- Quote:
  
  "Does it enable us to introspect...or does it substitute away from our own self-directed learning?" [14:54]
  "Students used it as a crutch and performed worse than students who were provided a chatbot with guardrails." [15:39]

7. Guardrails, Alignment, and Cultural Context

[16:22] Han Shang Chia:

Safety should be a foundational expectation, not the final goal—AI must also deliver on development-specific outcomes and pedagogical best practice.

[17:57] Temina Madon:

Cultural sensitivity is crucial. What is safe or appropriate can differ vastly by context.
- Quote:
  
  "There is a component of safety which is around culture, social norms, also language...and it may take a lot of work to develop different kinds of technologies, other foundation models that don't come out of the global north..." [18:38]
Inclusion means capturing local voices, dialects, and sector-specific language to make AI genuinely useful.

[20:34] Han Shang Chia:

Who should be responsible for this work—tech companies creating foundational models, or local organizations customizing for context?
- The question of feedback loops and ownership of cultural knowledge remains open.

8. Funding, Incentive Structures, and Resource Allocation

[21:39] Marcus Goldstein:

Notes the misalignment of resources: cutting-edge AI is focused where profits are highest, not always where need is greatest.

[22:44] Temina Madon:

Proposes an "API tax"—a levy on tech companies that could finance the development of socially beneficial AI products.

[24:07] Han Shang Chia:

Suggests usage credits as incentives for socially valuable use of major AI models; both ideas go beyond traditional corporate social responsibility.

9. Lessons from the AI & Development Accelerator

[24:52] Tamina Madon:

Accelerator with OpenAI credits supported nonprofits in health, agriculture, and education.
Key Insight:
- Understanding barriers and frictions is as important as finding early success stories.
Challenges Observed:
- Data scarcity for fine-tuning and evaluation.
- The difference in experience between digitally native and analog-first nonprofits.

[27:23] Tamina Madon:

Over-eager evaluation without first achieving product stability or user engagement is a common misstep.
Nonprofits lack adoption of digital tooling (e.g., Deep Eval, Langfuse); ‘vibe-checks’ replace rigorous evaluation practices.
There’s a need for user funnel tracking and a better grasp on digital product analytics.

[28:58] Marcus Goldstein & [29:23] Han Shang Chia:

Irony: digital products provide abundant data, but it's often not leveraged by NGOs due to lack of technical familiarity.

[30:01] Han Shang Chia:

Impact evaluations must account for the fluidity of AI models—products and outcomes change rapidly, so specifications and user engagement should always be tracked.

Notable Quotes & Memorable Moments

On Inclusion:

"Most of the foundation models...are from the global North...a lot of people not involved, not driving adoption, not getting access." — Temina Madon [03:38]
On Systemic Change:

"We have to talk about where our priorities lie. And is it electrification for AI chatbots or is it electrification to provide light in clinics where women are delivering babies?" — Temina Madon [08:36]
On Safety and Context:

"You can create a perfectly safe AI and yet it does not generate development outcomes that we care about." — Han Shang Chia [16:46]
On Personal Stories:

"That sense of dignity of a woman who has been given sort of a second chance at life, to me, that's quite compelling." — Temina Madon [33:29] "We went from in 2017 taking six months to identify, enroll, and pay just 5,000 households to several hundred thousand at weeks. And that's what motivates me…" — Han Shang Chia [35:42]

Timestamps for Important Segments

[02:08] – Current landscape of AI adoption in the development sector
[04:55] – Largest effects seen in health and education; user engagement stories
[08:36] – AI, electrification, and moral tradeoffs
[09:24] – Scaling and cost considerations in agricultural extension
[14:54] – AI, agency, and effects on self-directed learning
[16:22] – Safety vs. development-focused alignment; the need for contextual guardrails
[22:44] – Proposed API tax and funding mechanisms for inclusive AI
[24:52] – Overview of the AI accelerator, frictions, and lessons for nonprofits
[31:39] – Personal stories from guest speakers highlighting AI’s human impact

Tone and Takeaways

The conversation is both practical and philosophical, blending optimism about AI’s ability to empower users and scale programs, with a sober recognition of persistent bottlenecks and risks of exclusion. Guests emphasize the importance of localized, user-driven design, vigilant evaluation, and new funding mechanisms to ensure AI benefits are shared broadly rather than concentrated in the Global North.

Useful For

This summary is ideal for development practitioners, policymakers, technologists, and anyone interested in the intersection of AI and international development. It covers applications, challenges, and strategic questions shaping the field, offering both inspiration and caution.

Loading summary

Transcript48 lines

[00:01]
Tamina Mah Dawn
Health, education, climate, the United nations, the World bank, private actors, locally rooted institutions, locally rooted experts. What we're really talking about is a very complex ecosystem. How do we find the interlinks to innovate and come up with the best solutions to actually rethink what we do as a broader development community? You're listening to the CGD Podcast where.
[00:29]
Han Shang Chia
We explore smart policies for a better world.
[00:32]
Tamina Mah Dawn
Each episode highlights an important development topic hosted by a CGD researcher and featuring experts from around the globe. Tune in to learn how independent research.
[00:43]
Han Shang Chia
Can bring global prosperity.
[00:47]
Marcus Goldstein
Welcome to the CGD Podcast. I'm Marcus Goldstein, Vice President and Senior Fellow at the center for Global Development. Joining me is Han Shang chia, one of CGD's fellows and the director of our new AI initiative, and Tamina Mah Dawn, CEO of the agency Fund. CGD and the Agency Fund have been partners in an accelerator to support and study the use of AI in development service delivery, as well as on a broader effort to define new ways of evaluating AI powered interventions. Today we're talking about AI and its rapidly growing footprint in the world of development. Everything from nutrition coaching to advice to farmers to tutoring children. We hosted a recent CGD event looking specifically at how, and more importantly, why, to evaluate these types of interventions. And today we're going to have a broader discussion on AI development. What promising avenues we're seeing, how we should be thinking about safety and alignment and what needs to be done to nudge the sector in a better direction. There's a lot of interest in AI, like genuine interest. How should we think about this in the development space? Where's it being used? What does the picture look like? What and who are driving adoption?
[02:08]
Tamina Mah Dawn
Thanks so much Markus for hosting us. I would start by saying that civil society or sort of nonprofits and and tech companies are probably at the leading edge and they're doing a lot of experimentation. I've seen less engagement and direct delivery of AI products by government, although that could change. We're seeing adoption in the agriculture sector, getting advising to farmers, getting them weather information and more supportive insights to help them manage their farming businesses. We've definitely seen a lot in health and many donors go to health first because it does have a digital backbone in many cases. And so we're seeing everything from chatbots that support moms who are expecting a newborn, to nutrition coaches for people who may be undernourished and need advice on how to supplement their diets, to advice on how to adhere to TB and bots that kind of coach you on maintaining your adherence to medication regimes. We've also seen a bit of an explosion in livelihoods. The reality of no jobs for most of the young people entering the labor market today is, is one that AI threatens to exacerbate. Right. Everybody's afraid that AI is going to knock out jobs. But also some people think AI may play a role in it may help coach people toward creative livelihoods that they themselves develop through entrepreneurship. It's still yet to be seen whether those approaches are going to be effective. I would say it's been a mixed bag. Who's involved in AI is a pretty lopsided picture. Most of the foundation models and even open source and fine tuned models are from the global North. They are in English language or other languages that you find on the Internet. And what that means is that there are a lot of people not involved, not driving adoption, not getting access. And I think a lot of funders are trying to address this by making sure that AI is in the languages and reflective of the cultures of the majority of people on the planet.
[04:06]
Han Shang Chia
Just a bit on what Tamina is sharing. I think at CGD we are hoping to expand the conversation even further beyond hey, look how powerful these technologies are. And we should extend their access across the world to does the powerful technologies and the increasing access actually improve development outcomes? Because we know that powerful technology and access alone does not necessarily lead to improvements in infant mortality or increases in agriculture yields. And so I think that's where our collaboration with the agency fund that is supporting nonprofits make that final step. Deploy interventions powered by generative AI that hopefully moves those outcomes we care about is so critical.
[04:49]
Marcus Goldstein
Tavina, what is exciting you the most in the intervention space, we've seen the.
[04:56]
Tamina Mah Dawn
Biggest effect sizes, I believe in health and education. So in health, at least at the level of user engagement with health information services, when you make it conversational, people are more likely to listen, to ask questions, to participate, to engage. And so with maternal health bots that are coaching these new moms through safe delivery and neonate survival, you sometimes see two, three times the engagement of women. They're taking agency, they're asking questions, they're asking for help, they're getting referrals, and that relies on having a functional health system around you. An AI bot where there's no affordance to access care is pretty useless. But where you have those affordances, we're starting to see people really getting engaged. And there actually was an RCT during COVID Susan Athey is a co author suggesting that in Ghana, access to Covid information that was conversational in two way was much more attractive to users than these like information blasts that are static, the sorts that people are used to. It's too early to say if this is going to avert health, morbidity and mortality. I think these AI conversation bots are giving people a sense of connection to the health system and awareness and engagement around their health. I think similarly in education we've seen these kind of almost implausible effects of AI tutors who are coaching kids. I guess my spicy take is that what I'm actually most excited about are AI products that are built for people, not programs. So I don't love the idea of here is my education program. How do I ram AI into it to make it work better? But rather, here is a teacher who has to manage a classroom of 70 kids. She has to teach at multiple levels. She is pretty hassled. How do I improve her experience in her work? Both supporting her emotionally through a coaching bot or a supportive bot, a conversation that helps her, but also through providing maybe cutting edge techniques that we're learning are more effective for managing a classroom or for improving numeracy. And so I really advocate a deep focus on the user and solving their pain points as opposed to coming in with a central planner's perspective and asking how we get to numeracy. Straight off the bat, you raised this.
[07:14]
Marcus Goldstein
Issue that if there's a functioning health system, then this is good for connecting. What if there's not a well functioning health system? We know absenteeism is a pretty significant issue. I did some work back in the day which showed that absenteeism was often for training and there just wasn't a second nurse at this maternal health clinic. Right. So it's not just not showing up for work, it's a thinly stretched system, stretched even thinner when you try and do capacity. So absenteeism with a system that has gaps, is it possible that we'd see AI help fill some of those gaps too early?
[07:55]
Tamina Mah Dawn
I'll jump in, but I'd love to hear what Hans has to say. I do think we can fill those gaps to some extent, but we do have to identify the pain points. So if a nurse is absent, why is she absent? And how do we support her being present at the facility? I do think that bots for frontline workers should be a government's priority or even development bank's priority, because that is where the government is vesting massive resources. So much money goes into hiring and paying those nurses, not the community health workers, sadly. But where they're salaried, they're also an investment that the government's making. We need to help those people do their jobs and leverage that investment by providing supportive co pilots. But you raise an important question, which is you see AI companies and CHIP companies telling global self governments that they need to build data centers and they need electricity to build those data centers. Is that ethical when the local clinics and hospitals don't have electricity for safe delivery of a baby? We do have to talk about where our priorities lie. And is it electrification for AI chatbots or is it electrification to provide light in clinics where women are delivering babies? So I think there is a bit of a moral dilemma there.
[09:06]
Marcus Goldstein
I do wonder if there's an imperative to build a data center and there's private capital, we might actually get a positive external.
[09:16]
Han Shang Chia
Right.
[09:16]
Marcus Goldstein
We might get electricity for firms and for clinics and schools. Hans, over to you. What are you most excited about?
[09:25]
Han Shang Chia
I'm most excited by AI's potential to rapidly increase the scale of intervention delivery and to drive down the cost of traditional methods of delivering interventions. So for example, ag extension work, you know, having an extension officer go out in the hot sun, go farmer to farmer, can be quite costly, can be very, very time consuming. And so one of the questions that is most exciting for us is can a digital extension worker, delivered over a voice call or delivered over WhatsApp or SMS chat, allow the delivery of information to farmers much more inexpensively? And are we able to cover just a much wider geography at a fraction of the cost? I think that's incredibly exciting. And then you have maybe the same pool of human extension workers on the back end reading and tracking the bot's responses to the farmers and correcting when there may be errors. And so I think that's an incredible multiplier of your existing workforce that I'm quite excited by. But to come back to your point about what if there just isn't existing infrastructure? And if we were to stick with the pharma example, what if there were no agro dealers in the area that could supply the necessary inputs, fertilizers, seeds, et cetera. AI is not going to fix that. And I think this comes back to the idea that we often study in development economics, that there might be multiple constraints to addressing a development challenge. AI could address one of those constraints, but that's why we should not be thinking of AI as a silver bullet, but really evaluating if ultimately we are achieving the outcomes, because there might be other constraints that the AI is just Unable to help a user overcome.
[11:18]
Tamina Mah Dawn
I have heard people say that we need to find ways to connect AI to the physical world and that probably means AI along the supply chain. So yes, deliver support to people who are end users of government services. Yes, deliver it to the frontline workers, empower them to deliver in person services better, but also connect it to the pharmaceutical supply chain and the vaccine cold chain and connect it to the network of agro dealers or pharmacies or other physical entities in the world that deliver goods and services. And yes, that takes a systems approach. And I think, Markus, that's sort of what you were alluding to. We do need to think of where this inserts along the supply chain. Some of the cases people are not as excited about enabling the querying of complex logistics and transport data to understand where rapid diagnostic tests or vaccines are in a country. But that probably is as important an intervention and we've certainly seen that in the private sector that there are big unicorns doing AI logistics for warehousing and transport and getting goods to retail. And we probably need to do the same thing in the development sector.
[12:27]
Han Shang Chia
Yeah, absolutely. I think that's where we have to go. This combination of AI interventions plus a platform play. And what I mean here is you can imagine a digital agronomist providing advice to farmers saying, hey, I've detected a pest. Based off the photo that you have uploaded, I recommend pesticide abc and here are six agro dealers in your vicinity that I think has stocks and supplies for the pesticide. And so this combination of connecting you to a marketplace, connecting you to supply chains, I think is absolutely where we have to go next.
[13:06]
Marcus Goldstein
I was reading an article recently about why AI isn't going to be a super accelerator for growth and one of the arguments was it's just going to make bottlenecks more obvious, right? Or more painful, to your point. But here I'm hearing a more optimistic argument which is we're going to find these bottlenecks and pinpoints and then we gotta bring the sort of next gen solutions to start dealing with them. There may be limits, I think ultimately with physical infrastructure, like if there's no road to deliver those pesticide hunts, then at the end of the day all the AI in the world, unless we get at building roads, isn't going to help us. I'm intrigued to see what the second generation problems are going to be. And to your point, the systems approach is I think going to be critical there.
[13:57]
Tamina Mah Dawn
Marcus, what are you most excited about?
[14:00]
Marcus Goldstein
I. I think you both captured things that I'm really excited about the empowerment aspect that you alluded to. Tamina personally excites me and I think that could be a game changer for a lot of folks. I worked with some psychologists on this business training which was helping to develop personal initiative or helping people tap into their personal initiative. And the program's been scaling since I first worked on it pretty significantly. There's the day to day constraint of tapping into that which I myself experience. And a lot of times it's just asking you a question, you know, hey, Markus, what are you afraid of today and what are you going to do about it? To me, like bringing that kind of approach to helping people develop their agency, that's like super exciting.
[14:54]
Han Shang Chia
I think that is the question of our time, which is, does the use of generative AI enhance or remove our agency? Does it enable us to introspect, to be reflective and unlock our potential? Or does it substitute away from our own self directed learning and critical thinking and introspection and then just hand us the answers? We're already seeing some worrying signs in the education space. There was a study that was run in Turkey that provided an AI chatbot to students with no guardrails. And one you saw was that students used it as a crutch and they did not write self directed learning. Students just wanted the answer. They did not challenge themselves and they performed worse than students who were provided a chatbot with guardrails that didn't give the answer away.
[15:49]
Marcus Goldstein
So, Hans, you opened this question about guardrails and thinking about how we design AI to make the world a better place instead of a worse place. There's lots of discussion on this in the Global North. There is a fair amount in the global South. How is the question on what guardrails should look like and what's important to think about when we talk about development? So what's the development cut on this question?
[16:23]
Han Shang Chia
Let me give a very brief history of this question, of how do we get generative AI models to align to human interests? And the conversation so far has been dominated by very important conversations about safety. So you want an AI that doesn't encourage the user to engage in self harm. You want the AI to not tell you incorrect answers during medical emergency or a sensitive period of your life. But to me that's table stakes. And when I say table stakes here, I don't mean that it's necessarily easy to execute. There are a lot of incredible researchers who are working very hard to get AI to be more safe. But it's table stakes to Me, because you can create a perfectly safe AI and yet it does not generate the development outcomes that we care about. So you kind of have an AI that very rarely tells you to engage in self harm, but doesn't necessarily exhibit pedagogical best practices when it's tutoring the user that still gives the answer away. So to me, these traits that we want in development best teaching practices, like exhibiting concise and empathetic agronomic advice to farmers, these traits are additional and built on top of a base layer of safety. And that's where I think we need to take a further step beyond the conversation of safety.
[17:57]
Tamina Mah Dawn
What I would add is a shout out to some of the people in the Global south who have really been pioneers in thinking about safety in context. TATL and Tarunima is an organization in India that's doing academic networks that work in this space. Part of it is understanding that what is appropriate advice in one culture, for example, I've had a fight with my husband. How do I resolve this assertion? May be appropriate in some cultures. In other cultures there's a need for stepping back and not always asserting ourselves. And we, you know, I myself as a Western feminist would find that odious in my own life. But I live in an environment which gives me that affordance to feel that it's odious to not assert myself. But you know, for people who don't live in my society, I need to provide advising and coaching that is safe for them. So I think there's a component of safety which is around culture, social norms, also language, how we use language, when it's appropriate to use I versus we. So much of our Western global North culture is baked into these models at the most foundational layers. And it may take a lot of work to develop different kinds of technologies, other foundation models that don't come out of the global north that better suit the diversity of global south contexts that exist. I guess the other ethical issues I would raise are ensuring that the voices of people who are engaged in social service delivery and in the use of social services is brought into these models. And what do I mean by that? I mean that the language that we often use to train models and even fine tune models often comes from the written word. We're getting more and more data now with philanthropic investment of spoken word, more and more language models that are spoken speech to text, text to speech and speech to speech in local languages. But beyond capturing vernacular for every sector, we need to understand the kinds of questions people are asking. What kinds of help do they want the example that I've gotten from Rikkin at Digital Green is that an understanding of rust in the context of wheat versus rust in the context of other plants versus rust in the context of a rusty nail. Different models deal with that differently and it depends what they've been exposed to. What kinds of language? If I'm a farmer, a smallholder, I probably need my vernacular, my sector specific language represented in these models. This is less about ethics, but it's more about inclusion and making sure that we, we capture the voices and the needs and the asks of people who we want to benefit with this set of tools.
[20:35]
Han Shang Chia
I guess an open question here is who is going to be responsible for building these products? Should the responsibility lie with the major tech companies at the foundational model level? What is their relationship to the nonprofits or to the governments that will be the the ones discovering these six definitions of rust that Temina just brought up? And at the end of the day, should we be developing products that live in two separate worlds? For example, the agriculture specific product that has that use, case in this particular region in India where they refer to rust in this very specific way, or should that information be fed back to the tech companies such that at the foundational layer level, this unique cultural and contextual insight is then incorporated into the foundational product? I think that's an open question. And that relationship between the local actors and the major tech companies is yet to be defined.
[21:40]
Marcus Goldstein
I was reading an article this morning about how Google DeepMind is making progress on treatment for cancer. Right. Like, it sounded like somewhat fundamental ways. I don't see anything about malaria or other tropical diseases. I'm in agreement with everything you've both said, but it's not free. And I do think the resource allocation is skewed to where the profits are. And so I'm worried that this doesn't happen because of how resources are being allocated. And I'm not sure that concessional financing, philanthropic financing is going to move fast enough to do this before it's either too late or just sort of kind of add on. Because I think you're both raising sort of fundamental questions and issues. And if it's later, it's just going to be sort of a layer that's put on top of whatever we're using. And these issues are, I think, a little bit more core reactions. Tell me I shouldn't be so sad.
[22:44]
Tamina Mah Dawn
No, I think you're right. It took us a long time to tax carbon, right? And we're still not taxing it at the level we need to. What if any company that has taken more than $100 million in investment has to tax every hit to its API? So an API is an application programming interface. It's the portal between a set of software that a company might offer, like an AI model, and the user who in our case might be a nonprofit building an AI powered chat product. And what if we had an API tax and we started charging companies that are generating wealth and generating revenues, some tax, a public contribution that goes into building products for the rest of the world. You don't, as a company need to build the product, but you contribute through your revenue to somebody building that product. I do want to give a little plug for Google. They do have a TB cough detector that they've built a model that was trained on human sounds related to health, cough, sniffles, et cetera. And my understanding is that others have done this as well. And there is application to neglected diseases, to tropical diseases. As you say, it requires investment. Development is never free. Right? Development is an investment we make in ourselves. And yes, somebody has to pay for it. It's important to get it in the right path. Now, we didn't do it with electricity, with gasoline, until it was probably too late for the environment.
[24:07]
Han Shang Chia
On the flip side to the tax, you could also have a credit where the major tech companies, you know, if you're contributing a data set in a minority language, for example, or you're using their models for socially beneficial purposes, for every unit to be giving a credit back to the user, whether it's a tax or whether it's a per unit usage credit. These are novel approaches. These are newer than the traditional corporate social responsibility investments that are a tiny, tiny fraction of all the private investment that's going into this space.
[24:41]
Marcus Goldstein
So you both have been involved in this accelerator. Tell us a little bit more about the basic idea, and then I'm excited to hear about some of the experiences you've seen.
[24:53]
Tamina Mah Dawn
So CGD, the agency fund, worked in partnership with OpenAI, providing credits to help nonprofits think about how to incorporate AI into their service delivery. We worked in three domains with eight nonprofits. Healthcare, specifically maternal and child health, agriculture, and specifically smallholder agriculture and education. And I would say our interest from the agency Fund's perspective in going into this accelerator was not necessarily to come out with a bunch of success stories. Our interest is in to understand the struggle that nonprofits face in adopting this technology and bending it toward development outcomes, and really even in being able to evaluate and test and learn. And so we wanted to understand the frictions. What is it going to take for the social sector to mold this new technology to its purpose? And I think we've seen a lot of interesting and in some cases painful frictions arise. Obviously the scarcity of data for fine tuning or even for evaluation of models has been an issue.
[26:04]
Han Shang Chia
I think one of the investment thesis for the accelerator was that I think all seven or eight of the nonprofits draw from some existing evidence base that suggests that their intervention prior to AI had some demonstrated literature on effectiveness, that their pre AI version, whether it was a human delivering maternal health coaching to pregnant mothers, or whether it was an AI tutor delivering a phone call math lesson, they all had some indication that their programs were effective prior to AI. And one of the big questions was can AI be used to help potentially improve effectiveness, drive down cost, expand scale? I think even within the cohort you saw a handful that were digitally native. So prior to AI they were already delivering maybe a chatbot just without an AI component. And then we had more traditional analog first organizations who never had a digital engagement of their user, for example. And I think having a mix of both digitally native and analog nonprofits has been very instructive on what it would take to get the development sector at large more on board with AI applications.
[27:21]
Marcus Goldstein
And any lessons there?
[27:23]
Tamina Mah Dawn
Definitely a lot of lessons. I think a First is some NGOs are jumping straight into evaluating the impacts of an AI product or service without getting it to stability first and without understanding whether people are using it first. And so there is this rush to generate evidence about AI's effectiveness. The reality is you really need many iterative cycles of product development before you make that investment. Another thing we've seen is there are very well established metrics and benchmarks and strategies for evaluating AI model performance, and there's even tooling for it. But the tooling requires expertise in software development. And we're seeing that a lot of organizations are vibe checking and vibe evaluating the model based products they're building rather than actually using the tooling that exists. So there are things like Deep Eval and Langfuse for model observability, but also for model evaluation, testing whether the model is behaving as you expect it to. We would like to see more nonprofit adoption of these tools. We'd like to see adoption of a user funnel. If you're now using a digital product with AI and you're delivering it to a user, can you show me the stages of that product as the user moves through the experience from onboarding to first use to engagement and retention. And can you show me how many people convert to each stage as they move through the funnel? We're just not seeing enough inspection of how a product's getting adopted, how many people are adopting it, who's adopting it, and how do we refine the product to increase meaningful adoption.
[28:58]
Marcus Goldstein
It's kind of like this blind faith in technology. So if I'm an analog organization, I know how to do a program. I know if people aren't showing up to my health core clinic, something's going on and I'm going to go find out what it is. But now I bring out a bot and of course, like somehow I think it's just going to be great and everyone's going to use it and I'm going to assume that.
[29:24]
Han Shang Chia
So it's this switch to this over.
[29:26]
Marcus Goldstein
Optimistic sense that the technology will just be fine.
[29:32]
Tamina Mah Dawn
I think so. And the irony in that is that we have so much more visibility with the digital product. You can actually see whether people are using it. You can actually see what the model is doing. We can record the inputs and outputs. And so it's ironic that in this context, where there is something digital that is less physical, it's much easier to monitor and trace. People don't have the tooling to do that and they really need it.
[29:56]
Marcus Goldstein
Yeah. Oh, gosh, yeah. Leaving data on the table, ah, it's a cardinal sin.
[30:02]
Han Shang Chia
One of the things that worries me about a rush towards impact evaluation without tracking how the model is performing or without tracking how users are engaging, is that at the end of the day, there is no1 singular AI. These products move and change every day. There are knobs and buttons that you can press and twist to make the AI perform differently and to increase engagement or reduce engagement. And when we don't track this earlier stage performance, the model or an engagement, what are we really doing an impact evaluation on at the end of the day? One day we may find that an impact evaluation boosts crop yields or boosts learning, and then the next day, after we've changed the knob or an AI company has tweaked the dials a little bit, you find that the AI produces a different impact. I think we've all seen enough impact evaluations making claims about what AI can and can't do. And the reality is that there's very little specification on what exactly the AI system is under the hood and how it's been developed and designed. And so I think we need much more evaluation and reporting of what the product is actually doing and how users are actually Engaging or dropping off?
[31:22]
Marcus Goldstein
We're getting close to the end of our time, so I wanted to ask one of the CGD standard podcast questions. You both have worked in a pretty diverse set of organizations and experiences, so I was wondering, can you share a memorable story from your work?
[31:40]
Tamina Mah Dawn
I go back to some work we did with Breck when I was at the center for Effective Global Action. We had Breck. Brac is one of the world's largest NGOs. They're a non governmental organization established by Bangladeshis, headquartered in brac, but now with programs around the world. And they really focus on empowerment, which is, I think, in great alignment with our work at the Agency Fund. We had BRAC as a partner for research and BRAC researchers would come to the US and teach us and learn from us. And then we would go to Bangladesh and some of the countries in Africa where they were working. And I remember visiting a woman who was part of what would become the graduation program. This was maybe around 2009, maybe 2010. And sitting in the small, not even a room. It was a little space where a woman lived with her three kids. It was tiny, but she was an entrepreneur and she had been endowed with coaching from someone at BRAC who cared about her enough to invest in her ability to navigate. She had become a trader in fabrics and their bed was made out of a big pile of fabrics that she slowly, you know, that was her inventory, that was her startup grant, and they were selling these at market. And it was not an easy life at all. But this woman had been afforded a sense of dignity from the investment that was made in her in coaching and supporting and listening. And I don't want to say that AI replaces that. What I hope is that AI supports the coach in the graduation program to be a little bit more responsive and maybe nurturing or maybe inspires the coach in that program to think a little bit more with empathy for that woman's situation. Whatever it is, I hope that AI helps us make those investments in each other. Because that sense of dignity of a woman who has been given sort of a second chance at life, to me that's quite compelling. And I don't know if this woman had removed herself from extreme poverty yet. This was where I was doing some qualitative work, but that's a memory that sticks with me.
[33:53]
Marcus Goldstein
You're giving me flashbacks to visiting the adolescent girls in Uganda that were part of BRAX program. Like one of the roughest sets of questions I've ever gotten in a meeting was from those adolescent girls they were, yeah, I'm with you, Ans.
[34:11]
Han Shang Chia
So my story is from 2018, 2017, 2018, when I was an implementer of a cash transfer program and I was responding to a hurricane in the Caribbean. One of the big pain points that I experienced was not distributing the cash transfer itself, but it lay in targeting, finding and enrolling households that had been affected by a hurricane that had their homes damaged and lived below a particular poverty line. And that process of targeting and enrolling households was incredibly time consuming, very slow, and I felt like I was letting down the families that we were trying to distribute cash transfers to. It took us something like six months to identify, enroll, and pay just 5,000 households. And so after that experience and prior to the world of generative AI that we now live in, when predictive AI was just making its way into the development sector, I reached out to Google and reached out to professors at UC Berkeley to ask, hey, surely the machine learning techniques are getting advanced enough such that we can use satellite data and other big data sources to identify, based on satellite imagery, which households had their roofs blown off by a hurricane. And beyond that, whether an urban setting, you know, whether there are mud roads or tar roads, whether there is urban planning, whether roofs are made of grass or made of tin, surely all of those factors could form a composite index of wealth and poverty. And surely a machine could help us identify regions that were poorer than others. And over the next few years, we partnered to be able to use machine learning techniques to predict not just damage, but household wealth and poverty. And in 2021, 2022, during the COVID pandemic, I worked with a number of governments, including government of Togo and the government of DRC, to be able to deliver cash transfers to 200,000 households in weeks. So we went from in 2017, taking six months to identify, enroll, and pay just 5,000 households to several hundred thousand at weeks. And that's what motivates me with this new crop of AI technologies, to figure out how we can just go to scale at a fraction of a cost, and using and applying these technologies to proven interventions like cash transfers. And so that's my favorite memory of AI and development.
[36:39]
Marcus Goldstein
Thank you both for a really interesting discussion. I've learned a lot about thinking about using AI to help us boost programs to scale and increase speed, and also to think about the deep potential for possibly helping people tap into their agency and increasing it. And I've heard cautionary notes about building mechanisms now to make sure that we don't leave parts of the world behind and exclude them from what could be a really powerful technology. You can learn more about CGD's work on AI and development as well as the other things we work on on our website.
[37:20]
Tamina Mah Dawn
Thank you Markus. This is a lot of fun.
[37:22]
Han Shang Chia
Thank you Marcus. Thanks for listening to the CGD podcast.
[37:31]
Tamina Mah Dawn
You can learn more about the topics discussed on our website at cgdev.org@cgdev.org work.
[37:40]
Han Shang Chia
See you next time.

The CGD Podcast: Where AI Meets Development with Temina Madon and Han Sheng Chia

The CGD Podcast: Where AI Meets Development with Temina Madon and Han Sheng Chia

The CGD Podcast: Where AI Meets Development

Episode Overview

Key Discussion Points and Insights

1. The Current Landscape of AI in Development

2. Rethinking Impact: Technology vs. Outcomes

3. Where AI Is Making a Difference

4. Scaling, Cost Reduction, and Real Constraints

5. Bottlenecks, Systemic Challenges, and Optimism

6. Agency, Empowerment, and Education

7. Guardrails, Alignment, and Cultural Context

8. Funding, Incentive Structures, and Resource Allocation

9. Lessons from the AI & Development Accelerator

Notable Quotes & Memorable Moments

Timestamps for Important Segments

Tone and Takeaways

Useful For

The CGD Podcast: Where AI Meets Development

Episode Overview

Key Discussion Points and Insights

1. The Current Landscape of AI in Development

2. Rethinking Impact: Technology vs. Outcomes

3. Where AI Is Making a Difference

4. Scaling, Cost Reduction, and Real Constraints

5. Bottlenecks, Systemic Challenges, and Optimism

6. Agency, Empowerment, and Education

7. Guardrails, Alignment, and Cultural Context

8. Funding, Incentive Structures, and Resource Allocation

9. Lessons from the AI & Development Accelerator

Notable Quotes & Memorable Moments

Timestamps for Important Segments

Tone and Takeaways

Useful For