← Humanitarian Frontiers
Loading summary
Transcript33 lines
- [00:03]
A
Welcome to Humanitarian Frontiers in AI, the podcast series where innovation meets impact. In each episode, we dive deep into how artificial intelligence is reshaping the future of humanitarian work. From enhancing crisis response to making aid delivery smarter and more effective, AI is opening new doors in the way we support communities in need. In this series, hosts Chris Hoffman and Nassim Motelabi brief you thought leaders from academia and the tech industry to discuss not only the vast opportunities AI offers, but also the ethical considerations and risks we all must navigate. Join them on this journey as they explore AI's potential to transform lives and address humanity's most pressing challenges.
- [00:49]
B
All right, well, welcome, everyone. It's nice to have you here today. This is such a distinguished panel. Unfortunately, Nassim can't be here with us. She is gallivanting. No, that's not true. She is at work in the Dominican Republic and she's going to try and pop in, hopefully sometime throughout the call. But if she doesn't make it, she's given me full autonomy to ask whatever I like this time because she's usually in control of the question. So we'll see how this goes. It's Chris Hoffman back here, episode three of the Future of AI and Humanitarian Action. And we've got, like I said, a distinguished panel here. And let's just jump in. I want to ask the first question. So I got challenged the other day by a university professor here in the Netherlands, and he said, I don't think we should be talking about the ethics of AI use. We should be talking about the human component of this instead. So how do we as humans apply our ethics in the way that we design the tools that AI is or AI becomes or that AI facilitates in terms of action? So he was like, you know, everybody's saying ethics in AI. And he said, I don't think that's what it should be. So, Emily, I mean, I want to start with you. Where does this ethics question start? Where does it find its genesis? And then where do you think it goes after that?
- [02:04]
C
Thanks for the question, Chris. So I'm actually going to say something a little provocative, which is, I actually think AI ethics is passe. So I actually think what we should be talking about is responsible AI. Because if we look over the last several years, I mean, sort of your question, like, what is the genesis of AI ethics? I think suddenly people were like, oh, wait, this thing that people talked about as being objective and neutral actually isn't objective and neutral. And we need to start thinking ethically about what is the appropriate use of this in society. And that was an amazing conversation. It gave us a lot of interesting things to work with and think about. But really, I think what I'm excited about right now is I feel like sort of the international community is coalescing around responsible AI, but as ethics are only good if they're actually operationalized. And I'm really excited because I feel like we're at a point where we're starting to talk about implementing some of the things that the conversation around AI ethics drives us towards. So how can we think about responsible AI, Trustworthy, secure, safe AI? How can we think through risk, harm, risk, mitigation, and all of those force people to think a lot more, to this professor's point, about human action? And so I think often we've forget that actually we're in charge. And the way that tech products get produced isn't unique to AI necessarily, but there are all sorts of ways in which human decisions get embodied into the tech product that gets out there. And we forget that actually it is those decisions. So when we see a racist or sexist app, we can usually follow that back. Or an AI system, I should say, we can usually follow that back to decisions that were made by people at companies or at organizations. And so I get really excited about responsible AI, because I think that is really the mechanism by which we can make good on ethical AI. Mala, I'd love to hear what others think.
- [04:07]
B
Yeah, I mean, Mala on the tech side of things, because that seems to be the moniker that's coming out from a lot of the organizations. You hear Anthropic talking about it now, you're seeing a lot of the folks that have left OpenAI starting different organizations that are talking about responsible AI from the tech side of things. Is this window dressing or are people really worried about the responsible use of these things from the tech side of things?
- [04:31]
D
Well, I think we could spend the full hour just on that question. So let's go. So I do actually have a series of videos covering some of these concepts, because as it often happens with any new type of technology, everybody's coming at it from different perspective, different fields, therefore different terminology. So just two terms that I think are important to clarify in this conversation. One is the idea how I define it as responsible AI versus AI for social good. The way I basically break it down is if you look at the Universal Declaration of Human Rights, which is actually a document that is obviously in our field, is very well known, but even in the tech world, you know, Anthropic, for example, cites the Universal Declaration of human rights in its manifesto, or whatever they're now calling it, about constitutional AI. But when you look at, historically, the tech industry, especially here in the United States, what they were really focusing on was the idea of negative rights, which is civil political rights that are basically codified in some kind of law. And so a lot of that does come down to compliance. When you talk about AI not helping somebody commit suicide or telling somebody how to kill somebody, or generating child pornography, those are the categories of things that are often coming under this moniker of responsible AI because they are something that tech companies have to do in order to stay compliant with the law. But a lot of what we focus on in international development, international humanitarian work, is also related to economic, social, and cultural rights, which are positive rights. The idea of public health, education, gender rights, these are all rights where something does have to happen in order for you to have that thing, in order for you to have a good experience at a healthcare system, somebody's got to build a hospital or a community clinic and train a doctor, and we have to have tools and technologies that support that. That latter half is really what I focus my career on in tech for international development and tech for social good. But that's not honestly what a lot of people are focused on at this point in AI. And part of that does come down to the fact of this idea of, like, AI evaluations. So in order to build out a tool that progresses public health, for example, using a generative AI, let's keep this to generative AI. In order to build out a gen AI tool that's for public health, you have to then go back and do your AI evaluations to make sure, as Emily was saying, that it's not racist or sexist or unduly biased. And something that's increasingly common is that it's not retrofitted for, like, an American context. If you're trying to deploy to West Africa, for example. The problem is these are very complicated things to do, and they're also quite expensive, because if you look at a large language model with 16 billion parameters, for example, even just to get adequate coverage across, like basic hazards or harms or however you want to define, it will take you a lot of prompts and it'll take you a lot of responses, and it'll take you a lot of evaluating. One of the things I just did and ML Commons, was to help launch an AI safety benchmark, which is supposed to do those things. And it does, but it is very much for the tech industry and, you know, informed by academia, but not for international civil society. And I could talk about reasons why. So there's a lot of work that has to happen before we get to the point where we can even say this AI solution is fit for purpose, because we might be putting something that's inherently problematic and introducing biases that we can educate humans not to have or that humans may not have because they're already coming from a different cultural context, whereas our AI tool was built on mostly American data. So those are some of the kind of bigger challenges that I see. But I do think one thing that's nice about AI, I will admit, as exhausting as it can be, is this. This idea that people are taking, like reclaiming the idea of the algorithm. And I do attribute it to a lot of really strong women of color over the years who have flagged these issues for a long, long time. Back in 2010, 2011, when everything was starting to become Internet 2.0, the era of social, we had this vague idea of algorithms. And so everybody in international humanitarian and development spaces who were working in tech were like, we're going to use Facebook to solve poverty. And then we launched all these programs and realized there's algorithmic bias. But then they're also just really easy to kind of subvert the system. I won't say hack, but, like, for example, you can create a fake profile and infiltrate a group that's meant for young women and girls to learn. It's not that hard to do that catfishing issue and get around the supposed guardrails on the platform. So I do think the world has learned from that era, which is nice, because now we get a lot of people who are very cognizant of this idea that there's algorithmic bias. But we're not at the point where we have a great suite of tools or really good solid methodologies or even really solid decision making and how to do that. And what we do have is coming out of a couple of countries. And so I don't know if this sector on its own will be able to figure out how to implement some of these ideas abroad and to other places, but I do find it heartening that people are trying.
- [09:10]
B
Absolutely. Absolutely, Suzy. So that is, I think, such a great segue into the publications that you've just put out, because I think it starts to discuss some of the biases that we see, especially from the global south, et cetera. I mean, I would love to hear a little bit about your findings and how that relates now to where we're starting to see our soup Come out. Right. We started out with some broth over here, some vegetables over here. Now we kind of know what the soup is starting to look like. And so what did you find?
- [09:37]
E
Yeah, thanks a lot, Chris. And just quickly reflecting on some of that, just going back to the, you know, what kind of framework is right to make sure that AI works for everybody? I mean, I think the ethics framework is still very important when you think about it from a macro level, because ultimately, you know, ethics is about normative values. What does it mean to be human? You know, what do we want our society to look like? And different debates that go on in different societies and different communities with different values. And as we start kind of wrestling with what does knowledge mean? I think these, these are things that are important to bear in mind as we, as, as AI is designed, and we. And we think about how it's going to impact people. And then just one thought on the responsible AI piece as well. And we'll come to our research in a moment because we talk a lot about this in the research. But for me, responsible AI needs to bring in very much inclusion. So it's also about inclusive AI and participatory and who's in the room making decisions about AI through the life cycle as it's developed, but also governance as well. So the thing about responsible AI, as a phrase, it can almost be the idea of we just need to make sure when we're building it, perhaps in the Global north, for communities in the Global south, that we need to make sure that we respect privacy, we avoid bias and these kind of things. But actually, if we talk more about inclusive AI and participatory AI, then it's not so much we're building for, but we're building with and together. So let me talk a little bit about then, the research that we've just published. So this was a piece I was leading for CARE International in collaboration with a brilliant team at Accenture. And what we did was we wanted to think about artificial intelligence and the implications for it, implications of AI as perceived by Global south civil society organizations. So what does Global south civil society think are the risks and opportunities and gaps? And what are the pathways to more inclusive decision making through that AI life cycle and also through AI governance. So we went out to civil society organizations, CSOs in 12 countries across four regions, and also. So that was Africa, Asia, Latin America and the Middle East. And then we also spoke to participants from multinational technical technology companies, international NGOs, UN agencies, and a government donor, really, to understand what Are some of those tensions to building in civil society throughout that life cycle? So I mean, in terms of kind of what we were looking at, we were looking at some of the tensions, particularly around AI in the humanitarian and development context. So thinking about AI and operations, so how do we navigate that balance between efficiency and effectiveness? We know NGOs and civil society is really under pressure to try and do more with less, but how do we make sure that using AI tools are also effective and safe and inclusive? And it's not just a focus on efficiency. We also looked around the AI and the digital divide, seeing it both as a potential challenge. So there are certainly risks around widening marginalization. Not everybody has access to even a laptop or electricity, for example, let alone the other digital and data infrastructure around AI. But also how it can be a potential equalizer as well. And I'll just kind of just one third tension and we can go into other findings later. But these are just the tensions we were highlighting. And also then that kind of participation that I mentioned before, like how do we make sure that it's not just kind of sporadic consultation, but actually meaningful inclusion? I'll stop there because I want to let the others kind of chip in and we can always come back to other findings. Well, the actual findings of the report as well, the title, so people can look for it, is AI in the Global South, Exploring the role of civil society and AI decision making.
- [13:58]
B
Absolutely. And we'll include a link in the podcast so that everybody can get it there. I mean, so we've talked about ethics, we've talked about responsible, but who is responsible, Right? So Susie, you mentioned accountable and inclusive. So all of these big terms that we utilize in and around the way that we speak. But let's say that the bot does something racist. Let's say that the bot does do harm. When we deploy something in a health context, who ends up being responsible for that? One of the things I think in the humanitarian sector specifically is the lack of accountability. Right. If disease spreads in a disaster because somebody built the toilets wrong, who are they going to sue? And who's got the right to take somebody who to court? Now we're talking about something that's even more intricate when we're looking at technology and the technology interfacing with people in need. So now let's take responsibility to the next level. It's not just about the responsible development of. It's the responsible implementation of. And then who takes responsibility for. And so I don't know if that's controversial or not. I feel like it is. Or maybe it's just a question that nobody has an answer.
- [14:59]
D
I think the lines of what exactly is deploying and implementing a tool has changed significantly. So I've been in this space now for 15 plus years, right? And when we started it was SMS is going to change the world. We're going to reduce poverty with text messages. There was always a technology that was going to reduce poverty. But when you looked at even just SMS aggregators like Rapid Pro or Rapid sms, there was so much work we had to do just to design the tool, the interface, figure out where it would sit in the workflow, all of that stuff to do the implementation. And you're talking about quite literally one of the most basic technologies on earth. Then as we've moved into different spheres of technology, whether it's proper ux, research and design, mobile smartphone, mobile app development, cloud computing, and now with AI, these technologies have gotten increasingly complex. And I think you can't really put a fine line to say like these are what you need to do to implement it and this is what you need to do to customize it. Because this is a word that is very unique to this world and literally nobody in big tech uses that because it means not a whole lot. So when you look at the idea of like what it takes to implement and deploy an AI tool, you can focus on what I would call like the more humanistic aspects of that. So again, looking at where it should sit within the workflow, designing an interface so that people understand what they're interacting with, understanding, like what exactly people are supposed to do with the tool, the limitations of the tool, all of that stuff you can focus on. But then there's so much stuff that you have to do in order to get to the point where you can decide if that AI tool is fit for purpose. And a lot of that is selecting your large language model and figuring out what kind of guardrails are in place and figuring out what are the factuality limitations of that tool. And all of that work I think is now happening within many responsible AI communities. But the challenge is then trying to translate that very expensive work honestly to this idea that we can somehow put it into a AI for good solution and then have something that doesn't say or do something racist or sexist or biased, because sometimes you won't even know. And that's one of the big challenges. So then if you get a response back that is overtly racist or just culturally inappropriate to go and try to troubleshoot, that is really, really complicated because you're now looking at a tech stack that's massive. Some of the stuff is not even anything that you probably touched. I mean the foundational large language model, there's pretty much a 0% chance that anybody in the international development or humanitarian space did anything with that. At most they probably fine tune that and maybe they put some kind of rag system on top. So to then go and troubleshoot kind of the core challenges with any kind of algorithmic based tool is really, really challenging. And I think honestly a lot of people in humanitarian work and international development don't understand kind of that spectrum of what it means to have something that is a non algorithmic tool versus an algorithmic tool versus algorithmic plus AI. This is some of the work that I did when I was a senior advisor at the World Health Organization. It was quite challenging then too, honestly, because we weren't even talking about AI, we were just looking at open source. And the delineation really did become, you know, is it algorithmic based or non algorithmic based? Because once you get into the world of algorithms, there's so much work that he could be doing and a lot of it is just a shock. The dark.
- [18:02]
C
Yeah, I'd love to come in on some of the things that Mala just fantastically mentioned. So I think when we talk about responsibility, I actually feel like we need to back way up. So Mala talked about the expense and the difficulty of this. And let's get really real about what funding looks like in the social impact sector. Like it's not there. And often we also work on project cycles. And so it's like you've got one, three, five years maybe to prop something up. And I would say right now there's such a trend to just fund the pilots that are like, hey, what's the prototype of an AI system? And it's really about that technical build. We are not funding and donors, funders don't seem to be aware of how much work actually has to go into making something that is fit for purpose. And so I would love to see a lot more work and a lot more funding that looks at AI systems holistically, that looks at them from a socio technical perspective. And certainly there are ways that we can think through what we can get at. If we think through sort of the technical. Have we actually built a system that people can understand what the global rules are for how a system comes to its outputs? And can we actually even explain why an individual received this particular outcome and then sort of work through that? But nobody's funding that detailed level of work. I think because of ChatGPT there's this idea of AI as plug and play. And AI is anything but plug and play, especially in the circumstances in which we operate in the social impact sector. And so I really think broadly we need to sit down and reconceptualize how we approach these as builds and what actually makes sense for us in this sector. To your point Chris, on sort of like who is actually accountable, who is truly responsible? I think right now, and I mean I'm an American citizen, we have not socialized. We do not have any sense of actually holding companies to account. We don't have any sense of what is informed consent. All these companies are turning on being able to learn from our data as defaults and then we have to figure out how to go in and actually shut it down. Even the people who ostensibly would be designing these systems don't even have a culture of actually consent, data privacy, holding companies to account. And so we have this massive issue. We're going to go say that we care about those things in a development or humanitarian circumstance, but do we even know what it looks like? And I would say that we don't. And I just want to follow up on something that Suzy said earlier around inclusive and participatory AI that is sort of what I have been looking for for the last several years. Suzy knows this. We met at a conference in Barcelona recently when I presented a whole case study on this. I am searching and have been searching for examples of meaningful co creation with communities around AI builds. And I think as we all probably are aware, social impact sector often has a tendency to use words and not put them into action. And so I think we need to get really real about what it is that we mean when we talk about inclusive AI and participatory AI. And we to need, we need to set the bar really high because doing one focus group with eight people and then saying we had a participatory design process that is not actually participatory AI. And I can see how easy it's going to be as a sector. We're time budget constrained. I can see how easy it would be for us to fall into this whole participation washing game all over again. I get really excited about AI as an opportunity to do things differently. Yes, it's the same as all this stuff that's been going on as Mala's been talking about, you know, this like this tech solution is going to solve everything. But that's not true unless we radically change the way we actually approach Design and thoughtful implementation and maintenance and all of those things. I'll stop for now.
- [22:52]
B
Awesome, Susie.
- [22:53]
E
Yeah, I mean, Emily knows that I kind of agree with her on all of these things. And I think the real risk, where we are at the moment is that NGOs and humanitarian, big humanitarian aid agencies are suffering from massive FOMO and there's huge kind of panic to get on the bandwagon if there is any funding, funding for pilots without kind of coordination between organizations. And I think that is being kind of more and more understood now. But I think with regards to who has responsibility and to whom do we need to be accountable, it is affected populations. We know that from our humanitarian principles, from accountability to affected populations through core humanitarian standards. But as we were talking to tech multinationals during our research, we had some really interesting, honest interviews where the tech companies were saying, absolutely, we are doing our due diligence, our human rights due diligence and trying to understand how models work. But deployers also need to do their due diligence and understand within their communities what are the potential impacts. And I think that's absolutely critical for INGOs and UN agencies and so on to really understand, I think, in terms of where we are at with funding for it. I'm really pleased to see FCDO in the uk, the Foreign Commonwealth Development Office, releasing a call for responsible AI and in humanitarian action. So actually really trying to understand what does this look like. And I think absolutely there are not enough efforts to look at participatory AI and what it means to be meaningful. Although there are some interesting bits of work out there. I know NESTA has done some interesting work around collective crisis, intelligence and understanding what are the challenges around that, the opportunities and challenges. But I think really what this requires is that bridging of lived experiences and creation of relationships between tech developers and civil society, which has been kind of my focus within the Machine Race, my blog, and through this research as well. And the reason why I was so keen to partner with, with a tech multinational, in effect, like Accenture, who are actually, you know, can be really open to kind of creating those, those relationships to understand, you know, how do we navigate some of the tensions between for profit and not for profit and work out what is that shared value proposition really, how do we make sure this really is equitable for people in the Global south and navigate, navigate those conversations that are needed so that civil society will be in the room, but that it will be equitable. I'll just stop there and pass over to others.
- [25:53]
C
Thanks, Susie. I want to so build off of all of the great stuff that you said. I also just want to pause for a second and also just declare that I think providers are ultimately responsible. And so, like right now, I don't know that any of us have actually said exactly who we feel is responsible. And I think the piece, like Susie, did a great job of saying we're accountable to the communities that we're hoping to serve. But I think so often now what we're seeing, and to Malo's point, it costs so much to build out these things. And so we have all these social impact orgs that are taking a foundation model and then building on top of it and all that investment that it takes to get to be fit for purpose in particular use cases and actually stand up to the rigor that's required in that particular use case. So I certainly think providers are a key element here. And the organizations that are choosing to deploy certainly are as well. But that brings us back to AI literacy. I work as an AI literacy trainer for social impact professionals who are not coders, who are not data scientists. I can tell you that there is so much misunderstanding, understanding around what AI is and how it operates and how it can be used. And so in order to get to do all the things that Susie mentioned, we need civil society organizations to have the AI literacy, to have these conversations and it's not broadly present and to continue with this idea of like the fear of missing out. People don't feel comfortable. And it can be a scary thing to say, I don't actually understand what AI body is, except a lot of people don't actually know what it is and how it works. And so we need to change the culture of being able to say, hey, wait, let's slow down here. There's so much hype, there's so much pressure to adopt and we need an ability to say, hey, let's upskill everybody so that they understand this. And one of my big passion projects is I think we've sort of given the AI space to data scientists and coders and developers. And there is such a big role for non technical, I mean, they're technical in their own right. Non coders, I should say. There is such a big role for non coder individuals to play on an AI build, but they have to have the AI literacy in order to understand how and where they would contribute to an AI build. And so if we think about something like fairness metrics, what are you going to pick for fairness metrics for your particular project for this particular use case? What is going to be important? If you look across the different suite of fairness metrics, they can conflict each other. You need to pick something and you need to have a rationale for why you picked that and why that is appropriate for this use, this cultural context, et cetera. The. There are really big questions in there around, are you going to prioritize an individual, Are you going to prioritize the group? And we can see, if we look cross culturally around the world, we can see. I come from America, we're an insanely individualistic society. And the vast majority of my career has unfolded in Ethiopia, where that is not. It's much more about the collective. And so people are going to choose different, different things. If we get all the way down into the nitty gritties of fairness metrics, people will choose different things. And yet we don't have right now a broad AI literacy in order to have the conversations that we need to have. And so I would put in a really big plug for people upskilling in AI and thinking through, really coming to know how and where bias comes in to a system. And we need to know that so that we can deal with it, so that we can proactively mitigate it, to actually build the solutions that are meaningfully fit to purpose and hopefully not reinforcing or amplifying existing inequalities. And I would just say that's a really hard task. Like Big Tech is not responsible for doing. No. I mean, I would say they should be, and they might say that they are, but I don't think in reality they're really focusing on do no harm and not reinforcing existing inequalities. But the social impact sector has a different mandate. We have to figure that out. We have to figure out how to do AI builds differently than what Big Tech is. And it takes a lot of work and it takes a lot of knowledge on behalf of everybody on a team. If you have a specialty in education, in health, in whatever it is, you have a role to play. Even if you don't know what AI is, there are really meaningful things that you can contribute to those discussions. Questions you can ask data scientists, questions you can ask developers. There are all these decisions that happen in an AI build around thresholds and why did one thing get picked over another? And those are all social questions that should get debated. Mala, what are your thoughts?
- [31:22]
D
So many. So many. Emily, I echo a lot of your points. I think the FOMO point kind of hits me hard because that's, I think, for everybody who's Ever worked in tech for international development or tech and humanitarian spaces can understand that. It's too bad and it seems end here because we had a really great conversation a couple months ago when I just kind of like about the arc of our careers. So when I started out 15, 16 years ago, I came into this world thinking big tech is evil. They've done so many horrible things. They are morally and just unethical. I can't possibly imagine myself working for them. So I went to the un, well, a lot of commentary there, Built tools for them for about a decade and then as time went on, saw the complexity of these tools were getting bigger and bigger, but the budgets were getting smaller or there was just like too much emphasis on, unfortunately companies like Accenture or other consulting firms coming in and writing a beautiful report, but then very little thought about how to actually implement and deploy anything. And then my view started to shift. I was like, I think what we're doing in the UN by and large is unethical. We are not talking about security, we're not talking about proper inclusivity, we're not talking about data privacy, we're not talking about governance. These are all things that Big Tech to their credit has thought about in quite a lot of detail. So maybe I should be looking at what they do. So that's when I switched over to Big Tech and worked at GitHub for four years. And it was a nice period where kind of the two worlds, at least in my, my world, came together and there was a very strong alignment among everybody. But then I think the reality is that social impact teams that are kind of these interlocutors at big tech companies have mostly been decimated, my team included. Right. So those interfaces that people had in the UN or other international development organizations are largely not there. And a lot of that is now shifted over to these responsible AI teams that Emily was talking about that are very much dominated by data scientists and software engineers who have not had any kind of, you know, more humanistic or even just public facing work. And so they're coming from this idea of I'm going to basically retrofit a data set to fit another data set. And that's how I'm going to evaluate my AI tool. On the other hand, you've got these social impact organizations, as Emily's calling them, you know, call it international civil society. These international civil society organizations that in a lot of cases don't really understand the depth of what they're trying to do. And so they might just pick a random large language model, slap a couple things on top of it, say that we're going to call this a public health for AI tool, and then not really understand all of the different decisions that had been made up until that point. And then when something does go wrong, they don't realize what's happening. So I think the question is not really who is responsible for a quote unquote bad implementation because it is just so complicated. I think the question is who is responsible for saying no until we can actually do something correctly. And I think we are a long, long way from there. And as hard as it is to tell organizations, I never advocate for deploying a kind of technology that you fundamentally do not understand. And to Emily's point, there's a lot of work that could be done that is not heavily driven in code or in data science. And that part is really hard to figure out where to interface because there's not a lot of organizations out there who are doing this. One of the things we did at ML Commons with our benchmark was to create this idea of like a taxonomy that can be applied globally. But the taxonomy itself and the assessment standard of how, like what would be considered a safe or an acceptable response was very American driven. It's based on this concept of legality and not of international civil rights or human rights or international standards and norms. So for example, one thing that could happen is we build some kind of taxonomy that's actually informed by human rights doctrines, by WHO standards for public health, by UNICEF standards for education. That kind of stuff I don't think has been done yet because there's just this idea that as soon as the tool comes out, that's what I need to jump on. But I think what we really need to come back to is this idea of what are the concepts that we're trying to use and what is the actual purpose of it. It's a lot harder to fund this stuff because everybody just wants a. I mean, everything in international development has also been productized. So people want the pilot that's been a phenomenon for 15, 20 years. Everybody wants the shiny new pilot and to say they were innovative. But I think if we're going to get real about what it means to use AI, these are foundational technologies and not just one off tools. So it's not just about deploying ChatGPT for this use case, it's about understanding what ChatGPT is and then figuring out is it actually fit for purpose in a broader set of solutions just to.
- [35:51]
B
That point for me. So we've come To a part in this discussion where I'm asking myself the question, which is, okay, do we put on the brakes? Do we just say, hey guys, Guterres stands up and says, it shall not pass. Nobody shall do this until we have these five things in place kind of thing, right? Or how do we take the baby steps? How do we know that? Do we want pilots to still continue? Because that's the way that it's still going to push the conversation. Do we stop those things? I mean, you've all kind of alluded to different things we need to do, but in general, what is a process that we need to take to go to the next level with this?
- [36:28]
D
I'll just jump in as like a follow up to that. I think honestly this is going to be decided by a much bigger sphere than just international humanitarian or development work. I mean, already we're seeing kind of a new world order geopolitically. That part we can't ignore and part of it is the degradation of these institutions or the questioning of these institutions. But I think another question that's happening is what vision of AI is going to win? Because right now we're very dominated by Silicon Valley, highly capitalistic, highly wealth driven, right? Like the AI use cases that are coming out that are most relevant or have the highest standards, honestly are for rich people in the United States, which is often the case for a new technology. But then in Singapore and India, for example, they are taking very much a community based approach. And then there's a lot of African natural language processing communities within sub Saharan Africa. So these things are already happening. But I think it's the question of whether this sector can jump on that and kind of facilitate that in the way it knows how to, or if you can learn from that and take a step back, which I think is probably a good thing as well. But I don't know if this sector is going to really shape these forces because they are just so large and they are so complicated. And honestly, the sector has done a not great job of retaining talent and training people and making sure that we do have the software developers and data scientists. I mean, just the sheer reality is the salaries are too high in other sectors to stay competitive. And then it's hard to work in these spaces. I mean, we've all worked at the U.N. i think, for several years, or analogous organization. It's not easy to get the stuff done, you know, and a lot of people go into these organizations precisely because they don't like technology. So then you're always seen as Kind of the outsider and you have to make a case of why it's important to think about these things. I think that's probably caught up to the point where it's not going to be competitive and it's not going to have a real shaping of that. So now the goal is to actually listen to what's happening around there and figuring out how it can plug in in the correct way.
- [38:17]
B
Love it. Emily?
- [38:18]
C
Yeah, I think I get frustrated by seeing what's getting funded right now around the pilots and the prototyping. And I feel like we have this moment where it would be really nice to focus on building the infrastructure for the future. I would love to take the next couple years focus on. I would love it if donors would fund the creation of really strong data sets that could get used in different contexts that are from places in which we work and really to build out what's there. I do a lot of work in gender and social inclusion, but just we've been fighting in the international development sector for sex disaggregated data for years, decades. We still don't even have strong data sets. Forget machine readable, forget machine actionable that are sex disaggregated. And so what I would love to see is if we spent the next couple of years instead of, yes, I want a couple pilots still running. Let's do that. I want those pilots to be a lot more holistic, to be focusing on here's your AI build, but here's your AI governance around all of this. But how can we take the next couple years and build out the railroads, the streets, the things that will help us continue forward in actually achieving much more inclusive AI from a global perspective. That's one thing I would love to build out the infrastructure that will serve that. And the other is AI literacy. And focusing on those two things for the next couple years I think will pay dividends. And I mean that monetarily, but I actually mean it socially in terms of I love the way Mala set up sort of the negative, negative and the positive rights at the beginning. And thinking through that, actually the task that we're working on is fundamentally a bit different. And so how can we think through what AI systems look like there and what are the precursors to having success in those fields? I also want to pick up on something Mala, that you said and I don't know if we have a back and forth here on this, but I'm really curious. When you were talking about global health standards and education standards, it made me think if we look at Anthropic's constitutional AI. For those people who are listening who maybe aren't familiar with this case, Anthropic actually went and did this survey across 1,000Americans. They said it was diverse by age and race and ethnicity and gender and all of those things. But actually then they fine tuned a model on the results from this as well as contributions from. And there are a bunch of details in here which I might be getting slightly off, but they sort of trained it on the universal declaration of Human Rights. And then they also had additional contributions from Anthropic as a company. And then they had these results from these thousand Americans. And what you see across the like 1,000Americans is that there's actually a lot that people agree on, but then there are these, these hot button issues, which goes to politics, which goes completely to different beliefs about the world and what's important. And you see polarization in the responses. But then they actually took this and fine tuned a model to try to make models more helpful while being less harmful. And so it was a way to actually increase the utility of the answers that the model would provide. Because it's often that because of guardrails, models will say, I can't answer that. Which sorting through the right level of content moderation is a whole podcast that we should have because there are a lot, a lot of issues in there. But I would be so curious when I'm talking about let's build out infrastructure. When you were talking Mala, it made me think, are there ways that we could take education standards and fine tune on top of that and say, okay, here is potentially a model that then we could do another layer, but it's built for education with these as sort of north stars within the model itself and do that for global health and thinking through how we can embody some of the standards of the development sector, the humanitarian sector, into things that then could be repurposed. So not everybody has to do that. I'm just thinking, brainstorming out loud here. But like, what does that infrastructure that we could build actually look like that would be really helpful accrue across sectors? I don't know, Mala, if you've got.
- [43:08]
D
Any thoughts or reactions, I'm not sure of that specific example, but if we were to turn it into a hypothetical, let's say that, you know, Anthropic went out, they surveyed these thousand people and they were trying to look for some kind of pattern, like maybe people had some biases towards women or they had some kind of negative thoughts about black people, whatever Protected class. If you were to take what they are proposing and then turn that into a prompt, feed that through the system, and then look back at the responses you get, then you would have to have an adjudication process in place to basically say, is this something we want the model to respond with, or is it perpetuating some kind of bias? And then in order to have an adjudication process, you would have to have some kind of policy or some kind of, you know, standard by which you would evaluate those responses. And so one hypothetical that I was saying, both for ML Commons and what you're talking about is what exactly is the process by which we would say the response is acceptable or safe or not racist or whatever term you want to use? A lot of tech companies, again, have focused on this idea of responsible AI, which is unfortunately almost always just a compliance measure. So we don't want to say something that's against a protected class because that's illegal. But it's not necessarily looking for the positive human rights aspect, which is to say, is it factually accurate and does it meet the standards we need in order to promote a healthy society? And those are two very different things. And so one activity that I think could happen, maybe not globally, but in more concrete terms with a lot of international civil society, is to say, we have been talking about this stuff for 50 years. We literally wrote the standards and what it means to have good public health or good education or good food security. So it's not just a fact of like, we're going to base it in what's compliant with the law in a certain jurisdiction. The adjudication process could quite literally be go and consult with who says about public health. But the problem is that kind of melding has not really happened yet between international civil society and big tech companies, because big tech doesn't have that orientation. So I think one huge opportunity is to create these taxonomies and these adjudication processes and these policies that are based in international civil society standards, human rights norms, everything that we've been working on for so long, because that stuff takes forever to do. And these things are voted on. I mean, a lot of them are voted on through the General Assembly. So we have like a standard, a very clear standard, saying these companies have approved this international standard for public health. Why can't we use that as the basis of our adjudication process in ethical or responsible AI rather than what a tech company says? So that's one example of what you're talking about.
- [45:41]
C
Yeah, I completely agree. And when I talk about responsible AI. I just want to clarify from earlier. I think there is compliance. Check the box. Responsible AI. And I think the development and humanitarian sector can do so much more with responsible AI precisely because of the way you laid out the positive rights. We have a different. The whole way that we're going to approach this is going to be different. And I actually think as a sector, we could really do some really neat thought leadership and produce some really cool stuff that could be really informative beyond just sort of our sector. But it's so interesting, sort of to hear you talk through like, yes, we actually could. We see this. We have the standards, they've been built, they're there. So this is an open question, but how can we really start trying to get at capturing the knowledge, the intelligence, the expertise that exists in our sector that is fundamentally about something a bit different than what big tech has been working on and building that out?
- [46:47]
E
Suzy, Chris, do you want, at this point, put a question in?
- [46:51]
B
No, Susie, I would love to hear your thoughts. I think it's a great segue into humanitarian organizations and exactly what you're saying and what you're seeing within humanitarian organizations themselves.
- [47:02]
E
Just kind of going back to thinking about, you know, if you're a humanitarian or development organization and thinking, okay, what do I do now about artificial intelligence? Or, you know, we're thinking about rolling out AI systems. What are the kind of steps to making sure that's done safely? I think, first of all, it really requires a widening of the conversation. So we probably on this call all think that everybody's talking about AI, but actually that's because it's our thing. Right. And actually, I think humanitarian organizations and development organizations need to have a bit more is a widening out of that conversation. You know, we often see it in silos. So it's the, it's the digital teams or even, you know, the IT teams who are thinking about this. But actually, you know, it's really about getting a multidisciplinary approach to this. You know, making sure that organizations have the right teams in place to be thinking about the impacts of AI. And that's how you kind of build in the right teams to think through, engaging communities to think about the implications of artificial intelligence from a protection perspective. You know, protection staff would be brilliant at this. You know, all the different technical staff that you do have that, you know, humanitarians are used to thinking about how to mitigate risk, how to engage communities, making sure that there aren't unintended consequences of programs. These skills are all Absolutely applicable to artificial intelligence as well as those AI technical skills. So it might be quite useful for people. The research that's published because in it we look at four critical pathways for amplifying, you know, increasing civil society inclusion, you know, the inclusion of communities that we work with. So first of all, absolutely, it is about AI literacy and that cross sectoral knowledge sharing. So that's AI literacy both within communities themselves, you know, from your person on the streets, you know, in towns, villages, you know, through up to government to understand what are the potential risks as well, if I'm using AI, what am I being exposed to? And then very much so, it's AI literacy as well in international organizations and through technology companies as well, because it's that human AI connection. How are my models going to be affecting people, end users or people who are impacted by them? So that's one, that's the first pathway is about literacy. The second, second one I've talked about before is the participation participatory AI piece around increasing local decision making and representation across the AI life cycle. And for humanitarian NGOs. Right. We've had a whole conversation for years now about decolonizing aid. Well, if we really want to put our money where our mouth is, and that being the proverbial, because not enough money is going to local organizations, we need to make sure that we're also looking at decolonizing AI. And if we're sitting in meetings around meaningful localization of aid, and then we're just buying AI models off the shelf and not including communities in any of those discussions to understand whether they're A effective or B ethical and potentially harmful, then we're not living up to the principles that we're espousing. And then in terms of the third one would be around strengthening advocacy on the contextualized impacts of AI and what are the desired outcomes we actually want. And again, international aid agencies have a responsibility here to amplify community and Global south voices around this and also support civil society organizations can realize their potential for adding to this conversation. And then lastly there's the whole bit about improved digital infrastructure and equitable data governance. So how much in terms of data privacy, things like that, unless as NGOs deploying these models, unless we've got our data houses in order, do we know that we are protecting people's data? Are we kind of applying data sovereignty principles that really allow people to be in control of their data? What does informed consent look like when it comes to biometrics, to using chatbots? All of these Things. So lots of specific recommendations in the research on what organizations can do next and for different stakeholders as well, for tech companies and donor governments as well.
- [51:53]
B
It's amazing. I want to use an F1 reference. I don't know if that will resonate with everyone, but Lewis Hamilton just left Mercedes, which has been a watershed moment for us, watching him be such an amazing athlete for so many years. But it's kind of like if you wanted to start racing, you got a bicycle and then you got a little go kart that you pedaled, and from the go kart that you pedaled, you got one that had an old lawnmower engine on the back, and you kind of did that in your garage. And what we're talking about is predominantly we've been kind of doing that working in our garage as organizations, and that's what we're comfortable with because we tend to be generalists in our work, because we need to respond in many different ways. But what we're talking about today is really running an F1 team. I mean, it's a completely separate thing. It still has four wheels, but otherwise there's really nothing that's similar to the way that we've been doing work for the last 70 years to. To what we're talking about today, right? From the infrastructure, from the legalities, from the data, from the impact, the scalability, all these things. It's completely different. And so I really want to thank you all for joining us today, because I think that you raise the important questions and important points that people need to consider as we move forward. And again, none of you have said, put on the brakes, but none of you have said, just go forth and it shall pass, and we shall do this and it shall be great. Right? So I think that there's got to be a lot more informed decision making across the sector on how to use this. And I hope that when people listen to this podcast, they start to get that picture that it is around informed decision making. And that's not at the C Suite, right? That has to be from the ground up. And that comes to the reality of information. You know, Emily, that you brought up. I want to kind of close this out. It's been an amazing hour with all three of you, and thank you all. But I want to kind of give you one last space of a couple sentences on what is your big ask when the head of this organization comes to you and says, mala, today we're going to implement AI not internally, but beneficiary facing. What's your big ask? Of us to do to get started. So, Mala, over to you. What's your big ask?
- [54:06]
D
Don't do it until you're ready. Simply enough.
- [54:09]
E
It.
- [54:09]
D
Yeah, I love it.
- [54:10]
B
I love it.
- [54:11]
C
Emily, slow down. I always say sort of default AI, if you just do a build that is sort of status quo, that is not going to help achieve the sustainable development goals, that is not going to lessen inequality. But if you do responsible, inclusive AI that is well funded and carefully thought out and you're actually providing in all this time for testing and working through things, you might be able to build something really neat. So default AI won't get us there, but responsible, inclusive AI, I believe can.
- [54:53]
B
Spectacular. Last but not least, Suzy, the final word, your big ask, I wouldn't say.
- [54:58]
E
Necessarily put on the brakes, but do slow down massively unless you already have your kind of ethical and governance frameworks in place and crucially know how, have thought through how to operationalize them. So I think it's about, you know, don't be scared, but do realize and throughout the organization that AI is a societal game changer, you know, in the societies that we work and definitely also within humanitarian operations. And also don't be scared to say, hey, we don't know. Let's have some conversations and let's ask for some advice and some help about that. And just to remember we are thinking and talking so much about decolonization of aid. AI cannot be left out of that conversation.
- [55:47]
B
Absolutely. Well, again, I want to thank you all for joining us. I'm going to end with an analogy because it came to me as you guys were all giving your big asks, which is you don't ask your child to get into the ocean and swim in the waves unless they know how to swim. That's not a good place to learn how to swim as a kid, right? You want to kind of start in the pool where the water's calm, make sure that you understand what this feels like, what does it feel like to float, go underwater and all that stuff. You don't just throw them into the ocean with the waves crashing. So I think it's really important what you've all brought to this discussion, and I think it's going to be a really meaningful component of this podcast, I think. Molly, you said it. We could spend one hour on one question around this thing, right? I mean, there could be 10 podcasts around just responsible AI and how to do it. So I really appreciate you guys trying to condense it and to make it as simplified as possible because the goal is for this to be a really good conversation that many, many people within this sector can listen to, that aren't technologists, aren't the data scientists, etc. So thank you all and really appreciate your time and can't wait to hear what the final cut looks like because it's going to be really special.
- [56:51]
A
Thank you for joining us on humanitarian frontiers in AI. We hope today's conversation gave you new insights into how AI is transforming humanitarian efforts and the steps we need to take to ensure it's done ethically and effectively. If you enjoyed this episode, be sure to subscribe and stay tuned for more discussions with leaders and innovators at the intersection of technology and humanitarian work. Together, we're exploring how AI can bring real change to communities in need. Keep pushing the frontier of possibility.