A (5:33)

Okay, so you're, you're not busy at all?

B (5:36)

Yeah. Really not lots of free time? We did just get a puppy, so certainly trying to get more time with her, but yeah, staying busy.

A (5:46)

Oh my gosh. Wow. Yeah, that's a lot. And this feels like such a Great segue on this show. We've talked about AI a bunch, both with Jonathan as well as our VP of research in AI and broadly tracking this, like, what does AI for good look like? But I don't think we've really talked about responsible AI and like what that means. So I'd actually love to. Maybe this is like a dumb starting question, but for my sake and for our listeners, like, what is responsible AI and yeah, how do you define that? And how does that definition evolve when you're thinking about folks in the global self versus Silicon Valley?

B (6:22)

Yeah, absolutely. So there isn't one definition of responsible AI, but I would say that one that many have coalesced around would be designing, developing, deploying, and I add using AI systems in ways that are safe, fair and trustworthy. Now, of course, what does each of those words mean, like fairness and trustworthy? Those are words that can also bring debate around what fairness means or looks like and what exactly is trustworthy. Within that, I'd say there's five key areas that are important to think about. Those are fairness and bias. So how might this tool perform differently for different populations? What are the potential stereotypes or discrimination or inequality that these tools might replicate? So there's two pieces there. It's like performance issues. So, you know, maybe having higher inaccuracies for some groups are not working as well. We sit in Global South a lot with in terms of large language models not working as well for certain language varieties. But then you also see issues in terms of these tools perpetuating, say, gender biases around who should be caretakers and who should be a CEO or something. We're doing some research now around gender rise and text image models. That is from a global perspective. So we're looking to hoping to have more information and some compelling statistics that can help us better understand this from a global perspective. So anyways, that's one. The second issue is around data privacy questions and considerations around what are the privacy guardrails that are in place, the data security guardrails that are in place, consent mechanisms for data. The third is around security and safety. So these are considerations like how might bad actors try to break into this tool or how might they use it in ways that could result in harm. So in the context of generative AI, we see like concerns around, from a safety perspective in terms of leveraging tools for say, deep fakes or misinformation. But from a organizational standpoint, there could be cybersecurity issues, bad actors breaking into the model, and perhaps having some sort of like Prompt injections to have models output different things. So that's security and safety. Transparency is the other kind of category, responsibility. So the that's around, you know, do people understand how these models are leveraging their data to arrive at certain outputs? And also for model developers, are they being transparent around their training data, different components that went into how the model was being developed? So there's transparency to the user. There's also transparency for the developers themselves and about the models. And there's some underlying risks there, especially as more and more organizations, including in the development community, are leveraging foundation models and happy to. I don't know if the, if it would be beneficial to go into like the definitions of these different things, but as more and more organizations and development actors are leveraging foundation models like GPT5 or leveraging models from Deep Seq. DeepSeek's a little different because it's more open, but say GPT5 is a closed model. You're adopting this lack of transparency essentially in whatever you're building, which means there's not clarity around some of the risks that could be in place given the underlying training data or other components around how the model is built. And even Deep Seek is more of an open model, not an open source model. So there's differences there. Anyways, happy to delve more into any of these topics. And then accountability. So you know, what are the accountability structures that are in place if something goes wrong or to make sure that responsibility is occurring in the organization and the ways the tools are being developed and deployed? So those are kind of the five key areas that underpin responsible AI, fairness, slash, bias, privacy, security, slash, safety, accountability and transparency. You can think of those as like the responsibility lenses and they become important for any AI application. Right. And especially as we think about how AI is being deployed in the development sector, oftentimes for vulnerable communities, the purpose of using a tool for good is not sufficient because it doesn't actually mean that any of these other things are solved. For in fact, these are fundamental issues that exist in these technologies today. And it's really important to be eyes wide open around the different ways that those can manifest for communities globally if we want to get that for good benefit in ways that are ultimately what we're trying to achieve as development practitioners. I'll also add that there's more broad societal concerns in the responsibility umbrella. Those include things like future of work implications and environmental considerations. Future of work. Especially now we're in this period of AI agents and as we have more AI tools that are reducing work or making more productivity and efficiency gains, or agents that are taking on different types of tasks. What are the future of work considerations, including in communities and low and middle income countries, that. And how might that impact economic safety and security of different communities? And then the environmental considerations is. Currently we're in this period with foundation models where there's this idea, there's this scaling method that is being pursued, which is essentially more data, more compute leads to more capabilities. And that has had some really early success in that having more data, having more compute has really led to immense growth in capabilities. We are starting to see that curve a little bit. Instead of going straight up, it's kind of like curving off a little bit. But that scaling approach is still one that is being very aggressively pursued. And with that comes environmental considerations as we have more data centers with more energy requirements that are working to meet the growing demands of this scaling model for these foundation models? And what are the communities that are going to be suffering most from some of these environmental considerations? And data centers are projected to be the fourth largest consumer of energy by 2030. So this is not a small thing that's occurring and it has a lot of geopolitical might behind it because we're also in this period of time and you know, happy to delve into this in the podcast as well, but we're also in this period of time where there is these geopolitical tensions and struggles that are occurring in this race to win the AI war, so to speak, which is really something that's happening between China and the U.S. but other countries are coming into this as well, this sort of race to artificial general intelligence. And so there's this kind of full throttle approach that is certainly happening when it comes to foundation models that have some implications from the environmental perspective. So sorry to like drone on a lot there, but I think that's just a little preamble for some responsibility, for understanding what responsibility is, why it's important, and some of the landscape.

C (13:33)

Yeah, no, that's great. And a wonderful overview of those five attributes of responsible AI. One point that you closed on though, you mentioned this massive race that has political implications, national security implications, commercial implications. And a lot of the trends we see obviously are like, race as fast as you can. Like, we just got to win, we got to throw more compute at it, more energy, more everything. A lot, as you said of global health and development work is adopting frontier models or deep Seq or some of these, but kind of riding on the foundational growth and technology curve of the AI. Industry as a whole. And so if you're product manager or program manager, thinking about your specific use case of really trying to make a meaningful impact at a national level, at a community level, how do you think about how program managers or product managers can adopt those five principles with respect to responsible AI into their work in kind of a practical use case in a sense of there's the AI is going to save everything or kill everything. There's kind of these macro questions that are significant in scope and implication, but maybe not as much practical day to day of what a product manager should do tomorrow. And so I'm curious, how have people taken that broader framework that you have and applied it at a more tactical or practical level in their work so far?

B (14:56)

Yeah, I love that question. And as I mentioned at the beginning of the call, I have this dual hat, right? I sit in this responsibility initiative at the Berkeley Research Lab and at the business school. So I think a lot about how corporate governance approaches and product teams and product managers can operationalize responsibility. It's not just a technical solution. This isn't a solve these five issues in the more broad future of work and environmental considerations are really ones that trickle down from corporate governance and that trickle down from broader priorities that exist at the institution. Organizational levels and product managers have a really important role to play. We've done research around how product team our research has specifically looked at how product managers use generative AI and what they think of in terms of responsibility and how they operationalize it or not, and what are the tensions or challenges to operationalizing it and what are the kind of wins at their back that support operationalizing it. So we did survey, a large scale survey with product managers and we also did interviews with product managers and product teams and actually from that created a playbook for product managers specifically around how do you responsibly use generative AI tools? And these are good strategies I think for any folks who are working on these types of models to consider and product teams to consider. And I'll be happy to share that link in the chat. It can also be found through our Responsibly Initiative website at UC Berkeley. But essentially for product managers, choosing a model can be really important to assess, like what are the needs and potential risks that exist within that model. And doing some risk assessments and audits for generative AI products, including with cross functional teams, can be really important if you're using again for generative tools doing like red teaming and adversarial testing, which basically put yourself in the position of a bad guy to try to poke holes. And so those are a couple of strategies to say at the product level. But it's also really important that organizational leaders prioritize responsibility because they're ultimately the ones that set the priorities of the organization. And if leadership isn't bought into those five issues that I mentioned, and responsibly, more broadly being important for a product or a use case, then it won't be necessarily incentivized and it won't necessarily therefore be prioritized. So it's really important for leadership to set the tone by having principles around this and working on incentive structures that support responsibility, like having these questions come up and say performance reviews or using them and objectives and key results like okrs. And I also want to say kind of at a higher level, we are in this period of time where we see global health and development practitioners more broadly really developing or adopting different foundation models, fine tuning them, incorporating them in different contexts, and they can certainly provide value. But I think it's important that as development practitioners we don't think about like, you know, this hammer for every nail, nail for every hammer ever, that saying goes. But I think there's this and I would actually be curious for your thoughts as well around how you see this. But I think we're in a bit of this time period where now we're USAID no longer exists. And I forgot to mention in my introduction, I am also used to be the responsible AI fellow at usaid. So helping think about what are some considerations and how can we support this development practitioners incorporate responsibility into their for good applications. But anyways, what I was saying is that as some of these different funders are gone now, as the development landscape itself is shifting this hype around developing and using AI that exists more broadly in society in Silicon Valley and the Bay Area where I'm based is also transporting globally. And I think we need to be really mindful that we're not really just wanting to use AI because we can use AI, but we're actually using AI because it's the best way to solve the particular problem that we're trying to address. And so asking ourselves is this the best way to solve this particular problem? Is the first question. And that should be like the first starting and stopping point. And I know it's tricky because we're also in this time period where funders are supporting AI initiatives. We're looking for funding and development practitioners. You're wanting to think about where can I deploy AI? So it can be a bit challenging. But yeah, I Don't know. I'm curious actually, for your thoughts, like, how are you seeing that play out? And yeah, any considerations that you have.

C (19:43)

Great question. And obviously the amount of hype and energy going into AI, which is both good and bad, is causing a ton of use cases to be how do you apply AI to this? Back when mhealth or digital Health or ICT was the buzzword, a lot of times the answer was like, don't like adding. Trying to digitize this or turn into software is not the problem, not what you should be focused on. And so one observation that we have a lot is just don't do the project. Don't ask yourself how responsible to do the project. Don't do it at all because it's not going to add enough value. Even if it works or even if it does work perfectly, one of the things we see a lot is, let's say it's perfectly responsible, perfectly performant. There's no feedback loop that will cause that to be sustainable. Right. Like the industry is not able to afford to pay for that thing to go better. So even if you do a great job of using AI in it, are you really taking cost out of the system? Are you driving up quality in an area where people will actually use it? And so that was also a complaint back in the digital health days of we do all these pilots and then it was a pilot of a project that nobody was going to scale, no matter how good the result was. And we see a lot of that happening with AI where you're like, even if that is perfect accuracy, perfect everything, nobody's going to scale it up. Right? But in terms of responsible use, I do see a lot of people concerned in terms of how these will work, how we make sure communities that could most benefit from these tools are able to benefit them, but also acknowledging and recognizing the big disparities that could be further exacerbated if you don't do this correctly. I think one of the really exciting areas is a lot of that kind of middle layer of HR capacity, of supervising employees, of training up and coaching. There could be a lot of really powerful use cases where AI, I think, would do quite a good job for some of the people. And so the question is, if you have technology rolling out where you're like, yeah, half the people come in this way, half can't. For the half that can't, I'm going to do nothing. And for the half that can, they can get a huge bump. That's problematic from an equity standpoint and from a workforce Management standpoint. So I think there's a lot of questions on that. But I do think we've seen extreme amounts of value for subsets of the user base. And the problem is, now what? So if it works for 25% of your user base really well to do upskilling or coaching, but 75% don't like text, or 75% could do audio, but not in the audio that's available with any quality. So I think that's a big area that we're seeing in particular these less utilized languages that are critically important in almost every LMIC context because English is almost never the right language to be speaking in. That's a big area of research for us is saying, okay, let's assume that the frontier model or the foundational model is perfect in coaching CHWS community health workers in English. Now what? Because we know that's insufficient for almost every community health worker deployment that we support. And how do you do the extra work to make sure it is viable in local languages? And it's definitely going to be the case that it's not viable in all of them. So now what? What do you do for the community health workers who really want to access that tool? Maybe do have the digital sophistication to be like, yeah, I could type into the chatbot or things, but you're like, this is just bad at my language. So now I don't get to learn, even though this thing is totally viable for my colleagues who speak a more popular language or more commonly spoken language. So I think that's. These are all key questions that I don't think we have a lot of answers to. But I think there's tons of excitement. We're seeing huge amounts of value in these use cases in English. And then the problem is, okay, well, great, it works in English. How are you going to make it work at scale in a target country?

B (23:43)

You bring up a lot of really good points there. And maybe just some thoughts back going back to what we were saying in the beginning around what we saw in the kind of digital health days and you know, how tools were being deployed and there were some areas of refusal that food was designed for. I originally, as I mentioned before, is working with the UN foundation with the Clean Cooking alliance on their gender strategy. And it reminded me a bit around the clean cooking back in the 1970s and they were first starting to implement cleaner cooking technologies in low and middle income countries. And a lot of times these were tools that were kind of helicoptered in from labs in the west without really being adapted to the needs of the cooks themselves, the communities themselves. And they didn't really get used because they weren't really adapted to those local communities or they. There was a lot of waste and there was a lot of, you know, kind of you should use this, but not really meeting the needs of people. And I think the clean cooking space has kind of a long way, you know, the development space come a long way in terms of helicoptering and with like, you need this better understanding, how do we meet the, how do we. What are the needs of different communities and how are we making sure that we're responding to those and helping them develop or adapt these tools in ways that really serve them? I'm going to come back to that. But that was just like one thing that you were mentioning I was starting to think about. And certainly the point around the disparities potentially being exacerbated I think is a really important one. And that's something that the development space should be really aware of because as you said, there's a lot of positive benefits that these tools can have. And certainly I don't think we need to completely shun AI technology. I think sometimes this idea that like responsibility means refusal at all times and is kind of like anti innovation and anti, like use these, use of these. I don't think that's true at all. I think responsibility when it comes to these tools is how can we go in eyes wide open around what are the benefits and also limitations and risks of these tools and then how can we move from there to make decisions that ultimately support the missions that we're trying to achieve without unintended consequences that we know has riddled the development sector in the past. You mentioned disparities when it comes to English language. We did some research around linguistic discrimination in ChatGPT and the underlying models actually looking at different varieties of English. So we looked at Kenyan English, Indian English, Nigerian English with 10 different varieties globally because there's a presumed excellence in English, but then there's also disparities in the type of English. So basically what we found that research is that models tend to default to Standard American English. So outputs tend, whatever kind of English you put in, they tend to default towards specifically standard American English, even when you put in, say, British English, interestingly. And they also tend to not perform as well for varieties outside of the kind of quote unquote, standard varieties of Standard American and standard British English. And they have higher stereotyping and condescension for those other varieties as well. So like some Interesting. I think disparities, even in some of those places that we think they're a little bit better. In the research that I've done on machine learning based credit assessment tools, I find that they do enhance access to finance overall, but not in ways that are gender equitable, ultimately, in ways that kind of reinforce gender differences and access to finance. So more men get loans at higher amounts, even though women are deemed as better repairs. And that's linked to some of the proxies and features that are used to determine creditworthiness that are gendered. Things like income stability, formal employment. But it's also tied to how these algorithms can be optimized towards profit and larger loans and more late payments can be more profitable. So anyways, I think it's important that we think about even inclusion can actually come at the cost of equity sometimes. And we shouldn't also think that inclusion is sufficient when it comes to some of the goals that we're trying to achieve as a development community. So anyways, those are just some points, but I love the community health worker application. I think that is a really like, amazing application of these types of technologies and tools. And yet how do we make sure it's available in different languages is a really good question. And there's some really cool innovations happening here. We've just been starting some work around, like participatory data governance and data cooperatives. And so how do we think about collecting data, whether in different language varieties or different image data or whatever, in ways that's not extractive, that we can then use to make these models work for more people? And so we're doing some work now around kind of data cooperatives and participatory data governance, which I think there's a lot of potential in the development space as we work to make these tools work for more people.

A (28:38)

Genevieve, there's so much in here that I want to like, double click on, but to your kind of original question to us, which was like, how are we thinking about this? It's so timely, right? And I'm sure that every team in any organization right now is just like grappling with, with AI. And I think one of the things that I've been just mulling over really is from like a team perspective, like even just like how we use AI in our work. Like, I think there's a few different layers here, right? There's like, how do we say on our products, which maybe it's like a higher bar, but even just how do we use AI in our work, right? Like, how do we encourage experimentation and people to, like, test it out to see if it is a good fit or not, rather than just feeling, like, pressure to use AI everywhere? Right. And like, I've actually just. Just last week I was with the team that I work on, and we were kind of working through, like, what are some guidelines or guardrails we can give our team to better understand our expectations as leaders of, like, what do we want to see with AI use? Right. And it's so tricky because I'm like, I don't want to discourage it. Like, I actually want to be very encouraging of experimentation. There's so many things that I'm worried about happening Right. When, like, people use AI too heavily and there's like, so many layers of that. Right. So it feels like very, very timely that we're having this conversation.

B (29:47)

Yeah. And maybe just to speak to that quickly. I think first off, that's great that you're thinking about it from a leadership perspective. And what are the kind of principles or approaches that you want employees to be thinking about when they're leveraging and using AI? It's really important for leadership to kind of set the tone on that, though. We found that again and again through our research is that ultimately leadership sets the tone around responsibility and around these questions related to use and what are the values that the organization wants employees to adopt when it comes to developing or deploying these tools can be really important. Setting those principles up, it can be really important to help navigate those gray areas. And then what are the incentive structures that exist around it? Just as an example here, Meta recently changed its performance reviews have a question around how AI is being used in the workplace. So by doing that, signaling the importance of leveraging AI within organizations. But imagine if there was another question, which is, how are you now using it in consideration to support for responsibility or trustworthiness, Adding that extra piece then incentivizes, helps people think about it triggers this kind of different mental approach, mental model that can be really valuable in organization. So I think that's just one piece I would mention and I would say too, in my classroom, I encourage students to use AI. I think it's important that they experiment with it, that they're using it already. Like, whether I say that they can use it or can't use it, the reality is that they are using it. And it's good for different things. So it can be really helpful for summarizing things. It can be really helpful for editing and for writing purposes. But it's like a gentle balance between Wanting students to use it and experiment with it, knowing also that employers want them to use it. So I certainly don't want to prohibit it by any means. In fact, I want them to experiment with it so they can think about where are those responsibilities, tensions, what am I uncertain about? And so I require my students, whenever they're using AI tools, to practice responsibility by being transparent about the particular use case and like why and how they're using it there anyways, I think there's ways from an organizational perspective, how do you have transparency around it and don't make it like a shame thing, because these tools are really powerful and are great for a variety of different tasks. But how do you make it more of a. Maybe you have monthly lunch and learns where you chat about different applications that you use in the workplace. And what were some things you weren't sure about? What were some things you're maybe nervous to share? Like you put into the model some aspect around yourself your asking for, like health considerations for. And, and you know, how do you create a safe space for people to share some of the ways they're using the tool without feeling like their wrist is being slapped? Where everyone can learn together I think is really important.

A (32:40)

Some of us may be like a little bit further along, but we're all learning AI, right? Because it's new technology, right. So like normalizing the idea that we're all in this journey together and making sure people feel safe to both share what they're learning, what's going well, what's not going well. Right. And not like you don't want to create shame around your use or non use of it. I am curious actually this is was not in my list of questions, but I am curious, like in your own work, like how are you using AI and like where do you see it adding value for you?

B (33:06)

Yeah, for sure. So I would say coding definitely. So I'm a social scientist, but in a computational social scientist, so it's really helpful with coding and helping when I have puzzles there. I use it for editing sometimes too. And I also use it sometimes like a thinking partner. I think it can help me think through things that I'm stuck with. So I really like that. I will say that I am starting to use more and more local models, meaning models that are local to my machines, like open ones that I kind of download and use. Part of that is because of data privacy questions and considerations. You know, even if you're using a model that's going back to like a server, like a closed model, like ChatGPT goes back to OpenAI servers. Even if you've checked the box where your data isn't being used for training data purposes, we don't really know how it might be used for secondary purposes and how long that's being stored for, if it could be used on the line for like advertising things or stuff like that. I just had a talk in my class with a data privacy expert, expert Jennifer King, who's at Stanford. And she was noting how we've reached what's called peak data, which basically means that these big tech companies we have, they've gotten any and all data that is currently available. And so now the question is how do we get more data? And she's done some research into privacy policies of these organizations developing foundation models, and they're very vague essentially. So I think we need to be mindful as users around some of that. So I think about the data privacy stuff a lot of and in terms of like how I'm using it and the types of models I'm using it, those are things that I think about.

A (34:48)

I'm sure that listeners will be curious. There are specific open models that you're using locally that, that you recommend depends.

B (34:54)

On your computer and the bandwidth that your computer has. But Ollama is a pretty good one. There's some deep seq ones that can be pretty good too. And you know, just being mindful of like with those as well, like Deep Seek, there's certain things that you know, you know that if you ask it about like Tiananmen Square or certain things like that, you know, there's limitations, there's like, can be cultural things are embedded in these tools. So I think it's important to be aware of that. But I would say like Ollama Deep Sea are two great ones and you can explore what is possible based on what your computer can do. Basically, yeah.

A (35:26)

Awesome. Well, I want to pick up, you've started to touch a little bit about under this idea of like when tech gets kind of like helicoptered in or parachuted into a scenario, there can be challenges, right? Like we're just like bringing it in without kind of consideration to local context or culture. I would be curious just to hear a little bit more from you around. Have you seen instances where AI has failed or struggled because of that lack of understanding and how should we be thinking about that piece?

B (35:55)

Yeah, certainly. And so I think again, you know, it's like with discriminate. So there's discriminative AI systems or algorithmic decision making systems that are More prediction tools. So that would be like predicting who should get a loan or what should be like your lending limit or who should get hired for a job. So those are predicting decisions or informing decisions. And then generative AI systems that are generating text, image, creating something new. And so I think within both of those there can be instances where the AI has failed or struggled. And again, just like looking my research around financial systems, so those that are assessing credit worthiness and predicting who essentially repays loans, you know, I think some of those, I wouldn't say that the AI has necessarily failed or struggled, but I think they're essentially like predicting credit worthiness based on different features and proxies that are developed in these systems. And it's really important to think about how those are either are those correlated or are they causational to repaying loans. And so it's really important to have that lens around like correlation versus causation and thinking about some of these different ways that like data and features can be gendered. And so like being mindful of that. So in the research that I've done, I did a case study in Kenya which basically found that women received less loans and lower loan amounts and that rural women were over five times more likely to not get a loan even after controlling for other variables. And so this is kind of a question around like why is that occurring? It's really hard to kind of like dig into the meat of some of this, but it's likely tied to the proxies that are being used for assessing creditworthiness. And then also how algorithms are being optimized for lifetime value and profit motivations and how fairness is understood can actually reinforce some of those gender differences. So basically like there's a lot of normative things that go into AI models and decision making that I think is really important for us to be aware of. And it's not necessarily illegal. But I think this case kind of goes to show that businesses self regulating can make certain decisions that are in the interest of for profit firms. And again, it's not necessarily bad, but the question is, is society, is society on board for that? And so I think that's really important. And then yeah, I would say with other like applications like generative systems tend to have, especially with low and middle income countries, they have pretty big biases around different communities. And so it's less of like failing or struggling and more around like how do we want, what do we want these models ultimately to achieve for us and how do we get that there? And then who Gets to decide, you know, a lot of the work that we're starting to do now is more around participatory design, research and co creation. So how do we develop AI systems with or by communities in ways that are really meaningful for them and support their agency?

C (38:47)

So one thing you mentioned was the that loan example for the women in rural areas being five times less likely. I'm curious if you're seeing what might be perceived as like a false trade off right now with some responsibility, priorities might directly compete with profit. But I think one thing we saw with the energy sector that certainly, I think if everybody could go back in time would have made it less about perhaps climate or other issues and been like, this is a national security priority, that we get energy cheaper for all sorts of reasons. And oh, by the way, it's also good for climate. And so from a responsibility standpoint, I'm curious, in that case of the loan, was it a bad economic decision in addition to being a bad outcome for the world that those women are less likely? But in that case I'd be like, oh, that's, I bet they repay at a higher rate. And I bet you want to service that loan if you could get it. And so I'm curious if you're seeing this like it feels like you're trying to say trade profit for be more responsible, when in fact there's probably a lot of use cases where it's like, no, no, no, this is a better economic outcome as well.

B (39:53)

Yeah, absolutely. I mean, I think in this case with women not getting as many loans as men, but their repayment behavior being higher, it shows like maybe there's this market inefficiency that's happening and there's a variety of reasons that women could be not getting as many loans. That there's user side and supplier side.

A (40:10)

Right.

B (40:10)

Like user side, there's gender digital divides. So women don't have as many access to the apps. The digital literacy and financial literacy can be different. They may be not able to take advantage as much of these tools and those kind of things could be addressed. Right. We can think about how these tools could exist on basic phones. We could think about financial literacy and digital literacy tips that could help take them through the process to get more access to these. And then on the supplier side, what are the proxies and features that are maybe correlated with creditworthiness or some aspect of creditworthiness that are not actually causational? And how do we like, better understand that? Because yeah, there is this, you know, market inefficiency and potential discriminatory effect that's occurring. That's also holding back the potential of how many folks we can reach with these tools. So I think, certainly think that's one of it, one aspect of it. The other piece is if we're using like a profit lens from like an optimization or evaluating if something's fair, then it could reinforce this idea that like certain groups are better to lend to and then the tool like learn that over time. And if we don't really do the proper auditing internally, we don't recognize that that could be happening. So I think doing those internal audits and checking against different demographic groups is really important to hold ourselves accountable. And yeah, there's also this market inefficiency that could be happening too.

A (41:31)

I'm curious, I know you've mentioned now twice this idea of kind of developing AI within four communities. You mentioned a participatory data governance, like creating cooperatives to do that. I'm so curious. Like that just piqued my interest. Can you share a little bit about that work and what promise that holds and.

B (41:48)

Yeah, absolutely, yeah, I'm really excited about it. And I think at a, at a high level like AI is a tool, right? It's not inherently good or bad, it's a tool. It's a technology that can be employed in different ways and it's a really powerful, powerful technology. But it really matters who develops these systems, what are the incentives, priorities, perspectives, beliefs that are kind of getting built into these technologies. And because of that there is a lot of movement, I would say, especially in the ethical AI community and researchers in this space that are thinking about how do we build, develop, deploy, use tools that center the needs and agency and perspectives and priorities of community members, especially vulnerable communities. And there's a line of research is a big ethical AI conference called the Fairness Accountability Transparency Conference, ACM Conference on Fairness Accountability, Transparency or fact. And this is a huge topic there. And there's questions around participatory design research and human centered AI, which is essentially how do we make sure that we're understanding the needs and priorities of the people that we're targeting with these systems. And participatory design research is a bit of a spectrum. So like you can move towards more deeper participatory design, actually more co creation type research, which is then these systems being developed with, for and by these communities, not simply doing like a survey and then integrating that, that'd be like on the other side of the spectrum, which is great. But there's levels of deepness that you can go, I would say. And this can be applied in the machine learning life cycle. So when it comes to data sets, we can think about participatory design and participatory governance of data sets. And that's where this cooperative project is coming into play. And so for here in this particular project we've been doing research around gender bias and text to image models globally and recognizing that there is pretty big biases that can exist for different communities. And so we're creating a new data set of people doing different things globally that may be non stereotypical for their gender. Like men taking care of kids in different places globally and women doing like mechanic work, things that people really do. But these models don't represent people doing necessarily because they tend to actually exacerbate biases. And we find that through our research, it's not just training data, but the model itself can exacerbate the biases based on how they're learning Anyways. So we're creating this new data set that will be a cooperative structure. So that essentially means that the people who contribute their data will have governance rights over how it's used over time and be able to also have payments go to them. If it is approved for use in different cases, then they can receive payment for that over time. So we're really still exploring like how do we do this, how do we create like a global data cooperative structure? And then yeah, we're also thinking about and within like machine learning tools as well. How do you develop those in ways that you know, and center community agency and needs. So things like defining the purpose of the model with communities thinking about like proxies and features that are resonating with the community, defining fairness with community members as well. Anyway, so there's some different exciting applications there and I'm really excited to be doing more work in that space again because I don't think responsibility isn't just about finding the issues and being like this is a terrible tool in technology. It's also about imagining technology futures that we want to live in and how do we ensure that they represent people's needs and perspectives and priorities and bring out community voice.

A (45:38)

I love that example, Genevieve, because it's not just like saying like stop, don't do this. Right? It's actually this very thoughtful approach to like how do we bring this forward in a way that's going to make sense and help make sure that AI is representing things and people in a more realistic way, in a more fair way? That's yeah. What a beautiful example. Thank you for sharing that. One final question for you. We've talked about a lot. You've gone through just a really incredible framework you've outlined in terms of what is responsible AI. If you could encourage our listeners to do one thing differently on Monday morning, what would it be to make sure that like they're embracing AI responsibly?

B (46:18)

Yeah, I think just, you know, being aware of the risks that exist and kind of eyes wide open around that. I also say that perhaps it depends on, you know, as a listener, are you a funder, you're a policy maker or you're a practitioner? As a funder, I think the one thing you might want to consider is come next week or in the future is do we have a responsibility lens when we're reviewing tools to invest in applications? We did this at usaid, as I mentioned, as a responsible AI fellow there, we looked at a responsibility lens and grant making decisions and then provided TA support to grantees to help them build that out. Right. And also for funders, think about funding processes, not just products. So how do you support funding things like participatory design methods and that might result in like a different end product, but are really prioritizing the process of censoring community, agency and voice and how that product lives in the world. And so I would say like those two things would be one thing for funders, for policymakers, recognizing that these tools aren't objective and that leaving businesses to self regulate is flawed because they're businesses with responsibility to their shareholders, they are responsible to their shareholders and those responsibilities can be in conflict with equitable benefits to vulnerable communities. So be mindful of that. And then for practitioners, how do we think about building imprisonment design processes? How can we use those five responsibility components as a lens to make products that better serve our communities and have a responsible AI strategy, including through things like referencing that playbook that I mentioned can be really important or really helpful as well. So hopefully it's a tool that can empower the listeners out there to think about how to operationalize this in meaningful ways.

A (48:09)

I love that. And we'll include links to those in the show notes as well. Thank you so much, Genevieve. This has given me so much to think about. I'm really excited to share this out with our audience and even with Demagi internally, I think super, super valuable. So thank you so much for your time.

C (48:24)

Yeah, thanks so much for joining us, Genevieve.

B (48:26)

Of course, yes, thank you. Thanks for having me. I really appreciate the work that Dimaji is doing and excited to stay in touch.

A (48:34)

Thank you so much to Genevieve Smith. That was a fantastic conversation and gave us a really clear framework to navigate the intense hype around AI. Really grateful for her time and her expertise. For our listeners, I want to share share with you a few of my key takeaways from this conversation. First, remember those five key lenses of responsible AI fairness, bias, data privacy, security, safety, transparency, and accountability. Use this framework to audit your AI projects, moving beyond a simple AI for good label. Second, process over product. Funders and practitioners should prioritize funding and building things like participatory design methods and data cooperative. This centers community agency and ensures the models aren't helicoptered in without local adaptation. Third, inclusion is not sufficient for equity. Be mindful that AI systems can enhance inclusion, such as granting more access to finance, but they might still reinforce existing gender or social differences. The purpose of using an AI tool for good is not sufficient as fundamental issues like bias and unfairness still exist in the technology. Finally, leadership sets the tone. Tone leaders must set clear written principles and create incentive structures like integrating AI responsibility into performance reviews to encourage safe experimentation and learning across an organization. Check out the playbook that Genevieve mentioned and we'll include a link in the show Notes. That's our show. Please like rate review, subscribe and share this episode if you found it useful. It really helps us grow our impact and write to us@podcastemagi.com with any ideas, comments, your feedback this show is executive produced by myself. Prathana Balachander is our editor, Natalia Glowaki is our producer and cover art is by Suran Shukant.