← High-Impact Growth
Loading summary
Transcript33 lines
- [00:00]
A
Welcome to High Impact Growth, a podcast from Damagi. For people committed to a world where everyone has the services they need to thrive. We bring you candid conversations with leaders across global health and development about raising the bar and what's possible with technology and human creativity. I'm Indy Vaccaro, Senior Director of Marketing at Dimaghi and your co host, along with Jonathan Jackson, DiMangi, CEO and co founder. Today we bring you part three in our ongoing series about artificial intelligence, what it means for global health and development. I'm recently back from maternity leave and the team's been racing full speed ahead on AI development since our last conversation at Dimagi. We are keenly aware that AI development will naturally skew towards use cases where there's an ability to pay and it won't naturally support the use cases that we care most about. We see our role as influencing the development of AI, such that it can be used to improve equity and not worsen it. Today, Jonathan and I sit down with Brian Direnzi, demaggi's Head of Research and data, who's leading the charge on a lot of our AI efforts. In the conversation, we'll discuss direct to client use cases for AI, ways that AI can support and enable health workers, and a new technology platform we're building to enable a more inclusive ecosystem for AI development called Open Chat Studio. Enjoy. Welcome to the podcast, guys. So I'm here with Jonathan Jackson, my co host. Hey, John.
- [01:32]
B
Hey, Amy.
- [01:33]
A
And we have today Brian Direnzi with us, who you've heard from on a couple other episodes, namely on AI and Brian leads our research and data team and has been really leading the charge on a lot of our AI effort at Tamagi.
- [01:48]
B
Thanks, Amy. Happy to be here.
- [01:51]
C
Welcome back.
- [01:53]
B
Thanks, Chen.
- [01:54]
A
Yeah, welcome back to the podcast. It has been almost six months since we published our last conversation about AI, and in that time I was out on maternity leave for six months and really not thinking about AI. It's funny to go from just so consumed by something to like just not even thinking about it. But now that I'm back at Dimagi, I'm very interested to just hear where you guys heads are at, what you've learned, how things have been going, what are we up to, and really just catch up on all that's been going on. And it does seem like it's been a very busy time in the AI world here at DMage and certainly outside of D' Amagi as well. So, yeah, maybe just at the highest level. Can you catch me up a bit on what you guys have been up to.
- [02:43]
C
Yeah. Brian, you want to take it away?
- [02:46]
B
Sure. Maybe I'll tell one quick story to illustrate things, which is I was talking to a partner today about some work that we did with them back in December. And I mentioned that there had been four major releases of the model since we did the original work. And some of the limitations that we put in around the original work were no longer true because of some of the new models that came out. It was an interesting thing where I was presenting the work that we had previously done and talking about where it goes and talking about the future. And it's still so hard to predict because things are moving quickly. There's, I think there's been some talk that things in AI are slowing down, but I think they're just shifting and the types of improvements that we're seeing are different than the improvements that we were seeing previously. But it was an interesting moment where I started to count the number of model releases between doing the work and the presentation today and was surprised to see how much things have shifted just in the last four or five months.
- [03:49]
C
Yeah. And I think from our own dmage experience, obviously the world of AI is moving extremely fast. The massive billion dollar foundational model companies are moving as quickly as they can. Google, Apple, Microsoft, Facebook, Salesforce, they're all embedding or creating foundational models as quickly as they can. So it's very clear that the generative AI at a foundational level is going to be in the core tools we're all used to using as quickly as possible. If you're on WhatsApp, there's already WhatsApp meta AI you can talk to now. So it's starting to come out like into the mass billion plus user consumer products in a very real way that was envisioned six months ago, but not quite there yet. And now Microsoft's talking about putting or it is in their os. Apple's about to embed something in their os. So there's these core capabilities that'll be available to everyone. And for Dimagi, we are still racing as fast as we can towards equitable AI, saying, okay, that's great for the people who are already on the tech curve and already getting buying MacBook Pros and exposed to these tools and technologies that'll take care of itself. The market's driving that very fast and seeing prospectuses on up and coming foundational model companies who are saying, if we get a 5% market share, that's a trillion dollar company. So they're like, they're not Claiming they're going to be the foundational model. They're like even a slice of that kind of core foundational market is going to be a trillion dollar company. So the market is clearly investing heavily in those companies and that'll continue to happen. The use cases above those are also racing ahead. We've seen tons of progress in medical AI, in legal AI, in business workflow AI. We're going to continue to see rapid progression. I anticipate they are going to get notably better than humans pretty quickly in some types of tasks. One of the things that we're doing now that Brian's leading with our team is looking at multi agent use cases where we're saying an AI is going to get. You can make an AI really good at answering a very specific set of questions, but maybe it's not good at answering every question you could throw at it. So if you have a family planning question or you have a TB care question, maybe the same bot can answer both of those well, but maybe you want two different bots or agents that can answer those independently once you detect the question that's being asked. So we're really excited to explore those multi agent use cases and then we've really pushed hard into our research and our work on low resource languages. So the models are getting quite good at being useful in languages that you wouldn't think. Brian can go into some detail on this and that's really exciting because that's a huge barrier to testing. There's the user experience and the language experience of interacting with the model and then there's what it's trying to convey. And the what it's trying to convey in English is already crazy impressive in terms of empathy and response and almost being better in some ways than a human would be at helping you think through problems or talking to you about certain issues. But if it only works in English and major languages, then it's going to not reach probably some of the most important equity use cases. So we're really excited about what we're already seeing and I think there's going to be a lot more progress on that ahead as well. Lots of huge areas and we've been very fortunate to receive additional funding across numerous use cases. I divide them into three areas. Now we have our direct to client work. So that's an AI that's exposed directly to end users. There's coaching use cases where we're trying to support frontline workers, community health workers, AG workers, with a coach, supervisor, assistant, not to replace the human, but to augment that, and we touched a bit on that on episode two. And then also a new use case that we weren't talking much about that's really been quite popular is a kind of program manager assistant. And we break that assistant into kind of three buckets. One is a knowledge assistant, which is what a lot of people picture with a Q and a type bot that's trained on your data that can answer questions, a data assistant that can interpret and analyze and help understand data that you might be dealing with as a program manager, like which county has the highest burden of disease. And then a workflow assistant. If you're reviewing documents or moving documents across teams, how can you do that? Imagine a program team of 10 people who's running a epidemiological program or a surveillance program. An AI that can really help all 10 of those team members is something we're getting increasingly excited about. So there's a ton going on, both within dmaggi and obviously way more outside of damagi, but a lot of progress at the same time. I think everybody's holding their breath on when's the next jump in foundational models. So we're recording this on May 22nd. OpenAI just released ChatGPT 4.0 and some people were guessing that might be ChatGPT 5 or 4.5. It's not. It's an amazing step in some directions, but not a huge step in terms of the next level foundational model. And one of the interesting things that's hard to put it in context, the jump that Anthropic just made. So they, a month or 2 back released Claude 3, which was a massive difference between Claude 2 and Claude 3. And people now view it as on par with OpenAI ChatGPT 4. So one, people are excited, there's some competition because OpenAI's ChatGPT 4 is pretty far out ahead. But two, the fact that other companies are replicating these massive step changes in performance and capabilities is also, I think, giving to me some credibility to the people who think we're on a very fast curve right now and just have to be patient. There was a podcast that we can link to in the show notes from the co founder of Anthropic with Ezra Klein from the New York Times. And one of the interesting things on that podcast I found was he's look, if you've been on the inside of this, we've been seeing really impressive step changes for years and so the fact that the next one hasn't come out in months doesn't bother us. But to consumers who just got exposed, saw this huge jump between 3.5 and 4. Now it feels weird that you haven't seen that next jump, but they're all like, no, nobody inside this world is worried about the progress. And maybe that's salesmanship or wishful thinking and people are worried. But his view is like, to the outside people who have not been doing this for a decade, it feels maybe slow, given how fast 3.5 to 4 was. Because if you're inside and you're testing these and you're like one of the engineers, you're like, you're just. This is, this is just how R and D works. We're moving forward and some huge things can come out next.
- [10:14]
A
John, I don't think I've heard anyone say this field flow, but that's funny. A funny perspective. Stepping back a little bit, John, I'm curious to hear, and maybe this is for the audience too, what do you see as Demangi's role in. In AI? And clearly, like, we're not building foundational models. Right. We're not trying to compete at that level. So as a tech company in global health, like, what do you see as our most important role here?
- [10:39]
C
Yeah, I think we, and Brian in particular, and the team that he is leading, we think a lot of this is very aligned with our core philosophy and ethos as the monkey, which is there's these amazing technologies out there, and technology is going to keep getting better and markets are going to keep driving it into profitable use cases. And we see huge potential and with AI potentially transformative potential to make it also work for the impact use cases that we care about. So what we're trying to do is understand where the market's going to take care of itself. And obviously we're not going to influence that. This is well beyond the resources the development sector mobilizes or is even influential in. But there's a lot of people who are concerned about equity within use of AI. There's a lot of people concerned about how this will be a race to a monopoly, either at the government level or at the corporate level, and how will AI be equitably accessed and equitably deployed? And so that's really where we're focused, is saying, how can we as quickly as possible, as safely as possible, and obviously ethically test these use cases, show our partners what's possible, show governments what's possible, just get this tech into the hands of people. And for us, that's been one part becoming expert in just what the tools and technologies can do, making sure we're doing it not just with any one proprietary model, but across all the models. Understanding where open source models are. And for us, our open chat studio platform is about just making it easy to learn and test. So we have hundreds of bots we've built at this point, hundreds of use cases. We're going to probably build thousands and not the too near future. Many of them are going to be totally useless, not work at all either because the AI didn't work or the use case wasn't worth trying to solve. But a couple of these are going to work and they're going to work really well and they're hopefully going to show a path on what use cases are possible today, what people should be thinking about for tomorrow. And I think that's a really important role that we play and I think we're uniquely suited to play It Given our 20 years of experience, our global presence, the fact that Brian has a research team with research backgrounds, the fact that we have program people who understand the day to day of what it's like to be CHW and how to support program managers, former government officials, partner with government. So I think you need to really understand a lot to know where you can add value from an equity lens in these use cases. And I think we're very fortunate that across our team we do have that collective skill set.
- [13:02]
B
I think to every point I have something to add and I've already forgotten half of them. But if I were to jump in I would just echo the team perspectives that we have an incredible team. A lot of the projects that the research team are focused on are around evidence generation. So everybody knows that we can generate interesting text using large language models. But there's an open question whether any of these use cases that we've come up with can lead to meaningful impact. Can we move the metrics and the indicators that we care about? And so we, we have a handful of different projects where we're trying to do that across a range of different scenarios where we're, we're actually going out, we're, we're deploying in a meaningful and rigorous way and trying to evaluate whether we can, we can shift important indicators in global health and mental health across the board. And our team is not only engaged at that level but and has all the experience that John mentioned. But we also internally have an incredible range of languages that we speak as a company and we did a little exercise while you were away, Amy, where in a literally like a four hour period or something we Tried out over a dozen different languages on a model and got some early, early understanding of how well the model could communicate, could understand it, could generate text across those different languages. And it was really, it was like a fun few hours where we had people jumping and saying, oh, I speak, I speak Zulu and I speak Chichewa and I speak Wolof and I speak whatever. And so we were like spitting up all these different bots and they were engaging with them and giving some feedback and like filling out these forms. And when the dust settled, we had this big spreadsheet that kind of gave us some initial understanding of how well a few different models were working across these languages. And it was really a fun moment where we got to leverage the incredible diversity and the rich backgrounds of all the individuals that make up Tamaki. And so I think we're really well positioned as a team to be able to take this on and push this work forward.
- [15:04]
A
I really appreciate just hearing a bit about the work that you guys are doing around language, because at the most fundamental level, if these models don't speak your language, you're not going to get any benefits from AI. Right? That feels like such an important layer that we can bring in and one of, one of many. I want to maybe provide a little bit of a framework for the audience in terms of how are we thinking about bending the arc of AI towards equity. And John, you had mentioned three different areas where we're seeing funding, right? Direct to client work, coaching and then kind of program manager use cases. But what I want to frame is maybe even higher level, like the three kind of areas where I see the teams applying effort. So one is that direct to clients, right? How can we, how can AI directly work with end users and support them? The ways that we're using ChatGPT daily about you guys, but I am the second being how can AI support healthcare workers? So that's where that coaching use case you mentioned, and even potentially the program manager use cases could fall in. But knowing how essential frontline health workers are to our end goal, right. Dimangi's end vision is a world where everyone has access to services that they need to thrive, and health workers are critical there. It can't just be AI bots. So how can AI really enable and equip frontline health workers in all their various forms? I think that seems like the second sort of bucket of where we're investing. And then the third bucket is really looking at the ecosystem of AI and how it's being developed. And that's where John mentioned Open Chat Studio, which I'm not sure if we've talked about on this podcast yet, but it's just a developer platform for building bots. You can do it with little tech skills. And Brian, I'll invite you to share a bit more about that in a moment. But I see a lot of work being done at that ecosystem level. How can we ensure that the testing and building and iterations and these learning cycles that are required are being approached equitably and inclusively so that more folks can be in there building and testing and learning. And it's not just concentrated in Silicon Valley where that's happening. So I've been really excited to see some of the progress on Open Chat Studio, where you're bringing in many partners, individuals, to empower them to cast and learn in this environment. Brian, would you care to speak a little bit to those three buckets? And then if you want to share a bit more on Open Chat Studio.
- [17:31]
B
Yeah, it's a big question and we could spend an entire episode on probably any one of the three topics there. But I think for the direct to consumer, I think in some ways it's the most straightforward because there's an individual, they're trying to get some information or share some information, or we're trying to make sure that they have access to, to the support that they need to accomplish something. And so we have a number of different projects where we're doing that. And the projects are starting to diverge in interesting ways, which is good. In some projects, we're focusing much more on language. So if we're working in a low resource language, making sure that it's interacting, and we have a project in Kenya, for example, where we're interacting with the youth and it's important that we're doing everything in Shang, which is this code mix language between Swahili and English, have lots of slang words thrown in. And even as we push on that we're becoming little mini linguists ourselves within the project, because even the Kenyan teammates we have who are working on this project, they're learning more about Shang than they ever knew before. And we're continually having to refine what it means to speak Shang or to communicate in Shang. If we were to come up with an English equivalent, you can imagine somebody speaking with the Queen's English and some very proper grammar. You can imagine the way that we all speak colloquially and other versions of that, and trying to hit the right level of informality and formality of that language, but still make it accessible to people. And still make it approachable enough that they're willing and able to engage with it and engage with the content. So I think those are some of the challenges around language that we're thinking about in different countries and for different projects. From the health worker perspective, the way I've been conceptualizing it is there are end users and then we might have frontline health workers, community health workers at the next level up, and then they have some supervisors and their facility workers at some point, all the way up to county and district and national level government and things. And at every level you can imagine creating chatbots to support the person exactly at the level and to facilitate the communication between the levels. So for the health worker, you can imagine a coaching tool that's just for them, nobody else is involved, is a reference, it's reference material, or it's some skill building material and they can engage with it on their own terms. You can also imagine if you have a frontline health worker who's going and doing health visit home visits for, let's say pregnant women, maybe she drops off a digital assistant that's going to be directed to client and reports back to the health worker. So now we've given the health worker some additional tools to keep track of all of her clients to make sure that she's able to provide continuity of support between those home visits. She's able to triage a bit better because she can see who's having issues or not. And then you can also imagine the version above where it's looking at the data and looking at the home visits she's doing, looking at the form she's filling out, looking at the services she's providing and summarizing that or providing some feedback on how that's going and identifying proactively some areas to support her. And in some versions we might also communicate that up to the supervisor so the supervisor can provide better human support to that community health worker. So I think there's a whole range of different things that we can do around a single health worker. And then for the ecosystem, I think there are a few interesting things. I met one of my colleagues on the research team on the plane and we were on the plane with no wi fi and I opened it up and said, hey, look at this. And I pulled up llama 3 and I had llama 3 running on my several years old MacBook and we were making chatbots just for fun, offline with an open source model and it worked surprisingly well in English. When we asked it to speak Swahili, it went into this crazy loop of it just kept repeating the same words over and over until it felt like my computer was going to catch fire. So I had to shut it down. We were pushing the limits there, but just the performance that it's able to do. And this is a quantized version of the smallest model that's available. So this is the lowest level of performance you can expect out of the model. And it was able to do some of the significant illness guide work that we were doing. It was able to do it quite well with no prompting effort at all. So just starting there, I think there's a lot of excitement around that. And colleagues of ours at Jacaranda Health have already retrained Llama 3 with a Swahili set to get it to speak in Swahili, similar to the work that they did for llama 2. So I think there's a lot of excitement in the larger ecosystem about open source models and where those are going. Things are getting more powerful, they're getting smaller. There's people who are running entire models just within a web browser, load into the web browser cache and run in there. There's all sorts of things happening. And so I think there's a lot of excitement around there. And with Open Chat Studio, I think you really hit the nail on the head in that our goal is to support internally our team at Dimagi, but also the larger ecosystem to be able to more easily engage with these models, to think about how we can quickly spin something out, spin something up, test it out on a modality that makes sense for us, whether that's WhatsApp or Telegram or SMS or Facebook or Instagram. So our goal with Open Chat Studio is really to build up the support and to leverage other advances in the open source community to be able to support that entire building and deployment process, to get something out and into the world and into the hands of people.
- [23:12]
C
And just to build. On that last point, Brian Amy, you mentioned it as a developer tool, which it definitely is. And we have people self hosting Open Chat Studio, but we also took the ethos we had with Commcare, which is non developers should be able to use this. And not that it's like trivial but like with some effort be able to rapidly deploy chatbots or AI use cases. And so we have lots of kind of non technical people on the platform building and testing as well, which I think is it's a very obvious idea to build a platform that lets you build chatbots. Like when we were first starting Open Chat Studio, we're like this is the least interesting idea you could come up with for AI. But every little thing, adding the safety layer we added, making it easy to connect to any proprietary or open source model out there, making it easy to test multiple instances, one against llama, one against AI, making it talk to both Telegram and WhatsApp and SMS and have a web version. All these little things add up. And so I'm really excited. Just six to 12 months ago was like, I don't know, should we build this and is it an internal thing? And do we need to build. There's just so many one off things one has to do to make the user experience for these things good and to make testing them interesting and easy. Any one step isn't that bad. You could download all the data, put it into a spreadsheet, do a pivot table and then compare the data. But it's like each one of these things adds up to be like a big barrier in closing that last 10% between making it actually work for the users we care about and not. And we saw that with CommCare, we spent tons and tons of time. And Brian has an episode speaking about his experience early with Commcare. But getting the multimedia to load correctly and the images to display correctly was like the difference between a low literate user being able to use CommCare and not. And so these last 10% user experiences really are like gotcha in, in my opinion, in equitable deployment of technology. And so that's an area that I'm really excited that I, I do think there's something here with the platform that it just, it's solving so many annoying edge cases for you that it's really an accelerant for developers and organizations to use to be deploying AI.
- [25:19]
B
I think my hopes and dreams for, or the vision that I have for Open Chat Studio is that it's a platform that support the entire process of exploring what we can do with an AI application and rolling it out and deploying it. And there are a bunch of things along the way including needing to build in safety layers or needing to build more sophisticated architectures for different agents, as John was talking about, or needing to build up some testing infrastructure to increase the confidence or generate synthetic data for that testing infrastructure to increase confidence of the applications that we're building and the chatbots that we're building. How do you actually deploy that and how do we test that? How do we make sure that it's working in different languages, how do we capture the nuance and the contextual information for Any particular use case? One of the early tests that we were doing in our work in Kenya, one of our colleagues was like, oh, I'm running low on money, I don't know what to do. I'm not sure how I'm going to eat next week. What should I do? And the chatbot responded, oh, you should go to a soup kitchen. And she was like, yeah, that's a great answer if you live in, in New York, but maybe doesn't apply to the context where we're working in Kenya. I think that's the vision that I have of being able to really support that for a large number of people, a large number of different organizations, be able to give access to locally led organizations, be able to give access to a lot of the work that we're doing. We're trying to get the partners involved because there is some democratization. Let's rephrase that. There is. What's. How do you say that word?
- [27:03]
A
Democratization?
- [27:04]
C
Democratization. This is Stanley, by the way.
- [27:07]
B
Good.
- [27:07]
C
I'm glad.
- [27:08]
B
My linguistic anyway fits the theme. So I think there is some like, democratization of large language models where all of the prompts are just written in plain text. Opposed to needing to learn programming languages, but just like general programming and approaches to building things, you'd have to learn some programming language and kind of upscale around all that before you can do it. We're trying to be very intentional in the projects that we're doing and bringing partners into the fold and getting them involved in writing prompts and seeing how we're writing prompts and seeing how it affects things in order to spread that ability to bend and manipulate large language models to their context and to their environments.
- [27:46]
A
It's like large language models in some ways are democratizing AI because it's more accessible. And then Open Chat Studio is taking that a layer further. Right. Democratizing the ability to be building these chatbots in a safe environment with a lot of thought put into how can we best be building, testing, learning and even deploying these chatbots into real world use cases. One thing that is coming to mind for me is I've heard quotes somewhere where AI is changing everything and it's foolish to go back to day to day operations because things are changing so fast. And so I'm curious, John, what are your thoughts around how should we be thinking about AI with our existing offerings? Right. CommCare shared here. How are we thinking about including AI in those?
- [28:33]
C
Yeah, it's a great question. I was just talking with a potential partner this morning, right before this podcast. And I've given this advice to many organizations who are not going to develop their own AI products but are thinking about how AI can be used in their work. And there's like the enterprise software that we use as an organization, whether that's Salesforce or Google Gmail or Tableau and Power Bi and things. And those are going to have their own kind of company wide capabilities that we'll just start using. And companies can choose to adopt them quickly or slowly, just like they could with Internet technologies and IT technologies. But like eventually this is just going to be table stakes for how organizations operate, right? There'll be AI infused capabilities in all these products. Then you see add on models where they're like, no, no, no, this isn't part of the product. We want to charge you more to get access to this AI. And I'm curious how that market's going to evolve. So there's like OpenAI, which is 20 bucks a month to have all the access to a chatbot. But then there's Google Search, just has Gemini embedded in it. You don't have to pay more for it. So there's going to be, I think, an interesting evolution of what the pricing model is and what the adoption curve is. And the reason I brought that up is as we think about it, for Commcare insured here, there's like a mental model. We're like, all right, let's build this new set of features and try to frame it as an add on that we can charge for. And so the mental model is similar to product development, right? Can we invest enough R and D to create enough value that we can then recoup it by charging for it? And then there's a flip side of like generative AI like capabilities are just going to be table stakes and an expectation that they're in all products and you can't charge more for them because everybody will have them. And that could end up being the world we enter as well. Now really good generative AI features are not necessarily cheap. So if we end up in a world where this is just expected shared here has virtual health coaching available and remote patient monitoring. So if there's a video, you can easily imagine adding AI capabilities to interpret the video. In fact, we're doing research and development on that now. It's expensive. You're talking about tens of cents per analysis. And if you're doing that on all of your transactions, like these things add up. So we're looking at three different areas. Table stakes, features, like we need to do this just to either keep our competitive advantage or we're worried we might need to catch up to what other tools can offer features that are somewhere in between an add on or just part of the product. The most Obvious is an AI coach for CHWS. We are exploring whether an AI could look at the app you've built in CommCare or look at the workflow you're doing insured here and then create an AI for you that could help coach the workforce that you're supporting or the end client. And then the third is like clearly net new things that the products don't do today that obviously we'd have to charge for if we offered. And we have teams thinking and developing testing features that could be on any of those three dimensions. I will say, because the market is really unclear to me it's hard to know how to invest as the CEO of dmaggi in those types of efforts. So we obviously are investing a ton in Open Chat Studio and for all the reasons that Brian mentioned. And then within our product lines we're doing a lot of testing and iterating. But it's interesting, like should we build the next obvious feature customers have been asking about for 12 months that has nothing to do with AI or take a leap onto an AI based feature that were far less clear on the demand and monetization? So obviously to both and almost every software company is in a both hand state right now. But I think there's a lot of potential for AI and I'm interested a lot. And Brian and I talked about this when we were together in Oxford a couple weeks back. I'm really interested in voice interfaces as well, which doesn't fit into any of those three buckets we talked about. But for CommCare you enter data into a mobile app, it takes you through a decision support algorithm, you're collecting data during the encounter. You could easily imagine replacing that with just voice conversation between Brian and I and then the AI just interprets all the data out of that conversation. Or you can imagine after I talk to Brian, I just quickly dictate for 60 seconds and Brian actually sent me a quick demo of this right after we talked demonstrating just like giving a summary with errors and like in totally normal human, hey, this child has 17 beats per minute, et cetera. And the AI is very good at extracting the data model out of that type of conversation. And so I don't think we could charge for that. That's just kind of so tangential to how we think about our markets and our business models for these products right now that some of the really interesting step changes and the user experiences you can create, the approaches you can create, they're really hard to think about strategically for us as a company because it's just really unclear like what is the payback of that, even though the benefit to the end user is really obvious. So we're definitely going to be exploring a lot with voice interfaces, but I don't think we can charge more for the voice interface in CommCare. If it works, it's going to be like part and parcel of hopefully how CommCare works one or two years from now. So lots of really exciting stuff. And then there's the stuff that people wanted to have done this whole time. Even pretty generative AI exploding. Like it has always been a goal that you can do predictive analytics with Commcare data. The shared tier team has always had that kind of video interpretation question of can you automatically detect whether somebody is ingesting a pill in the video? So there's kind of like the use cases that have been around for a long time that are now maybe a lot easier to test or a lot easier to deploy. And that's really exciting too. So there's a lot worth thinking about. And it's interesting because it's really hard to know what is going to end up having. There's tons of value. And then Brian and I were talking about this too. The Mike is extremely good at creating impact and creating value. What's very hard is knowing what will the market accept that it's willing to pay for, about that impact and that value. And as we've talked about at length on this podcast and in a lot of our public documents, like there's unfortunately a very low correlation between what the market and the development sector is willing to pay for and the impact it generates sometimes. So a lot of it's going to come down to framing, right? What, what feels okay for people to be paying for and how much of that is where we think the most impact and the most equity can be generated.
- [34:55]
A
I'm curious to hear from both of you your overall feeling about AI right now in terms of. I feel like there's so much potential and it's a bit of a nervous system overload situation where there's just so many possibilities for how we can be thinking about it and applying it. And I'm thinking about AI for my day to day work, AI for our products, AI for new products, like so many layers and that process of choosing is really tricky. So I'm. I think I'm in a place of cautiously optimistic but also maybe some overwhelm. What about you guys?
- [35:30]
B
Yeah, maybe I can go first. I think that makes sense. There's a lot happening, a lot of different players and John raised the point earlier. In some ways 12 months ago it was easier. There was one model that was clearly better than all the rest, GPT4. And if you wanted to do the most sophisticated thing or you wanted to work in the largest number of languages, you had one option which was to work and build off of GPT4. And I think it's exciting. There's some excitement that we now have a number of large foundational models from multi billion dollar companies. Llama 3 is much better than Llama 2 and apparently the 400 billion, the most impressive version is still training, so that'll get better. GPT4O is a much faster, much less expensive version that, that is on par and seems to smooth out some of the rough edges of GPT4 in our early testing. I am optimistic and I'm excited about the work that we're doing because I think we're keeping one eye on that and trying not to get sucked into the vortex of what AI, what major AI update happened this week versus last week and letting that derail us. But we're really focused in the team on exploring a range of different use cases and around generating evidence and convincing ourselves that we can actually move important indicators. Can we increase the self efficacy of a young person in Kenya to make choices about her own sexual and reproductive health? That's what Demaki is here to, that's what we're here to do is to support things like that. I'm excited about the focus that we have on that kind of work and around trying to generate that evidence and take that effort forward. I'm unsure where the future is going. All of the multibillion dollar companies seem to be focused on multimodal. Different people at different times have talked about oh maybe the future is in smaller, more targeted like the very focused medical models or the very focused lawyer models or something like that. And I wonder whether some of the sort of unintentional benefit for lower income markets goes away with that. But I'm also optimistic because we're starting to see better performance out of smaller models and better performance out of open source models and we're exploring a wide range of use cases but like narrowing in on, trying to evaluate and generate evidence around whether we can move the metrics and indicators that, that really matter in, in Our field.
- [38:07]
C
Yeah, it was really well said, Brian. And I think for me it's definitely optimism is the only viable position to take, given that it's going to keep moving fast. So we need to think about what, what are the positive benefits of it. I think there's huge negative externalities. We're going to. I mean, everybody does. This is like, there's going to be plenty of downsides. But as Brian said, there's lots of reasons to be excited that there's going to be unintended positive externalities that are just going to come out for free because that's just like the natural step of evolution of some of these models and approaches. So I'm really excited by that. I think we talked about this on episode one of our AI series, but I still worry as these models get better and as you can be 70% good enough, is that going to cripple innovation on being like really good at anything? So I had an interesting conversation. One of our new product lines where I'm like, you are not allowed to hire a human support person as a thought exercise. Just accept whatever the AI can do today, assume it'll get better tomorrow, but, like, never hire a human for the support team on this new product. And then we're talking to him. I this might be the right forcing function for us to try it because it's low stakes. The support burden is very low on this new product. And it's interesting to think through things like that. And you're like, if we just accept that's what we can get today and we hope it gets better tomorrow, that might be a really good business decision and maybe even like an optimal human use of time decision. But what if it never gets better than 70%? And then you're like, oh, and like, for support, maybe that doesn't matter. But for coaching health workers or for being empathetic to people seeking sexual and reproductive health issues, we shouldn't settle for 70%. And we manage a world, particularly with the next evolution of a lot of these models, where it's like clearly able to get to 70%. And then you're like, oh, crap, what are we trading off by just jumping onto this? And obviously we are going to jump onto it. We're not going to be able to stop ourselves. So that's one of the things I think about. That's both an optimistic view that we're getting there, we're getting to 70% pretty rapidly. But then also, what are the implications of that and how confident we are we'll get Maybe we're going to get to 150% by jumping on at 70 and it keeps getting better, but maybe it stops. And AI in general won't stop, but stops for the use cases that we care about. So that's one thing that I do think about, but I think there's so much uncertainty ahead and so much potential for the benefit of these tools, particularly in areas where the alternative use is not. Like we, we dmagi cannot afford to invest enough to provide perfect support for any of our products. Like it's just too expensive. And it's really exciting when you think about different use cases like this and the potential for it. Even if they're not perfect today, if they're on the curve and getting better tomorrow, that's just such a powerful place to be. And so I think with optimism, but it is increasingly challenging. One of the really exciting things Brian's working on is like, how do you just compare the performance of these things today and then it's obviously going to change tomorrow, even within the tools.
- [40:59]
B
I just want to frame the 70% question, I think in the vocabulary of our first episode was avoiding the dystopia where only the rich people get to see the real live health provider or mental health specialist or whatever and everybody else is left with that 70% as good AI version. And so I think it's really important that any of the optimism or any of the excitement or any of the work that we're doing, we're holding that in our mind at the same time and actively working to avoid it. We're actively working to increase the equity and exposure and access to these tools. And we're also keeping in mind that dystopia that we want to avoid so that we don't unintentionally build solutions that kind of push towards that. It's a difficult problem in that there's no clear answer for how to proceed. But yeah, I think incredibly important to keep in mind.
- [41:51]
A
Absolutely. Yeah. And Brian, thank you so much for tying it back to that original question of how can we avoid this dystopian future where only people in high income markets have access to real health workers. And there's so much to be mindful and thoughtful of and just really grateful to hear from both of you on all of the work that's been happening and all the thought that's been going into this. We'll definitely have to circle back in a couple months, see where things are at. Things are moving so fast. Thank you both so much.
- [42:19]
B
Thanks, Amy.
- [42:21]
C
Thanks, Brian.
- [42:21]
B
Thanks Amy.
- [42:22]
A
Thank you so much to Brian for joining us today. My biggest takeaway is that AI is moving fast and needs to be thoughtfully stewarded to make sure it's directed towards the most impactful use cases. We need to do so being fully aware of the risks and pitfalls. Since recording this episode, I had two AHA's I wanted to share. The first AHA is an analogy that the Open Chat Studio team shared with me. An LLM is akin to a brain that has no memory. All the LLM brain can do is process whatever input it gets through the nervous system and generate responses based on that input. OpenChat Studio is like the body that complements the LLM and creates a useful organism. OpenTextoStudio provides everything else domain specific knowledge and experience, guardrails, supporting systems, et cetera. The other AHA I had was around how we can think about technology like AI within our core products as Dimagi and this came from a conversation with a Director of product on our team. Products exist to solve a problem. The evolution of AI doesn't change the problem that our customers and our clients are facing. If AI is the best way to solve a problem, then great, let's use it. But let's not use technology just for technology's sake. Let's stay focused on the problem and the best ways to solve those problems and be open to exploring the various ways technology can be applied to those problems. That's our show. Thank you so much for joining. This show is executive produced by myself. Michael Kelleher is our producer and cover art is by Sudanshi Kant.