← Becker’s Healthcare Podcast
Becker’s Healthcare Podcast
Premera Blue Cross’ In-House AI Chatbot Boosts Customer Service Efficiency
00:11:54
Loading summary
Transcript15 lines
- [00:00]
A
Hello, everyone. This is Jacob Emerson with the Becker's Payer Issues podcast. Thrilled today to be joined by Dr. Nathan Cronk, who is the manager of AI engineering and operations at Primera Blue Cross. Nathan, thanks so much for taking the time to be with me on the podcast today.
- [00:15]
B
My pleasure. Thanks for having me.
- [00:17]
A
Yeah. And before we dive into everything we want to talk with you about in a special new project that's been happening at Primera, can you tell us a little bit more about yourself and what it is that you do at the company?
- [00:29]
B
Sure, yeah. So I have a. My background is in academia. You know, I was researching AI, you know, back in 2016 for my PhD, and it kind of sort of taken off, making some big splash out in the world. And I thought, you know, maybe I'd like to get out of the. The rubber room, the ivory tower of the. The crazy space with the academics and go make an impact with some of this stuff. And healthcare seemed like a really, you know, it transcends race, gender, ethnicity, politics. Everybody needs health. So it seemed like a nice place to go try and make a contribution. So I joined Primera, you know, about three years ago, and I've had the really enjoying opportunity, like, the enjoyable opportunity to help kind of encourage and engender this. This adoption of AI technologies by, I go, do, like, AI roadshows around the company, presentations, you know, on jailbreak attacks, you know, trying to help people become familiar with the capabilities of this technology, building prototypes to show the art of the possible. Um, it's been a really fun role that I've had here at the company, but since then, we've actually moved from, you know, roadshows and education into production. We've got some pretty impressive governance standards in place. We've got, you know, an AI security gateway that's pretty cutting edge and some real gen AI products in production, making difference in the healthcare space. So that's a bit about me.
- [01:46]
A
Fantastic. Well, like I said, we really appreciate you taking the time to be with us today. And let's talk about some of those generative AI products that have been happening at the company. There's a new generative AI chatbot called Alice. So can you walk us through what exactly that is, how it was developed, and some of the timeline, from concept to the actual rollout. Ultimately, can you give us that sense of scale in terms of what's happening with this at the company?
- [02:15]
B
Yeah, there are some really important details to make these timelines make sense. Right. So about two and a half years ago, we saw, I mean, you Know, myself and my team, we came from academia so we kind of knew what the cutting edge tech was and what the art of the possible was. And we saw this agentic future coming, or at least we believed that it was coming, and we started building an infrastructure very early on. So some of the timelines and things I'll share with you sound pretty quick, but I want to couch them in the reality, which is that we started building an AI infrastructure for the company that was reusable and reproducible about two years ago. So, but the reality is, is the CSR team, you know, they had the dynam, the 365 tool advertised to them as a nice tool to help support the csrs and finding information quicker. And they, they went and tried it out and it unfortunately, right, very early on these, these generalist AI solutions that try and like boil the ocean for everyone, they, they can, they can work, but not fantastically, right? I mean when you try and have a generalized, you know, a vector database, right, which basically takes any type of data of any form and chunks, it partitions it and makes it searchable for your AI age agents, you're losing a lot of domain expertise that can be exploited by people who really know what the application is for. So that was where we came in. They came to us and said, hey, we were trying this Dynamics thing, it isn't quite working. Why not? And we said, well it's not because there's a lot of idiosyncratic domain expertise that needs to be baked into the chunking processes and how the text is formatted and how the agent reasons over it. Let's show you. And they said, okay. So honestly, it took us about a week to make a prototype, given that we had the infrastructure and everything in place. And we had a reasoning agent that was built on a technology called the LLM compiler. And it searched through thousands of methods and procedures and would give you answers, would reason over them and give you answers. And we said this is the kind of thing that we can do. And they said, great, let's build this out and go into production with it. So it took us about three months to get into product like the UAT phase. So that was when we rolled it out to a couple of different lines of business to start experimenting in a small, in some small waves. And then over the past about two to three months we've been slowly rolling it out to more and more and you know, we're up to hundreds of CSRs that are now using, using this tool to help them Find methods and procedures, information more quickly to help members and people who call in.
- [04:44]
A
Wow. No, I mean, that's amazing. So a lot of time went into this, but in terms of the infrastructure. But then it sounds like it really, a lot of it came together quickly once you started developing that prototype. You know, you mentioned, you know, this is for customer service operations. So can you give us a sense of some of the impact you've seen so far, what you've heard on the ground from, from people actually using this tool?
- [05:06]
B
Yeah, that's, that's some of the most rewarding part of it here. I mean, you know, we are, we love the AI and the building stuff, but that's, you know, hearing the impact on the front lines has been great. Yeah, there's, there's so many quotes that we've heard. Firstly, you know, just allowing the CSRs to hear that the members are happier, like on average because they can get information faster. So that sit there and get, you know, frustrated on the phone. Well, please give me a minute to go look this up. And you know, they're calling in on their lunch breaks or maybe, you know, in a break in work, they don't have a lot of time. That's a bit of a luxury. Right. So for the CSRs to be able to get them answers faster has been rewarding both for the members and then also the CSRs to give them more job satisfaction, which is, which has been pretty great. The leaders are also quite happy because we're seeing trending down and call handle times, which is, you know, as leaders are well aware, that's a pretty important metric for success in call Sanders. So average handle time has been trending down as more and more adoption of the tool is used. And in addition, something else that's pretty neat is we have the TSS themselves, which are the technical support specialists that help CSRs when calls get tricky. Usually they're inundated with questions. But now that the CSRs are able to go to Alice for some first level triaging of challenging questions, the TSS now have a bit more, more time to go actually improve the documentation that ALICE is using to give the CSRs the answers, which has been great because, I mean, there's thousands of documents and they've been maintained by humans. So as you know, there's of course going to be a few errors and mistakes in them. So Alice has been surfacing those where maybe Alice gives a quote unquote wrong answer and they go, well, Alice made a mistake and they dig in there and they're like no, actually our, our policies or things have, have evolved or changed over time. So now we can update this and this just making everything better where the documentation is more aligned more quickly. CSRs give answers more quickly and the leaders have been pretty pleased with it too. So it's been a pretty rewarding turnaround to see how people have been adopting it.
- [07:10]
A
Absolutely. So really this sounds like a win win dynamic both from the member and the internal perspective. Especially with that average handle time trending down, the technical support burden being lifted so they can focus on other things for the other health plan leaders that are listening in. Nathan, from that operational and that ROI perspective, I know you touched on a few metrics here. Are there any lessons you would share in terms of scaling an in house AI solution that could, that they could apply to their companies all over the country?
- [07:39]
B
Yeah, that's a really important question and I think we're at a very critical point where these decisions need to be made now and they need to be made correctly. Right. So everybody's got this build versus buy idea and it seems attractive, right? Well why would I try and get an in house AI engineering team to build these things out when pretty much every company out there now is, you know, advertising AI, AI everything. You can buy AI powered pizza on the way home, right? Well, of course, you know everybody's going to advertise that because that's the new attractive hot topic that people you know, want to want to market. But what I would encourage leaders to acknowledge is the following. Most of businesses is institutional knowledge, right? Your, your real intellectual property and value is in oftentimes your data of course, but all the knowledge and expertise that your staff and your employees bring, all the processes and, and procedures that you've distilled over decades of, of honing all this, this knowledge and expertise. So if you can build AI agents internally that crystallize all of that expertise that you've honed over time, you're basically locking down that intellectual property and that's now an asset for your enterprise and you can evolve it, grow it over time, allow those agents to cooperate with other agents to solve more complex problems as we start emerging, evolving towards these emerging like multi agent systems. So my advice is early on, if possible, if it makes sense for you really do try to think through, if you can try to build your own custom agents or at a minimum ensure that you have full IP over the agents that you are trying to build internally. Yeah, that's my advice.
- [09:24]
A
Sure. It makes a lot of sense and I think that's great advice for our audience as well. The last thing I wanted to ask you Nathan, is how you see AI enabled customer service tools like the one you've built, how is that going to continue to shape Primera if you look forward the next three to five years? I know this is very quickly evolving technology and that is a long time frame. But what's your predictions here for both your company but also for the wider industry?
- [09:49]
B
Yeah, I mean, I can tell you, right, So I mean we have across all of our organizations, right. Cybersecurity, it, corporate data and analytics. You know, we've been coming together and we've got a core group of people that have been making, you know, our enterprise AI architecture and trying to ensure that our, we're future proofing our adoption of AI and where we're scaling towards. So the main thing to acknowledge is that, you know, health insurance, I mean it's, it's a math problem at the end of the day, right? So first and foremost is you got to solve the math problem. Computers are better at solving that math problem. And we want to acknowledge that that is the core capability that we want, you know, technology to support. And. But then also there's the human element. I mean health is really important and we need to be able to reach out to people, be sensitive and be able to connect with people as well. So what we're thinking is that this insurance is basically a process driven industry. So the vast majority of the math problems that are being solved and the input and processing of documents and output of processing of documents, these are things that AI agents can help automate. And what we would then be able to do is to leverage all of the human expertise to really touch on the human side of things and make sure that all the members are really feeling cared for, that they have the support that they really need. So we can automate the laborious, slow, sort of procedural factory line type work and allow humans to connect more on the health element which is, you know, I think at the end of the day what, what we really want, that's one of the things that is going to separate the health industry from others. And AI adoption is that it's very personal and it's very, very important. And so how AI is integrated, there is going to be a pretty important thing that these companies need to think through carefully.
- [11:33]
A
Wonderful. Well Nathan, I want to thank you so much for taking the time to chat with us and with our audience and for sharing our insights with all the leaders list in. We, we really appreciate it.
- [11:43]
B
It's My pleasure. Thank you for having me.
- [11:44]
A
Yeah. And. And to our listeners. If you'd like to listen to more podcasts from Becker Healthcare, you can visit Beckershospital Review.com.