B (3:41)

Wow. There's just a lot here I think, especially from healthcare. But I think other folks who are in different industries can even see some parallels to some of the challenges that they're thinking about. So when you were addressed with these challenges, how did you think about it in terms of implementing AI solutions?

A (4:02)

Yeah, it is a big problem, but it's also a very personal problem. Healthcare is very personal. Even when you're simply talking about getting a bill that you don't understand why or having to wait a few weeks to get an answer for to get an imaging, it's very, very personal. So as we think about AI solutions, it really has to do with trust and transparency. It is the utmost important thing is the trust. And remember I was just reading to my kids Bernstein's books, Bernstein's faer books, love those books. Right. And that's this book that talks about once trust is broken, you can get it back. So I think as we as technologists and think about building and rolling out these AI solutions, we have to get it right the first time, which is fair and forgiving in this notion of stochastic and deterministic world. So that's something that it's top of mind for us. So what does that mean? We need to make sure what we do is reliable and consistent. There's no tolerance for hallucination, like none. Also there are a lot of important experts opinion we have to take into account. Right. It has to be clinically sound, a lot of literature that we lean in, a lot of guidelines that we can count on. But the funny thing is when you put two doctors in this room for the same case, chances are they don't agree on everything. That's why we love to get a second opinion as we think about building AI, we have to take all that into consideration. What do we mean when a system is performant based on whose opinion is that and what are the nuances that we have to account for where we really need to say human needs to be in the loop.

B (5:52)

Yeah.

A (5:55)

I think last but not least is the security and privacy aspect and we can get into some of that as we think through. When we kind of get our hands dirty in building a system that's a lot of technology safeguards but also process safeguards that we have to consider.

B (6:13)

These are some themes that I think a lot of businesses are thinking about regardless of what industry they're in. Especially with AI, I think the bar just keeps getting set higher and higher, which is great because now the what the what customers want to be able to implement, to be able to serve their end customers is going to be an even better, more accurate, more performant application for them. So I'd love to understand how are you thinking about no hallucinations, ensuring that it really is a safe and reliable application?

A (6:51)

I think let's put it in terms of the software development life cycle. Yeah. So when we are doing design and kind of the reference architecture, it's easy to have the technologists in the room. I think especially in healthcare and I imagine in many, many, you know, nuanced domains, we must have the experts in the room on the get go. It is common for many healthcare startup to talk about performances, talk about, you know, clinicians in the loop. But I think there is a difference when you engage your domain expert in the beginning versus at the end when we simply ask them to do validation. In career health, we do every single development project, have clinicians on the team and in the loop as opposed to waiting until we already built the prototype or already about the launches solution, asking them to validate. And I think that domain expertise is key to ensure that we're measuring the right things. I think that leads to my second point. I grew up in an era, we talked about test driven development and I think now, especially with a lot of this agentic solution, we really have to move to the mindset of evaluation driven development where eval comes first. What are the metrics that are important? How are we going to track? We talk about no tolerance for hallucination. It took us a month to iterate on exactly how we quantify hallucination in particular clinical settings. All those upfront work is more important than ever. I think that's key once the solution is launched. Now we are at the, you know, monitoring and tracking phase. Obviously having 24, 7 monitoring is key, potentially using, you know, the gen AI itself to help as a judge. But I do believe in the importance of human audits. Having that regular, intelligently sampled human audit, it's really, really critical. And I'm going to say one last thing and Kenji, you may have Something to add to because we've been working together on this is we're learning that there's different Personas that will use AI solutions, especially since healthcare is such a personal space and even learning about how do you roll out to different Persona groups over time help us build a more trustworthy and useful applications. So

C (9:33)

yeah, the one thing that I would add there is the, the key focus for us at a lower level is having strict guidelines and standards so that all patients get unified care while allowing for some level of user preference when it comes to how that care is received.

B (9:54)

There's so much to unpack, so I'm glad that we've got more time. I'm going to ask so many different questions, but let's dive into that. What Kenji was just talking about, that user, you'd use the words I think user standard care, but really like focusing on the end user. And that sounds like you also need to be able to have that domain specific data in order to be able to bring it back to providing the best care or the best experience for that end user. So maybe you can tell us how companies can really think about incorporating their own domain specific data into in terms of AI.

A (10:33)

It's a great question. I think technically there are many ways case specific context or pre training of models, the spectrum is wide. But I think instead of talking about the how, I want to talk about the why and the what just for a moment. Right, like why, like ask ourselves why do we want to incorporate domain specific data? Is it because I want the solution to run faster one cheaper, be more accurate, be more contextual? Is it a matter of that you want more code control over your system's output? Especially given the industry we operate in. And to Kenji's point where there's a standard of care that we want to maintain. I think understanding the motivation behind of using this domain so specific data will help us pick the right data and at what point do we incorporate them? So there are instances that would make sense to incorporate more of a knowledge system, right. Like in our world it's more around medical society guidelines, standard of care ontology framework. Incorporating them will allow us to have a more consistent framework in how the AI operates. But then when the goal is to provide more contextual, accurate response in every single interaction, then we need to, you know, have our AI be able to access very specific individual case notes and they all, they all are relevant and they are important. It goes back to why. Yeah. And last but not least, put a plugin to the notion of data use rights. Is really critical when we entrust it with patient data and when we entrust it with business sensitive data. What rights does Cohere Health and our partners have to use the data for what purpose is really, really critical. And it's quite nuanced. Right? Are you using it for operations versus are you using it for learning or are you using it for education or training? All that nuances have to be accounted for.

B (12:43)

That is such a good call out because I know I've seen companies already start going down the route of maybe using a specific data set for an example and then it's the engineers who get really excited about the problem, wanting to find out later on that like, oh, sorry, we can't use it because of like the. The rights to the data for maybe it's like compliance, licensing, whatever kinds of reasons. So that's a such a good call out that I think will definitely help a lot of the listeners. Same with what you were saying earlier about having those experts really from the early stages of the process instead of the human in the loop just being the end part. So maybe you can help us really share your thought process of how do you decide when to actually automate versus actually having that human oversight? That's part of the process is the million dollar question.

A (13:41)

Right? It's risk and reward. I think human in the loop makes a lot of sense when we're working with high risk, high reward cases. And like for instance, in our case, we have a good number of solutions that target prior authorization, automation, trying to get the patient in the right care faster and with less paperwork. But we've made a clear decision, regardless of regulatory and which geography we operate. AI would never use to deny a patient's care or to even deny a provider's request. Right. Anytime there's any doubt that we cannot say yes, we will always have a expert of the same specialty to review the case and even sometimes have a conversation with the requester and bringing that really important human conversation in the loop. So to us we draw a clear line of when we will not. We will always have. Let me take that back to Cohere Health. We always draw a clean line on when we will not automate is when we have to say no to a patient's care. But going back to your question earlier, Julian, about human oversight, even in the cases where we choose to automate, I think oversight is still essential. I think it's just a matter of at what point does it come into play. Yeah, oversight should always happen with the system design, reference architecture and how we design the eval and oversight should always happen with audit. But I think it's in a high risk scenario where human oversight is in terms of every single case and every single nuance detailed. And figuring out that middle ground, I think is the key to figuring out how to scale.

B (15:35)

Yeah, maybe you can give us like an example that I think can help the listeners maybe visualize what that could kind of look like in their own business.

A (15:47)

Yeah, sure thing. So for instance, going back to the prior authorization automation example, we could look at a variety of clinical areas. A lot of us have experience getting a prior off for imaging, trying to figure out what's going on, a diagnostic reason, or you go get a prior off because you have to get a knee surgery. Depending on the clinical use case, the risk tolerance is quite different. It's something when you need to bring me to your operating room and cut me open versus getting an MRI which is taking half an hour of your day and with minimum radiation exposure. Considering that I would, you know, the way cohe health approach it is for a knee surgery decision, we better be absolutely confident before we automatically, you know, say yes to the case without a human review. Whereas for a diagnostic imaging we will likely say as long as there's no contraindications, as long as patient risk is considered, as long as it's covered by your policy so there's no financial risk, we'll go ahead and say yes without human intervention. So that nuance, that's why the human experts, the domain experts in the loop as we design the system is so critical.

B (17:09)

I love that really having a framework for, with the experts assessing the actual risk and then using that to be able to design how the human in the loop, human oversight is part of the entire process. Let's get into how this has actually been built. So Kenji, I'd love to understand really what are some of the factors that your team was thinking about that ultimately led you to choose Amazon Bedrock and Amazon Bedrock Agent Core?

C (17:45)

Sure. Yeah. So it, I think timing was a huge factor for cohere. We this past year have invested a lot of time and resources into building out a platform around our AI And a lot, a lot of that incorporates. How do you scale with the new agentic services that are, that are out there in the market? And so we were attending workshops with AWS for Agent Core. We were evaluating the different components. I think the key thing that stood out right away was the speed of innovation here. The ability to build these agents quickly, effectively and safely. A couple key components for us that I think most developers can get held up on are memory and mcp. And so it was clear to me that these were paramount for the service teams at AWS when they were building out Agent Core. Because the tenancy concerns that we have in a highly regulated space like Healthcare are covered with some of the components of Memory Client out of the box and implementing them only takes a couple lines of code, which to me is a huge, huge benefit. The gateway is another thing. So we had started building out our own MCP servers, but with the identity built on top of Gateway and the different targets that the gateway provides, we've been able to at least iterate on our research and scale out the potential use cases for, for our agents. Because again, it only takes a couple lines of code to implement an entire MCP server target.

B (19:42)

And that speak, it says a lot that you're saying you're able to build it quickly, effectively and safely because Bedrock Agent Core is relatively new. So it sounds like there must have been some folks at AWS that really helped you. So maybe you can share some of the some how AWS was really able to help you with everything you were saying earlier, like the technical challenges, the business challenges that you had and to be able to actually build it in production today.

C (20:13)

Yeah, I think what helped us was a little bit of hand holding around understanding our use case. Right. Our top concern is always the security of the data on when it comes to implementing a solution like this. And so the, the first thing we brought to them was how do we transition our short term memory to Agent Core so that we have tendency separation. The, the, the workshops that we went through with some of the jupyter notebooks that they had available had everything that we needed right out of the box. And so going through some hands on experience in a test environment was super helpful. And then understanding the documentation was also key. And I think the namespaces that the Agent Core memory client provides are very intuitive to set up.

B (21:14)

I'd love to know, based on your experience between the workshops, the documentation, is there anything else that kind of stood out to you that can help folks who are on that journey of implementing agents?

C (21:33)

Yeah, so I touched on it a little bit, but I think the amount of experimental research that we've been able to do has far exceeded what we thought we would be able to do by, by this point. And a lot of that is due to the ease of development here. So I'm trying to think of a good example to share. There are two different use cases that we have. One was Transitioning an existing service over to Agent Core. We had spent all of this time evaluating and setting up memory for a chat agent and transitioning over to the memory client was a relatively trivial task for our team to implement. And a lot of that was due to the documentation and the workshops that they were able to attend. But then there's also net new development. And what's clear to me is that Agent Core really takes away a lot of the ops concerns from the mle. So the developer on the ML side, who typically wouldn't be a DevOps expert, doesn't have to evaluate those concerns as heavily because a lot of it is handled by the Agent Core service.

B (22:56)

I want to go back to something you were saying earlier, because I think this will resonate with a lot of folks who are listening. MCPS are really a huge hot topic right now and a lot of people are thinking about building it themselves. So I'm curious from your experience when you were at that stage of you had started building it yourself and then you would start then use Bedrock Agent Core, if there was any other learnings that you had from that experience that you can help someone else who's really thinking about building it myself or should I use Bedrock Agent Core to make that easier?

C (23:32)

Sure, yeah. So cohere has been around since before a lot of this technology existed. And so some of our applications have tendency built into them that the agents need to follow. Right. So we want to make sure that the agents are following the same patterns that we already had in place pre. Pre agentic implementation. And so we were building out our MCP servers and ensuring that we had auth proxies to send through the same sort of approach that we would follow on the core application side. But the Agent Core Gateway handles this implicitly with the identity provider. So it's one of those things that with a couple lines of code, like I was saying, you can. You can just pass through the same authentication method that, that we would typically set up on our own with the configuration enabled by Agent Core Gateway.

B (24:34)

So I'm curious now that you went from before you started to really like go down a path building yourself. Then started with Agent Core, did that change at all? Maybe like your time frame of when it was that you were able to put into production your. Any. Any other areas of like your velocity?

C (24:54)

Oh, 100%. Yeah. We were talking about this all week. I think going into this quarter we had maybe an agent or two that we had planned to develop and looking ahead at Q1 and the rest of 2026 it's full of agents, it's full of agentic systems, multi agents and I think Gigi touched on this a lot but the evaluations come first so a lot of our focus has been on building out the evaluation framework this quarter so that we can scale and continue to build new agents that we know are going to be successful on the first past using using Agent Core.

B (25:34)

I've got a few questions that I want to get your both of your opinions on. So Gigi, I'll start this one with you. So model choice, this is a super hot topic that I know a lot of businesses are really thinking about so I'd love to hear how you think about looking at evaluating different models and the term. I love that you chose earlier evaluation driven development and based on your experience maybe some insight that you can share on the business value of that approach.

A (26:08)

Yeah actually it could be helpful for me to go back a few years of history. Kenji talked about how Cohere Health had started a few years ago before this agentic revolution and in fact we were using our own transformer models to do NLP months if not quarters before ChatGPT came out. So the company was on already a accelerated trajectory on adopting this cutting edge tech. So I think we have a unique perspective because it's always the question build versus buy or do we tune like so I think the decisions between do we continue our own journey in building our own model from scratch versus adopting one of the frontier model with pump tuning or maybe go down the path of fine tuning slash maybe pre training. So we've been having these internal healthy debates for for quite a few months. So I think it's a unique experience that I love to share more widely and I think one one thing is change is constant and the only the best thing I could do as a leader for these amazing technologists is to set very clear metric right? Accuracy, cost, latency, reliability and honestly how much eval data is needed for each approach. We cannot push any AI out without publishing eval data, especially in the industry we're in. So really thinking through all those metrics and I can be honest with you Julian earlier this year is still leaning very heavy toward self training and more recently it's leaning more and more towards fine tuning or perhaps using one of the frontier model at least at the get go so that we can get really good coverage. So that I think it's keeping an open mind. What has really helped us is once we agreed on these are the business and operational metrics that are important to us and setting up a leaderboard so that we have the ability behind the scene, not as part of the constant spring planning that we have, you know, a way to keep monitoring and tracking which models are winning and what sense may make sense for us to make the switch.

C (28:30)

And I think what makes mine and Gigi's lives a lot easier is the investment that the company has made in experts that help us label our data too. So Gigi touched on it earlier. We do have a ton of licensed clinicians that not only help us when it comes to evaluating our decisioning modality, authoring policies and those components, they also label data for us so that we can use that in our evaluation data sets. So on top of an automated LLM as a judge approach which we have in place, we also have clinicians labeling the data for us behind the scenes 200% Kenji.

A (29:13)

And change is a constant. Having a leaderboard to keep watching against metrics are so helpful. And what we've learned too is as this ground truth data keeps coming in like every day for us it's 50, 60,000 labels a day. The answer will change, right? Because when you have more and more ground truth data, your ability to fine tune something or self tune something change as well. So I think being, being clear on again let me take that back. So for those who's editing probably get rid of what I just said that last five seconds. But I think going forward is being really cognizant on how we collect ground truth data so that we can have that use case specific insights to make these decisions.

B (30:05)

There is a lot to really unpack there that I think every single listener can take get a takeaway from metrics, having a leaderboard. I know a lot of businesses out there that I speak to have a model valuation framework. They're usually just sticking with one large language model and they stick with it until maybe there's a reason not to. But I love that you're really assessing all these different options that are out there I think and obviously it's very clear that you're looking because you have all these different metrics that you've defined ahead of time, you're able to then and you've got this process, you're able to have the best cost possible at the lowest latency, at the best performance which at the end of the day that's what companies are all looking at. They are just don't have a system that can be able to help them get all of the benefits that they're really looking for. So I would love to Hear your advice for a company that right now they're using a single large language model and they're curious of there's probably other models that there are definitely other models that are out there. But how do they go from one to being able to assess others so they can pick one or more that are going to be best for their use cases.

A (31:32)

I think you summarized it well. Right. Let's make sure you know what's important to you, what are your metrics. And then investing in that automated eval framework, both human in the loop and using large language models so that these decisions can be made with data driven decisions. That's a big part. And when it does show that there may be value to switch, my personal experience says that it's not always one size fits all. It's not that you switch from one frontier model to another for all your use cases or even within the use case with every single piece of your pipeline. So it goes back to your architectural conversation, working with the architect and thinking through how you design an architecture that allows you to have that composability, the ability to, for some, in our case, certain use cases rely more heavily on smaller models that that we host internally. In certain use cases we rely more heavily on frontier models. But if your architecture doesn't support it and every time it's a new build, then it makes the adoption cost very unbearable.

C (32:48)

Yeah, Gigi touched on it. I think having the platform, investing in the platform is key to this style of development. Both the evaluations framework and the gateway. I think agent core gateway is meant to be a gateway for the agents. I think having an LLM gateway or an AI gateway is also valuable to put on top of this framework so that you can iterate quickly, change targets and evaluate these at a much higher pace.

B (33:22)

So I'm very curious about really the business impact that you've been able to see in AI. I know there's still companies that even struggle to be able to measure the ROI of AI. And so hearing, I think from your experience will certainly be able to inspire them. I know in addition to everything else you said earlier, that definitely has, if not already.

A (33:46)

Yeah. Oh well, let me go back to the example I started earlier in this conversation. In the prior auth space, we've worked with clients who have to deploy dozens and dozens of nurses and MDs and doctors to review these cases in order to just meet the volume and meet the turnaround time requirement. It's a highly regulated industry. We have 14 days to decide. But starting in the new year you Only have seven days to decide. So it's easy to just try to full bodies at the problem. But with the AI system that we build out with our clinician in the loop, we able to achieve 85% automation. So 85% of decisions are made within minutes. And for those 15% of cases that require high touch human review, with our agentic system we've seen about 30 to 40% improvement in productivity. So and something that's tough to measure but we are hearing feedback from the users is it makes that it helps with the job satisfaction because they spend their time making clinical decisions as opposed to trying to figure out where the information is or trying to dig through all the requirements. Right. Everything is surfaced, all the deep research is done on their behalf and they can really just focus on the area expertise. So. So yes, we save time but also I think we have better retention.

B (35:19)

That is definitely a testament to the operations that your team has done. I mean to be able to get to 85% automation. Your end customers being even happier with the solution. That really speaks to I think for all the listeners who are thinking about how to be able to get there and especially those I know who are really in a monolithic type of architecture right now where they have one LLM and they're not able to maybe experiment with a number of different models that are out there to be able to maybe have a certain use case that's for the. They can get away with like a lower cost, one that'll give them better latency, all those different factors that you were talking about. So I think I just love those metrics because I think it just shows others who are in the early stages of their journey what's possible. Okay. Some other areas that I think I'd love for your opinions on. All right. So Gigi, based on what your your experience, what are some like what are three things that all companies should really be doing doing when they're in the early stages of being able to build agents before they actually push into production? I think I'm going to know your answer, but maybe I'll be surprised

A (36:43)

and a little context. Right. I've been doing this line of work for 20 years. So agents or not, right. The movement from a proof of concept or a pilot to a large scale solutions. I think McKinsey says that 95% of AI prototypes and pockets even go into production and only half of the ones that go in production actually stay in production. So this has been a age old problem regardless of agent or not. I think agent does create Extra layer of pressure because on one hand you can innovate and experiment faster but on the other hand there's more unknown where that you have to manage. So you actually exemplify the challenge and you write we're going to start with evaluation driven development. If there's one thing you want to take home from this podcast, that's the one line that I will really encourage us. And as we think about these metrics, to Kenji's point, having the right experts to label, provide the label data, provide the ground truth, even if it's weak ground truth, right? They are valuable, but make sure that they tie back to your business and operational metrics. So they're not pure functional non functional eval, but tying back to the overall company or your client strategy. The other piece I've personally seen a lot of struggles going from POC to scale is not having that, not investing that time. Let me put it the other way, I should put it in a positive way. Let me try again. Another part I've seen successes in taking from pocket to large scale deployment is taking the time to define and letting everyone know what must be true for the agent to be successful at scale. Because the nature of POC is to simplify, right, Is to not consider certain edge cases and assume certain level of integration and operational efficiencies. So in order to kind of flip that switch, we must be very, very clear on what are those important criteria for it to be successful. And they're not usually AI related, they're usually about the people who are going to be using the tool. Are you going to give them the right training? Are you going to give them the right transition plan? Are you going to bring in the right advocates? Because as I mentioned earlier, everyone look at technology differently. There are different Personas. Who do you bring on board to help you evangelize. And oftentimes things fail because of processes. Because if you don't change your processes but you get a new tech, you're not going to see the benefits and you can quickly fold, the impact can quickly be minimized. And then I think the third thing is classic system built, right? We always assume data integration is easy and it's never easy. So I think that's the piece that we always have to take a step back and say amazing AI system. Let's make sure the people, the process and the data already and having that clarity so that everyone's on the same page and marching to a single. It's key.

B (39:53)

I think you just saved people a lot of time on that because People get so excited about AI and already start thinking about like oh what LM for example are we going to start using but getting the domain experts really at part of the process making sure you really understand like the that you were talking about earlier like that what you're building is going to change the entire process for your end customers and really having clarity on what that means for them. Are they going to even use it even the data ingestion part as well? I think these are all prerequisites that people really need to think about. I've seen that as well as they often become overlooked and then you have to take two steps back before, before you can go ahead. Anything you would do differently if you were starting over today?

A (40:52)

You stumped us with that one. Let's see. I think the jury is still out. We don't know yet. We don't know yet. One thing that I'm still thinking through is is it more effective to have a small team that focus on the agent development in a larger org or is it better to plug the agent development into every single development team and let me try to say it the other way, is it better to have a centralized agent development team, agent development team and let them drive the innovation and then dissimilar what they learn. Is that more effective or is it more effective to have each development team to start adopting and doing more of a federated model? Jeremy is still out on that one.

B (41:51)

That sounds like that'll be a part two episode.

A (41:54)

We will let you know. I don't know. Kenji, anything you might want to say?

C (42:01)

No, that's a good point. I don't have anything there. It's a tricky problem.

B (42:10)

All right, I've got one last question for each of you. Kenji, I'll start with you. So we've got listeners here that are in all different industries at all different stages of their journey within aws. So for those who are considering building AI agents, what's one piece of advice that you have for them to get started?

C (42:34)

I think AWS has so many resources out there right now to get, to get to make it self serviceable. So I would. My best advice is to just start testing out and trying the tools and at least in my experience asking the questions early and often to the service teams, to our account managers, it's helped, you know, uncover the solutions that we would have spent more time trying to figure out ourselves.

A (43:07)

Kenji is right. Being able to have hands on experience is the best thing to make good decisions. And I guess one, one more thing I'll add is don't let FOMO get in the way just because everyone seems to be deploying and benefiting from agent systems. Really always start with the why? You'll be surprised. There's still a class of problems that may not be agentic, right? And there's a class of solution that can be very well solved with a single LL and just really kind of going back to the business and success metrics to make these decisions is key. And I'll add one last one, probably not in the right order. So if the person who's editing can help us out here, I think my advice to the folks who are wanting to dig into agent and so forth is think hard about the team you assemble. To make this real think agentic work truly does require a new profile of developers and a development team. We mentioned earlier about the importance of having domain experts in the loop on day one. Those who are technology savvy domain experts, they are gold in the team. And we've had amazing success seeing the hardcore platform developer working side by side with a data scientist and working side by side with a clinical MD and a nurse. That combo really allows us to iterate super quickly. And that's a new framework. I don't think we've had that kind of dependency in terms of diversity and skill sets. Being in the same room at the same time before the agent.

B (45:11)

Wow, this was seriously a masterclass on, I mean, evaluation driven development, AI driven development. There clearly was something here for everyone. Thank you so much Kenji, Gigi. This was phenomenal. Really appreciate both of you. Thank you spending time here with me on the AWS podcast.

A (45:35)

Thank you for having us.

C (45:36)

Thanks Jillian. This was awesome.