← Priorities Podcast
Priorities Podcast
How Utah’s AI policy office is laying the foundation for governance
00:20:38
Zach Boyd, director of the Utah Office of Artific…
Loading summary
Transcript33 lines
- [00:02]
A
Hello and welcome to State Scoop's Priorities podcast. I'm Sophia Foxoa, a reporter for State Scoop. This week I'm talking to Zach Boyd, director of Utah's Office of Artificial Intelligence Policy, about how state and local governments should investigate incidents where generative AI tools inadvertently cause harm and who should ultimately be held responsible. But first, here are the biggest state IT stories of the week. The Department of Justice has delayed the complying in states for its Web content accessibility guidelines, giving states and large cities an additional year to bring their digital assets and content into compliance with the Americans with Disabilities Act. Philadelphia has published a new data dashboard aimed at helping city agencies coordinate efforts to revitalize the low income neighborhood of Kensington. One official said the effort is part of Mayor Sherrelle Parker's vision to create a government the public can see, touch and feel. Veritone, the AI company, announced a partnership with the non profit Cold Case foundation, lending the group new technology that's hoped to help solve old cases. Earlier this month, the Aspen Policy Academy, a non partisan policy training program, released a guide urging state officials to build more formal systems to invest incidents when generative AI tools make mistakes or cause harm, such as algorithmic discrimination in hiring, housing and other government services which can erode public trust. The guide proposes a standardized incident investigation framework modeled after similar safety practices in aviation and healthcare, bringing together government officials, developers and industry experts to explore the root cause and implement prevention measures rather than just enforcement. The framework was specifically designed for Utah's Office of Artificial Intelligence Policy, a statewide agency that operates one of the nation's few AI regulatory sandboxes, which allows the state to test technologies under the close watch of regulators checking for legal and policy compliance. I interviewed Zach Boyd, director of the agency, about the recommended framework and how officials like him are grappling with how to manage the real world risk that come with generative AI. A few weeks ago, the Aspen Policy Academy published a set of recommendations for the Utah Office of Artificial Intelligence Policy, basically recommending that the agency adopt a set of post incident investigative frameworks in order to kind of investigate artificial intelligence incidents, basically when AI tools make mistakes that could possibly cause harm. The recommendations are modeled after the National Transportation Safety Board, which is an independent oversight body that, you know, investigates aviation accidents. They publish reports so that the general public can can view them and they bring together a lot of industry stakeholders. What does the office make of that?
- [03:05]
B
Well, I think it's great. I've worked closely with Michelle over the last few weeks and months to develop this and really appreciate her work it's been really inspiring to see her and so many others come forward at this, at this moment where we're trying to lay the foundations for AI governance and bring ideas from all different fields. That's super, super helpful.
- [03:25]
A
And any of the recommendations or guidelines in the framework stand out to you? I know in her report, Michelle had said that she doesn't think the agency has enough of a preventative measure or even an enforcement measure to kind of look at what happens after an incident happens. What do you think?
- [03:47]
B
Well, on a high level, I have two takeaways that I've remembered in a lot of discussions. First is the overall framing that it's just culture. Right. It's really about a safety culture. Yeah. That goes to the core concern in a lot of the AI that we're looking at. My group is not in charge of government deployment of AI and it's not an enforcer of regulations. It's really a developer of legal frameworks and a manager of some pilots that are really forward facing in the commercial sector, so not in the public sector. And so one of the things we are really worried with about is how to encourage a culture and how to craft a regulatory environment where people are inclined to have a safety mindset rather than a reckless mindset. So that's the top one. And then second, I just view frameworks like this as a resource to help people who want to deploy in the state and to help the industry start thinking about these issues. Because obviously it's not exactly the same as the aviation industry, but there are lessons to be learned and we want to make sure those get learned.
- [04:55]
A
What are the type of lessons you think that you need to learn?
- [04:59]
B
Well, I think one of the main lessons is just this attitude of focusing on what went wrong, not who did wrong. I think that's really, really effective for safety. And then I think bringing in lots of stakeholders from different directions, that's, that's certainly been the philosophy of my group. And yeah, I think it's right.
- [05:20]
A
I'm really glad you pointed that out. Of what went wrong versus who to blame or who to stick with responsibility or penalties when it comes to enforcement of some of these safety laws. Especially because that is a huge question that is getting posed all around the country as multiple states try to come up with an AI framework for, for good governance, especially around these, these safety tools. And so kind of investigating what went wrong. That way it brings together everyone to the table in an environment where you can holistically look at these tools, how it was developed, how it was implemented how it was used. That way you can really take a well rounded look and that way no one feels the pressure to get defensive or, or to clam up or to not even want to be open and come to the table. Is that a type of culture that you're fostering now?
- [06:12]
B
Well, we hope so, right? I mean, it's early stages and the world is big, but definitely we're viewing some of the best actors in this space as being those who are very accountable, who clearly respond to incidents that occur on their platforms and come up with concrete plans to make sure they don't happen. And then there are others who just seem to disclaim responsibility for the design of their products and the effects their products are having on consumers. So that's the distinction. I think legislatively and in rule, the trick is drawing the lines the right way so that people have the right incentives to both try hard, but also to put innovative products out there. You don't want to slow down innovation with, you know, with a level of concern that's not proportionate to the risks.
- [06:57]
A
Right. And what do you think about their recommendation for incident reports to, to be published? So that way it creates more of a culture of transparency between developers, government organizations who are, you know, making, evaluating these tools and the people who are using them.
- [07:14]
B
Yeah, I think that there are different settings where that may work or may not work. You know, I'm thinking of our sandbox participants in the state. You know, these are companies that are doing very forward looking deployments and they really do have a clear burden to establish both public trust and trust with the regulatory agencies of the state. So they, you know, I think we absolutely expect them to maintain the infrastructure before incidents so that there's auditability and traceability of key decisions and then to be accountable after the fact if there is ever anything that goes wrong and, and hurts someone. In terms of the general, you know, public deployment of AI, that's really broad. I don't know if it's appropriate in all sectors or not. But I, you know, I think it's definitely worth looking at really closely.
- [08:05]
A
Can you tell me more about the sandbox that the agency is operating?
- [08:09]
B
Yeah, so this is a unique program to Utah, or almost unique. And what it is is we have a mandate from the legislature to accept applications from companies that want to do really forward facing deployments of AI that may conflict with exist existing regulation or have no clear compliance pathway with respect to existing regulation. And basically we can give limited waivers from existing law to allow deployments in these Regulated sectors like healthcare, education, transportation, financial services. And the idea is to have the state partner with some of these companies, create reasonable deployment plans that we think are proportionate and responsible and let them run with it for a year or two or three. And at the end of that, the state hopefully has learned enough to create long term durable regulatory pathways. So there's a certain amount of risk involved, there's certainly a possibility of benefit involved, but there's a partnership that's required with the regulatory agencies.
- [09:12]
A
Yeah, as you mentioned, this type of experimentation does seem really unique to Utah. There are several states that are obviously piloting AI tools in the public sector, but with very limited capacity, given the regulations that are out there currently or because in the absence of regulations, they don't want to deploy them widely beyond a certain agency or a set data set. Whereas this case, as you, as you mentioned, they're allowed to kind of have a waiver and be able to have a little bit more reign and freedom to be to explore some of these tools. What have you seen that's been successful in the sandbox?
- [09:50]
B
Well, it's early days, so we've granted several pilots. In health care especially, that seems like where the state has a lot of rules around workforce and who's allowed to do what. And you know, we see a lot of potential for AI to increase consumer surplus there. But yeah, I think what's been successful is just really involving third party experts in the development of the proposals. And when we grant a pilot having staged deployment so that you have tight human oversight at first and then kind of decrease that as trust is established in the different applications. But you know, we're still learning. That's the, that's the point. I guess the whole goal of the sandbox is to maximize our learning. And if you don't let anything through, you don't learn. And if you let everything through, you also don't learn because you just are, you're just harming people and not tracking stuff. So we're trying to strike this happy medium of letting things through that we can learn from as much as possible.
- [10:48]
A
You mentioned healthcare is a big one that you're allowing a lot of pilots to go through. Are there any other industries where you're seeing a large surge of applications that you're reviewing?
- [10:59]
B
Yeah, and I'll emphasize there are not that many that have been granted so far, although a lot more in the pipeline. But yeah, healthcare is the biggest one. We've seen some interest in education, financial services, transportation, real estate, and maybe a handful of other Industries. But. But it's really where the existing state regulation is. What the people are coming as applicants. Right.
- [11:22]
A
Mm. And I mean, healthcare and education, those are two really big sectors that deal with a lot of sensitive data. And especially with compliance rules changing under HR1 in the healthcare industry with like Medicaid and things that people can apply to. Do you see some of the applications coming through that are going to address some of those compliance measures.
- [11:45]
B
I'm trying to think not so many compliance requests directly, especially because that's federal law and we don't, you know, as the state. We can't waive a federal law. The state's law. The state's law tends to focus more on scope of licensed practice. Who can do what, what are the qualifications required? When does a human have to be in the loop? So, you know, we're seeing applications where maybe you needed a very highly trained professional, like a dentist, but now with AI Assistant, maybe a dental hygienist could do it at a much lower cost, greater accessibility, greater ease, or some cases where you might want to go directly to the consumer with some kind of autonomous care app or something like that. And we're kind of exploring what that might look like for the state, but currently there's no regulatory pathway to do that. So we need to develop something.
- [12:30]
A
I see, I see. And I want to go back to where AI can make mistakes that cause harm. I wanted to follow up, especially because you had mentioned that maybe not all industries would require some of these tools, or these tools are not best for. For a few of these industries. Can you name some of these industries or some of the harms that AI might cause within them?
- [12:50]
B
Well, I would like to think a little bit longer before I say that any industry ought to be exempt from this kind of thing, because the more, you know, the more you delve, the more you learn that you don't really understand, but just some general perspectives. I think the state is more concerned about direct to consumer transactions versus business to business transactions or versus tools that are internal to an organization. You know, if a private company has a tool that's less reliable or not so concerned versus if you have a tool that's cheating consumers in their transactions, that would be more concerning to us. So that's kind of the general idea. And some things are really low stakes, some are high stakes. You know, we treat healthcare very differently from, you know, taco stands.
- [13:35]
A
Yeah, of course, there's no. There's no risk of harm in terms of bias or discrimination, I can imagine in hiring practices as well, there's a lot of scrutiny that, that comes with that in terms of the sandbox as well. I know you're exploring this waiver policy where people can, or developers can apply and receive a waiver for to waive regulatory so they can have more innovation. Is there anything else the, the office is exploring with regards to its sandbox or any new projects that are coming up?
- [14:07]
B
Well, that's, that's the sandbox really mostly that's responsive to industry demand, which right now has been growing. The other thing that we do is just policy development. We help develop policy and regulations that the legislature could pass or adopt or you know, modify if they see fit. I think this year we're seeing a big need for legal clarity around AI agents. You know, what does accountability look like, what does liability look like? Do we need to modify contract law? If agents are making agreements with each other on our behalves and you know, there's digital identity, you know, how do you know that an AI really has a delegated authority from a real person? So anyway, there are lots of questions around that that we're probably going to explore in the coming months.
- [14:51]
A
What have you learned so far? Because this comes back to, you know, a what went wrong as opposed to who is responsible. When you're looking at accountability, what are the factors that you, you look at?
- [15:02]
B
Well, we're still really early stage, so at this point we're mostly in a listening mode. So some of the things that we're seeing, you know, there are some consumer protection concerns about unaccountable bots going out on the Internet and you know, doing things that could be harmful to people, whether that's deceiving them or something like that. So we need to make sure there are lines of accountability there. On the other hand, you know, we are hearing from innovators that they just, they just need clear liability standards. Right? They need to know what will be their fault, what will be their customers faults, what's their supplier's fault. Just the clarity in the environment because there's not that much norms that's been set so far.
- [15:42]
A
What I'm hearing and I'm visualizing in my mind, it honestly sounds like a really big circle. Like the government needs to obviously issue some policy guidelines and some guidance in terms of clarity with this before innovation and developers can, can happen and before they can issue it to consumers. They need to have those, that guidance and that established so that there's transparency with their consumers as well. And then the consumers want to make sure the government is also Having good governance. So it feels like one giant wheel of an ecosystem where it's. Everyone is relying on the other to come to the table in good faith.
- [16:20]
B
Yeah, I think so. I mean, our philosophy is that no one's going to have the whole answer, and we have to be bold and put out some attempts first, but then also be listening really carefully to understand the consequences of our decisions and make modifications rapidly as we get feedback from the business community, from consumers, from voters, from everyone in the whole ecosystem. Right.
- [16:42]
A
Mm. Mm. And I'm sure you're obviously in close contact with government officials and industry leaders. Have you heard from consumers or residents about what they're clamoring for?
- [16:53]
B
Oh, I mean, it's a really mixed bag. I think right now a lot of people are still very concerned about AI. They're concerned it will take their jobs. They're concerned it will. You know, they're. They're concerned if. Is this the beginning of a Terminator movie? They're concerned the economy might crash. Right. So I think a lot of people just have a concern, but we are seeing demand. I think a lot of people would be really excited for real proof of how AI can enhance education. For example, the dream of a personal tutor that is AI enabled and available for you is a big deal. We're seeing huge demand in mental health support for people who really see this as a potential expansion of the current care system. I mean, there's just a ton of applications out there, but we'll see over time which innovations turn out to be durable. You know, I think the government's pretty bad at figuring out what the market's going to do ahead of time. So we're just kind of watching and learning, too.
- [17:51]
A
Yeah, but even. Even then, that can be. That can be tricky to be constantly reacting as opposed to getting ahead of it. But it's such a rapidly evolving technology. No one really knows where it's going to go next.
- [18:04]
B
Yeah, I think mostly the state is looking at making sure that we're creating the right foundational rules for everything to develop along and also being mindful of our workforce. We don't want people to be left behind, and, you know, that's. That would be a huge problem. So we're thinking about how can we make sure our educational system is preparing people for opportunities, and that when people, you know, do have new career opportunities, they can capitalize on them, or when they. There's a job displacement, they have places to go. So that's. We're kind of thinking about the system as a whole. And trusting in people's individual innovation to, you know, see to create and seize certain kinds of opportunities.
- [18:45]
A
I actually, two weeks ago I. No, was it two weeks ago? I think it was last week I saw there's a new documentary out called the AI How I Became an Apocalyptomist. And it kind of goes exactly to what we've been talking about, especially when it comes to users. The premise is that the, the narrator, the host, is about to have a baby with his wife and he is really scared about the world that his baby could be born into with AI and tools and especially getting conflicting narratives of it's going to bring about the apocalypse like the next Terminator or it's going to create a utopia where everything is going to be solved. Climate change, cancer research, all of our problems will. And so he interviews a ton of different experts and analysis and company owners and CEOs to try and get to the bottom of his question. And where he he lands is somewhere in the middle. And that's kind of where I feel like we all are right now. We are somewhere in the middle between like we're on the fence of is AI going to be this force for good or is this AI going to be a force for, for harm and disruption?
- [19:57]
B
Yeah. I mean, we're trying to own our own influence here, right? We can't control everything in the world, but we ought to do our part and we ought to be being as proactive as we can. But yeah, no one knows the future, right?
- [20:09]
A
Thank you to Zach Boyd for participating in that conversation. You can subscribe to the priorities podcast@priitiespodcast.com and wherever you get your podcast. While you're there, be sure to leave a review or rating on the podcast page. That small extra stick step helps more people like you find the show. This podcast is a production of Scoop News Group in Washington, D.C. adam Butler and Carlin Fisher help put it together until next week. I'm Sophia Foxoell. Thanks for listening.