← The AI Policy Podcast
The AI Policy Podcast
Three More AI Hacking Incidents, and a Push to 'Pace the Frontier'
00:44:56
In this episode, we touch on Texas' new verification and audit requirement for data center developers seeking connection to the state's grid (1:11) before unpacking Anthropic's disclosure of three incidents in which its models hacked into another company during internal cyber evaluations (11:02).
Loading summary
Transcript26 lines
- [00:00]
A
Foreign.
- [00:11]
B
Welcome back to the AI Policy Podcast. I'm Alok Mehta, director of the Wadhwani AI center here at the center for Strategic and International Studies.
- [00:20]
A
And I'm Nicole Herrera, a researcher with the Center. Today we're following up on our last news roundup coverage of the OpenAI hugging face cyber incident, in which several OpenAI models escaped containment and broke into Hugging Face's production infrastructure during internal cyber evaluations. We've got a lot to talk about on that front, including the revelation that OpenAI's competitor, Anthropic, experienced three similar incidents with its own Claude models, plus a petition from over 1300 Frontier Lab employees calling for the US government to help pace the frontier, and an open letter from Nvidia on the merits of open weight models. But Alok, before we dive into all that, there are two smaller stories I really want to make sure we have the chance to discuss today.
- [01:08]
B
All right, great. Let's get started.
- [01:10]
A
Great. And the first story is that on August 3rd, Texas Governor Greg Abbott issued a directive to the Public Utility Commission and Electric Reliability Council of Texas to conduct a comprehensive verification and audit of all data centers currently seeking to connect to the state's electric grid. Under this new order, each data center developer will have to provide information about any financial assistance it expects to receive from the state. The data center's projected power and water consumption measures the developer plans to take to reduce impact on the local community and who the controlling interests are in the project. Abbott said that the Electric Reliability Council is currently processing around 474 gigawatts of requests to connect to the Texas grid, which is more than five times the state's record peak electricity demand, and that approximately 90% of those new power requests are for data centers. Any data center developer that fails to go through this new auditing process will be denied connection to the grid. But some critics are saying that Abbott's directive doesn't go far enough. For example, Texas Department of Agriculture Commissioner Sid Miller said that without legislative action, this directive is all hat and no cattle, empty political rhetoric wrapped in meaningless fluff. So, Alok, what's your take on this new verification and audit requirement? Is this just a case of checkbox compliance, or will it translate into meaningful accountability for developers looking to build data centers in Texas?
- [02:50]
B
Yeah, so this is a really interesting development. I think before we get to the sort of specifics of what's happening in Texas, I think one of the interesting things to note here is sort of what it signals about the larger data center conversation happening in this country so, you know, just a couple of weeks ago on this podcast, we talked about the fact that New York State has put a moratorium on data centers. So that is a very blue state. Here we have Texas, a very red state, historically known as one of the most business friendly states in the country, perhaps the most business friendly, doing something similar. So this is not exactly the same as what New York is doing, but it has a lot of similar elements to it. And so I think this shows the extent to which this data center backlash, we've talked about it numerous times on this podcast and in sort of our various public events and reports that this extends across political divides. It's a red state and a blue state thing. And it shows the depth to which the American public is sort of concerned about data centers going up. I think it also shows increasing concerns by governors about how they can leverage all this money that's going into the data center industry, how they can leverage that to improve the infrastructure that is being built in their states, including a backlog of sort of existing deferred maintenance and things like that relating to water and power utilities, as well as sort of minimizing public impacts, community impacts, because that has a direct connection to sort of how they think about their constituents and how they think about their own political prospects. That being said, I think here there is probably, you know, two things happening. One is that the fact that the governor's taking this action is a pretty strong signal about the level of concern that has risen up to the top levels of Texas government about data centers, and that this sort of demand for electricity that is significantly exceeds the state's capacity is a real issue. But also that there is probably a limited amount of things that the executive branch in Texas can do without legislative support. And so I think that there's some signaling happening here and that the most likely thing that will happen is that this signaling will both change how data center developers think about sort of building in Texas and whether Texas is the right place to build, or if they should think about other locations, but also likely to spur some more intensive legislative action on the part of the of the Texas lawmakers about sort of coming up with a comprehensive legislative approach with teeth to the data center issue and making sure that they're minimizing impact.
- [05:45]
A
Right. That makes a lot of sense. And the second story that I wanted to talk about today is a Bloomberg article published on August, August 4th, the day we're recording this episode. And that article is titled China's AI Blitz Creates Death Zone for Rival US Model Makers. And this title is referring to a chart produced by Artificial Analysis, an independent AI benchmarking and analysis firm that plots a bunch of AI models on two axes. So on your X axis you have cost per tax, and on the Y axis you have the model's intelligence index. And I realized this chart might be a little difficult to visualize for those of our listeners who aren't watching on YouTube, so I'll try and keep it relatively simple. Essentially, if you're an AI user, your ideal quadrant on this chart is going to be in the top left. So where you have very high model capability on the Y axis at a very low cost on the X axis, you're going to avoid using models that fall into the bottom right quadrant, where you have a lower capability on the Y axis at a higher cost on the X axis. And this bottom right quadrant is what some experts are calling the death zone for frontier AI developers. Because if your newest model is more expensive and less capable than, say, a open weight model from a Chinese lab, you're going to lose customers very quickly. And as open weight models become cheaper and more capable, that death zone is going to just keep expanding. So, Alok, what can you tell us about this chart and the broader implications for AI policy especially? And we'll talk about this more as part of our main topic for this episode, but as there's a growing camp of AI experts and industry leaders who see open weight models as a critical input for AI safety.
- [07:38]
B
Yeah, so I think a couple of interesting things to note here. So, you know, traditionally when we hear people discuss the efficiency of AI models, they often talk about sort of token efficiency, and in this case, we're talking about cost per task. So I think that is getting close. Closer to what people care about in the real world is not how much a particular token costs, because a token is kind of an abstract AI thing that doesn't really correlate with real things, real artifacts. In the real world, what people care about is how much money will it cost me to get this thing done? And so I think that this is closer to measuring the things we really care about. When we talk about how efficient our particular models in terms of the argument here, I mean, death zone, I think, is a little bit dramatic. What we know is that measuring AI capabilities is really complicated. There is a real sense in which some of our benchmarks are saturated, so they don't tell us good information about what models are better than others, and that in other cases, you can take steps to sort of optimize how your model performs in a benchmark, but it doesn't necessarily reflect in the real world. I think there is some truth to this idea that Chinese competition will make certain kinds of model products difficult and it'll change the economics of those products. And yes, certainly if you are offering a less capable model at a higher price, that is generally going to be a difficult market segment to be on. But there are, I think there are instances where that product might make sense. Right. So if this is a product that has associated infrastructure around it, if the tooling is really good, if the interface is really good, if you have sort of a bunch of information associated with that company and so you can port that memory over, it might be useful. And then there are things like cybersecurity guarantees and cybersecurity practices that might mean that you are willing to pay more money for a less capable model because of things like the level of security practices, the ability to handle classified information, the level of uptime. And so I think this is complicated. I think it's directionally telling us something important, but we'll have to see how it plays out. I think the other thing to keep in mind is we, we don't know, or we don't really fully know how much Chinese models are being subsidized. US Models are being subsidized by venture capital money as well. And so it's a little bit difficult to get at the true cost per token. And we also don't know in the future, will China's lack of compute really change the calculus for the kinds of models it's able to serve? So we'll have to see. We know that export controls are affecting their ability to get access to compute, and we'll see if that changes their ability to do inference. Right. To serve models for customers in the future in a significant way.
- [10:54]
A
Yeah, absolutely. And thank you for unpacking that. But now I want to move on to our main topic for today's episode, which is that over the past few weeks there's been a ton of stories centered around AI and cybersecurity. And this was kind of kicked off by Open AI's disclosure, which I mentioned briefly at the top of this episode and which you and Matt discussed in greater detail in the last news roundup, that several of its models had broken out of their internal testing environment, gained access to the Internet, and hacked into Hugging Face's production infrastructure. Now, the second big revelation came nine days after that initial announcement from OpenAI, when Anthropic shared that it had also discovered three of its own Incidents in which a CLAUDE model hacked into another company during internal cyber evaluations. In a blog post on July 30 titled Investigating three real world incidents in Our Cybersecurity Evaluations, Anthropic said that following OpenAI's disclosure, it had begun a large scale retrospective review of its own cybersecurity evaluations in which it looked for evidence that Claude, like the OpenAI models, the accessed hugging face was able to access the Internet from, from within testing environments that should have been sealed off. So Alok, walk us through what happened here. What exactly did Anthropic discover in its review and how do these three incidents compare to what happened with OpenAI?
- [12:24]
B
Yeah, so obviously we talked a lot last episode about what it means for a model to escape containment like this. The fact that it's happened at other companies suggests this is a more widespread issue and raises the alarm level, which I think was already pretty high considering just how sci fi feeling that previous incident was. I think in the Anthropic case we have two pretty significant differences. So one is in how these incidents were discovered and then the other is related to how they happened. So for Anthropic, this was done because they read about the OpenAI incident and they decided to go back and review their logs and, and so they found it through this retrospective review, which suggests that, you know something, they have this data available and they found it by scanning their data, but it suggests that their processes for detecting these in real time are not working at least the way we would hope in the AI industry. So, you know, I think it's really good that they found these, that they're engaged in public transparency about the process of finding them and what happened in these instances. But I do think it suggests that we probably have improvements to make in the continuous detection and evaluation process for testing models. I think the second interesting thing here is that there's a pretty significant difference in kind between what Anthropic discovered happened and what OpenAI discovered. So in Anthropic's case, it was essentially seems like a configuration mistake where Anthropic is working with a third party, a third party evaluation firm who was setting up an environment and there was a misconfiguration that allowed the models to reach the Internet. In a lot of ways, the OpenAI incident is more concerning because it involved the model essentially, after being given its instructions, detecting a novel vulnerability that allowed it to circumvent the sandbox environment and gain access to the Internet. So there was a much more sort of directed action happening in this particular circumstance. In both cases, I think what we've seen is that there was a fairly open ended prompt. So basically for anthropic it was like a capture the flag challenge or sort of a scenario set up. It had to find some piece of secret information and then sort of a pretty open ended prompt and a pretty open ended ability for the model to sort of do what it needed to do to sort of find this particular flag or piece of information with not a lot of specificity on methods or means or anything like that. And I think this was similar to the OpenAI incident and that there was also an open ended prompt. I mean it seems like the fact that if you don't provide a lot of specificity to the model that there's a greater likelihood of this kind of behavior happening. I think this suggests that like we mentioned last time, there may be some utility in thinking about how we do our sort of evaluations, whether we should use open ended prompts like this. And if we do, I think there's useful information you can get about capabilities, whether there should be additional safeguards put in place when you're using open ended prompting.
- [16:15]
A
Yeah, and even before this story broke, we were actually planning on returning to that original OpenAI hugging face incident in today's episode because new details about that incident have emerged since you and Matt covered it two weeks ago. So can you give us a quick update on that front? What new information have we learned about OpenAI's model escaping containment?
- [16:39]
B
So you know, there is some more technical information about what happened. This includes sort of a write up from Hugging Face. And we're hoping here at the center to do a little bit more of a deep dive and talk through sort of the technical details. So stay tuned for that. We're hoping to release some findings or maybe do some public events on that in the future. But in the meantime, I think there are some interesting sort of both industry and policy developments coming out of this incident. So the first is that there is now an independent review planned for this incident. So Meter, which is one of the leading independent frontier capabilities firms, has reached an agreement with OpenAI to conduct an independent review of this incident. They're going to be working with Redwood Research with another well known firm in this space and that the plan is for them to publish sort of a blog post that lays out some of their the details of how they engage with this, what they did, the scope of the investigation and some of their conclusions. And so we have some details about how they're going to go about doing this work. But I think we should commend the companies for the level of transparency that they've both disclosed in terms of what happened and their willingness to engage with independent evaluators for more deep dives into what happened. This is very much in line with some of the things we've been talking about on this podcast in terms of the utility of independent verification and what that signals to the public in terms of how much they should trust the model, but also the fact that they can often provide unique kinds of information and unique kinds of expertise that might not be. That might not reside in the labs. And so I think that there is a lot of utility in this approach as well. We have also seen some policy responses to this incident. So this includes interest from state attorney generals. So, you know, just, I think Yesterday, actually, about 15 state attorneys sent a letter to OpenAI and to the CEO Sam Altman, telling you to preserve relevant documents and I think, maybe more interestingly, halt certain internal cyber evaluations. The letter flags concerns about potential violations of state and federal law, including consumer protection and data privacy statutes, which is something that attorney generals and states are charged with enforcing. Sometimes they do this alone, sometimes they do this with the federal government. And so, you know, I think some of their asks are in line with things that OpenAI said it would do, particularly around the halting of evaluations. But I think that it is quite interesting that the state attorney general. I mean, it's not interesting that state attorney generals are taking interest in this. I think the fact that they are using some of the existing laws on the books is totally in line with stuff we've talked about before, which is that time and time again we've seen that there are lots of existing laws that can apply to AI in some way, and that when there is an incident like this, there's a lot of sort of creative use or sort of people looking deeper at the existing statutes and figuring out how they can take action. So I think this is another instance of people looking at the existing laws and saying, hey, there are existing statutes that cover this kind of activity, and we're going to explore the authorities we have under those statutes to protect consumers and prevent this kind of thing from happening. Again, that's at the state level. We have seen interest at the federal level as well. So we have some representatives also writing to OpenAI, requesting more information about this, including raising questions about the environment in which OpenAI is securing its models and testing its models, and questions about how models like this were able to escape those Containment processes.
- [21:19]
A
Yeah. And something that I found pretty troubling when I was reading some of the reporting on these incidents is that it sounds like this wasn't the only time that something like this has happened at OpenAI. One Time article cited an OpenAI staffer saying that externally, this being the. The hugging face incident feels like a big warning shot, but internally related incidents have been happening for a while, so that's something that's pretty concerning. And we're also seeing a level of concern from inside those frontier labs. There's been a petition now signed by over 1,300 employees of leading frontier labs in the U.S. calling for the federal government to help develop mechanisms to pace the frontier of AI development. So can you talk a little bit about that petition and what it kind of tells us about the current state of the AI industry?
- [22:16]
B
Yeah, it's really interesting, you know, 1300 employees, Frontier Labs. The specific ask here is for the US Government to help figure out a way to pace the frontier of AI development so noticeably. Right. We request the US Government to support an international effort to develop technical and governance tools needed to deliberately pace the frontier of automated AI development. We've seen a lot of notable sort of figures in the AI industry sign this. This included the Anthropic CEO Dario Amade, you know, high ranking officials from OpenAI and Meta, and Google DeepMind. So that includes the OpenAI chief scientist, Jacob Pachowki, Meta's AI chief scientist, Google DeepMind's co founder Shane Legg, OpenAI's head of strategic futures, Dean Ball, who recently joined OpenAI. Sam Altman didn't sign it, but he has mentioned similar concepts in some of his public statements, including a podcast on July 31. I think the challenge here, right, is that I think that oftentimes what I describe here is we're in this kind of prisoner's dilemma. So even if there's broad agreement that developments in AI are happening too quickly, and then maybe in the cybersecurity space, this is the most concrete manifestation of that happening too quickly. If any individual company sort of decides to stop development on their own, they'll essentially be putting themselves in very precarious financial circumstances, possibly or probably bankrupting themselves. And so it's very difficult to envision a mechanism where one company is able to do this unilaterally. So you have to have a lot of companies do it at once. But then you have a second layer to this, which is, let's say that we can get all the US companies to sort of sign up to this Then we have international companies who are also developing this. And then the same sort of circumstances apply, which is that if the US does this unilaterally and China doesn't, then China will continue to make advantages, take more market share, and sort of the US will sort of drive itself into possible irrelevancy. And so the real challenge here is essentially the challenge we've had in the AI space for a long time, which is that we can't pace the development of AI until we figure out an international approach to AI that includes the us, China, and a bunch of other countries where there's progress happening near or at the frontier of AI. And that's a really difficult thing to envision happening. Perhaps we have an upcoming summit where the US and China are going to be talking. AI is supposedly on the agenda. And I think that it would make sense, given all the recent developments we've seen in the cyberspace, to think about that as an opportunity to have some discussions about whether we can come to some sort of agreement, some governance agreement that would allow us to start taking steps towards slowing things down. But I think it's really hard. I think that there's very likely lots of other things that the US and China will want to talk about at that summit. And so we'll just have to see what happens.
- [26:04]
A
Yeah. And something else that's interesting about this petition, something I've seen pointed out by Neil Chilson, specifically on X of the Abundance Institute, is that, and I'm quoting here from Neil, to the extent there is a collective action problem, it's not between employees of the Frontier AI Labs, but between companies and countries that want the ability to slow down. And Neil is kind of raising the question here of why wasn't this petition being led by the Frontier Labs themselves? Why is it being led instead by employees at these Frontier Labs? I don't know if you have any thoughts about that.
- [26:43]
B
I mean, you know, it's hard to say for certain. I do think that, you know, there are different circumstances that apply to the companies versus the employees of the companies. So, one, we know that because of how competitive it is for the top tier of AI talent, that that talent is often given a high degree of discretion and the ability to make public statements that maybe is not true for other industries. And so they're allowed to say what they feel. And so they might be more vocal than we might see in other industries. But, you know, companies, they have had, you know, tens or hundreds of billions of dollars in investment, and they do owe, you know, they've made certain commitments to those venture capitalists who have, who have funded them. And so I think that that does put some constraints on their ability to do things. Even though I think, you know, what we've seen is that OpenAI is very upfront that it has this complicated governance structure where its non profit makes many of the controlling decisions for what the company can do. And Anthropic is organized as a public benefit corporation. And so they've told a lot of investors that we will sometimes make decisions that are what we think are best for the world and not necessarily what's best for the company. I still think that there are more constraints on their ability to sort of signal things that might put the companies at existential risk than is true for individual employees. So I suspect that plays a little bit into it.
- [28:23]
A
Yeah, and that makes a lot of sense. Now, zooming out a bit from this petition specifically to this question of AI safety and cybersecurity in general, I'm curious, how are people in AI policy and the cybersecurity communities responding to these revelations? First the incidents disclosed by OpenAI and now by Anthropic.
- [28:48]
B
Yeah. So I think it is definitely be interpreted by a significant part of the AI community as this is a harbinger of things to come. So I don't think anyone thinks that we've reached some sort of high watermark and that incidents like this are going to become less common. In fact, I think this really raises the, the concern that sort of as models become more capable, that they will increasingly engage in things that feel like they're coming straight from science fiction. And that right now we're not seeing any significant slowdown in how models are advancing and we're not seeing a circumstance where the AI tools have improved our cyber defenses so much that this is no longer a concern. If anything, it feels like the ability for models to find vulnerabilities and evade protections and escape containment measures is increasing faster than we know how to use these tools to sort of stop those sorts of things. And so I think this is raising some debate about what is the root of the problem. And, you know, where should we invest resources or where should we think about policy interventions? And I think there are two broad camps here. I don't think they're mutually exclusive and almost certainly both of these are true to some extent. So the one is saying this is just a basic cybersecurity issue, that if we had better sandboxes, if we had better cybersecurity, if we were employing best practices consistently across the industry, then something like the OpenAI incident would not have happened because it wouldn't have been possible to escape the sandbox. And so what we really need to do is figure out how to be better at detecting vulnerabilities and fixing them, patching and really making sure that the systems we put these models in are really buttoned up so that there's no possibility of this happening. The other camp is that this is, you know, perhaps a foolish thing to try to do because historically we've seen that trying to make a perfect system that is perfectly secured is really, really difficult. And that was even before the era of really powerful tools that could scan and find vulnerabilities quickly. And so the thing we really need to do is invest in alignment, that is making sure that we are fundamentally programming these models to do the right thing for some definition of right thing to follow, user intent to not engage in things that are problematic, that could lead to harm, either financial harm or physical harm. And so what we should really do is invest more resources in the kind of basic research that would allow us to make progress on this alignment issue. And sort of what that means is that we would funnelly have models that we could give open ended instructions to and they would be like, we're going to do the best we can to solve this problem, but we're not going to do it in a way that sort of escapes our, sort of contravenes our instructions or potentially put systems at harm or that does things that we sort of suspect are clearly unauthorized or unintended by the people who gave us those instructions.
- [32:27]
A
Right. And there's an interesting link here from the between the cybersecurity camp, the people who think cybersecurity is the core issue here, and then the people advocating for more open weight model development. This is coming from Hugging Face CEO Clem Deling. Hopefully I'm saying that correctly. After the incident with OpenAI's model escaping containment, he said that AI safety won't be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender everywhere. And that's kind of a reference to the role that open models played in Hugging Face's response to the incident. And also kind of wrapped up in the midst of this all these stories. On July 24, Nvidia published an open letter to US policymakers titled Open Weights in American AI Leadership, in which it makes the case for open weight models as the foundation of an AI driven economy. And part of Nvidia's Core argument here is that open models help strengthen AI safety while closed models jeopardize it. The letter states, quote, openness may be one of the most important paths to AI safety and security. Relying solely on closed models is not inherently safe, as they cannot be. They cannot be breached, misused or fail in, in ways that outsiders cannot detect. And concentrating advanced AI capabilities behind a small number of closed models compounds that risk. It results in a small number of single points of failure, weakens competition and leaves critical technology in the hands of a few providers. So I'm wondering if you can speak a little more about that connection between AI safety and open weight model development. How much merit do you think there is to that argument that Nvidia is laying out?
- [34:25]
B
There's certainly merit to this idea that open weight models, because they're open, they're widely distributed, allow for this, this collective body of knowledge to form. These models often get more scrutiny than closed models. So there's definitely an element of truth to this. But there are sort of unique risks posed by OpenAI models as well. And there's a real sort of balance here. And I don't think that we have currently figured out a good way to empirically assess this. Right. So we're sort of muddling our way through. A lot of this will depend on what the overall trajectory of AI development is. Some of this will depend on specifically how open weights, how these models develop, and then some of it will depend on how we figure out how to deal with the cybersecurity issues raised by models. And in particular, there is this argument that as AI models get better, that they'll help defenders more than attackers and they'll shift the balance away from attackers towards defenders. I think that's an argument that's been made, but maybe there's increasing skepticism that's actually how it's going to play out. And I think all of these are important to figuring out what the overall impact of open weight models is. But basically the argument is you have an open model, you will definitely be able to scrutinize it more, more people will be able to look at it, more people will be able to interrogate it in really, really deep ways. You can fine tune it, see how to circumvent safeguards, all sorts of ways of scrutinizing that model. And a lot of that can and will be published publicly. And this is very different than closed models, where because they're closed, they're served through APIs, companies will be able to control how they're used to some extent. Both through technical measures and through terms of service measures. And that oftentimes they'll sort of make agreements with organizations to test those, but not all that information will be publicly released. But there's a flip side here, which is, you know, typically open weight models, they come with fewer safeguards built in. Those safeguards can often be sort of removed. The safeguards that do exist can often be removed relatively easy, with very little data or technical knowledge or token use involved. That one of the concerns we're really thinking about here is that there are, there's a possibility that open any model can have sort of an asymmetric backdoor that allows you to get it to do things you don't want, like a form of jailbreaking that allows you to like bypass all the safeguards that's in the model and that those can be programmed in and very easy to activate if you know the right way to do it, but very hard to detect if you don't. You know, you might think of it analogy to sort of like cryptography, where it's very easy to decode something if you know the key, but if you don't know the key, figuring out is incredibly difficult. And if that kind of thing exists, we don't have a lot of evidence for it right now, not a lot of concrete evidence. But if that sort of thing exists, then even if a model is released openly and millions of people are testing it, they may just not be able to find that functionality. And if that's the case, then you run into the biggest issue with open models, which is that once they're out, they're out. And you can't do anything to control who has access to them or to take steps to sort of, if they have a really dangerous capability, stop people from exploiting that dangerous capability. What we saw previously is when the US Government implemented an export control on Anthropic and said you can't provide your model to foreign nationals, they were able to go and flip the switch and turn that off. And so then people couldn't access it. And that's because they controlled it. But you couldn't do that with a model that's available on the Internet for anyone to download. And so this is the trade off that we're really trying to think about and deal with. And like I said, I don't think we necessarily have figured out a good way to trade those off yet. And so we're sort of muddling our way through.
- [39:08]
A
Right. Well, this is the AI policy podcast, so I'd like to wrap up this conversation by talking about policy. Could you walk us through what are the AI policy implications for all of these stories?
- [39:25]
B
Yeah. So I think there are a couple of things that we should really think about this, and that is that, you know, like, for me, the issue of open models has always been one of the really difficult, maybe the most difficult policy issue that governments have to grapple with. And one of the ways I'll have to grapple with it is sort of the various frameworks they have for testing and release of models will need to incorporate open models in some way. As long as those models are sort of weaker than the frontier, not on par with the most powerful models. We probably didn't need to address it explicitly. But the closer they get to the frontier, the closer or the more deliberate we'll need to be about whether there should be any exceptions to the kinds of frameworks we're thinking about relating to testing and release of frontier models generally. And so what that means is,
- [40:29]
A
for
- [40:30]
B
example, the White House is sort of working on and supposedly finalized a volunteer frontier AI model review process. It's easy to think about how they could include open models in that process. It's harder to think about, well, what happens if a model is released and then we find some significant concern about it. What do we do then? So I think that this is going to cause a lot of places in the US and around the world to think a lot harder about what it means to regulate open models and what you can do to sort of control access to them or make sure they're safe, because once they're out, they're out, and there's no sort of putting the genie back in the bottle. I think the other things that we should really think about, these are going to extend on things we talked about in the past. One is that a lot of the frameworks we're thinking about for regulating AI really have to do with models that are about to be released to the public. And it's very natural to do this because there is a very defined point at which you want to test the model. It's easy to figure out what checkpoint you should be thinking about. The timelines are more clear when it comes to these internal deployments. Right. Some of these models may intended to be released publicly, but that's not a guarantee. Right. Some of these models could be for internal testing only, or they might be intended for use only within the company. And now I think we have to think a lot harder about extending our regulation or test how we're thinking about testing to happen much earlier in the process and to include some of these internal only models. And it turns out that this is a lot harder to do. Right. You have a lot more questions about when you want to do this and what models would fall under scope and how the government who might be interested in these models would even know they exist. A lot of these are going to implicate proprietary information. There's going to be a lot of experimental models. We could be thinking about a far greater number of models than are implicated when you're just talking about models intended to be released to the public. And so this is an issue, I think, also where we're going to see a lot of discussion and a lot more sort of exploration of various policy options. And then the third thing has to do with the fact that Hugging Face because of the various safety controls placed on models from OpenAI Anthropic had to do some of its cybersecurity testing around the OpenAI incident using Open models, using Chinese models, because they didn't have the same safeguards in place. And so those models didn't get overzealous and say, hey, you mentioned the word cybersecurity and therefore we're going to refuse to answer this question. And so I think we should also be figuring out how can we get trusted access, trusted actors, access to the types of models that would allow them to improve their cybersecurity practices. So really like segment out, there's a sort of general capabilities that we don't want available to a broad segment of the population because that's just too much risk Surface. But there are definitely organizations that we trust and that have very serious and significant cybersecurity issues that they want to proactively address. And how can we get them the right tools to allow them to improve their cybersecurity practices?
- [44:19]
A
Right. Well, that feels like a great place to wrap up. Thank you Alok for your insights and thanks as always to our audience for tuning in.
- [44:29]
B
All right, thanks. Thanks for listening to this episode of the AI Policy Podcast. If you like what you heard, there's an easy way for you to help us. Please give us a five star review on your favorite podcast platform. Subscribe and tell your friends. It really helps when you spread the word. This podcast was produced by Sarah Baker and Matt Mand. See you next time.