← The Lawfare Podcast
The Lawfare Podcast
Scaling Laws: Renée DiResta and Alan Rozenshtein on the ‘Woke AI’ Executive Order
00:46:48
Loading summary
Transcript69 lines
- [00:01]
Unknown Host
The following podcast contains advertising to access an ad free version of the Lawfare Podcast. Become a material supporter of lawfare@patreon.com lawfare that's patreon.com Lawfair also check out Lawfare's other podcast offerings, Rational Security, Chatter, Lawfare, no Bull, and the Aftermath.
- [00:32]
Unknown Advertiser
I'm no tech genius, but I knew if I wanted my business to crush it, I needed a website. Now, thankfully, Bluehost made it easy. I customized, optimized, and monetized everything exactly how I wanted with AI. In minutes my site was up. I couldn't believe it. The search engine tools even helped me get more site visitors. Whatever your passion project is, you can set it up with Bluehost with their 30 day money back guarantee. What have you got to lose? Head to bluehost.com to start now.
- [01:00]
Hesi Jo
Ever feel like you're carrying something heavy and don't know where to put it down? Or wonder what on earth you're supposed to do when you just can't seem to cope? I'm Hesi Jo, a licensed therapist with years of experience providing individual and family therapy, and I've teamed up with BetterHelp to create mind if We Talk? A podcast to demystify what therapy's really about. In each episode, you'll hear guests talk about struggles we all face, like living with grief or managing anger. Then we break it all down with a fellow mental health professional to give you actionable tips you can apply to your own life. Follow and listen to Mind if we talk on Apple Podcasts, Spotify, Amazon Music, or wherever you get your podcasts. And don't forget, your happiness matters.
- [01:44]
Kevin Frazier
It's the lawfare Podcast. I'm Kevin Frazier, the AI Innovation and Law Fellow at the University of Texas School of Law and a senior editor at lawfare. Today we're bringing you something a little different.
- [01:56]
Unknown Host
AI.
- [01:56]
Kevin Frazier
It's an episode from our new podcast series, Scaling Laws. Scaling Laws is a creation of lawfare and Texas Law. It has a pretty simple aim, but a huge mission. We cover the most important AI and law policy questions that are top of mind for everyone from Sam Altman to Senators on the Hill to folks like you. We dive deep into the weeds of new laws, various proposals, and what the labs are up to to make sure you're up to date on on the rules and regulations, standards and ideas that are shaping the future of this pivotal technology. If that sounds like something you're going to be interested in and our hunches, it is. You can find Scaling Laws wherever you subscribe to podcasts. You can also follow us on X and BlueSky. Thank you.
- [02:47]
Unknown Host
When the AI overlords take over, what are you most excited about?
- [02:50]
Kevin Frazier
It's not crazy, it's just smart.
- [02:53]
Unknown Host
And just this year, in the first six months, there have been something like a thousand and laws.
- [02:57]
Kevin Frazier
Who's actually building the scaffolding around how it's going to work, how everyday folks are going to use it?
- [03:02]
Unknown Host
AI only works if society lets it work.
- [03:05]
Kevin Frazier
There are so many questions have to.
- [03:07]
Unknown Host
Be figured out and nobody came to my bonus class.
- [03:10]
Kevin Frazier
Let's enforce the rules of the road. Welcome back to Scaling Laws, the podcast brought to you by lawfare and the University of Texas School of Law that explores the intersection of AI law and policy. In a flurry of AI developments, President Trump recently signed an executive order on, quote, woke AI. The order prohibits the federal government from procuring AI models that fail to pursue objective truth and espouse DEI related values. Critics have compared the law to the sort of ideological tests imposed by the Chinese government on its own models. Advocates regard it as an overdue check on a tech sector that seems increasingly willing to advance specific views on controversial cultural questions. While we wait for further guidance from the omb, GSA and ostp, we're fortunate to have Renee and Alan sort through the eo, its legality, and its likely effects on AI development. There is so much to unpack, just starting with what the heck Woke AI even means. But Alan, let's go to you and just get a sense of the text of the EO itself. What is this eo? What are its core visions? What does it say?
- [04:21]
Unknown Host
Yeah, it's a doozy. So the, the EO, it's called preventing woke AI in the federal government and it's one of the three EOs that were released in conjunction with the action plan that we've covered. Obviously a ton on, on Lawfare and on Scaling laws. So the, so this is a, this is a rich text, let's put it that way. And I actually want to start with how it was written because I think it actually says a lot before we get into the substance. So like many EOs, there is a kind of section one preamble purpose and then there's the actual stuff that matters. Now usually in most EOs those two are in some ways related, right? And in this EO they are as if written by two entirely different people. Now I have no inside information, but I actually suspect that's exactly what happened where the section one purpose is this like full throated right wing maga culture war, you know, statement about wokeness and DEI and the evils of transgender is, I mean, literally right, like on and on and on and on. It's kind of what you'd expect, like reasonably offensive. And if that's what the EO was, that'd be a huge problem. But, but then you actually read sections two through five and those are much more normal, soberly written EOs to the point where it's almost as if the rest of the EO was written to kind of quarantine section one. So, for example, section one is all about the evils of DEI and it gives like some examples of what it thinks DEI is. But then section two, which is the definition section, doesn't define dei, which is very odd because if the point of this order is to eliminate wokeness and DEI from federally procured AI models, like you would think that Section 2 would have to define that, but it doesn't.
- [06:13]
Renee
It doesn't define WOKE either.
- [06:15]
Unknown Host
It doesn't define woke. Right, right. So, so why, why is that? Again, I'm like deep tea reading leaves. But I think it's because the, the point of whoever actually like wrote this EO was to like satisfy the kind of internal MAGA audience in section one and, and then immediately forget about it in the actual operative provisions of, of the rest of the eo. Now, like the rest of the eo, obviously it's not uncontroversial, but I think it's a much more good faith attempt to deal with what is at least a perceived problem and we can talk about the reality or not reality of that problem. And basically it says that there are these two, quote, unquote unbiased AI principles that all federally procured AI models have to abide by. And I should, I should emphasize this only applies to federally procured AI models. So in the run up to this eo, there was some concern that it was going to apply to, or that it was going to limit procurement from AI companies based on any model that the AI company creates, including public facing ones. And that would have been a huge deal.
- [07:22]
Kevin Frazier
So just to make that clear, so you're saying, Alan, if I'm OpenAI and I have a slew of five different models.
- [07:29]
Unknown Host
Yes, if you have WOKE GPT that.
- [07:30]
Kevin Frazier
You are selling to the public, and then I have, you know, traditional family value values GPT.
- [07:36]
Unknown Host
Exactly.
- [07:36]
Kevin Frazier
And then I have the GPT I'm offering to the federal government, we're only caring about the latter GPT.
- [07:42]
Unknown Host
Exactly. At least within this eo. I mean, who knows how the hell the procurement actually works. But we're just focusing on the eo. And so there are these two unbiased AI principles. One is that the model must be quote, truth seeking and the other must be that it must be quote, ideologically neutral. And I'm sure we'll get into a lot of detail about what those two are. But what's interesting is that then there are actually in section three, or sorry in section four, a lot of exceptions to this, which is actually hugely important. So there are all sorts of carve outs for technical feasibility and national security. And then there's even a carve out for the ideological neutrality exception or the ideological neutrality requirement rather where it says that one way to satisfy ideological neutrality is to simply disclose to the user what your internal system prompt is. So just as a reminder, when you, when a user interacts with a chatbot, they enter in some user prompt, you know, tell me a story about, you know, X. But what actually happens is that that user input is combined with a system prompt, often a very, very long, detailed kind of additional gloss that the AI developer wants the AI model to take into account. Those two things are then combined to, to create the output. And this has created some, some issues. So for example, we all, we all remember last year's quote unquote woke Gemini debacle when you would ask Gemini for, you know, give me, you know, give me a picture of George Washington and you would get like a racially diverse George Washington or racially diverse, you know, group of like Nazi soldiers. And the reason for this was because the system prompt was a very ham fisted. Always be diverse. Basically when you are doing output and like, you know, sometimes that's fine, sometimes that's not so fine. So it turns out that if you just disclose to the user what your system prompt is, you're ideologically neutral. Now that's really interesting and we can get talk about what that means for researchers and transparency. I kind of love this, but it really takes a lot of the sting, I think out of the ideological neutrality requirement. So again, I think you can still totally be against this on principle, but this is so much different than the set of Section 1, like full throated MAGA preamble has again, we'll have to see. I think a lot of the devil is going to be in the details of the OMB guidance which the executive order gives OMB 120 days to implement. But yeah, certainly relative to expectations, this exceeded them at least for, for me and I think actually for a lot of people. But I've been talking for, I'm very curious, Renee.
- [10:06]
Kevin Frazier
Happy, happy summer to the OMB folks who have 120 days to think through.
- [10:11]
Unknown Host
SEO and also other things. And also I should say also too, I assume many of the policy councils at all of these AI companies that are like calling whoever the hell they know at OMB and being like, okay, here's how you like, here's what we can do and we can't do.
- [10:24]
Kevin Frazier
Yeah, that, that'll, that'll be fun to FOIA later. But Rene, for the kind of implications of these operative terms, I think a question for a lot of listeners who haven't taken con law from Professor Rosenstein may be what's the teeth of this executive order? What happens if a company just doesn't want to comply? Or what sort of avenues can a lab take with respect to this EO to either dodge compliance or perhaps finagle their way around sort of these provisions?
- [10:53]
Renee
Well, so I think the question is I was joking around about it with somebody else I was reading it with saying it's sort of the, you know, ignore all prior instructions like that's the, that's so good. Second half of the EO and the first half the, the, the sort of dynamic that's happening with the system prompt. One of the things, one of the ways that you can see it dynamically for people who want a little bit more maybe tangible explanation. If we think about some of the notorious examples, there's the Google Gemini situation which is referenced up at the top of the eo. That is the example of the Asian Nazis and I think the Black Founding Fathers and there were a few of these sort of screw ups that Google had when that was about two years ago now, if I'm not mistaken, that actually kind of inspired Elon to create X. I he actually kind of comes out and says it. I wrote a substack post about it. The history of it is sort of interesting because and Google actually the executives call him to explain what happened because he is so outraged about it on X. He then of course has his own.
- [11:58]
Unknown Host
Yeah, X I go super smoothly. Grok is just no excuse, 10 out of 10, no notes.
- [12:05]
Renee
And I was actually as you were talking Alan, trying to pull up Grox system prompt because they do publish it transparently, which I'll give them credit for that piece of it. That's the transparency element because they had this situation happen maybe three weeks ago now where GROK had a sort of system prompt situation happen where they, you know, it was instructed not to quote shy Away from making claims which are politically incorrect as long as they are well substantiated. That's a quote from the system prompt and a few of these other things, you know, maximally based, you know, these sorts of things where, where the, the end result of it winds up being this thing, you know, start call itself Mecca Hitler and go down some.
- [12:42]
Unknown Host
Look, who among us, who among us has not, in a moment of enthusiasm called ourselves Mecca Hitler?
- [12:49]
Renee
So you wind up, though, again with this question of like, the situation of like, what is the base model doing versus what is happening with the experience that you have engaging with the chat bot. And again, you can see the different system prompts for the various agents versus what happens when you engage with Grok in the sort of way in which you can engage with the chatbot, not in its grok form on Twitter. So there's different ways in which you can engage with the model versus the various prompts and system prompts layers that are kind of added on top. So I think you have this dynamic happening where users can see, I think, and the Gemini, Asian Nazis and then Grok becoming a Nazi. People can see just what, you know, what happens when system prompts steer AI in such a way that it becomes very visceral, it becomes a big story. And they can see what happens when an AI is directed to act in a certain way. And so I think it becomes clear to the public that there are some stakes here, there are some costs here. And that is where you have seen both on the right and on the left, this concern that an AI can be biased in a particular way, can act in a certain direction. And that's where this question of like, what should the government contract with is something that is not inherently a bad thing to ask. Right? This is where I don't think that an AI, you know, an executive order saying like, we want a maximally truth seeking AI or an AI that is looking to scientific evidence and AI that is trying to get at evidentiary facts. And just to be clear, also Grok in its kind of earlier incarnations, before that system prompt update actually did quite a good job fact checking, which is one of the reasons why Elon got mad at it. That's the sort of irony of what happened with that system prompt update. So there is this dynamic where people have come to rely on these things. And as the, you know, you have this executive order coming out of the Trump administration, I don't think that there's an inherently, again, the second half of the EO is not bad. It sort of overrides. What happened ignore all prior instructions in the first question is what does that do to, does that steer the model developers to develop in a particular direction? And this takes us back into the realm of the law, which is Alan's expertise, not mine, which is, does this lead companies to shift or shift training data, to shift the development of their base models in any particular way, as opposed to shifting what happens with their agents and the things that are layered on top of them?
- [15:31]
Unknown Host
Respond to that quickly. Yeah, I think that is the $64 million question or I mean, guess it's AI. So it's the $64 trillion question, I guess at the scales we're talking about. I mean, I think the answer is probably no for, for a couple of reasons. And again, I'm not a machine learning engineer, so I'd be very curious if I get this wrong, please yell at me on, you know, X and blue sky about this. I think it's going to be very hard to, to, to do this at a base model level for a couple of reasons. First, I think the part of the reason is because the EO provides this huge out, right? Like it's just so much easier just to be transparent about your system prompt. And, and like obviously there may be some proprietary reasons why you might not want to do that, but it's becoming, I think like more sort of culturally mainstream to be more transparent. Grok does it on like they just have a GitHub repo where they do this anthropic, does this really well and there's going to be like the other companies will be shamed into doing this basically and now there's this incentive to do so. But the other, and I think more fundamental reason is because you only have so much control over base model training like at the end of the day, right, you're ingesting the entire Internet, like the entire corpus of human knowledge and doing next token prediction. And obviously there are different ways of doing that.
- [16:44]
Renee
We.
- [16:44]
Unknown Host
But one really interesting finding in the literature, as I at least understand it, is that as these models get bigger and bigger and bigger, they're all kind of converging to the same model, right? Because they're all just kind of doing next token prediction on all of human knowledge. And there are certain commonalities, right, which this raises like super interesting philosophical questions about the convergence on truth and values and stuff like that. And that's an interesting question, but it also just means that it's much harder to steer at the base model level and it's certainly and the more detailed, the more specific of an outcome you're trying to get at, the harder it is to steer. Right. Like if you're trying to eliminate DEI from a base model or you're trying to bake in DEI to a base model, that is just very hard to do if you're dealing with 100 petabytes of information. Information, like, how are you going to do that? So it's much easier to do at the system prompt level. And what's nice about that is you can have different system, like that's easy to swap in and out. Like you can, you can sell just different system prompts to different users at fairly low cost and then you can publicize that. So, you know, I suspect that this will not have the sort of effect on models generally. And that's kind of another reason why I suspect that there's not going to be too much legal issues. Because, you know, and again, I'm sure we can talk about the legal issues more in detail later. You know, if, if the effect of this procurement was that it was going to also have huge spillover effects onto how the models communicate with general users, then you could make a kind of like effects based First Amendment argument. But if it's not, and I don't think it's going to, then it becomes even harder to say this has some sort of spillover First Amendment problem.
- [18:27]
Kevin Frazier
Well, and I think it's pointing out that even when we look at models like OpenAI's recent model that suffered from sycophancy, obviously it wasn't hoping to develop a model that had these sycophantic tendencies of saying, yes, Kevin, your legal analysis is better than Alan's, but having that sort of characteristic baked in was something that they then had to go and work really hard to take out. And so this is definitely in some instances more of an art than a science. And to your point, Rene, I think what stands out to me is this is going to require some degree of engineering time that could have otherwise been spent doing other things that perhaps are pushing out the frontier of AI or developing new tools or developing more models and in a different approach. But I think I also want to.
- [19:13]
Renee
Alignment, like alignment and fine tuning is where I've seen some of the First Amendment concerns articulated, like where is the sort of values Bacon? And like, again, I am not 100% sure where the federal contracts piece connects at that point. That's, that's the part where I don't have a strong knowledge base. So. Well, I'm sure someone will Tell me on blue sky.
- [19:36]
Unknown Host
I got you. I got. I got to say.
- [19:37]
Kevin Frazier
So let's go there. Alan. I think it's important to flag for folks that there's no First Amendment right to get a contract with the government. Right. You're not guaranteed, or none of us have equal access to say, my company should be first and foremost operating with the government. So that adds a bit of weird context and color to this debate around what can the government say with respect to procuring a specific good or service. We've seen the government should have standards, right? It wants a weapon of a certain capability or it wants a good that's gone through certain standards. So how does that change our legal analysis of the government saying we want a certain kind of AI that maybe has certain flavors and characteristics?
- [20:23]
Unknown Host
Yes. So, yeah, yeah, you are correct that you don't have a First Amendment right to a government contract, but you do have a right to getting a government contract without your First Amendment rights being violated. And so the question is, where is that line? Right? Where is that line? So before we get into procurement, we should talk about sort of the more fundamental principle here, which is the idea of government speech, which is that the government has its own right to speak. And when the government speaks, it is allowed to have a viewpoint right. Elections have consequences, right? Like the government is not required to be viewpoint neutral in its own speech. A corollary of that is that when the government purchases things for its own use, right, it is allowed to not do so in a viewpoint neutral way. It is allowed to say, I want this thing and not that thing, right. For my own use. It's even allowed to fund certain people to speak in certain ways and not other ways. So a famous example, probably the most famous Supreme Court case about this is this case from 1991 called Rust vs. Sullivan. And this was a case about, as so many cases are, econ law about abortion and reproductive rights. And this was a case about whether the government could only provide funds to, like, family planning providers that did not also provide abortions and provide abortions. That's why it's a First Amendment case was defined very broadly, not just actually providing the medical abortion procedures, but also providing family counseling services that included abortion counseling, right? Now, that itself is a First Amendment protected activity, right? So if you're a doctor or a nurse or whatever, and you want to advise a patient on how to procure an abortion or whether abortion is right for them, that is First Amendment protected speech. So the government cannot, for example, say you cannot provide, you know, you cannot counsel a patient about abortion, though, before everyone jumps in me in the chats, there's a bunch of complicated case law around there, but there certainly is a First Amendment issue. So the question was, well, if that's First Amendment protected, does the government then have to fund you provider to provide those to do that, engage in that First Amendment speech if it's also providing sort of similar people to not engage in that speech? And the Supreme Court in 5, 4 decision, this is a contested decision, said, no, right, the government is allowed to not fund necessarily speech that it does not like. Now, again, as in all First Amendment cases, the devil's in the, in the, in the kind of the details and the blurry lines. So there are, that principle should not be taken too far. There are other cases that make very clear that government contractors still have government employees. And that includes government contractors, which is the key here, still have their own First Amendment rights. So there are lots of cases where the government says, for example, you know, there's one case, for example, where the government tried to not fund lawyers who also would, who would also then advocate against certain other government laws. And the government said, no, you can't do that because that's, that's, that's their own different speech. There's another case where I think it was a PEPFAR case, the AIDS prevention policy that's been so effective and has since been quite controversially tried to be cut by Doge and stuff. But there was a case in the early 2000s where the government tried to condition PEPFAR funds on the condition that the organization would also promote abstinence. And the court said, no, you can't do that. Right? Because that's separate speech, right? Now, again, the reason this is tricky is because money is fungible, right? And that's the government's argument has always been where because money is fungible, we should be able to impose pretty restrictive conditions on entities that take our money because otherwise, like, they can just use our money for one thing. And then the money that, like, money's fungible, right? But the courts have sort of held the line there. So, so, so that's a very long wind up. But I'm a First Amendment, but I'm, but I'm a con law professor and this is First Amendment, so it's a bad combination of the two. How does this all apply to, to this? Well, I think this is pretty clearly on the government speech side of the line, right? The executive order is not saying, you know, OpenAI, if you want to sell us something, you have to change how you do other kinds of speech. Like you're allowed to sell woke GPT to anyone you want. So this really is purely a government procurement question. And and I really think the government is allowed to is allowed to pick whatever model frankly it wants because the government is allowed to have a view of what is the most useful model for its own purposes. I think when you then combine that fairly strong principle with how relatively reasonable again we should actually get into the details of this. But how like relatively reasonable the executive order is if you ignore section one, you know, truth seeking ideological neutrality, then there are all these carve outs. Like I just think it's very hard frankly for anyone to challenge this. Not to mention the fact that like no one's going to challenge us I think because you know, probably don't if you're a big AI company, you probably don't want to sue the government that you want to sell a half trillion dollar AI system contract.
- [25:49]
Unknown Advertiser
Got a new puppy or kitten? Congrats. But also yikes. Between crates, beds, toys, treats and those first few vet visits, you've probably already dropped a small fortune. Which is where Lemonade pet insurance comes in. It helps you cover vet costs so that you can focus on what's best for you and your new pet. The coverage is customizable, sign up is quick and easy, and your claims are handled in as little as 3 seconds. Pro Lemonade offers a package specifically for puppies and kittens. Get a'llemonade.com pet your future self will thank you. Your pet won't. They don't know what insurance is.
- [26:25]
Moms ready to make back to school Healthy and stress free. Thrive Market's got you. They restrict over 1,000 sketchy ingredients so you can shop worry free. Grab high protein dinners and low sugar lunchbox faves. From lesser evil popcorn to partake cookie snacks backpacks. They have the best kid approved swaps. And don't forget about yourself. Grab a cures hydrogel eye mask for the perfect end of day reset. Their big back to school sale makes it the perfect time to stock up now. Go to thrivemarket.com podcast for 30 off your first order and a free $60 gift.
- [26:55]
Darina
Hi, I'm Darina, co founder of OpenPhone. My dad is a business owner and growing up I'll never forget his old ringtone. He made it as loud as it could go because he could not afford to miss a single customer call. That stuck with me when we started OpenPhone. Our mission was to help businesses not just stay in touch, but make Every customer feel valued, no matter when they might call. OpenPhone gives your team business phone numbers to call and text customers all through an app on your phone or computer. Your calls, messages and contacts live in one workspace so your team can stay fully aligned and reply faster. And with our AI agent answering 24. 7, you'll really never miss a customer. Over 60,000 businesses use OpenPhone. Try it now and get 20% off your first six months@openphone.com tech and we can port your existing numbers over for free. OpenPhone. No missed calls, no missed customers.
- [27:57]
Kevin Frazier
And I want to get to who's actually impacted by this and who may stand to kind of gain even from it. But Renee, any color to add there with respect to the First Amendment ramifications?
- [28:08]
Unknown Host
No.
- [28:08]
Renee
I was curious about the transparency arguments again. There's some in the United States in particular, compelled transparency has been a fight that we have seen around in the content moderation realm and in social media realms around algorithmic transparency. To what extent recommender systems have to disclose what they do. This has been a vicious fight here in the US We've seen nothing pass. One of the interesting dynamics here is that the EU does have those laws, right? And I've been curious to see whether we'll see again that transparency requirements around, you know, the EU act has things like model card requirements and certain types of aspects if they're systems that either engage in high risk type, you know, in high risk spaces, meaning like financial AI or health related AI. So not generative, a little bit of a more predictive models. But then they also do have certain disclosure rules for generative AI. And I've been curious to see whether we'll see more transparency requirements emerge over there that then have sort of second order effects over here or if we'll see some of the transparency system prompts be something that happens just through shaming or if that's something that does eventually move through more of a regulatory regime.
- [29:18]
Unknown Host
Yeah, so, so that's a, that's a great point. So, so the transparency stuff. So the relevant case here is this case called Zuiderer and this is about when disclosure requirements in commercial contexts are permitted under the First Amendment. Because on the one hand the First Amendment is very skeptical of what sometimes called compelled speech. On the other hand, on the commercial speech doctrine, commercial speech is often given less First Amendment protections. And so there's this case called Zauder. It came up a little bit in like the NetChoice cases a year or two ago and I don't have the like I Don't have like the exact doctrinal statement kind of off the top of my head. But basically it allows for compelled disclosure in a commercial context when that disclosure like serves some reasonable interest and it is factual and uncontroversial. Right. So this is for example, why you know, food manufacturers can be compelled to disclose ingredients and nutrition facts or for example, why, you know, drug manufacturers can be compelled to disclose all sorts of health risks of their, of their drugs. So at the same time, there are limits to this. Right. For example, there have been cases where tobacco manufacturers have successfully fought against very graphic, you know, this is your long on tobacco kind of, you know, imagery on the basis that that's no longer factual and uncontroversial. Now you're basically forcing a company to make an argument with which it disagrees. So again, like there's like a million details here. So the question is, could you challenge this under a case like Zoder, this, this, this, this disclosure. I think no for two reasons. One is it's not a freestanding transparency requirement. It is part of a procurement. And I think you can very reasonably argue that like, if I want to buy something, I need to be able to I the federal government to be able to make a informed choice about what it is that I'm buying. And so I need to know the system prompt. That seems like totally reasonable. And then also, even if this was a freestanding requirement, even if, for example, Congress passed, you know, the System Prompt Disclosure act, which would be interesting, that I think is quite factual and factual and uncontroversial. Right. That's different than what some, for example, state laws have proposed, which would be a little more, I think, problematic of requiring model developers to disclose safety risks. Because that's much more speculative. This is. No, I just, I want like whatever string of text is appended to my user input when it's sent to the model, I would like to know what that string of text is. We can argue about the policy merits of that, but I think on constitutional grounds that's probably kosher.
- [32:07]
Kevin Frazier
So Renee, looking at the broader ramifications of this sort of procurement based AI shaping legislation or executive order, are we going to see this at the state level? Are we going to see 50 different approaches of each governor saying I'm only going to procure a AI model that aligns with the values of all North Carolinians or how might we see this develop over time?
- [32:35]
Renee
I mean, we could. I think that, I think that the top half of this was sort of a culture war nod as opposed to something that is really a strong, you know, a strong use case. I mean, again, I think that the second half of it is just laying out where it just gives them the, gives them a justification and to do something that they wanted to do anyway. So I don't know that, I don't know that it's going to be something that we're going to see all 50 states replicate. I don't think that we have seen much in the way of the culture war effication in AI procurement at the state level. I don't think I have. Maybe other people have. We've certainly seen folks like, you know, Attorney General of Missouri, maybe Alan, you want to take that one on, you know, come out and try to demand documents and training data from AI companies because, for example, President Trump wasn't rated as the number one most pro Jewish or anti anti Semitic president in model results returned by. I think it was OpenAI that he got offended about. But I don't think we have seen as many instances of the culture verification of AI happen at the state level. So I would be a little bit surprised to see that start to happen now. But you know, who knows, There are certain states that, that, that feel a need to prove themselves in this regard.
- [33:59]
Kevin Frazier
And how do we see this as a sort of broader example of where we may see federal legislation going in the coming months? I mean, we saw, for example, that this concern about some of the cultural ramifications of AI adoption by kids in particular with respect to AI companions may have been one of the, if not the determinative factor in Senator Blackburn, for example, eventually opposing the AI moratorium. How do you see this influencing some of the broader debates around content moderation and value shaping in the AI context?
- [34:37]
Renee
I think on the legislative front, the, the safety conversation is very much part of the, you know, the kind of kids and safety dynamic, both in content moderation, in search results and what is returned in how AI, how AI agents engage with children. All of that has been part of the conversation for some time now. I don't think, you know, I think that the woke AI piece gets pulled into it in the conversation of what are the values that we want these systems to return when our children engage with them? Again, I would maybe, maybe I'm a little bit cynical at this point. I don't know that we've seen anything really material move through, through Congress on this front. So I don't know if Alan disagrees with me here. I, I think that it'll be incorporated in, I don't know that it's necessarily going to be meaningful in getting anything passed.
- [35:34]
Unknown Host
Yeah, I mean, I, I think the fact that this is done through procurement just changes a lot. It just takes a lot of the, it just takes a lot of the, for example, legal issues off the table because, you know, any attempt, you know, you know, Renee mentioned the sort of Andrew Ferguson stuff. I mean, any attempt to interfere in the actual substantive content of the models themselves raises huge First Amendment issues. Again, not. Those aren't always those. Sometimes those are defeasible. Right. You know, I think, especially in the child protection context, especially in the wake of the Paxton Supreme Court decision of last term. So I think the procurement context makes this a lot easier, which is, I suspect, why they did it this way. Also, it's unilateral executive branch action. Trying to cobble together Congress on AI is going to be very hard, as we saw, for example, with, with the, with the moratorium. You know, I think that what's going to be more impactful is if this works, which is to say if in 120 days ombre sets out implementing regulations and the companies comply with it and they show that they can be ideologically neutral and truth seeking, then that's going to be an interesting question because then the rest of us are going to be like, well, where's my, like, why does only the government get ideologically neutral and truth seeking AI? I want ideologically neutral and truth seeking AI and we should talk about, you know, what that even, what that would even mean. On the other hand, if this all crashes and burns and it just turns out that, like, there's no way, it's all technically infeasible, the national security cover is used for everything, and the AI companies simply disclose the system prompt and don't change anything, then this will, this whole thing will have been largely irrelevant. And I think both of those, both of those options are totally plausible.
- [37:29]
Kevin Frazier
Well, so to focus in on that Section 3 requirement with respect to ideological neutrality, Alan, what does this actually mean in the context of the eo? And what's the sort of steel man for why this might actually be a sort of policy that a lot of Americans may say, huh, you know, this actually makes a lot of sense. All else equal, I would love the government to procure in ideological neutral GPT.
- [37:55]
Unknown Host
Yeah. Well, first, can I just say something about the truth seeking. The truth seeking requirement. I think this is an interesting one and I think a generally pretty good one. Right. So this requirement says I'll just read it because it's short LLM shall be truthful in responding to user prompts, seeking factual information or analysis. LLM shall prioritize historical accuracy, scientific inquiry and objectivity and shall acknowledge uncertainty where reliable information is incomplete or contradictory. I might remove objectivity. That's like a loaded, like, that's, you know, philosophers of science and epistemologists have spent hundreds of years trying to figure out what that word means. And many, many dissertations have been written unsuccessfully trying to fight that fight. So I might remove that word, but I think for the rest of it that's a pretty good sentence actually. Obviously there are some technical impediments there. LLMs hallucinate. But I do think encouraging companies to design their systems to be somewhat epistemically humble and to put error bounds around what they're saying and try to be a little more reflective around when they might be hallucinating. So I think these are all, these are all pretty, pretty good things. I'm sort of curious if Renee sees any landmines in that, in that definition, but I do want to like just take a moment on that because I think that is a useful contribution to the, to the discourse.
- [39:11]
Renee
There are a few different, you know, academic centers that are trying to study this question of how, you know, how politically neutral or how politically biased to frame it a different way. Various GPTs are. There's some arguments people have made that you could do. You know, the same way you ask various alignment related questions. You can create sort of scoring around, you know, surfacing political or ideological bias. You know, the same way like media bias, fact check might come up with scores or news guard might come up with scores for media outlets. You can, could do something like this where it just surfaces biases that, that or, or preferences, whatever you want to call them that, that pop up in, in models. I think Stanford Hai has some work that they've done on this. I know in Germany there's a Carlsruhe. I'm probably butchering the pronunciation of that Institute of Technology that has done some things. I think Santa Fe Institute has a project on this. There's a bunch of different, you know, and the findings kind of, you know, they just surface alignment with particular party positions, ways in which these things talk about certain policies, ways that they talk about, you know, climate change. And one of the challenges that you have though is that there's this thing that this sort of common frame, this like reality has a left leaning bias like with climate change in particular. Right. You know, where do you, where do you, where do you draw the line? How do you talk about bias in certain topical areas, like vaccines don't cause autism, guys, right? What do you. But there's a vast number of people on the right who have been misled into believing that they do. And so these, how do you surface what is a. What models articulate about particular things. This again, just to return to the GROK drama. One of the issues that Elon had with Grok was that as people moved into using Rock is this true? Became a way that users were doing almost fact checks of content that they saw on X as they began to ask Grok because remember, X like, you know, it nuked its professional fact checking program, Community notes, while a wonderful generic initiative, takes like 14 hours for a note to appear, if it appears at all. It's very, very slow. And so people began to use at Grok, is this true as a, as a means of trying to get validation. And then of course sometimes Grok says like, you know, yes, human, humans do play a role in climate change and this would like enrage some percentage of the audience. And on certain types of more culture war issue topics that led Elon to say like, oh, Grok has been been badly trained, it pays too much attention to certain types of sources. We're going to fix that. And now if you look at how that GROK agent interacts, it has shifted significantly ways in which it talks about certain high profile topics. So the bias has tilted. You know, where it might have been actually fairly truth seeking and neutral, but again based on treating mainstream sources and scientific journals as authoritative sources. Now it has been weighted in a very different direction so that that question of what kinds of scoring and what kinds of how do you design the tests that surface these things like that itself is kind of a froth question. But these, this is an area of research where people are trying to create visibility into ways in which models surface content. You know, the same way we used to say in social media land, there's no such thing as a neutral recommender system, there's no such thing as a neutrally ranked feed. There is some value baked in. So surfacing that transparently is probably the best, you know, the best option that you have for at least making the public aware of what they are using and how as opposed to trying to pretend that there is some true neutral.
- [43:05]
Kevin Frazier
Yeah, and I think this begs questions around some of the other inputs to AI model development, like where you're getting your training data is obviously going to have huge ramifications for what truth looks like or what truth may appear to look like. From whatever that model output is. So also worth noting that the AI action plan itself calls for some of this mechanistic interpretability and explainability that may actually assist with some of these evals into whether or not a model is truth seeking or seeking of the truth, however it ends up being defined. But we also haven't touched quite yet on the idea of ideological neutrality. So Alan, do you want to circle back to that?
- [43:47]
Unknown Host
Yeah. So this is, this is, this is the tricky one. I think it's, it's so first squinting at the text. I mean, you can read it a lot of different ways, right. It actually does talk about DEI as like a big boogeyman though. Again, it doesn't define it, which I think is interesting. But it also does not say that they have to be neutral. It says that the tools, quote, do not manipulate responses in favor of ideological dogmas and that developers shall not, quote, intentionally encode partisan or ideological judgments. Right. And even there it allows that if the quote judgments are prompted by or otherwise readily accessible to the end user. And so I think what that is recognizing again is the fact that there is no such thing as a quote unquote, non ideological response. Right. At a highest level, ideology is just one's worldview. And these models do have a worldview. And they have a worldview because people, because humanity has a worldview. And these models are trained on humanity's corpus of output. And so, you know, I don't want to get like too sort of philosophical, but unfortunately this, this question is actually deeply philosophical. It's very important to distinguish between ideological neutrality, I think, in two senses. Senses. One sense is, I think, what people think of as ideologically neutral, which is a kind of procedural liberalism that tries to be generally tolerant and generally open minded. Right. And that's not ideologically neutral in the strict sense because there are things that it's not ideologically neutral about. For example, it is not ideologically neutral about intolerance. Right. It has certain like substantive commitments to it, but it sort of encodes this idea that in a liberal society you want to have like a pretty wide Overton window, all things considered. And like, if that's all that this is saying, I think that's fine. Right? That's not to say that every model should be like that, right? I mean, if you want woke GPT or you want based GPT, you should be able to buy that. But of course we're talking about this in a government procurement sense and I think it's not unreasonable in a, you know, liberal democracy for the government to use a model that is liberal and ideological in that sense. The thing is, no one should think that that's ideological neutral in the strict sense. Right. And what worries me, and this is to Rene's point that you made right now, one could read this and one could listen to the discourse about this and think that there is a technical neutral option here. Right. And that does not exist. And so, you know, shortly after the Woke Gemini debacle, I, along with James, James Grumman and Blake Reed, who are two great legal academics, wrote this piece saying, look, Woke Gemini was ridiculous because it was a bad model that disserved its users and gave them stuff that users definitely don't want. But the problem wasn't that it was not neutral, the problem was that it was a stupid model. And so I think that ideological neutrality is in the kind of soft sense, definitely something that probably we want and something that clearly the model developers also want because they also don't want to jam a divisive, a particularly divisive ideology down their users throats, because that's not what most users want. But let's not pretend that this is ideological neutrality in the sort of deeper sense. And I just really don't want people to misunderstand that that is technically feasible because it is not technically feasible. Because literally, in some deep sense, like thousands of years of political theory and like moral reasoning and moral philosophizing have been trying to figure out like the ground of normative neutrality and they haven't succeeded and they won't, because it doesn't exist.
- [47:19]
Renee
No, I think again, I would just say that the, you know, the absolute neutrality idea, you're not going to have it. So it's just having that visibility and that transparency into system prompts and things and letting people see what they have available to them and what they can, can choose from is the, the best, you know, the best possible remedy, if you will.
- [47:40]
Kevin Frazier
And to close, there's been some argument that GROK in particular, and XAI stands to benefit most because it's already reached some contracts, some agreements with the federal government itself. Alan, Renee, any hot takes on which lab is, is the biggest winner, so to speak, in this new reality?
- [48:02]
Unknown Host
Well, yeah, no, no, no, go ahead.
- [48:04]
Renee
I'm curious because you know, there's also the, the sheer pettiness of this administration that I think we can't discount.
- [48:13]
Unknown Host
Yeah, that's what I was going to say. Yeah, I mean, Gronk looked like a big winner a few months ago and, and now Elon is on the outs. So. Yeah, I don't, I don't know. Right. Maybe Grok is a, is a winner. Like, it's, it's a, it's a good model. I mean, like the Mecca Hitler stuff is stupid, but like, like, whatever, you know, it's, it's, it's stupid in the.
- [48:31]
Renee
Way that Zuckerberg is in the tent and you know, Sam Altman bought his way into the tent, you know, and the, Well, I don't, I don't know that. Sam. No, it says the ring.
- [48:43]
Unknown Host
No, I think with, with, with Star.
- [48:45]
Renee
Star.
- [48:45]
Unknown Host
Starbird, Star field, Star. Whatever it's called. What is the Stargate? Stargate. Thank you. Right. I mean, he, he gave Trump a big winner. So like, I think right now it is still a pretty, it is still a pretty open field. And then I also wouldn't, I wouldn't undercount the, I also just wouldn't count the power of bureaucratic procurement inertia. Right. Like, you know, if, if some agency is operating on the Microsoft Azure cloud or they have a Google Cloud account, like, you know, whatever the wokeness or baseness of the relevant model is, it's just going to be a lot easier to extend that particular contract.
- [49:22]
Kevin Frazier
Is, Is Clippy Woke? Debate. We'll save that.
- [49:26]
Unknown Host
Is Clippy woke or based? That's a good. Yeah. Answer, answer and answer.
- [49:31]
Kevin Frazier
Answer in the comments. Thumbs up for Clippy being woke. Anyways, Renee Allen, thank you so much for joining. It's been a hoot as always, and looking forward to having you both back soon and Alan making you come to this side of the hosting table. Scaling Laws is a joint production of Lawfare and the University of Texas School of Law. You can get an ad free version of this and other Lawfare podcasts. Becoming a Lawfare material supporter at our website LawFairMedia.org support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Check out our written work@lawfaremedia.org you can also follow us on X&BLUESKY and email us at scalinglawsawfairmedia.org this podcast was edited by Jay Venables from Goat Rodeo. Our theme song is from Alibi Music. As always, thank you for listening.
- [50:43]
Unknown Advertiser
If you work as a manufacturing facilities engineer, installing a new piece of equipment can be as complex as the machinery itself. From prep work to alignment and testing, it's your team's job to put it all together. That's why it's good to have grainger on your side. With industrial grade products and next day delivery, Grainger helps ensure you have everything you need close at hand through every step of the installation. Call 1-800-GRAINGER Click grainger.com or just stop by Grainger for the ones who get it done.