← The Lawfare Podcast
Loading summary
Transcript48 lines
- [00:02]
A
The articles of incorporation under which OpenAI was set up as a nonprofit make very clear that they have a legally binding fiduciary duty to do certain things. And what we chart in the piece is this kind of years long process of the company kind of converting itself out from under those restrictions.
- [00:21]
B
It's the Lawfare podcast. I'm Kate Clonik, senior editor at lawfare, with Ronan Farrow and Andrew Morantz, writers of the New Yorker and authors of a huge 16,000 word piece published last week in the New Yorker titled Sam Altman May Control Our Future. Can He Be Trusted?
- [00:40]
C
Sam Altman is one figure. He's a consequential one, but he's especially consequential as a window through which to view a wider dynamic that critics in the piece describe as a race to the bottom.
- [00:54]
B
Today we're talking with Ronan and Andrew about the legal and national security implications of a number of things that are revealed in their article and discussing what some of the policy options might be going forward. So I want to start for, just to take a moment for the Betteridge's Rule of headlines, which is that if you ask a question, a headline, as you do, the answer is always no. So in theory, this could be a very quick interview. No, we can't trust Sam Altman. But there is actually so much here, much more than just kind of profile, so to speak, of Sam Altman. You guys spent over a year Reporting this, conducting 100 plus interviews and it really brings the receipts on a lot of information while putting it into a full narrative. And so there's a couple of things that I think have been underreported as news and might have legal ramifications. So I wanted to unpack those because it is lawfare. First, the general volume of lies or the lack of public disclosure to the corporate board and the possibility, if not legal fraud, certainly fraud in the colloquial sense to the public. Second, I want to talk about alignment and trust and safety, safety issues if we have some time. And then third, the national security implications, obviously and the reveals that you bring up. And so let's start with the fraud, not necessarily in a Therano style. It's not like you have a product that doesn't exist here. Right. But what you do have is a product that Altman has been putting out there with OpenAI that continuously makes promises on a level of safety and a commitment to safety as you expose over and over again in the piece that he doesn't make good on, backs away from. And there's comparisons in your piece to SBF or to Elizabeth Holmes. So what is your takeaway and what you were hearing about potential kind of public accountability for the way Altman was saying one thing and doing another, both in terms of product safety and then also in terms of conversations with regulators?
- [02:47]
A
Yeah, I mean, there are many layers of this and maybe I'll just sort of take a couple and then Ronan, you can take whichever ones I don't get to. But I think I should just say really plainly, we are not alleging in the piece or here that we know that OpenAI or Sam Altman broke this or that law at this or that time. Right. The purpose, and I think the reason I bring that up, is that this is sort of why a lot of this stuff has been in the realm of kind of, I mean, the accusations, as you say, of borderline fraud or potential fraud are many in the piece. And it's an interesting mix of like, what you don't have is this sort of smoking gun thing where the reason Altman was fired in 2023 is that he was caught red handed embezzling funds or it's not that kind of allegation. What you have is this kind of very slow buildup and accumulation of facts such that they're very hard to kind of put together in one place, which is why this took so long. And it is, I mean, since you point out the fact checking and since you have experience with it, we literally did have the best fact checkers in the world on this and we really did get it so bulletproof that it's kind of hard to contest. But to the question of, like, which frauds rise to the top or allegations of fraud, one of them is we talk to a lot of economic experts who talk about, you know, what they call circular deals in the industry. And this is not unique to OpenAI. This is kind of a lot of these, what are called hyperscalers. You know, there's really just a few of these companies building this incredibly highly leveraged, risky product that requires massive, unprecedented amounts of capital expenditure in order to build the infrastructure to make the next model. So even if you are extremely bullish on the economic potential of the product, you could still think this is a bubble. And in fact, Sam Altman at times says this is a bubble because even if the product is, as you say, it's not a figment of the imagination, it's not like a tulip craze, it could still be a cause of this frothy investment. And you can get into the details of how when OpenAI and Nvidia strike these deals, for example, they're kind of both investing in future products that don't really exist yet and that depreciate very quickly. And so there are all these sort of microeconomic allegations of what people call borderline fraud or circular deals. And then to your question about the safety stuff, I think as a baseline, it's very easy for people to look at a tech company and say, oh, are we so naive that we're shocked by a tech company saying they won't do bad thing X and then they do it right. And I think this is a bit of a different case where, you know, the articles of incorporation under which OpenAI was set up as a nonprofit make very clear that they have a legally binding fiduciary duty to do certain things. And what we chart in the piece is this kind of years long process of the company kind of converting itself out from under those restrictions such that now they're one of the biggest for profit companies in the world. That is not something you can do without angering a lot of people, including people who you recruited and who took pay cuts to work at your company.
- [06:14]
C
I'd also point out on the sort of fraud question, and I want to be clear, you know, we're wearing our lawyer hats here, and I'm not going to talk about fraud as a matter of law, but when you talk about, in a colloquial sense, things that feel materially deceptive and may have real legal ramifications of one kind or another. It's really important to note the way in which this company was behaving amidst this backdrop that Andrew just described right of you have promises that don't always line up with what's being delivered. You have that on a scale that critics in this piece are saying is beyond even the baseline of Silicon Valley being founded on that kind of a hype model. You have material assurances to board members about real concrete safety testing requirements that are just turning out to not be true. And then you have a superstructure of handling of this that also raises question marks according to many people around it, stakeholders and outside legal experts that we spoke to. So one great example of this that hasn't been previously reported is that a part of the mechanics of Sam Altman coming back after board members fired him several years ago for this alleged pattern that we document of apparent deception and manipulation is that he got rid of some of the board members that had moved against him, but as a condition of their exit, they demanded an outside review and he was reluctant. We report that people at the time recall Altman saying, you know, I don't want any review. Any review, just by its existence, regardless of its conclusions, could make me look guilty. An understandable concern. But ultimately, after a lot of tense negotiation, he acceded to that demand. And the way that it went down was that he got rid of two keyboard members who had moved to fire him. He, in close consultation with their major outside financial backer, which was Microsoft, basically selected two new board members. Larry Summers, which is a name we can talk about, but at the time, you know, he was designed to confer legitimacy because he had been a cabinet member and he had been the president of Harvard. He's since stepped down because of his role in the Epstein files. But, you know, he was a big cheese who was brought in alongside Brett Taylor, who had been at Facebook and was also viewed as a steady hand. But, you know, we. We uncover how, during a previous round of Brett being considered for that board, actually, there had been people who were concerned that he was too close to Altman to be independent. So you have these two people that Altman selects, you know, pretty closely. He's in all the conversations basically proposing these names. And then you have a process where they go to, you know, a very tony, legit law firm, Wilmer Hale, to do an outside investigation. And the interesting thing is Wilmer's legitimacy for these kinds of investigations. It comes from a lot of transparency in many of the prominent cases where they've done it. You know, Enron, WorldCom, these are cases where they publicly released voluminous details of investigations. In this case, we report there were a ton of people around this investigation. I think we cite six people close to it who really felt it was designed to limit transparency and as one put it, to reach a foregone conclusion that they went in, in the perception of these critics. And of course, you know, we have the lawyers involved defending this. They say, no, it was. It was comprehensive, it was independent. We. But there were a lot of people around this who felt that they were not interested in the core questions of integrity that were underlying the firing. It seems they did deal with them because people wouldn't stop raising them. You know, I want to be careful about this. It's not that they totally scoped it away from that, but ultimately, it really seems like what outside investors who wanted Sam back were looking for was, did he sexually assault someone? Did he embezzle? Much more bright line things. And with respect to the actual findings, we actually have someone involved saying, well, you know, on the honesty and integrity question. The review, I'm only lightly paraphrasing here, did not find that Altman was a George Washington's cherry tree of integrity. So, so you have that underlying content of a review and then the really extraordinary thing in my view, which is yes, private companies sometimes keep these things out of writing. It's worth noting that there are plenty of legal analysts who say that's a red flag. It's a way to prevent liability and avoid scrutiny. When you get to a case like this where it's a high profile scandal that engulfed Silicon Valley and where all of the, the stakeholders around it, senior executives at this company forget the public, expected to see findings, they on the advice of those two new board members that had been selected in close consultation with Sam and their personal attorneys, you know, the Wilmer lawyers obviously were also in on this decision. But out this particular private advice fed in kept it out of writing. It was only oral briefings. And look, the piece is very sober about appraising all the arguments some of the lawyers make that this is fine and appropriate. We actually have one of those new board members who oversaw it, Taylor saying on the record there was no need for a written report. So they're acknowledging this now. There are a lot of people around this who understandably say that defeats the entire purpose it was supposed to achieve. They released only 800 words of essentially a press release on the website saying there had been a breakdown in trust, nobody knew what the hell had happened. So I hope this can bridge some of that divide of misunderstanding. And this brings us full circle to your question. You know, under Delaware corporate law, there's like Section 220 obligations. For any nerds that are watching and know the mechanics of this, if this company IPOs. I think one reason why you can imagine the pushback and comment seeking and fact checking process on this point was very intense with the parties involved. Is there is real concern we have, you know, board members telling us well if there were these deficiencies, there may need to be another review. You have a law firm that does gigantic business on the basis of the legitimacy of these kinds of reports. And you have a, a legal system that entitles future shareholders to go back as they often do in these kinds of scandals where things are insufficiently documented and demand the internal underlying records. And you know, judges have often ruled in favor of those demands. So that is something of a potential powder keg I think in the view of a lot of people around it.
- [13:11]
B
I mean I read this and that was exactly what I was thinking. I was like, this had to go through so many lawyers to be able to basically put this stuff into writing. Because for exactly the reason that you state, I mean, what Andrew talked about is exactly correct. Some of this is Silicon Valley business as usual, you know, like lying about your ping pong score. Like, who doesn't do that, right? But some of this is just, you know, is salesmanship. No one's, no one's saying that that is the stuff that people are like. It is the actual rubber meets road of putting these things into a corporate structure, binding board members with fiduciary duties to those ideas and then kind of backtracking and like reverse engineering the entire nature of your company and everything that investors kind of had bought in on while simultaneously actually deceiving your board. I mean, I was kind of baffled and thought that. I mean, and it was again, and this is a lawfare podcast, so we have the right audience for it. It was a nerdy kind of point, but like, the idea that Wilmer did not release these details, the idea that they gave these oral briefings, like, how bizarre. Like, why give briefings at all?
- [14:21]
C
And we really sort of stumbled into the full extent of this. Like, we had Sam on the record saying, oh, you know, I don't know about it being just an oral briefing to those two new board members that I helped pick. You know, it was, it was, I believe, very strongly it was given to all of the subsequent board members who, who joined in the aftermath of this. I'm paraphrasing, but, you know, that's. You can find his exact quote in the piece. And then, you know, we looked into it and that's simply not true. And we have, you know, other people around us saying that is a lie. And really it was just this extremely limited, deliberately undocumented briefing process over the course of it. And, you know, Wilmer is also very careful about saying that their findings were not exonerative. You know, that their role was just. All they'll say is as summarized in that press release, which is based on whatever they found, those two board members Sam helped pick decided it was appropriate to have him continue. So it's all very cagey. It's very deliberately undocumented. And as you say it, it really, in the view of, of many around this, defeats the purpose of. Of bestowing the li. Legitimacy that you would get from a proper investigation. I think it's a great parable about the need for transparency in these kinds of corporate situations, especially with these vast existential and safety stakes for all of us, right? This isn't even normal business. This was a 501. You know, it actually is a disservice to Sam Altman that the investigation was handled this way because that's how you wind up with a 16,000 word New Yorker investigative piece years later. And there's a lot of legal authorities that talk about this very question in this very way that if you keep things out of writing for short term game, you know, and ostensibly it's to protect privilege and whatever, you know, to avoid as much liability as possible. But it has these longer term costs that people can speculate that there are much worse things that they found. So I again hope that we've done something to A, restore a little bit of the transparency there about what the underlying complaints were and B, send out this message loud and clear that there are these costs that companies and law firms doing this kind of work should reckon with, I think in a, in a more acute way.
- [16:50]
B
Andrew, do you have anything to add?
- [16:52]
A
You do have some instances of like, for example, Greg Brockman writing in a digital diary, like, I would like to make a billion dollars or something. So you do have a few people sort of writing down the thing that they.
- [17:02]
C
Yeah, and also Greg Brockman in a diary writing, you know, essentially like wondering whether it's a lie that they're saying that they're a nonprofit and then turning around and maybe becoming a B corp, which they were planning to do very early on. It's truly remarkable. And the fact that all of these guys keep such voluminous journals. This piece is so full of journaling.
- [17:23]
A
I know.
- [17:24]
B
It's insane. I was, I'm like, who is telling these people to keep a diary?
- [17:28]
A
I know.
- [17:29]
B
Do they think that they're like, kind of someone is going to like, be finding their notes like Archimedes someday or something and like wrapping them into like, that they're going to be part of this genius historical record.
- [17:40]
A
I mean, as, as, you know, they absolutely do think that. And. Yeah, and that's why, you know, this is not. I mean, obviously this is a piece about a person, you know, sort of being used as a lens onto these structural things. But I don't think anyone should walk away from this piece thinking like, oh, Sam Altman shouldn't be the AGI dictator like Elon should or Demis should or Greg Brockman should. Right. Like, obviously, I think the structural takeaway here is closer to like, who is vesting these dudes with this amount of power and how could that possibly end well? And yeah, as you say, a lot of people, I think sometimes take the easy way out sort of mentally from this. People who are dismayed by the accelerating power of these technologies. And they sort of say either it's a parlor trick or it's stochastic parrots, or this is all hype and an attempt at regulatory capture. And I think among many things, what that misses is you can think that sort of sitting at home and watching from the sidelines, but the people involved in this do not think that. They think they are playing like a Ring of Sauron level competition and their actions back that up.
- [18:49]
B
Yep, there is a huge piece of this and we'll kind of get to this kind of in a little bit, which is just that it's happening. Like it doesn't, it doesn't matter whether you think it's real or not. It is like there is enough money and enough buy in that. This is just becoming the reality that we're swimming in. And it doesn't matter if they're, they are like stochastic parrots there. If there's a trillion dollars invested in it, then we are all going to be run by stochastic parrots.
- [19:15]
A
And if the, if the autonomous military murder drone is run by a stochastic parrot, like that's still a problem.
- [19:22]
B
Right? And so it does hit me and Andrew, I know that you have spent a lot of time kind of, you and I both have spent a lot of time and the speech platforms kind of looking at social media and it just feels to a large extent exactly the conversation we were having in like late teens, early 20s, which was like. And this feels like AI is just speedrunning these issues. Like all of a sudden we're going to have these incredible, incredible megacorps, these transnational companies that transcend state power, right. Running these things. And there's no one to, to curb that. I think that we should just like kind of skip to the country plan, which is the idea that I guess Paige Headley, who has come in, was a former nonprofit public interest lawyer and she was going to be OpenAI's policy and ethics advisor. And she made a talk about OpenAI could basically avert a catastrophic arms race by building a coalition of AI labs that would eventually coordinate with an international body akin to NATO to ensure that the technology was, you know, deployed safely. And you have in like your piece that Brockman, Greg Brockman just like could not grok this. He could not understand why someone would engage in this type of like, new governance in this type of multi stakeholderism in this type of like, do the right thingism. He was like, where's the money? Like, how do we make money in that? And then begins this thing called like that you call the country plan, which is essentially like, why not pit selling this technology between China and Russia?
- [20:54]
A
Why not go wrong? Yeah, yeah.
- [20:56]
B
What could possibly go wrong? And then you, obviously, the rest of the piece is this like incredibly detailed conversation about how Sam's already in the process of doing that to like the way of selling the technology to the Gulf states. And so I don't know, I guess we'll kind of skip to the big question, which is like, is there any way to stop this if you have this kind of coalition of the willing between these giant moneyed countries and the rule of law is falling around our feet? And like, at the very least it feels, I, like, I, you know, you kind of like are hoping that the IPO gets cancelled. You're hoping. But like, I have to say, I finished your piece and I was like, in another world, in another time, this would have canceled the ipo. And I'm like, does it even, Is this going to matter?
- [21:42]
C
This is for, for me, the, the bigger point. You know, Sam Altman is one figure, he's a consequential one, but he's especially consequential as a window through which to view a wider dynamic that critics in the piece describe as a race to the bottom. And that's a kind of confluence, right, of what you described of legislative and regulatory guardrails on private industry, and particularly big tech, falling away more widely, an anti regulation bent in American politics under the current regime, and a backdrop of money in politics that increasingly allows Silicon Valley to really have its hands on all of the levers of policymaking and power. If you're running for office right now in the United States, you are contending with a flood of PAC money that is specifically focused on quashing AI regulation. Greg Brockman, Altman's second in command, has contributed to one of these major Trump friendly anti regulation pacts. So there is just a massive amount of machinery that is pushing away from safeguarding us from any recklessness, any lack of trustworthiness of the type that we write about in the piece. And the tragedy of it is these companies and individuals are the ones best positioned to clock what the risks are. And they keep telling us the risks are grave, right? You know, autonomous weapons in war zones, you know, the potential for mass surveillance, the devastating misinformation potential of this technology in electoral contexts, all of the Kind of more individual mental health stakes where you have all of these wrongful death suits against OpenAI alleging that ChatGPT contributed to suicides and a murder potentially. The list goes on and on. And yet there is just very little framework to deal with any of it. And the inflection point of the structures these companies themselves set up, right? OpenAI being this safety focused nonprofit, having a board that could fire its CEO if they thought he was lying serially about material things. That all fell away when Sam Altman came back. Sam Altman is not the only one participating in these broader dynamics. But very often OpenAI has been a first mover, and I think we both viewed the story as one that just tells the bigger story as well. And we can talk about some of the kind of shortfalls in policymaking steps in terms of regulation and legislation that get at a whole range of the issues that we've already raised. Some of them are being kind of trialed elsewhere in the world. Some of them have come up in proposed state regulation, though that too faces an uphill battle. We talk about that a little bit in the piece of how there's also a lot of machinery to get rid of any state efforts. But I think I was eager to have this conversation because that is the part of it that I feel is most gravely urgent, that we all need to marshal whatever resources we have as Americans to get at these broader shortfalls of accountability for companies that really do. As one person said of the Sam question in this piece, you know, it's people with their finger on the button in a very consequential way. And, you know, forget the more acute example of Sam and his alleged issues with honesty. You know, you look at the. In the. In their own eyes anyway, good guys at Anthropic, right, They've got breaches right, left and center. They're also weakening their safety commitments. So I think the point is we are placing all trust in unaccountable private sector actors. And I fear that we're not going to wake up to the consequences until something really devastating happens.
- [25:41]
A
Yeah. There are also major public sector actors. They just tend to be like Sheikh Tanun on his yacht in the uae. So I should say two things on this, just to piggyback on what you're saying, Ronan, it's often sort of painted as naive or kind of like, why are these sci fi nerds even talking about all of these risks? And for one thing, these mega dystopian sci fi risks are being discussed because people like Sam Altman told us to be afraid of them. Right. So this is not something that Ronan and I invented one morning to scare readers about. And we're very kind of, I think, sober about it. In the piece of the people who founded these companies told us to be afraid of this. And moreover, again, there are people who think that's just a move to try to attempt regulatory capture or to attempt to gain attention for the next round of investment. And there may well be some of that. But you bring up people like Page Headley. These are not all competitors who work for Anthropic or who work for Google or who are trying to scope the next round of investment funding. I don't know, maybe Page will have an IPO before the end of the year and I'll be proven not cynical enough. But there really were many, many people who we talked to in the hundred something people we spoke to who were not, you know, were very careful to not be just laundering competitor gossip through our piece. Like, there are a lot of people who really are concerned whistleblowers now. Their fears may be proved right or wrong, and unfortunately, we're going to live to see whether they're right or wrong. But it's not just all competitor hype and, you know, the normal stuff of capitalism.
- [27:21]
B
Yeah. So I wanted to say that that's another thing that I absolutely value about this piece, is that it is soup to nuts. It is. You talk to people on the factory floor of these companies and you talk to people. I have a list, actually of just like all of the different ranges of perspectives that you got in this piece. And one of the things that I think would have been a disservice and made this piece less valuable is if you had just focused on the Game of Thrones aspects of, like, these kind of companies, the C suite, the people who are trying to be the narcissists and to, like, run the world and who are writing their diaries so that one day people will study them and talk of their genius. But one of my really big takeaways from my time in Silicon Valley and my time inside companies is that the majority of people, like outside the top 100 people in a company are there because they think that their job matters and that they're trying to do the right thing. You talk about how in late 2022, four computer scientists published a paper, motivated in part by concerns about, quote, unquote, deceptive alignment, in which sufficiently advanced models might pretend to behave well during testing and then, once deployed, pursue their own goals, that this moment actually compels altman to create like a super alignment project where he like purports to care about safety and like going to set up this huge team and recruits on it. And then of course it just falls away. It just goes away.
- [28:44]
C
And he. And the announcement actually says specifically, like, we need to create this to prevent the extinction of humanity.
- [28:52]
B
Right?
- [28:53]
C
Yeah. So again, to Andrew's point, like, it's not us introducing those stakes. I think that's a great example. There are many others in the piece. I was struck by the thing you just said about the gap between the kind of C suite leadership and they're sort of writing down their own history for the books. And then, you know, the working level people where I share your impression. You know, again and again Andrew and I were encountering people who really came to this company, this nonprofit at the time that they joined for the early ones because they believed that there was a need for the thing that Sam Altman was pitching, which was something that was not just governed by profit and growth. And that would stand really defiantly against that, that had a corporate structure that was uniquely designed to prioritize safety. And very often those people at a working level are much less, less heard. And you know, that's one of the most important equities in this reporting to carry the concerns of these researchers which continue to this day. And you know, some of the lower profile, safety minded people who had concerns, they actually, you know, raised them or attempted to raise them within the corporate structures in formal ways. We have a lot of emails complaining to people from, from safety leads saying, you know, the company is going off the rails on its mission. You know, essentially sort of whistleblower material that we surface and we even talk about. There's, there's a case where in terms of the corporate transformations into a more for profit mode, there's a moment where there's a board vote on that conversion and there's a board member who appears to vote against it, you know, according to the materials and sources that we have, and argued that the way this was being done was not appropriate, that the nonprofit was being severely undervalued, and said, I can't do this in good faith.
- [30:51]
B
And then it's memory hold.
- [30:52]
C
Well, right. An attorney says point blank, if we record this dissent, it might be a flag to investigate further the legitimacy of this new structure. And the vote then gets recorded as an abstention, apparently without this board member's consent. Now there's a disputed factual record here we carry OpenAI's pushback. They say several employees recall it being a deliberate abstention. But I found that to be a fascinating case. You know, we don't, as Andrew rightly pointed out, come to legal conclusions here. You know, I'm a lawyer, but I'm not your lawyer. OpenAI and I'm not going to come to conclusions on it. But I think it's worth raising because it has meaningful legal ramifications. Whatever the conclusion might be, it is certainly something that raises questions about potential falsification of business record charges under state law. You know, that would be a misdemeanor in most jurisdictions. But more practically, the lawyer for the, for the board made exactly the right point, which is that vote was consequential. It authorized a corporate restructuring worth hundreds of billions of dollars. And a tainted vote could ultimately create grounds to challenge the legitimacy of that conversion, which is really relevant to pending nonprofit litigation, the Musk suit, and to IPO due diligence. So all of these cases where, you know, to use your frame kind of very often the little guy, the less profiled people sound alarms in one way or another. And then to use your term again, sometimes it does seem like there's some memory holding it. It just, I don't think the math on it is smart, even from a pragmatic standpoint. For the people who are trying to make the complaints go away, I think it creates more problems. And I, and I really do want to talk before we part ways about the failures of structures around this company and the regulatory and legislative failures that just, they don't create any recourse, like to use some of these examples. There should be more robust federal AI whistleblower protection legislation. You know, it's crazy at this point when you have these companies themselves saying this is the most dangerous technology ever to not have the kind of regulatory regimen that we have for, you know, pharmaceuticals or for food even, you know, and for whistleblower protection associated with that, there should be something modeled on Sarbanes Oxley for AI whistleblowers. You know, people were very fearful to talk to us for this story, and people who are sounding alarms about this technology are doing a public service, and they deserve to be protected. You know, that's, I think, just, just one of the many regulatory steps that would make, or legislative. You know, you could come at it in multiple ways. That could really make a difference.
- [33:44]
A
Yeah, I mean, just, just backing onto that. Like, Kate, you were talking about the ways in which this is continuous or discontinuous with the trust and safety debates around social media. And I think it's interesting. There's this one moment in the piece where I think to some people it maybe seemed like we were doing a gotcha thing where we were saying, okay, let's talk to existential safety researchers at the company. And a representative from the company is like, what's that? That's not a thing. And first of all, I think just to be clear, we absolutely did not do that as a gotcha thing. We went in saying, please let us talk to the research teams that you have working on the original ostensible raison d' etre of this organization. And the reason that we talked about existential safety versus trust and safety is that the word does mean different things and there are legitimately different people who work on different aspects of safety. Some of it is anti doxxing, some of it is privacy stuff, some of it is making sure the models don't, you know, turn into Mecca Hitler and start spewing racist stuff at you. And some of it is this stuff that is in the more hypothetical sci fi realm of making sure they're aligned and don't kill us all. That to your point, Kate, the deceptive alignment research, we kind of just very quickly say in the piece, this all sounds crazy, except that under certain experimental conditions, it's already happening. And again, best fact checkers in the business, no one disputes that as a factual claim. These things are starting to happen under certain experimental conditions. Now, we don't know what that means. We don't know how fast this stuff is going to accelerate. But as Ronan is saying, the fact that the people closest to this technology are trying to make everybody wake up and start to panic a little bit, and we have a political system that seems incapable of that is a deep structural problem that, I mean, we just document in the piece. Like a California bill comes up. And as we report in the piece, an investor, Ron Conway, who's close to OpenAI, calls up Nancy Pelosi, calls up Gavin Newsom. The bill gets killed. Right. So this is the current system. It's not shocking to anyone that that's the system we live under, but it is unprecedented. The scale of the acceleration.
- [36:05]
B
No, And I think that we should switch to kind of and talking about this because I do think that we should have a conversation about what next? Does the law have any type of recourse? Can we push back on these things? I mean, after reading your piece, I'm incredibly bullish, and I was before, but on kind of regulatory potential, legislative potential, doing something to curb some of this and to make it so These companies are accountable to anyone besides their kind of megalomaniac CEOs. And like, I just don't understand how that's possible in this current government. It seems like a perfect storm of the closeness and the coziness between kind of this current administration and these companies.
- [36:51]
A
Well, and in fact, when, when there was almost a moratorium on all state regulation for 10 years, this isn't even in the piece because we had so much to get to, but it seems like Steve Bannon and Mike Davis and a lot of MAGA ish people were standing in the way of that. So it's a very strange bedfellows kind of situation.
- [37:09]
B
Yeah, I mean, I think that that's completely right. And then like, you know, I watched in Europe just from like afar, like people try to kind of create the AI act in Brussels and everything to create real kind of strictures. I mean, the other thing is that the terminology that is being used in this is so vague and so untethered to like mean technical meaning or you know, or even non technical meaning. To be totally frank. Guardrails. If I had a nickel for every time some asshole said guardrails to me, like we should just have guardrails. Like, okay, well that's really nice. Like on what? For what? Saying what. Yes, exactly.
- [37:45]
C
Well, let's talk about what guardrails could mean, by the way, because let's talk about it. The UK had an AI Safety institute long before we've tried anything of this type of the US tried to do this under Biden to basically have like some form of mandatory pre deployment safety testing that's actually meaningful.
- [38:05]
A
And Sam Altman was all for it at the time, but that.
- [38:08]
C
Right, and part of the, the pattern with Sam, as Andrew's pointing out, is that we document how he was the regulate US regulate US guy and he proposed a federal agency to regulate AI and he talked about wanting this, these kinds of testing requirements and then simultaneously was very often working against these very kinds of provision in regulation. To your point about what is actually possible, even thinkable in the current environment, I mean, the flood of money from AI into politics is enabled by this post Citizens United landscape where you can have Greg Brockman putting $50 million into the leading the future pack and he and his wife putting 25 million into Maga Inc. And that's just free speech now. And I think we are learning that the world we're living in, where this is possible and where the flood of money into politics really makes it unthinkable to meaningfully push back that that has real deleterious consequences for all of us. That that is endangering all of us. So that to me is why it's worth us putting a year and a half of our lives into pieces like this. I had just come off of, when Andrew and I started this a year and a half ago, a body of reporting about Elon Musk, which published at a time when people were not raising the red flags that they now openly raise about him. And the reason there was also not that I was, you know, so fascinated by this one man. It, it was that Elon Musk is a really leading example, sadly, of the ways in which these tycoons in Silicon Valley have developed supra governmental authority. And that story was about someone who had, you know, real geopolitical ramifications where he was the sole vendor in a lot of key areas to the defense sector and where basically, you know, senior officials had to scurry around doing his bidding. And he could, at a whim, in that case, turn off communication access through Starlink in the battlefield in Ukraine on the front lines, and people's lives were in the balance. And so that's the bigger story. And it's what we see in AI too.
- [40:22]
A
I mean, one way to kind of connect these threads, right, is exactly like you're saying about this kind of supra governmental power. When you go back to these early days of people sitting in a conference room with a whiteboard saying, what if we were to auction off AGI to China and Russia? Or what if we were to grasp the Ring of Sauron and become an AGI dictator? Or what if this wakes up one day and kills us all? In some sense, the best case scenario is that was all hype and bullshit and it was just trying to get investors attention. Anything other than that is kind of the worst case scenario, right? What if there's some even slim chance that something like that is true? And so I think to Ronan's point, it's like, yeah, it would be one thing if it was just someone like Elon or Sam Altman kind of having these sort of like self aggrandizing conversations about one day I'm going to own all these satellites that will be useful in war zones, or one day I will build a technology that the Pentagon can use to maybe conduct mass surveillance, or maybe one day I'll build a technology that can be interwoven into markets and transportation infrastructure and hospitals. And, but, but like the scarier thing is what if some portion of that comes to pass and Obviously, that's the timeline we're currently living in.
- [41:36]
B
Yeah. So I'm going to propose one thing. As long as we're on the topic of kind of what we can possibly do and who will save us from this, which is, I mean, it's a little idealistic, but the courts, and I know I would point exactly to kind of they are less viable or at least were than, you know, the other kind of branches of government and these other types of structures. I think one of the things that really struck me and you know, you don't. You just said before, like, that Anthropic has its, its problems as well. Like, it's not as if it's like a perf company, but you do have. And you end the piece with. And it must have kind of been one of the provocations to finally get this out the door. And kind of like a perfect coda to end on is the supply chain risk designation that the administration let, like came down. And with Anthropic, I have to say that reading everything that you wrote, I am just like, how on earth is anthropic the problem? Like, I mean, like, how could they possibly pivot to OpenAI as a substitute? You go through everything from like Altman failing to get a clearance. Right. To all of his dealings with the uae, with Saudi Arabia, you know, these back and forths and promises of safety and then not making good on them. And then you have like these kinds of conversations about, you know, geopolitical maneuverings, launching an unchecked safety OpenAI products in India, you know, just various types of things. Are you following kind of closely the anthropic case in the Northern District of California? Are you following closely the case in, like that we just got kind of Anthropic lost, at least initially with in the D.C. circuit. So, like, what are you thinking? Do you think that there is hope with the courts on any of this?
- [43:13]
C
Well, you know, it was striking to see after the Pentagon standoff that we narrate in the piece that the initial ruling from a California judge is this Judge Rita Lynn, who wrote actually a really powerful decision and talked about the designation being classic illegal First Amendment retaliation and called it Orwellian. And Anthropic was restored to federal procurement schedules. The latest ruling, I would not overstate its significance just yet. What it does is create kind of a mess. It's a procedural ruling. I mean, basically the argument that the circuit came up with was it's primarily a financial standing that Anthropic has and that the EQUITABLE balance is in favor of the government's interests on this. So it just. The D.C. circuit didn't say the blacklisting was legal. They just said anthropic didn't meet the very high bar for emergency relief while the case plays out. So the merits are still live. There's an evidentiary hearing on the 19th. It'll be interesting to see. I think you're exactly right in the big picture question, the courts can play a pivotal role at this time when, for all the reasons we just discussed, the legislative branch is so disempowered. And, you know, the courts also have in some ways been denuded of a lot of the legal thinkers that would be into holding private sector actors accountable. There are real problems with the judiciary right now as a source of accountability for a Silicon Valley run amok. But I do think you're right to identify that as one of the core areas of hope, because you do see rulings like the one I just described where someone said, you know, no, this is irregular and wrong and violates the First Amendment.
- [45:01]
A
Yeah. And I think just to add to that, all the conversations we're having about guardrails and regulation. Right. I think all of us would not want to give short shrift to how complex all of these discussions are and how there are good faith arguments. I mean, you heard a lot of arguments around when this supply chain risk designation was happening. Like, okay, for all you pro regulation people, is this the government you want regulating this technology in this way? Like, so there are good faith arguments that need to be worked out by people who are approaching these things from first principles. The problem is when it's all drowned out by industry fueled, you know, muddled sort of propagandistic talking points, you can't have that good faith conversation.
- [45:43]
B
Yeah, well, and I guess we can end on the fact that the, the muddledness. This is an incredible piece of journalism. It was a tremendous amount of work at, just full of, you know, just such information that the public needs, frankly, before you have a public company that is valued at a trillion dollars, which is more than the GDP of many countries in, in Europe, for example. And it's just, it's just, it's amazing that like, you know, that all of this wasn't available from like, everything from the Wilmer Report to, you know, knowledge about kind of some of how these alignment goals have been abandoned and the safety goals have been abandoned to, like, how this is getting released in other countries to the country plan, kind of the natsac and geopolitical Maneuverings. I think that this is going to be a piece that has incredible effect for a very long time. So.
- [46:33]
C
And I just want to say also in terms of the impact, I really appreciate the policy discussion, which very few people take the time and space for. And I hope anyone listening to this, I mean, this is a specialty audience of people who are smart enough to care, but they all have friends in the wider public, too, where they can explain why this matters. So I hope people, people start a meaningful conversation about this because the stakes really are that vast and I hope it extends to how we engage with our representatives. You know, you talked about the hope that the judiciary will play a role in creating, to use the term you love so much, guardrails. But also, I haven't given up hope about the legislative branch. You know, you see politicians already across different parts of the government clocking that it can be advantageous to play a role in creating accountability. Florida today just announced their AG's office just announced new investigation into OpenAI around some of this stuff we've talked about today. So I think that people are looking at the numbers in Washington as well. And there was polling last month from, I think NBC showing that a majority of the public now views the costs of AI and the risks as outweighing the benefits. So I do think we're all in this moment where we're kind of mesmerized by our screens and in the thrall of these companies. But if Americans join arms and really tell their representatives, we want you to do your job and make sure that we're kept safe from this technology, that could eliminate our jobs and threaten our safety and all the other things we talked about. Talked about. I do think the simple math of Washington can still lead to meaningful oversight and results if we really, really press for that.
- [48:19]
B
We have the midterms coming up. I mean, like, we've talked about how this is going to affect the IPOs, but like, I absolutely think this piece could play an AI existential risk and like fear, basic consumer and citizen fear of some of these technologies could absolutely play a role in how people vote.
- [48:38]
C
Before you vote, find out whether your representative is getting AI money and, you know, do scrutinize their policy positions related to that.
- [48:49]
B
Yeah. Well, guys, thank you both for coming on. Andrew Ronan, it was wonderful to have you.
- [48:55]
A
Thank you, Kate. This is awesome.
- [48:57]
C
Thanks, Kate. Great to be here.
- [49:02]
B
The Lawfare podcast is produced by the Lawfare Institute. It's if you want to support the show and listen ad free, you can become a Lawfare material supporter@LawFairMedia.org support supporters also get access to special events and other bonus content we don't share anywhere else. If you enjoyed the podcast, please rate and review us wherever you listen. It really does help. And be sure to check out our other shows, Scaling Laws, Rational Security Allies, the Aftermath and Escalation, our latest Lawfare Presents podcast series about the war in Ukraine. You can also find all of our written work@lawfairmedia.org the podcast is edited by Jen Patya with audio engineering by Kara Shillin of Goat Rodeo. Our theme song is from Alibi Music, and as always, thanks for listening.