← The Lawfare Podcast
The Lawfare Podcast
Lawfare Daily: Page Hedley and Gad Weiss on OpenAI’s Latest Corporate Governance Pivot
00:47:02
Loading summary
Transcript54 lines
- [00:01]
Podcast Host
The following podcast contains advertising to access an ad free version of the Lawfare Podcast. Become a material supporter of lawfare@patreon.com lawfare that's patreon.com Lawfair also check out Lawfare's other podcast offerings, Rational Security Chatter, Lawfare, no Bull and the Aftermath.
- [00:32]
Advertiser
Support for this podcast and the following message is brought to you by E Trade for Morgan Stanley With E Trade you can dive into the market with easy to use tools, $0 commissions and a wide range of investments. And now there's even more to love. Get access to industry leading research and insights from Morgan Stanley to help guide your decisions. Open an account and get up to $1,000 or more with a qualifying deposit. Get started today@etrade.com terms and other fees apply. Investing involves risks. Morgan Stanley Smith Barney LLC Member SIPIC E Trade is a business of Morgan Stanley Optimize your nutrition this year with Factor America's Number one Ready to Eat Meal Service. Factor's Fresh Never Frozen meals are dietitian approved. Ready to eat in just two minutes. Choose from 40 weekly options across eight dietary preferences like calorie smart, protein plus and keto. Eat smarter@factormeals.com listen50 and use code listen50 for 50% off plus free shipping on your first box. Factor meals.com listen50 code listen50.
- [01:37]
Paige Headley
I'm not surprised by this. I am surprised that people are not more exercised given the fact that this is the whole point of those safeguards. This is the moment they were put in place to guard against the risks that they're put in place to guard against.
- [01:56]
Kevin Frazier
It's the lawfare podcast. I'm Kevin Frazier, the AI Innovation and Law Fellow at Texas Law and a contributing editor at lawfare I, and Alan Rosenstein, University of Minnesota Law professor and lawfare senior editor, were joined by Paige Headley, senior advisor at Forecasting Research Institute and co author of the not for Private Gain letter urging state attorneys general to stop OpenAI's plan restructuring, and Gad Weiss, the Wagner Fellow in Law and Business at NYU Law.
- [02:27]
Gad Weiss
If investors come up with a structure that allows OpenAI to attract capital and talent on better terms and be a stronger competitor, which will allow OpenAI to build AGI in a way that it wouldn't be able to do otherwise, society could also benefit from that.
- [02:45]
Kevin Frazier
Today we're talking about the twists and turns of OpenAI's corporate structure. The money people would likely not put corporate governance near the top of their list when it comes to AI governance. This group makes clear that the legal incentives shaping OpenAI's operations as well as those of other labs will have a major impact on AI development. So thinking about the headlines we've been reading for OpenAI and its change in governance structures throughout the years, starting way back as a nonprofit, then pivoting to this sort of hybrid structure and and now allegedly and reportedly considering a for profit structure, we need to kind of get back to basics and understand what are the different pros and cons of various corporate governance structures. So gad, can you just start with an overview of the nonprofit structure? What's the traditional rationale for opting for that approach over some of the alternatives?
- [03:44]
Gad Weiss
All right, so thank you Kevin. Trying to avoid too much of a business associations. 1 of 1 answer I'll go over this very quickly. So when you want to start a new business and you can choose different kind of business entities, one of the questions you have to ask yourself is how free you will be to promote a social purpose or a charitable purpose over the interests of other financially motivated stakeholders that you might have. So one option that you have is to go for the complete pure nonprofit structure where you have an entity that might be funded by donations from third parties, but these donors cannot expect to make should not expect that business to be run to promote their financial motives. Right. A nonprofit should be run to promote a certain charitable purpose. As we go down the scale, we can choose different kinds of business entities where that social or charitable purpose can be balanced with stakeholders financial motives. And we can think of the Public benefit Corporation which is also highly relevant both in the context of OpenAI and other business entities. So these are entities that in which managers directors are required to balance between stakeholders, financial interests and a social purpose defined in that entity's charter. And on the complete other end of the spectrum you have the regular plain vanilla corporation where you have a whole discussion about what does that structure actually require in terms of how much weight should you give to maximizing shareholder value as opposed to broader stakeholder values. But certainly does not give you the same kind of freedom to put the social or charitable purpose above all else as you could have in a nonprofit structure.
- [05:30]
Alan Rosenstein
So Paige, we're going to get to you about the OpenAI specific history here, but I do want to ask one follow up question of GAD first, which is can you talk through the implications of changing from one corporate structure to another? Specifically when a nonprofit tries to go and become a for profit, either a regular for profit or a public benefit corporation? Why is that such a big deal? Because at least as I was reading the news reports and commentary, there was this real concern, at least among some critics of this. This was a massive. They almost talked about it as a theft of assets. And I was really trying to understand why is it so profound and why are there such complicated rules? What is at stake when you go from being a nonprofit to. To being a for profit? And why is that potentially maybe more controversial than going from one for profit form to another for profit form?
- [06:17]
Paige Headley
Right. So just to add some gloss on what GAD said, under the nonprofit structure, it's not just that the company can pursue its charitable purpose above shareholders interests, but it must. It is legally required to. And if it doesn't, it can be sued, it can be brought to court by the attorneys general. That is a big deal. There's a clear primacy of the charitable purpose above all else. So that is the deal with the nonprofit. If you switch to even a public benefit corporation, that is very different. So under the public benefit corporation, the company is permitted to consider a public benefit goal in addition to shareholder returns, but it is not required to. It's required only to do some vague balancing. And there's literally never been a case of any public benefit corporation being held to account for that. In part, that's because the attorneys general can't sue. The only party that can sue are shareholders. And how often are shareholders going to sue because the company has favored them too much relative to the public benefit mission? So that by itself is a pretty big change for people who think the charitable purpose is important, because what it means is to the extent the interests of shareholders and the public diverge here, and that is the premise of OpenAI's founding, that these interests are going to be very different in the case of a transformative technology like artificial general intelligence. If they're different, then reallocating the organization's interest from the public to the shareholders either endangers the public because they're incurring risks they wouldn't have otherwise incurred, or they're literally reallocating expected wealth and power from the public to the shareholders. So if you believe in the importance of the charitable purpose and also just protecting what's owed to the public, you should be pretty exercised by this plan.
- [08:07]
Alan Rosenstein
And so gad, back to you then, sort of more generally then how do these nonprofit to for profit conversions tend to work? Sort of what has to happen, because my understanding is that this does sometimes happen though.
- [08:19]
Gad Weiss
So I think a main purpose to consider here is that you, when you have leadership of a nonprofit considering switch to for profit structure. At the moment that they make this decision, they are still a nonprofit. The decision to switch, the reasoning behind it should somehow serve the charitable purpose. And you have to take into consideration that if you did raise funds from donors, they did that based on a certain expectation and understanding that they are contributing to a nonprofit and not to a for profit business. So these are all things that the management of a nonprofit should bear in mind when they're considering this kind of restructuring, which in some instances could make sense. It could promote a charitable purpose.
- [09:03]
Paige Headley
Yes. Just to piggyback on that, I think the example of organizations doing this, that is the relevant precedent here is the hospital conversion cases. And without talking about the specifics of those, there are a lot of goals that would be more effectively pursued in a for profit structure. If your goal was to make awesome smartphones at cheap prices, don't incorporate as a nonprofit. If you did that by mistake and then you realized, oh my gosh, there's this incredible market for this, you should switch. And switching would be consistent with your charitable purpose. So that is possible, provided switching advances your charitable purpose. That is the key question. And so the fair market value question that I think many people have been asking, I think is the wrong question. It might be relevant in some cases, but ultimately the question is, does this change in structure advance or undermine the charitable purpose?
- [09:53]
Kevin Frazier
And with that key purpose in mind, Paige, in your letter, the not for Private Gain letter, signed by yourself and a number of former employees of OpenAI, a number of academics, a number of researchers, you all go into great detail about the original understanding and mission OpenAI had when it decided to choose this nonprofit structure. So can you walk us through why, given the known costs of developing a massive AI model, given the amount of financial capital you need and some of the traditional incentives that you would think about for creating a large language model or a large AI system, why opt for this nonprofit structure? And in particular, if you could dive into the weeds of what was that explicit charitable purpose that OpenAI initially adopted?
- [10:45]
Paige Headley
Yeah, great. So at the time OpenAI was founded, the only company explicitly trying to build artificial general intelligence, which, for those not familiar with it, could be the most impactful technology humans ever create. This is certainly the view of OpenAI's leadership was Google. And this was very concerning to OpenAI's founders, including Sam Altman and Elon Musk and Greg Brockman, in part because they thought that the interests of a company and its shareholders very much not the same as the interest of the public, and this was too important of a Technology too big of a deal for it to be under the ownership of a company like Google. So they founded a nonprofit in 2015. It was very unclear how we would actually get to AGI, as we call it. This was before ChatGPT. This was before the first paper about GPTs being a paradigm that might end up yielding the kind of results it's been yielding for many years. And it was unclear that it would be so expensive to build this technology. So OpenAI knew it needed a lot of capital to be at the cutting edge, because being at the cutting edge is an important part of its mission. It could not do what it wanted to do by being an academic think tank, by doing research. The donors who supported it could have written checks, were writing checks to academics. What OpenAI offered to do that no one else was doing, was be in the game, be a player, have the cutting edge models so they could figure out how to make them safe, so they could actually steer the policy and governance conversations to ensure this technology is developed in a way that benefits the public. A couple years later, it turns out that building this tech is even more expensive than they thought. And I think that wasn't obviously foreseeable. They had a lot of capital. They raised $130 million from donors. They could have kept raising money at that level. That was plausibly enough for them to be at the cutting edge. But it turned out it wasn't. They needed extraordinary amounts of capital.
- [12:38]
Kevin Frazier
Yeah, just to pause there, dad, if you could dive into how that issue around the incentives of a nonprofit structure also may impose a sort of cap on your ability to stay at the leading edge of AI development in general, when we're thinking about nonprofits in a sort of abstract manner, if you could go really into the weeds of why that may not be the proper vehicle for staying on the edge of something as capital intensive as AI development.
- [13:07]
Gad Weiss
So if you're going to develop AGI, there are two main kind of resources that you need. One of them is financial capital. It is very expensive to develop this kind of technology. The second is human capital. You need the best and brightest on your team. You need people like Paige to help you build it. And in order to try to attract capital and talent on the best possible terms, you need to offer the right incentives for them to join you. If you're going to go and check and try to raise money from investors, you need to offer to them a business structure that makes sense. And when startup investors think about investing in startups, the only way for these investments to make sense is for them to have an ability to rake in disproportional profits from the few good investments they make, because we know the startup investments fail in famously high rates. And when you try to attract the best kind of employees to your startup, you need to always consider the alternative. These are people that can go work for big tech companies or for other hot startups. So we need to be able to offer them the kind of compensation that would attract them to come and would incentivize them to do a good job and to stay for the startup for longer term. A major aspect of that compensation package is your ability to give them equity compensation and the kind of equity compensation that has a significant upside potential that can really allow them to participate in the startup success.
- [14:30]
Kevin Frazier
All of this.
- [14:31]
Gad Weiss
These are structures that are either impossible or complex to create under a nonprofit structure.
- [14:37]
Kevin Frazier
Great. So we know these key limitations of a nonprofit approach, especially with respect to the needs, the essentials of AI development. So, Paige, you mentioned that we started to see that despite substantial donations coming into OpenAI, we needed to maybe rethink how OpenAI was going to be structured. And this all came to a head in 2019. So what were the key factors where suddenly we saw this pivot? Maybe a traditional nonprofit is, we're reaching the end of the road with respect to that vehicle for achieving our goals. What was the rationale for moving to a new structure and what did that structure look like?
- [15:20]
Paige Headley
Yeah, so it's a balance. I think there's a quote from, from Sam Altman. I don't recall the specifics we have in our letter about, you know, the nonprofit gives them some of what they, what they need. The for profit structure gives them some, some. Some other parts of what they need. But they go all the way in that direction. You know, they would end up creating the problem they tried to solve. So they wanted to strike this balance, this grand compromise. And this was the 2019 restructuring. And on paper, I think it's pretty reasonable and happy to go over the basic components of it. So OpenAI restructured from a normal nonprofit relying on donors. And donors would receive no returns on their donation to a hybrid capped for profit. And what that means is they had a for profit vehicle, an LLC that offered investors a return with a caveat. I'll come back to that caveat in a sec. And the LLC had an unusual operating agreement. The operating agreement stated that its primary fiduciary duty, its primary objective was the charitable mission. And not just the charitable mission, also the OpenAI charter, which is an operationalized version of the charitable mission. So it's not quite so vague and there are some important provisions in there and that everybody, investors and employees, before they joined, before they invested, had to basically sign a document acknowledging that this was the constitution of OpenAI's commercial entity. That wouldn't be enough because that wouldn't be legally enforceable by the attorneys general. So on top of that, the manager of the llc, the entity with the authority to run day to day operations, complete exclusive authority, is the nonprofit. So the nonprofit and the LLC have the same duty to put the mission first. And the attorneys general have jurisdiction over what the nonprofit does. And so if they fail to discharge their fiduciary duties, the ags can step in. So that that is what allows the current structure to put the mission first and for that to be legally enforceable, which I think is like the most important part of its current structure. A couple things on top of that that are important. One, I mentioned that there's a caveat around how investments work. They're capped. And the reason they're capped is because OpenAI believes that AGI could generate extraordinary amounts of wealth, more wealth than any technology humans have ever created. And they don't need investors to be incentivized by that. Like pretty ordinary returns would be enough for them to raise the capital to be at the cutting edge. So they just need to create enough incentive for people to invest in their operations without promising some extraordinary, unprecedented level of future wealth to these shareholders. So what they said is you'll get a cap on returns. For initial investors. It was 100x, which is extraordinary, right? It's certainly high enough for people to feel pretty good about this investment. Despite that, investors are now unhappy about that cap. And so some people are skeptical that 100x is a meaningful limit at all. Well, if the investors are unhappy about it, that should be some reason to update your views about whether this is a meaningful limitation. So currently any profits above that cap belong to the nonprofit to benefit the public. Under the new plan, it would all just go to shareholders. And the last one is, currently you have partners like Microsoft. Part of the deal is they have rights to the intellectual property that OpenAI creates. But again, this is all part of this grand compromise just so they can get to the goal. And once they get to the goal, the goal, according to their founding documents, is to use this technology to benefit the world. So they said, Microsoft, you don't get access to artificial general intelligence, you get access to the pre AGI tech. But once we get there no one has any right to it. It belongs to the nonprofit. That means they can give it to the US Government. The un, a consortium of nonprofits, use it so scientists can solve, I don't know, mortality. That would also be gone. So all of these things, the legally enforceable primacy of the charitable purpose, the profit capsule, and who actually owns controls AGI, are in jeopardy under this restructuring.
- [19:36]
Kevin Frazier
And what stands out to me, as you note in your not for Private Gain letter page, is that again, the mission here that I think has gotten lost in all of the discourse around OpenAI's various pivots, wasn't necessarily to be the first to AGI, but to assist humanity with the adoption of AGI. So it wasn't necessarily we have to be the first and only. But okay, if someone else is first, let's make sure we can help out with this adoption by society writ large. So, gad, I want to come to you for this new hybrid structure to get an understanding of what may be some assumptions that OpenAI had about how that structure was going to work and how it might not work in practice. Right. So you have this idea of a new hybrid system that maintains the charitable purpose of the nonprofit, but there's a reason, or you tell me there's usually an understanding of why we don't see this hybrid structure copy and pasted in most contexts. So what were some of those assumptions that you think OpenAI or some other lab may have held when they were adopting this unique approach?
- [20:50]
Gad Weiss
Well, I think the question of what OpenAI had in mind while adopting this structure kind of depends on how cynical you are about this whole project. But I think that an important aspect to consider here is that we're discussing about the cap profit structure and about who controls OpenAI, what exactly the relationship are between the nonprofit level and the for profit level, and what kind of interest investors hold or the founders hold. We need to consider that in a company like OpenAI, the governance structure has another important aspect that does not build of board seats or formal control rights. There's a whole other layer of informal control. So OpenAI is not an early stage startup, but it resembles an early stage startup in the sense that it is highly dependent on both investor capital and on one of its founders, in this case Sam Altman, which is essentially inseparable from OpenAI, as OpenAI's board had to learn the hard way at the end of 2023, you can't really separate the two. Sam Altman is OpenAI in a sense, and OpenAI is Sam Altman and just.
- [21:57]
Alan Rosenstein
To explain, just for those who may not remember, this is when in November 2023, the board tried to, I mean, it did in fact, fire Sam Altman. The details around that seem to be continuously extremely opaque as to why they did that. It seemed to be concerns about Altman's, honestly, candor, basically, and his willingness to tell them things. And then the board really backed off when there was, it seemed to be a total full scale revolt from OpenAI's employees and its outside partners. And I'm just curious, sort of what that tells you, if anything, about whether this structure worked as it was intended to, that whole saga.
- [22:35]
Gad Weiss
I think what this saga tells you is that there's a reality you have to acknowledge. And the reality is that OpenAI's investors and Sam Altman will hold a certain extent of control over OpenAI regardless of how you allocate the board seats or how you structure its governing documents. That's just a reality. And you have to face it if you want to build AGI. And the first part of OpenAI's mission is to build AGI, right? It's building AGI that will benefit society. But the first part is building it. And if you are going to do that, you have to acknowledge certain realities. Now, does that indicate that OpenAI's structure is not working or. Or working exactly as planned? That depends on how cynical you are about this project, which is what I said. So one way to think about that is that we try to have the nonprofit board have complete control over OpenAI, but obviously that doesn't happen. And another way to think about that is we didn't really want that to happen at all. We acknowledge that there are informal and formal control structures. And by placing all formal control with the nonprofit, we try to create some.
- [23:37]
Paige Headley
Sort of balance just to jump in just with going back to what Kevin said about the mission and to put a gloss on what Gad said, OpenAI's mission is not to build AGI very clearly. Its mission is to ensure it is built safely and for the benefit of humanity. It states in its charter that one way we'll try to do that is to build it, because who's better placed to ensure that goes well than the company building it? But it would consider its mission accomplished if someone else builds it. And there are specific risks to racing to build this tech that can make the situation more dangerous. And that's why the charter actually commits, under some of those circumstances, to stop competing and basically fold up shop and assist another organization in order to achieve that. Same objective. I think this is a very important distinction that people have not sufficiently focused on.
- [24:27]
Kevin Frazier
And Paige, sticking with you for a second there, we saw this structure set up. It was unique. It was a hybrid structure not replicated by many entities. You have a bunch of board members who are in this confusing world of at once managing a nonprofit's mission while also trying to maximize profits of this LLC that you're kind of stewarding. And so weird incentives going on there. And at the same time, we've seen the level of competition domestically increase in terms of who are the leading AI labs, what's their user base, what products are they offering? And then, of course, we have things like the deep sea moment where we realize perhaps the moat people thought existed between China and the US Was way smaller than previously anticipated. So all this pressure is heating up around AGI and being the first to reach AGI. And then we started to get some rumblings of, hey, maybe this hybrid structure has now outlived its longevity, just like the traditional nonprofit structure outlived its longevity. So can you walk us through that evolution that OpenAI had, realizing, huh, maybe this structure isn't working. Let's consider the for profit structure and what alarm bells started to go off as a result of that consideration?
- [25:47]
Paige Headley
Yeah. So first, of course, you know, I was there when they were converting from the nonprofit to the current hybrid structure. I have not been there as they've discussed converting to the, you know, a complete for profit. So I can only speculate. But I'll say that when we were discussing what the world would look like, what opening eyes, incentives would be if it were successful building AGI, this is what we envisioned. These are the temptations that we were trying to plan around the governance safeguards we discussed. The mission comes first, legally enforceable, etc. That is OpenAI's effort in 2019, 19 to tie itself to the mast. It was predictable that when it got close enough to the big prize, the amount of money, the amount of power, the influence from investors, employees even, who have equity interest would be extraordinary. And so absent very strong unremovable protections, this is what we would expect. So I'm not surprised by this. I am surprised that people are not more exercised given the fact that this is the whole point of those safeguards. This is the moment they were put in place to guard against the risks that they're put in place to guard against.
- [27:07]
Podcast Host
Hey, do you insure your car, your home? Do you have a personal liability policy in case someone sues you unless you're elon Musk. It's a good idea because if something bad happens, you want to be protected. But what about you? Are you protected? What happens to your income, your family's future if something happens to you? Hit by a bus, plane crash, heart attack. Stuff happens. Policygenius makes finding and buying life insurance simple. Ensuring that your loved ones have a financial safety net they can use to cover debts and routine expenses. You can compare quotes from top insurers and find coverage that fits your needs and your budget. With Policygenius, you can find life insurance policies starting at just $276 a year for a million dollars in coverage. It's an easy way to protect the people you love and feel good about the future. Okay, this is a true story. A couple of years ago, my cabin in the woods flooded and was almost completely destroyed. And and the insurance company, my homeowner's insurance didn't pay a dime to rebuild. We were stuck with the whole cost of it. Can you imagine doing that if say, you had a mortgage to service on a property and your partner was gone? Life insurance can cover loved ones expenses if something happens to you. Mortgage payments are a common cost that could be covered by life insurance too. So I have life insurance because I don't want my family to have to worry about money after I'm out of the picture. Policygenius is a great way to get the right policy for you. Combining digital tools with the expertise of real licensed agents, you can compare quotes from America's top insurers side by side for free. Policygenius's licensed support team helps you get what you need fast so you can get on with your life. They answer questions, handle paperwork, and advocate for you throughout the process. Life insurance is not a one size fits all product, and policygenius doesn't treat it like one. They lay out all your options clearly. Coverage, amounts, prices, terms. No guesswork, just clarity. So check life insurance off your to do list in no time with Policygenius. Head to Policygenius.com or click the link in the description to compare free life insurance quotes from top companies and see how much you could save. That's policygenius.com.
- [30:00]
Advertiser
Every idea starts with a problem. Warby Parker's was simple. Glasses are too expensive. So they set out to change that. By designing glasses in house and selling directly to customers, they're able to offer prescription eyewear that's expertly crafted and unexpectedly affordable. Warby Parker glasses are made from premium materials like impact resistant polycarbonate and custom acetate. And they start at just $95, including prescription lenses. Get glasses made from the good stuff. Stop by a Warby Parker store near you. You were made for going the extra mile for your pet, and Farah Pets was made to do the same. Farrah Pets supplements are vet created with innovative ingredients that combine Eastern and Western therapies so your pet can thrive. After all, you would do anything to help your pet feel their best. And so would we. Visit FarahPets.com pod to get 20% off today. That's F E R A Pets.com pod Farah Pets. We were made for this.
- [30:59]
Howie Mandel
I can't tell you how often I hear, oh, I'm a little ocd. I like things neat. That's not ocd. I'm Howie Mandel and I know this because I. I have ocd. Actual OCD causes relentless unwanted thoughts. What if I did something terrible and forgot? What if I'm a bad person? Why am I thinking this terrible thing? It makes you question absolutely everything and you'll do anything to feel better. OCD is debilitating, but it's also highly treatable with the right kind of therapy. Regular talk therapy doesn't cut it. OCD needs specialized therapy. That's why I want to tell you about NO CD I. NOCD is the world's largest virtual therapy provider for ocd. Their licensed therapists provide specialized therapy virtually and it's covered by insurance for over 155 million Americans. If you think you might be struggling with OCD, visit nocd.com to schedule a free 15 minute call and learn more. That's n o c d.com.
- [32:01]
Alan Rosenstein
So what is it then that you wanted? Right? You and your colleagues that signed this letter, like, what were you hoping for? Just to raise sort of public outrage, to guilt OpenAI into acting in a different way to get state attorneys general involved. Like, what was the specific ask here? And do you feel like you've succeeded?
- [32:19]
Paige Headley
We've not succeeded. The audience, first and foremost of our letter were the attorneys general of California and Delaware. Those are the attorneys general that have jurisdiction here to hopefully help them see the issues as we saw them. And I think we collectively have perspectives on both the history of OpenAI and the risk from the technology and the corporate governance and law issues that we thought we could combine to provide an accessible framing of the issues. In part, we're motivated to do this because we've seen so many people talk about fair market value. The framing in the media had been the nonprofit has some economic interest in this for profit entity. The relationship between the nonprofit and for profit will be severed. Well, that nonprofit must be paid well. What is the fair market value of their economic interest? And I think that is the wrong question, because even if they got the fair market value, that would not mean they were advancing their mission by selling their economic interest. And you know, you can think of lots of silly analogies. If you are a nonprofit with a mission of protecting a specific patch of rainforest, and you sell that patch of rainforest to a lumber company for fair market value, you're not advancing your mission even if the sale price was fair.
- [33:32]
Alan Rosenstein
But let me just put you on that for a second, because it occurs to me that if the goal of the nonprofit is to make sure that whoever builds AGI, it will be built, or at least to be implemented and adopted into society in a beneficial way. Well, if you have a nonprofit that is running a for profit and that for profit wants to go and do something right, and the nonprofit is going to get a trillion dollars in return, I'm making up numbers here, right? But let's say the nonprofit actually gets the proper fair market value, right? And if you're upset about this whole thing, it's because you think that OpenAI might actually succeed, presumably, and so therefore the fair market value in such a situation would be extremely high. Well, why can't the nonprofit then say, look, now I have a trillion dollars now I'm a pure nonprofit profit. I'm finally properly capitalized and I'm going to go. And now without the distraction, frankly, of having to run a for profit research lab, social media, chatbot company, or whatever OpenAI wants to become, I'm now going to do the God's work that I was meant to do. What is wrong with that? Was what I just said.
- [34:42]
Paige Headley
I think there are two responses to that. First is, imagine you're a donor considering donating to OpenAI in 2015, and you have this vision, this goal. What you want to do is invest in frontier AI companies, OpenAI, Google, and then all the returns you get from those investments will go to a foundation to be used in the manner you're describing. If the goal were to get a slice of the economic upside to be used to charitable purposes, there are far more direct ways of doing that. That was not the goal of the donors here. Relatedly, on the first thread, one of the main reasons for this structure, for having these guardrails and OpenAI to ensure that it's not just racing to develop this tech for competitive reasons, but taking the public's interest into account is because people believe that companies can proceed dangerously and irresponsibly in ways that endanger the public. There will be lots of trade offs when they're deciding how thoroughly to test a model for safety, whether people can create bioweapons when that model is released versus getting it out quickly before a product launch to raise more money. And those are exactly the context in which you want to have a board with a legal duty to say no. I understand that the investors want you to release this next week, but you need to wait two more months so we can finish the testing. So that's one reason why simply investing in the companies wouldn't be enough. Another more cynical way of thinking about this is if we're just thinking about it in terms of expected returns for the nonprofit, why would the investors want this change? So right now the investors, they have their investment with profit caps. They want to change the structure, remove the profit caps. The nonprofit gets some amount of shares. The exact amount is still tbd. We don't know that amount. If the expected wealth to the nonprofit was higher after the restructuring, why would the investors want this? And I think a simple way to think about this is the investor's interests and the public's interest are creating some sort of zero sum game here. If the investors get something, it comes at the expense of the public, whether that's wealth or risk.
- [37:00]
Kevin Frazier
Yeah, and what stands out to me too, building off of that conversation is the fact that it's often neglected that corporations, nonprofits, these are state created entities. In the old days, way back when, you actually had to get an act of a state legislature authorizing you, for example, to assume the corporate form or to have this authorized nonprofit status. And so these are in many ways a contract. You are a state created entity. And so, as you all point out in your letter and argue forcefully, is that by taking this nonprofit structure and proposing to go to the for profit route, it's essentially an ultra viras and beyond the law, outside of that purpose act, where if you agreed to the state, we are going to exist for this reason. Right. So way back when we had sewage companies, for example, that said we exist to build pipes, and then all of a sudden they wanted to pivot into horse poop pickup. Well, hey, actually that wasn't within your explicit purpose. So we're going to yank your charter. You no longer get to exist. So I think it's important to really see that this is a specific relationship that OpenAI has with Delaware, that it has with California, which is why the AGs could be the Actors that say, no, I'm sorry, you don't get to go that route. This is what we agreed to contractually to allow you to operate. So gad, we've talked a lot about for profit structures and nonprofit structures and you talked briefly about public benefit corporations. Can you tell us more about how those exist in California and or Delaware? What's the traditional structure look like and what are some of the rationales and pros and cons of that structure?
- [38:49]
Gad Weiss
If I can just take a minute before that to respond to some of the things Paige said. So first, I don't think we should really view on investors interest at a public interest here as a zero sum game, not in terms of wealth and not in terms of risk. I think that from an investor perspective, in many cases being mindful about AI safety as you build your products will be the right business decision. And on the other hand, I also think that if investors come up with a structure that allows OpenAI to attract capital and talent on better terms and be a stronger competitor, which will allow OpenAI to build AGI in a way that it wouldn't be able to do otherwise, society could also benefit from that. And so this is not necessarily a zero sum game. And also think that investors have other legitimate reasons to want to affect these kind of changes in OpenAI's both economic structure and governance structure. That has nothing to do with mission drift, right?
- [39:50]
Paige Headley
Yeah, I appreciate the question and that's a good opportunity for me to clarify what I meant right now. In the current structure, OpenAI is required to put its mission first. That does not mean it can't consider investors interest. It does not mean it cannot go out and try to raise a bunch of money. That does not mean they cannot make a bunch of products what it cannot do, because there are lots of overlaps in what that achieves and what the mission is designed to achieve. What they cannot currently do is advance investors goals when they are very much not in the public's interest. So there's a delta there. All that's at stake if their interests were entirely aligned. We're back to the smartphone case. You made a mistake by incorporating as a nonprofit. Just trying to make money. That's the best way to benefit the public. So the premise of OpenAI is the public's interest and the shareholders are not the same. And shareholders are saying you're doing some compromising, whether in practice or an expectation that we don't like. So we want to shift the balance. So in that context it is zero sum. The delta is what's at issue here. There's a ton of overlap. That's fine, that's unaffected.
- [41:00]
Kevin Frazier
Let's pivot to the current proposal. So, Paige, can you walk us through? We had these early considerations of perhaps OpenAI is going to go full for profit. They're going to send it for profit instead. They've kind of walked that back theoretically in, in some ways to this public benefit corporation approach. How do you, as a signatory to this not for private gain letter feel about this scaled back proposal? In some ways, is it adequate? Is it problematic? What role do you want ags to play now?
- [41:32]
Paige Headley
Yeah. So first I want to question the premise of the question that this is a scale back at all. So what it was going to do before is convert the LLC into a pbc. That is still the plan. Before the nonprofit, which is currently the manager of the llc, has an economic interest in llc, it was going to sell that for some unknown price in return for shares in the PBC. That is still the plan. The only update is OpenAI has said that the nonprofit will have some form of controlling shareholder interest. Most likely that's special voting rights. This is a plan that was already under consideration. There was a Financial Times article from February that speculated that OpenAI might do this for self interested reasons because it would help Sam and Crew fend off hostile takeover bids from Elon Musk and from Microsoft. So my cynical best guess is they took this thing that they're already considering and they're repackaging it as a major change as a big concession. But on the merits, the question is what is the import of the nonprofit having voting control, shareholder voting control of the pbc? And the answer is not much. By default there might be all kinds of non standard contractual obligations that they could implement. They haven't shared the details, but by default it wouldn't change any of the things that I mentioned being concerned about. By default, the PBC would still have different duties than the nonprofit. The PBC would have this balancing obligation that's not enforceable, that is not affected by whether the nonprofit is a controlling shareholder. Their duties are not affected by that. The amount of control of the controlling shareholder is very different than the control they had as the manager of the llc. It is rumored that they might not have the authority to fire directors, which is the only relevant tool as far as I understand that they might have as a controlling shareholder. And the attorney's general authority to intervene only extends as far as the authority of the nonprofit board. So if you have a fairly powerless nonprofit board that can appoint directors but cannot fire people, which is most likely outcome. There's little they could do to actually stop the PBC from completely ignoring the public's interest. And there's virtually nothing the attorneys general could do if the mission is completely jettisoned by the company.
- [43:59]
Alan Rosenstein
So in your view, is it the case that the world is not so different today than it was two weeks ago, that the battle is still very much ongoing, that this is largely a PR move to get people like you, or maybe not people like you, but the broader media, to the extent that it has cared about this, to say, okay, I guess this has been resolved, we can move on to something else. But from your perspective, the battle that you are fighting is still very much being waged and none of this changes that fundamentally?
- [44:30]
Paige Headley
Yeah, I don't think it changes things fundamentally. I want to be careful about speculating about OpenAI's motives. I think that the headlines have wildly mischaracterized the importance of this update. I think the fundamental question that people should be asking OpenAI is, you know, right now in the LLC's operating agreement they state contractually that the charitable mission comes first and the nonprofit has the extensive day to day complete control. As a manager, they could do that by contract in the PBC's certificate of incorporation. They have not said they would do that. So until they commit to doing that, I assume they're not going to do that and people should be asking them about that.
- [45:09]
Kevin Frazier
And gad, from your perspective, is this a meaningful change going from the for profit to a pbc or do you agree with the general assessment that perhaps this isn't as substantive as some have been led to believe?
- [45:22]
Gad Weiss
I think that the most important thing that we have to keep in mind when we are thinking about a privately held PBC and going back to some of the things that Paige said before, defaults don't matter much. Privately held startups are built to a significant extent on all kinds of private, ordinary solutions. So even though shareholders in a PPC by default may not have much means to control the way its directors make decisions on AI safety issues on others, in fact, the way that OpenAI might look like after this incorporation might be very different. And we have today recent changes to Delaware corporate law that has even made it easier for shareholders take more and more responsibilities that ordinarily would be placed with the board and have to and to hold more control directly on how a company is managed. This is one thing we have to keep in mind. But I think that maybe even more important than that we always have to ask ourselves not only if this new structure allows for a mission drift that previous structure did not. If we're really trying to understand what the risks are here, we also have to ask ourselves two other important questions. First, how likely it is that investors do really have that kind of mission drift in mind, which I think there are good reasons to believe that they don't. And second, how much of this structure really reflect a change from what we currently have in OpenAI today? Also, again, taking into account not only the allocation of formal controls in OpenAI, but also the informal controls that are incredibly powerful in a company like OpenAI, which is again, highly dependent on both investor capital and the active involvement of its founder.
- [47:03]
Paige Headley
I mean, so regarding the investor mission drift point, the charitable mission would not be the investor's mission. It just wouldn't be. They would be investors in an LLC with, sorry, PBC with different goals. So it would not be a question of whether they will stay true to the mission. They just have no responsibility to the mission absent very non standard things that they could do in the incorporation.
- [47:24]
Alan Rosenstein
All right, so I think one important takeaway from this conversation is that this question over OpenAI's corporate form is still very much an ongoing live question. And we'll have to see for the reasons Paige outlined and the reasons you gad just outlined about informal organization, we'll just have to see how this plays out. And we will sort of revisit this, I'm sure, going forward. The other question that I have had, and I want to finish our discussion with this because it's really been lingering as I've been thinking about this over the last couple years actually is why does this matter? And I want to pose the question in the following way. If OpenAI was the main player in this space, then I could understand the real focus on corporate control. And I generally understand it. I mean, even if OpenAI wasn't important, if you ask people to donate assets, like the rule of law matters. But the stakes really were, I think in part because there was this perception for a long time that OpenAI was going to get to AGI first, or even if that wasn't necessarily OpenAI's mission, they were going to be one of the leaders here, and obviously they still are. But as Kevin mentioned earlier, just in the United States there seems to be enormous competition. And Google in particular, which I think is notable because I think it was my sense that Google was in part the reason why OpenAI was formed, has I think really just in the last six Months hugely accelerated its progress, I think it's fair to say, at least my sense of the industry is that Google is probably fundamentally in the best position right now, in part because of its deep expertise, its massive compute advantage, is one of the few companies that's not as reliant on Nvidia because of its own sort of stockpile of, you know, TPUs and its own custom custom silicon. It's. It probably has the best data just given, you know, what access Google has, right? And then, of course, you have Meta, you have X, you have a bunch of other companies, you have the Chinese companies. It just seems very plausible to me that OpenAI is not going to get to AGI first and that therefore this just matters a lot less, right. In a world where other people get to AGI. So why am I wrong, in other words? Right. Why should we still care about this issue beyond the sort of pure corporate governance rule of law question, which totally accept as important, but that's not the reason the New York Times cares about this. And so that's what I want to finish with.
- [49:45]
Paige Headley
Yeah, great. I think there are three things. The first one is the one you mentioned that you don't want to get into and arguably says a bad precedent. Sam Altman has made commitments to the public in Congress for a decade. That should be worth something. Bracketing that second, to the point you were saying about there are a lot of competitors. OpenAI isn't necessarily the front runner. Who the front runner is changes. By all accounts, OpenAI has been the, or arguably the front runner for a while. So they're not just one of many companies. They have a very likely chance of being the first company to develop technology that is extraordinarily powerful, whether we call that AGI or not. So I think they are in a select group of companies that we should be paying particular attention to. But I think the most important reason here is OpenAI's theory of change, or at least one of its theories of change, was to be a role model organization, was to be in the thick of things, to be a competitor, to be at the cutting edge and take the lead by example on what responsible AGI development looks like. And we see lots of examples right now because we have no regulation in the space of companies putting out a document explaining how they'll go about testing their models before they're released, or making transparency commitments, or making commitments to give government agencies access to a model before they're released. And what that does is it sets an example. It creates incentives for other companies to follow suit and criticism. If they don't, that creates this race to the top. And OpenAI was supposed to be taking the point, taking the lead on being a role model. It is not assuming that role in the way it was designed to assume. And I think even if it's one of many competitors, there would be great value in it actually assuming that role.
- [51:31]
Kevin Frazier
And gad, any final thoughts you want to share before we end here?
- [51:35]
Gad Weiss
I just want to say that I completely agree with Paige. I think that this is just not how innovation usually works. It is important that OpenAI stays and important player in this competition to reach AGI. Think for example about the market for GUI, right? Xerox was there first, Apple was the first one to package it into a personal computer that many people wanted and Microsoft at some point had the advantage of putting on the market a product that is more friendly for developers and had better compatibility. So I think it's important for us to a society that OpenAI stays in this market and is still competitive and offers a product or technology that offers a different balance between capabilities and AA safety. We can all benefit from that even if it's not the first one to reach AGI.
- [52:24]
Kevin Frazier
Well, the only thing I feel comfortable predicting at this point is that there will be new news in the near future. So let's leave it there and look forward to having you all back soon. Paige and gad, thanks so much for joining.
- [52:37]
Paige Headley
Thank you so much for having me.
- [52:38]
Gad Weiss
Thank you.
- [52:42]
Kevin Frazier
The lawfare Podcast is produced in cooperation with the Brookings Institution. You can get ad free versions of this and other lawfare podcasts by becoming a Lawfare material supporter at our website lawfairmedia.org support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Look for our other podcasts including Rational Security, Allies, the Aftermath and Escalation. Our latest Lawfare Presents podcast series about the war in Ukraine. Check out our written work@lawfaremedia.org the podcast is edited by Jen Pacha. Our theme song is from Alibi Music. As always, thank you for listening.
- [53:36]
Alan Rosenstein
If you're alignment in charge of keeping.
- [53:38]
Howie Mandel
The lights on, Grainger understands that you go to great lengths and sometimes heights to ensure the power is always flowing. Which is why you can count on Grainger for professional grade products and next day delivery so you have everything you need to get the job done. Call 1-800-granger. Click granger.com or just stop by Granger for the ones who get it done.