← The Tech Policy Press Podcast
The Tech Policy Press Podcast
Should AGI Really Be the Goal of Artificial Intelligence Research?
00:42:36
Loading summary
Transcript37 lines
- [00:01]
A
Your film is now ready to be shown.
- [00:11]
B
Good morning. I'm Justin Hendricks, editor of Tech Policy Press, a nonprofit media venture intended to provoke new ideas, debate and discussion at the intersection of technology and democracy. The goal of achieving artificial general intelligence is shared by many in the field. OpenAI's charter defines AGI as highly autonomous systems that outperform humans at most economically valuable work. Last summer, OpenAI announced its plan to achieve AGI within five years. While other experts at companies like Meta and Anthropic quibble with the term, many working in the field of artificial intelligence recognize AGI as either an explicit or an implicit goal. Researchers at Google's DeepMind went so far as to set out levels of AGI, identifying key principles and definitions of the term. Today's guests are among the authors of a new paper that argues the field should stop treating AGI as the North Star goal of AI research. They identify obstacles it creates to more productive pro social goal setting.
- [01:11]
C
My name is Eric Salvaggio and I am a visiting professor in the Humanities, Computing and Design Department at the Rochester Institute of Technology and a Tech Policy Press Fellow.
- [01:24]
A
I'm Boran Villier. You can also always call me Beau. I just this week started a new role as a data scientist at TD Canadian Bank. I have to say because of this this work was done prior to joining td. It does not it's not in any way connected to td. And the content of the paper, but also of today's conversation are entirely my own opinions. They don't represent views of the bank.
- [01:47]
D
I'm Margaret Mitchell. My main role right now is at Hugging Face as a chief ethics scientist. So I'm a computer scientist by training, but I work on operationalizing ethical values within the tech industry. And we don't all speak as a whole in Hugging face. We're distributed. So similar to veo, I don't represent the views of everyone, although I may represent the views of some people at Hugging Face.
- [02:11]
E
You also don't necessarily speak for all of the authors of the paper we're going to discuss today, because there are more than a dozen authors on this paper. Stop treating AGI, artificial general intelligence, as the North Star goal of AI research. Which caught my eye because this is the heart of the AI narrative. At the moment when Masayoshi Son stood next to Donald Trump in the White House announcing the Stargate initiative with Sam Altman and Larry Ellison, he promised Donald Trump we'll get to AGI within your term. Mr. President, Artificial General intelligence is nearly here and we're going to do amazing things. I don't know quite where to start with this, other than to say, why is that wrong? Why is that goal the wrong goal? Perhaps? Meg, I'll start with you.
- [03:08]
D
So I wonder when people hear that, what they think he's talking about. Part of the problem that we're getting at in the paper is that this term, this AGI term, doesn't have a very concrete meaning, but it does function as a narrative for people to prioritize their own interests. And so there are different kinds of definitions of what AGI might mean. But when we're in a position of talking through what we're going to achieve in a position of power, then this is more about just advancing technology that meets the needs or the interests of the people in power and giving it a. A positive sentiment by calling it intelligent, by calling it general. While what general means is not well defined, and what intelligence means is not well defined, in psychology, cognitive science, neuroscience, all these different sort of fields, it's just functioning as a narrative to move forward the kind of technology that the people in power just want to move forward.
- [04:13]
C
I would just say that I think there's an element of this AGI frame which is, as noted, quite vague. And I think there's, from my perspective, a lot of utility in that vagueness, in that by not defining what artificial general intelligence is in real terms, you get to do all kinds of things and pretend that it's AGI. You get to point at all kinds of outcomes from various types of models and say, this is on the path to AGI. This is sparks of sentience or whatnot, which is a different conversation. But just to be clear, it's a similar trend. And I think for me, this points to a kind of almost like a form of faith, whether you take it seriously or not, in terms of do people believe it or are they just utilizing it? But there is a form of faith that serves as an orientation for the AI research industry that often comes at the expense of a focus on real material needs, focus on social concerns around the technology, and oftentimes can be used to serve as a substitute for real political deliberation and sometimes as an outright replacement for the types of political conversations and participatory conversations which you may think are the same thing or not. For me, they are. And so I think AGI really deserves to be clearly specified so that we could say, what do you mean, Meg?
- [05:50]
A
You know, the way you were describing, the vagueness in the language enables people to do all sorts of things. And that's a concern. And Eric, the way you were describing, like, people are saying AGI is a goal or they're not saying it, what does that mean? Should be the question. And when we're faced with this question, we should just be, I don't know what you're talking about. The thing I want to add just to this question of, like, why is it the wrong goal is. And I just want to take a little bit of a step back, this wasn't a topic for me personally. That was very salient until relatively recently. The backstory for this paper, but also just personally how I became interested in the topic was so Talia Ringer, amazing researcher in the spring of 2023, was like, hey, folks, there's a lot of interest in this topic. Why don't we bring a group of people together, a very large group of people together, think about the topic, and to write a position paper that's trying to think about AGI and critique AGI. And for me, my way in was not having been interested in AGI, it was instead having been interested in the very, very surprising parallels between critiques of human intelligence research and critiques of AI valuation. It was a surprising parallel, not in the sense that folks have obviously been thinking about everything that goes wrong. When you start with imagining human abilities and then trying to measure very bad proxies or all sorts of things in machines that you make you think, sound, or look at the human stuff, there's all sorts of things that go weird. But the way we got into this, it was Liv and I was Liv Hank Oxley. And I was thinking, instead of what is similar in the structure of the work that goes into making assumptions about some capacity that you're trying to measure, some property of interest, call it intelligence in the case of humans, call it whatever you will in the case of AI evaluation. And it wasn't coming in. We really weren't looking at this from the perspective of you're trying to measure something like general intelligence in machines. We were looking at the structure of how you define what you're trying to measure. And the thing that surprised us is that the AI evaluation community, on its own terms, came to the exact same kind of conclusions that folks had come to in thinking about everything that goes wrong, but also how we should think about what goes wrong in the case of measuring human abilities. And for me, that was the entry door. It's like, having done that kind of comparison, why is AGI the wrong goal? For me, the question of what intelligence is at its core has that feature of Always specifying things that are desirable. It's a value laden concept is the way I like to think of it. So things that are desirable, things that are of value, things that are of ethical, social, political importance, together with things that you can look at, things that you can describe, things that you can observe. And when you're dealing with notions that have that quality of both specifying what does good look like, but also how do you observe the thing you're calling desirable or good? And when you're looking at concepts that have those two notions, you always, at the end of the day, at some crucial layer of the topic dealing with disagreements about what is politically, socially, ethically desirable, that feature of disagreements about what matter becomes a feature of the topic. Just thinking about why is AGI the wrong goal? The first question on my mind is, what disagreements are we having? Or are we maybe not having? Because we're using this very vague language that masks the underlying question of what priorities are we bringing to the table in talking about AI research? And not just the social priorities, but also what research priorities, what engineering priorities. All these questions of prioritization require explicit consideration. And for me, like the first like, step where I just come off is like, we need to be having a conversation about these underlying disagreements that aren't happening. And even before I come around and say, don't ever talk about hei, which like, personally, like in this group, I feel like I'm more on the side of the. I've been surprised coming in, not knowing much about the topic, looking at accounts of AGI, I've been surprised reading how many accounts I found incredibly thoughtful. There's a lot of surprising work to me on this topic that I don't end up finding unrigous or uninteresting or unimportant because of their focus on the concept. I was surprised by that. For me, that was a huge surprise. But what disagreements are we not having? And what questions about what matters and to whom are we jumping over? For me is like the thing that's super front of mind is like, why is it the wrong goal?
- [10:53]
D
One of the things that Beau's really getting at here is what we call the illusion of consensus in the paper, where you are asserting something with this kind of assumption that everyone knows what you're talking about and agrees on it being good. And that drowns out all of the other possible ways of contextualizing the AI problems, all the other ways of thinking through what's worth pursuing. And so by putting forward this concept of AGI, we're moving everyone down the sort of same path. They don't really know where they're going, but there's this kind of, this illusion of consensus to the detriment of sort of critical analyses of what AI might really be useful for and what it might not really be useful for. So it's creating sort of these exclusionary effects. It's creating this kind of thoughtless moving forward in a, in an ill defined direction that really leaves out a lot of the technology that, for example, I care about, coming from someone who worked at a company that did assistive and augmentative technology. This kind of thing where AGI is not the goal of AI in that context, the goal is assisting people. And all of the critical analyses you need to do about what the technology is doing relevant to that get sidelined in favor of this other sort of apparently great thing that we really don't have a clear conception of.
- [12:33]
E
An illusion of consensus is one of the six traps that you say hinder the research community's ability to set worthwhile goals. I do want to go through to some extent each of these, but this illusion of consensus seems like the really big one. It's the one you put first, of course. I think for a reason. I connected my mind to generally the illusion of consensus that I think Silicon Valley wants us all to have about not only the direction of AI, but the direction of the planet, where we're going as a species, what we want to accomplish, why we need this technology to save us from all of the various existential harms that we might face, including climate change. So it feels to me that this illusion of consensus goes a little further than just contestations around the term AGI or even the goal of AGI.
- [13:29]
C
I think that comes to what I was talking about before, about this idea of AGI being not just a technological orientation, but an ideological orientation. And to me, the orientation is a fantasy about the salvation of concentrated power ultimately, right? Because it's a dream where we. There's kind of thing that gets thrown around with AI and AI research, of solving, right? We solve creativity, we solve writing, right? And here I worry that what we are solving is the process of consensus building that goes into politics, which is inevitably a contestation site, right? Democracy is contestation democracy. If you solve democracy, you ultimately end democracy because you are favoring somebody's consensus or omitting the entire method of consensus building. There's that question of who decides what that consent, which consensus we're going to choose, whether that's Silicon Valley elites or a Machine in the true science fiction sort of sense. Right. There are versions and definitions of the AGI sort of mythology which says we'll ask the AGI how to solve climate change, for example. Right. But that is a real techno centric solution. And we see this a lot in not very fancy AI. We see it in large language models.
- [15:02]
A
Right?
- [15:03]
C
There's this misconstrual of product, as if it is the goal of the process. But there's a lot of endeavors where process is the point. And I think process is the product of a democracy, much as, say, a term paper is the product of grappling with your thoughts.
- [15:19]
A
Right.
- [15:19]
C
Which is why an LLM is not good for that. For the same reason an AGI is not good for the product of democracy, which is the process, which is that contestation, which is. I kept bringing up Chantal Mouff agonistic pluralism, right? Like you need to have the site for contestation. And as soon as the contestation goes away, democracy goes away. So if AGI is used to reach that goal, do we actually want that at all? And is this right? Are we building systems that do not allow for political participation in goal setting that sort of solve that problem? If we are, then that's a very dangerous thing. And I will say many people are not. Right. But this looseness of the goal means that even if you don't think you're building that, you might be. This is to me why the laying out these traps was so important.
- [16:16]
E
You never know when you're laying the pipe for authoritarianism until perhaps it's too late. Yeah. Let me ask about the second of these problems. Supercharging bad science. You lay out multiple kind of sub problems here in this area. Why does pointing towards AGI lead to bad science?
- [16:39]
D
I think that one of the things we're trying to get at here is that speaking to Eric's point, there's this sort of theological belief or this sort of religious belief in AGI as being some wonderful thing to work towards to the detriment of critical thinking about all of the pieces at play. So there generally is an under specification of the concrete goals of why AGI should be around or what specifically that would be. There is a lack of scientific rigor. So I think most people in like middle school in the US at least learn about like the scientific method. So you put forward a hypothesis and then you test that hypothesis and that sort of thing under the umbrella of the pursuit of AGI, all of that sort of rigorous science is abandoned and justified by this belief that we're working towards something inherently good. So a lot of the sort of rigor that other sciences, other disciplines have really done a lot of great work on developing are left to the wayside when it comes to AGI development in particular. And then I think another one we mentioned is around yeah, the ambiguity between confirmatory and exploratory research. That's that thing that has to do with our confirmation biases. Being able to actually rigorously test things that we think might be true versus just exploring to see what would be true. All of this stuff gets conflated as people are working towards developing AGI because there's just this general belief that this is a good thing to be working for independent of scientific method or independent of scientific processes.
- [18:26]
A
There's three things about this section that feel like, like also like great context to add. The first one is Leif Hancockley shout out to Leif. We made the decision, Leif specifically wanted to not be a lead author for the paper. But in practice like with the last stretch like actually writing the paper, Leif like just played an enormous role. And this was one of the sections that where like, like Leif just like played such a big role. The second thing is the, the little bit of context for why in the context of this paper which was really intended to reach the audience of AI researchers, there's different papers we can write on this topic. We can write papers that are meant for a policy audience for decision makers. We can write papers that are meant at more of civil society audience and trying to rally people together behind what should the goals be. But with this paper in particular, we wanted to target the research community, people who often, you know, do much more technical work and don't necessarily care about all these debates about big words. And for us the key, like the backbone of the paper is the thought that the problem of distinguishing high prime reality what is actually true but also what can we establish on the basis of evidence based research that is an area where communities don't all play the same role here like that question of providing all sorts of other communities in this space with very good evidence based information that you can then rely upon making decisions and helping distinguish hype from reality is an underlying problem across the AI space. This is not specific to AGI. The pain points of distinguishing high from reality is one of the top topics that the UN identifies as an obstacle in global AI governance. That doesn't come from AGI, that comes from the current things that are happening with the field of AI. The speed at which it's being deployed, the range of contexts across which it's being deployed, the fact that so much AI development is happening not just at a fast pace, but in ways that are hard to distinguish from marketing, that are hard to distinguish from like, often well motivated, but like the self interest of actors who are trying to achieve their own goals and making claims. So that responsibility to distinguish hype from reality for me is. It's a special responsibility of the research community. Other communities have a huge role to play, but the research community is really like falling asleep at the wheel if it can't do this. This question of what is happening with bad science in the space of AI becomes really front of mind when you start with this question of the disresponsibility for the research community. That's the second point I want to make about this section. And the third point about the section is that every single problem we highlight is one that exists independently of AGI. So we talk about goals being poorly specified. That doesn't just happen with AGI, it happens all over the place. We talk about the failure to distinguish the much more engineering oriented mindset of a lot of AI research. I'm not going to say of all of AI research and the ways in which that's fundamentally different from research that is aimed at things like hypothesis testing, that is aimed at things like figuring out how do our understanding of the world line up with reality. How can we rely on evidence to. To figure out whether our understanding of the world lines up with reality? And that is the core. It's one of the. There's many ways of thinking about science, but that is like part of what makes science distinctive and important. And that pain point of all things that go wrong in the AI space through jumping over or oftentimes just saying we don't care about these questions that have to do with. And I'm not saying everyone does that, but this is a pervasive problem that so many researchers have been thinking through. Pseudoscience in the AI space isn't a. Or language people use at this point. And the same with the distinction between that can be like a more granular and pointed way to think about this. But there's also just. It's a very important question of like if you're trying to figure out does our understanding of the world line up with our observations, then questions about where do you sit in relationship to this process of figuring out whether the evidence you have, the things you can observe, the things you can test line up with your assumptions. It's really crucial to ask yourself, where do I sit in that process? Am I at a stage where I haven't yet figured out what assumptions I'm even making about the world? That can be more the exploratory stage? Am I at the stage where I've very thoughtfully, pointedly, in ways that I can go on to pressure test and ask, does this hold up? I'm in a position to do that. Okay, I can ask, does this hold up? That's more the confirmatory stage. And again, that's a problem that's pervasive in the AI space, but that AGI just makes much worse through the vagueness.
- [23:35]
C
Of the language, if I may, I also, I have to say, I think one of the particular astute aspects of this section of the paper, for me, something that I learned from this process was this reminded me of this anthropologist Diana Forsythe, who in the 90s, went and studied expert systems and found out that really what people were doing with this form of AI back then was answered by the question, does it work? And as opposed to the many types of questions that you might be asking. And she, as an anthropologist, had alternatives. But I also think that the sort of scientific research community has a different question, right? There's a different orientation to questions. It isn't, does this work? It's does it work for the reasons you thought it would work? Is it doing the things you hypothesize it would do based on the thing you are testing? And those are very different questions from does it work? And yet, whenever we get a new AI model, a new LLM, a new whatever the question is, it works. Or I guess the answer is it works. And then there's all kinds of backward reasoning as to why it works, much of which is way out there as far as I'm concerned, in terms of we're this much closer to it being a human child. And that is, I think, part of what contributes to this hype cycle of misunderstanding is that the thing does something. And so we assume that is a verification of any theory that might be out there that might exist about why AI operates the way it does. And so that's why I thought this is a particularly useful trap to identify.
- [25:22]
E
And think through the next traps you identify. You have one around presuming value neutrality. That's framing goals as purely technical or scientific in nature. When political, social or ethical considerations are implied and or essential. Feels like we've talked about that to some extent in this discussion already. And then this gold lottery, this idea of incentives, circumstances, luck, driving the adoption of Goals even without scientific, engineering, or societal merit. Seems like, Eric, you've just touched on that a little bit there. I want to get to this idea around generality debt, because I think you're also hinting at this with the comment around the human baby. This is. I think the thought of AGI is that eventually we'll be able to essentially mint versions of our own brains, or maybe brains that are even better than ours, and that ultimately that's the. The thing that we're trying to do is it's to. To get something that's as pliant as the human mind and that can really solve any. For any type of problem. You say that essentially allows us to postpone engineering, scientific and societal decisions. What do you mean by that?
- [26:34]
D
This is something that I really have a bee in my bonnet about that people use this term general, even though we know that how models are trained are not in a way that they have access to something general. They have access to a lot of different things. And those lots of different things are what's called general. I don't know about how much people who listen here know about the details of machine learning training, but basically you take data from newspapers and data from magazines and data from blog posts and social media posts and NSync fan fiction and pictures of cats and all these things, these are all things that you put into a system to train on. And by not specifying what those things are, by not looking at them and not using any kind of preconceived notion of what they should be in order to guide curation, you just call it general. So we have this concept put forward in AI AGI research of making systems that are general, which is really just putting a blanket over a lot of different things, a massive diverse variety of different kinds of concepts that we're just not defining and not critically engaging with. And so what happens there is that you have systems where we can't make reasonable predictions about how they might work because we haven't done the homework of actually looking at what specifically they've learned. It's just been under this umbrella of general, of generality. This term generality is something that should be composed and is composed of a bunch of different components. Generality isn't something in and of itself that exists. It is a function of many different components. But by not actually grappling with that, by not actually dealing with what those components are and what that might mean for system behavior, we push off all of these questions about what we should be curating for, what are the Best things to. To be curating in order to have the kinds of systems we want to have. What are the specifics of the systems we want to have? We just sweep it all under the rug of generality and so don't critically engage with the very diverse components that make up what a general thing would be.
- [28:51]
E
The last one of the traps that you identify is what you call normalized exclusion. So this comes from excluding certain communities and experts from shaping the goals of AI research. It feels like some of the individuals who have experienced that are among the authors of this paper. But let's talk about that one just for a moment. AGI pushes out a lot of ideas, a lot of potential science, a lot of other types of goals that we might have for technological development, but it also pushes out people.
- [29:24]
A
Eric, I have a question for you here or I don't know, I want to reflect back, like something you said when, and I hope it's okay that I mentioned this. We just released a big report on generative AI red teaming, and one of the amazing people who asked to not be anonymous, you know, who we interviewed and had so many amazing things to say and was incredibly helpful to our research was Eric. And one of your observations about red teaming, which is a testing practice that's become incredibly popular, but also that's become oftentimes very misconceived in the context of generative AI. One of your observations about red teaming, it feels very relevant to this question of who gets to decide what we're even aiming for here and what happens when, instead of figuring out what are the priorities, but also whose priorities, but also who and what process should we rely on in setting priorities? When you just carve out a special and very abstract, but also maybe a topic that many people don't care about, Right? The thing about AGI as a topic, and if that's what we rely on in defining what we're even looking for, is you're also going to just lose a lot of people's interest, assuming that's where you start, right? But also you might give yourself the opportunity to not ask the right questions. So one of the observations in that setting was when you're doing red teaming, you need to start by asking yourself, and I might be misquoting you here, do I understand how this person thinks? And if you do, you've got the wrong person. It's just like a wonderful observation. I don't know, I feel like it's relevant here.
- [31:02]
C
I think this is actually a really useful sort of case study to be Honest, because this is a frame in which we are talking about exclusion from industry. This is an example, we're talking about red teaming, which is you get a bunch of people together, you tell them to try to do something, to. In this case, it was a large language model. And a lot of that was the. The contours of it were predetermined. Who was able to participate was self selected by who was in attendance at the conference. The methods that they were able to do was play determined with certain guardrails that were placed on access to the models and who could access them, how long and what was prioritized. And we were there as artists, we were. And as people who engaged in hacking diffusion models and large language models. And nothing in that setup made any sense to us in terms of how we approached these models, how we approach these systems, as people who are engaged in trying to understand what harms come out of them. And it was illustrative of a lot of the stuff that does come across, I think, in terms of who do you talk to about the goal setting. But then there is also this bigger sort of issue that is being framed in this section of the paper, which is entire disciplines. It's not just people, right? Specific people to tired disciplines of thinking that may have a different frame on artificial intelligence. There are certainly aspects of academia and academic research from not this sort of hallowed interdisciplinary enclosure that has become AI and AI research. And then there's also in the sort of technical development space I think, which is mentioned in the paper too, which is who are the people who have the access to do these like large scale training? Who are the people who have the expertise to pay, people who could or the funds to do that? And who has the blessing to be able to access these resources? That narrows down the field significantly. So it's self selected by interest. Like you've got to be interested in AGI to start working in AGI. And to be interested in AGI you have to buy into some of the myths that are already out there. And then who do you reward? It's people who have bought into the myths and who gets to work on it, People who've been rewarded. So there is this siloing. Even though it is definitely a transdisciplinary research field, there is a real siloing about which disciplines are the trans disciplines in this case, sorry, the interdisciplines. Transdisciplinary would be the ideal.
- [33:59]
D
I think that speaks to some of the other problems that we were highlighting in the paper as well. So there's the ideological orientation towards AGI. So if you're a believer, you can work on it, but if you're not a believer, if you're questioning it, then you're not really invited to participate. And also this idea of generality, where if you don't break down generality into its subcomponents, then you don't see a need to include other disciplines, because general means it can do medicine and math and reading and arithmetic, all these kinds of things. But without, like, critical consideration of these different subcomponents and disciplines, then you don't actually need to interact with these people at all or learn anything from them at all, because the system is general. It does all of the above. So there's really like a disconnect between what goes into making something that's like, critically well engaged with all of the roles it should be playing, or people hope for it to play, and then what's being sold and put forward by those who follow this sort of ideological idea of AGI as the North Star goal.
- [35:07]
E
In this paper, you make multiple recommendations. Some of them, I think, won't surprise most tech policy press listeners. You call for greater inclusion in goal setting. You say that pluralism of goals and approaches should be considered worthwhile or more worthwhile. And of course, you want folks to be more specific about what goals they're pursuing, not just deferring to this sort of squishy concept of artificial general intelligence. But I have to ask on behalf of my listeners who might be wondering, what's the bottom line for policymakers here, assuming that there are any policymakers that want to listen to this right now, it does seem like some extent, especially in this country, and perhaps now maybe in Europe as well, that there's a kind of tilt towards just buying the kind of corporate line, and that this is in fact the North Star goal, whether you like it or not. But what would you tell policymakers about the ideas that are here? What would you hope that they would take from your recommendations?
- [36:16]
A
The first one for me, for policymakers, is instead of asking, what do the people who have a lot of clout, a lot of sway, who are maybe the loudest voices in the room, who also maybe have a story that feels palpable, a story that feels exciting, instead of asking, like, where are. And who is telling me an exciting story that gives me dreams for my country and so on and so forth, instead of asking, like, where are those shiny stories being told and what are they, and what can I latch onto in terms of shiny story? Ask yourself, what kind of consensus matters to you as a policymaker? And also when you're confronted with these shiny stories, because fundamentally, this question of AGI and like, what's happening with goals, AI research, like, we're not talking here about formal organized structures. With some exceptions, there are companies who have in their charter AGI. So there are situations where all of a sudden there's a process, all of a sudden there's formal documents that make AGI part of, like, a very tangible structure. But that's an exception, that's not the rule. For the most part, this topic is really part of the intangible, informal ways in which all sorts of actors in AI space approach their relationship to goals. So it's like part of the softest, most squishy components of organizing our relationship to goals. Another way to think about it is it's part of the most informal dimensions of governance, of how groups organize, achieving their goals. So ask yourself as a policymaker, not what are. Where are the stories that I can latch onto? Ask yourself instead, what kind of consensus matters? When does consensus matter and how do I get there?
- [38:13]
E
It's a powerful elixir for politicians, right? I'm going to give you the ability to instantly mint brains. Artificial general intelligence will give you the ability to have an infinite number of scientists, soldiers, workers. We're going to solve all these big hairy social problems, we're going to address climate change, we're going to fix all the problems that seem so complicated. If you're some politicians who's dealing with the poly crisis, right, you've been to Davos and they've sold you on the polycrisis. This is some powerful medicine. I don't know the billions put behind this vision. Are you in any way confident that either the AI research community and, or the broader political society will put aside AGI as a kind of goal?
- [39:07]
D
I think that given everything we've discussed here, people will declare that AGI has been achieved. There's a massive incentive to do that, if for no other reason, because of all the money that has gone into it already. And so I think we're in a position now where there are going to be organizations in the foreseeably near future that say that they've reached AGI and they're going to try and monetize that in various ways. I would encourage policymakers to instead think about what should this technology be useful for specifically and for each of those things, what needs to be demonstrated in order to assert that the, that the technology is useful for that thing? Regardless of the grand claims and this notion of intelligence being wonderful and generality encompassing everything, get down to the brass tacks. What are the specifics of what this technology should be useful for? And for each of those, what needs to be demonstrated so that we know it is useful for that? I think policymakers can really help guide the technology industry there.
- [40:15]
C
I would just say that it's important to remember that AGI is literally not a technology at the moment. What AGI is a political organization. It is a way of organizing society. And if you look at definitions of AGI, you will often see that they tend to reflect the vision of political order that they are supposed to bring by anyone who is building that, ranging from a evaluation of a machine that can raise a million dollars out of a $10,000 seed fund. Right. That tells you specifically not about the technology, but about the vision of the organization of society that this technology is supposed to be able to bring about. And so if I were a policymaker, the question that I would ask to anyone who's talking about AGI is what is this as a political idea? Stop treating it like a technology, start treating it as a political proposal and ask yourself if the proposal is something you would buy if they were bringing any other technology or any other excuse to your desk.
- [41:25]
A
I feel like the two of you in different ways are bringing about this question. Politicians are saying they need help distinguishing hyper reality from reality. Where has that gone? Keep asking for help distinguishing I'm reality.
- [41:40]
E
We'll continue to do that on this podcast, I'm sure. Do our best at least, and hopefully with each of your contributions generally and also occasionally in Tech Policy Press. I'm grateful for those. We'll keep at it. I thank you all for taking the time to speak to me about this work. Beau, Margaret, Eric, thank you so much.
- [42:00]
D
Thanks for having me the opportunity.
- [42:02]
A
Thank you so much.
- [42:14]
B
That's it for this episode. I hope you'll send your feedback. You can write to me at Justin TechPolicy Press. Thanks to my guests, thanks to my co founder Brian Jones and thank you for listening.
- [42:33]
A
Tech Policy Press.