← Digital Disruption with Geoff Nielson
Digital Disruption with Geoff Nielson
From Dumb to Dangerous: The AI Bubble Is Worse Than Ever
00:57:03
Loading summary
Transcript55 lines
- [00:01]
A
Hey, everyone. I'm super excited to be sitting down with the authors of the AI CON, Dr. Emily Bender and Dr. Alex Hannah. Dr. Bender is a professor at the University of Washington, and Dr. Hannah is director of research at the Distributed AI Research Institute. What's cool about these two is that they're maybe the most vocal critics of AI you'll hear anywhere and think the whole thing is bullshit. My words, not theirs. What I want to ask them is just how far the distrust goes. What do they really think this technology is good for? And if it's as bad as they say, what do we need to do to get the future we actually want? Let's find out. I'm so excited to be joined today by Dr. Emily Bender and Dr. Alex Hanna. Thanks so much to both of you for joining today. I wanted to start by just asking a little bit about the message that you two have been sort of roadshowing these days. You've got a very clear perspective on, on AI, on kind of the future of this technology. And maybe for those who don't know, could you lay that out for us?
- [01:04]
B
Yeah. So the book is called the AI Con, and it is what it says on the tin, that AI is a con. First off, notion that AI itself is not a coherent set of technologies. It is a marketing term and has been from the beginning, from the initial convening in 1956 in which Don McCarthy and Marvin Mitsky invited a bunch of folks to Dartmouth College to have a discussion around, quote, unquote, thinking machines. So that's one part of it. The second part of it is that the current era of AI, the generative AI tools, including large language models and diffusion models, really are premised on this idea that there is a thinking mind behind that. That itself is a con. They fly in the frame. I'm so sorry. And so then that is. And we do that for one, you know, a few different reasons. One of them being that there is a human desire to impute language to a. To the synthetic media, but specifically the synthetic text that is an output of these models, and that leads host to a whole bunch of different things. If there's a potential mind behind these technologies, then that means that they can be a. A replacement for so many different types of things that require humans. Things like white collar work, social services, medical services, teaching, and the like.
- [02:30]
C
I was just going to add quickly to amplify the last point that. Because especially with the large language models, and let me back up one second and say I will never use the term artificial intelligence to Refer to technology, because I think it is a misnomer and I think it just confuses things. And so I will talk about automation, to talk about things in general, or name the specific technology. So in the case of large language models, models, especially when they are used as synthetic text extruding machines, we experience language and then we are very quick to interpret that language. And the way we interpret it involves imagining a mind behind the text. And we have these systems that can output plausible looking text on just about any topic. And so it looks like we have nearly their solutions to all kinds of technological needs in society. But it's all fake and we should not be putting any credence into it.
- [03:18]
A
I think that's so interesting. And I'm absolutely of the same mind, by the way, and I found myself laughing when I was kind of reading through your book. First of all, Artificial Intelligence. I completely agree. First of all, I do have to give credit because it is great marketing. It's so evocative of something, but nobody can really seem to define exactly what that is. And of course it has all these ideas and can be used for any purpose. But one of the things you do early on in the book is you kind of just pop that balloon by saying, well, you know what if it wasn't called artificial intelligence? Can you share a little bit about what that sounds like and why you encourage people to do that?
- [04:01]
C
Yeah. So we have a few fun alternatives that we call on. Early on in our podcast, Alex coined mathy maths as a fun one. There's also due to the Italian researcher Stefano Cintarelli's self, salami, which is an acronym for Systematic Approaches to Learning Algorithms and Machine Inference. And the fun thing about that is if you take the phrase artificial intelligence in a sentence like does AI understand or can AI help us make better decisions? And you replace it with mathy math or salami, it's immediately obvious how ridiculous it is. Does the salami understand? Will the salami help us make better decisions? It's absurd. And just sort of putting that little flag in there, I think is a really good reminder.
- [04:40]
A
No, I think that's. To me, that just hits it right on the head and was exactly what I was getting at there. There's so much people. Once you have the idea of a mind, it just gets in your head. Right. And it doesn't actually get to the idea that in some ways, like these algorithms we're describing, as you said, they go back to the 50s, right. As long as we've had computing, we've had these notions and it seems like it's really an extension of that. I wanted to expand the question though. There's so much misinformation, dare I call it disinformation, around artificial intelligence. In this whole sphere that benefits big tech, that benefits specific people and organizations. What do you think is the most dangerous or the most dangerous myth, single myth or multiple myths right now in this sphere?
- [05:32]
B
Yeah, it's hard to rank the different myths behind all this because all of it is so upsetting and harmful. I'd say that one of the things that I think underlies a lot of this is that there's a notion of, let's say, a singular type of intelligence. And so that's something that's not really well supported by science. First off, that there's a notion that intelligence can be reduced to a single number. And some of the stuff that has, you know, some of the pseudoscience that has supported that is based on a lot of eugenicist thought. The idea that like you can have an IQ test and the IQ test can be used to rank order people and put people and machines kind of in a line and you can order them. That's why you have people like OpenAI. And I think Mir Murati had said that we're going to develop GPT5 is going to have PhD level intelligence, or there's going to be a PhD level agent that people can use for $20,000 a month. And so it does a few things. It reduces the notion of what it means to be human to something that is more computable or automatable, which also has historical antecedents. It also does this thing where we're getting into this really dangerous notion of what intelligence is and what consciousness is, as if one needs to be just smarter to be conscious and therefore human. And so that's a very dangerous line to go down too. And so, you know, of the kind of rank ordering, I mean, of kinds of applications in the world, it is probably not helpful to do that. But that kind of foundational myth around AI itself is very dangerous.
- [07:23]
C
Like Alex, I have a hard time ranking these things. And I want to add another piece of the puzzle, which is another. Another one of the myths that I think is feeding into this is this idea that if you take a system that has unfathomably large data inside of it, it must therefore have an unbiased sort of bird's eye view into what's really going on in the world. And that is definitely a kind of wishful thinking. Like we Want there to be some peak we could climb to from which we could just understand everything. But that's not how science works, that's not how society works, and it's not a path towards building technologies that are functional, let alone fair.
- [07:59]
A
Right. So, you know, just, just, you know, following that thread, it sounds like, I mean, there's, there's a few things at play here. One of them is that we're, we have this kind of false foundation that intelligence itself is anything more than this kind of nebulous abstraction. Right. As soon as we say, oh, it's this, we're already kind of kidding ourselves and going down a dangerous road there. And then it seems like there's almost, and tell me if I'm misrepresenting you, but it seems like there's this drive to pretend we're creating something objective. But this objective thing just also so happens to look an awful lot like what objectivity for those who don't have a visual in heavy air quotes looks like from the office of a California technology leader. Is that fair?
- [08:51]
B
Yes. And I mean, there's so much in terms of scholarship that's showing that the use of these systems is anything but objective, right? From the foundational work from Bolomini in Dubru to folks like Eileen Keliskan, to my own work that's focused on data sets and their politics. Really, any kind of these systems are making judgments of some kind and they are encoding majoritarian notions of what it is to, you know, have a face when it comes to, specifically to Woloweenie and Jabru or to what it looks like to be someone in a certain kind of job, you know, which tends to have higher status jobs that are, that are whiter and mealer. And so, you know, there are those things that get encoded that we have a, and we have a word for what it means when we trust machines to have an objective view. And that's called automation bias, the notion that like we're going to seed certain kinds of operations to something that's automated. And so that objective view is certainly not objective. It is just reifying views that are in the majority and really to the determined people who are not in that majority.
- [10:11]
A
Along those lines. One of the themes or sort of theses in your work seems to be this notion that we can't actually disentangle the technology from the creators and owners of the technology. And I wanted to maybe ask you to expand on that a little bit for people who are, you know, just kind of learning about this concept for the first Time.
- [10:32]
C
Yeah, absolutely. I think one of the parts of the con in using the phrase artificial intelligence is that it's a way to displace accountability. And if you look at the language that's used around this, oftentimes we put the so called AI systems in agent position. We'll say, you know, ChatGPT has scraped the whole web, for example. ChatGPT didn't do that. The engineers at OpenAI did and some other people whose data they datasets they repurposed. Right. And also you can't scrape the whole web. That's a whole separate conversation. But to basically always keep the people in the frame and say, who built it, for what purpose did they build it, who's using it, who are they using it on? And also whose labor was either just flat out appropriated or otherwise exploited in the production of the systems really helps keep the conversation grounded.
- [11:21]
B
It's also helpful to note that in I think so much of this conversation, there's a notion that these companies that are building this are somehow magnanimous, that they're doing this for the benefit of all humanity. This is something that's really well underscored in Kieran Howe's book called Empire of AI that focuses on the kind of drama, Empire of AI that focuses specifically on OpenAI and the kind of drama behind OpenAI and specifically on how so many of the people behind that technology are very fallible specifically around Sam Altman and many of the folks around him. And so it's really one thing that highlights, and that dynamic highlights is the way that you don't come up with magnanimous technologies. I mean, technologies that actually work for people are ones that are closer to those people and the people themselves that are built by those people or for those people, ideally by them. And so, you know, like the idea that there is no human in the frame is a mechanism not only to displace accountability, but to displace where the power really lies.
- [12:37]
A
Right. And one of the, you know, as I think about that and I think about this notion of, you know, benevolence, the other sort of party line that I've been hearing more and more come out of that, you know, from come out of those organizations is that there's, it's described as kind of a race and oh, we have to keep doubling down, we have to keep investing in this because, you know, it's winner take all. And like there's this conflict here, right, that oh, it's for everybody, but we have to do it for everybody. And you know what's your reaction to, you know, that narrative?
- [13:11]
C
The race framing is just ridiculous. It's based on a misconception of how science and technology work. And this is some, some thinking that I learned from Beth Zingler, who's looked into this in detail, says, look, you can look backwards and trace the path of sort of what built on what to get to where we are now. And you can draw a straight line if you want, but looking to the future, the future doesn't exist ahead of time. And the people who say, you know, artificial intelligence or artificial general intelligence is at the end of this path and it's just a question of who runs down it the fastest have completely misunderstood how science works. Right. So first of all, there's, it's an ill defined notion that they're running towards, but they are asserting that it's there and it's definitely existing and we can get there. And also asserting, and I think is a very Silicon Valley brain way of thinking about things, that the person who gets there first is, as you said, winner takes all. And you'll sometimes hear people say, well, we have to build the good AI or the good AGI, lest evil, you know, opposition builds the bad one. As if somehow building one technology could prevent the building of another. Like it actually just doesn't make any sense.
- [14:14]
B
And just to build on that, I mean, specifically the focus here is on that the US has to build this before China. And so the boogeyman here is, is the kind of sinophobic idea that the Chinese, like AI or AGI, whatever it is, is going to be by virtue authoritarian, given that political environment. And I'm not going to say anything about what Chinese builders are doing, but as if what we're doing in the US isn't authoritarian by being built by one company that is commanding so many resources, that has so many implications with, with national security organizations, with, with large energy companies. I mean, these are things which are centralizations of power. And that itself is not as if it is a democratic version of AI. Right. And so I think there's a real way in which those, even the claim around the race dynamics also obscure what is happening in terms of power centralization and how that's undermining democratic dynamics in the US and in the West.
- [15:22]
A
The centralization of power comment is, it's really interesting to me, right, because that's another one of these tensions and another one of these narratives about it's benevolent, it's for everybody, it's going to make everybody better. Oh, except we own the platform, right? And by the way, we're charging you 20 bucks a month to have this new better life, which I find really interesting. Is there merit in your minds to. There's a lot of talk about maybe generative AI, maybe automation. Some of these functions might not be as good at professionals at any given task, but they extend the entire market. And market's a bit of a dangerous word there, but they extend the number of people who services can be delivered to and they go after the historically marginalized in a way that is empowering or makes their lives better. Do you buy that? Do you have optimism there? Or do you think that's just part of the con?
- [16:22]
C
It's really just part of the con. The, the argument is always, well, we don't have enough to provide good education to everyone, we don't have enough to provide good healthcare to everyone. And so these poor people are left out. And so this is better than nothing. And anytime you hear this is better than nothing, the question should always be why was the alternative nothing? Because we have enormous resources. If you look at the resources that are being poured into these systems and imagine instead those resources were used for shoring up education systems and healthcare systems, imagine what we could do with that sort of rich and rich investment view of social services rather than an austerity view.
- [17:01]
B
And very much it's tied into, I mean when you get behind this, it's tied into this idea that those with means are going to get these bespoke human oriented services that are by humans. And we already see this in other kind of technological domains. Adrienne Williams was on our podcast and she's a former charter school teacher and one of the things that she talks about as being a former fraternity school teacher is that when it comes to ed tech, things like Google Classroom and CMSs like that, or content management systems specifically in education, they are disproportionately offered in places where the students are lower income, black and brown and lower income communities. If you go to private schools, there's very few screens or those screens are used in a much different way. They're not used for surveillance, they're used, you know, they're pretty optional or you know, they're, they're much more pen and paper oriented. And so we're already seeing that dynamic. And that's not actually providing more opportunities to people in those communities. It is more heavily surveilling those communities. And it's being used as a way to also say that there's being an offer of that technology that they have access. But that's not actually education. The same way where one of the things that AI boosters are very excited about is following Ilya Sutskever wildly cheap and effective therapy because people cannot afford therapists that can speak to them specifically. But people aren't getting therapy. They're getting some kind of a machine that might just be telling them what they want to hear or leading them down to dangerous kind of particular behaviors or maybe encouraging them to engage in self harm. And so it's very. What really gives away the game is this comment from Greg Corrado who is health of head of health AI at Google Health, very confusing term. And he was debuting Med Palm 2 at Google IO in a press junction. He had said to the Wall Street Journal, you know, this thing is not something I'd want in my own family's medical journey, but I'm very excited for it to be available to everybody else.
- [19:15]
A
Wow. Yeah. What more do you need to say about a product than I wouldn't have my family use it? Right. It's like the quotes from the heads of social media networks saying, well, I certainly wouldn't want my kids on this. It's like, okay, well that's the whole ball game then.
- [19:35]
B
That's right. Yeah. I think Sam Altman at one of the recent, I think the recent commerce meeting was asked by someone, you know, you wouldn't want your, your new kid to, you know, be friends with a, with an AI agent. And he was like, no, I, I wouldn't. So that didn't get a lot of attention.
- [19:54]
A
Very, very telling.
- [19:56]
C
I just want to add that I think that that shows that, that these folks don't see the rest of the world as really people. And it then sort of reveals the lie of saying, well, we're doing this for the benefit of humanity, but these services are not services that we would consider good enough for our families. And so everybody else doesn't count the way our families count.
- [20:14]
A
Right. There's almost like this like self deification or something. Right. Like there's, we're, we're in some way outside humanity. Humanity is this project and we are these, you know, saviors who are going to come in and tell you what's good for you.
- [20:29]
C
Exactly.
- [20:31]
A
So, you know, in a couple of minutes I want to come back to this, this notion and what we do about it. But just, just before we get too deep into you, each of us can do. I wanted to come back, Emily, to something you mentioned earlier, which is AGI. And as we look at some of the, you know, the big Promises or the big, you know, stories around, you know, technology. Right now, two of the big ones we talk about are AGI and on the opposite end of the spectrum, this, you know, P doom or probability of doom that it's going to, you know, wipe out the human race. What, what's your outlook around both of these in the next. In any sort of reasonable time horizon.
- [21:12]
C
I mean, any time horizon at all. Right. So it's all fiction. And one of the things that's very frustrating to me is the way that the doomers, the people who think this is going to be the end of humanity, and the boosters, the ones who think it's going to solve all of our problems, present themselves as like two ends of a spectrum. And the media picks this up and sort of amplifies it and gives the idea that there's just one spectrum. So you're either fully doomer, fully booster, or somewhere in between, but in fact it's two sides of the same coin. So the doomers say artificial intelligence or artificial general intelligence is a thing, it's imminent, it's inevitable, and it's going to kill us all. And the boosters say, AI AGI is a thing, it's inevitable, it's imminent, and it's going to solve all of our problems. And putting it that way, I hope it makes it very clear these are actually the same position and there's no daylight between them. And it's just once you've gone down this fantasy path, which turn you take at the end.
- [22:06]
B
Yeah, so, and just to be put a finer point on it too, there's a notion of P doom and there's a notion of P hope. And I think what we initially want to say is we reject this probability framing at all. As if one can one imagine this as a question of completely fake probabilities. And the thing that really sets me off, especially about folks like the rationalists, so folks like Daniel, I completely, just completely mangled his name. But the. One of the Authors of the AA 2027 document and others of his colleagues is that they're putting completely fake probabilities to fake events. And it's really, in fact, for folks that really envision themselves to be very empirically minded, it's just very much just made up kinds of time horizons. And it's really. And Emily and I are both social scientists and are both empiricists, so that's actually very weird to see. So that's one thing about that. And then AGI itself serves as this very nebulous concept. As well, this kind of idea that there is some kind of a intelligence that's going to be very capable at many different tasks and we're not really sure how well defined that is. I mean, it doesn't seem really well defined at all. We just recently published a piece at Tech Policy Press called the Myth of AGI that was focusing specifically on the idea. On the idea that this notion is nebulous and that it has a very particular view of the world, is wildly unscoped and somehow there's supposed to be OpenAI or anthropic being the particular organization that is going to ensure that we receive this. And so that itself is very wrong headed.
- [24:17]
A
Right. So I'm just. No, it's interesting and I'm just reflecting on that a little bit because if you take all that and sort of synthesize it that most of what we're hearing in this space is as you described, a con. Right. It's marketing, it's nonsense. I mean, I'm not going to ask you if you think this is a bubble. I think the writing is on the wall there. But I did want to ask you, do you think this bubble is going to burst and is it already on a trajectory to burst? Do we as individuals and maybe as leaders need to do something differently for it to burst? And assuming it does, what do we think that's going to look like?
- [24:59]
B
Yeah, I mean it's going to burst. And I think it's not a question of if, it's a question of when. And it's not a question of and moreover, it's a question of how. Right. And so there's a few different ways this bubble could burst. It could burst big. And we got sort of the inklings of that when we saw the freakout around Deep Seq in which Nvidia lost a whole bunch of value in its stock and they said, oh, maybe we don't need these massive data centers to do so. But the bigger worry here is not that there's going to be necessarily more efficient models, it's that it's not going to solve problems, productivity or solve wages in the way that the boosters think it will. And so a lot of, you know, a lot of people have already said, well, there's a huge revenue bubble that needs to be, you know, that needs to be sorry, there's a huge debt bubble in terms of all the, all the GPUs and the data centers and all that infrastructure needs to get paid for were. And at some point there the facade is going to come. And is that going to be something like the first AI Winter in which, you know, there was the, the Lighthouse support, in which they said, okay, this isn't, this isn't panning out, no more government funding? Or is it going to be something that looks a lot more like Uber, where more money gets thrown at it over and over and over again until finally a profit gets turned? The thing is, so much money is going to get thrown on it and it's not going to turn a profit. The revenue margins have been so low and the investment has been so high. So it's either going to happen slowly or it's going to happen quickly.
- [26:48]
C
To your question of what we should be doing about it, I think that certainly we should be resisting the hype. And that's part of our goal in writing this book, is to help people articulate their objections and to the hype. But I think also it's really important to not let systems that we rely on get rebuilt around the false promises of artificial intelligence, because it's going to be harming us while the bubble is still going. And instead of getting, you know, actual thoughtful medical notes, at the end of a doctor's appointment, we get the output of a synthetic text extruding machine with some errors and the doctor saying, well, I, I, it's not my fault if it's wrong. The system did it. And, you know, I. Medical providers are under a lot of stress. And in many cases this would happen because their employer would say, well, you've got to see three extra people in a day now because you're not spending time doing the clinical notes or whatever. So it can be harmful while it's still going on. But if you think ahead to when these systems fall apart and the vendors aren't there anymore, say, how much has things been restructured? How much have people's jobs been changed? Either people being asked to do more in the same amount of time or, or people's jobs being turned from stable jobs into very casualized gig work jobs because the AI supposedly could do it. And so the more we can resist that restructuring, the better off we're going to be, sort of, regardless of when the scales finally fall from people's eyes. And we're not forced to have this everywhere.
- [28:10]
B
Moreover, one thing, and we mentioned in the book, and Emily took a piece of that, is we have a piece called the Grimy Residue of the AI Bubble. And the job loss is one thing, but we can think of two other things are going to be left over if we don't try to do some mitigation. Right now, one of them is the environmental damage that's already been done both in terms of the carbon that's been put into the atmosphere, the water that's been used for data center cooling, also the types of externalities in terms of air pollution and forever chemical that are put into the from semiconductor construction. So those are hard to reverse. And we're already on track to blow past the goals of the Paris Climate Agreement and then the kind of spills in the information ecosystem. The idea that we already have so much synthetic text out there, it's going to be hard to suss out, you know, what is synthetic text and what is not synthetic text. And we already see that battle being done at in places like Wikipedia, which is trying to fight the onslaught of so many LLM generated and therefore not very trustworthy outputs of synthetic media, text media machines. So that's one part thinking more of what we can do about it. One thing is trying to be more cool on investment, especially in data centers. I mean, that is harming communities in the here and now. And whether it's the data centers operating in southwest Memphis that is polluting Boxtown, predominantly black and poor neighborhood, or whether it is areas like Northern Virginia and London county or the outskirts of Atlanta where more and more data centers are being built, specifically relying on fossil fuel powered power plants.
- [30:06]
A
Yeah, again, just kind of reflecting on that, Alex. I have the same concern about the environment and I'm tying it back to your comment about Uber because the thing that really gets me concerned about this is Uber is kind of throwing good money after bad for a long time. Right. It's the old joke of we're losing money on every interaction, but we'll make it up at scale. Right. But the thing that worries me about this, when you combine it with that kind of arms race narrative, is it feels like it's not even linear, it's like exponential. Right. It's like everybody's saying we need more and more data, more and more data centers. I can very easily picture kind of an asymptotic curve where while we still haven't found the solution, it's still not profitable. And so, well, let's just try throwing 10 times as much energy or 100 times as much energy and either this whole thing collapses as fake or we do an awful lot of damage in the interim there. So that's certainly a piece that resonates with me and Emily. I was certainly thinking about your comments on automation and the need for us to be more Thoughtful about where we automate and how we make sure we're not just replacing a solid foundation with a flimsy foundation.
- [31:21]
C
One of the things about these ever larger data centers, data sets and so on is that it becomes a metric that the people who are spending the money can say, see, look, we made it bigger. Because what they're trying to build is actually not well formed, it's not well conceived. And so you can't evaluate how close you are to building the thing you're trying to build. And so they've got something they can measure instead. And that thing is in fact, environmentally quite ruinous and built on stolen labor and so on.
- [31:49]
A
Right. It's more parameters, more computations. And, you know, that must be good because it's. Because it's bigger. I wanted to come back to the, you know, and put a finer point on sort of the, the adoption piece. And I'm thinking specifically for, you know, leaders of industry, for, for organizational leaders who are, you know, very much, you know, everywhere you turn, I'm sure you are exposed to it too. AI this, AI that. If you don't do it, you're behind the curve. What. Do you have any specific guidance, you know, for these people in terms of what they can do to, I don't know, maybe be more responsible here? Does that mean just fully saying no automation, none of this, or is there an approach they can take that's just going to yield better results and protect them from some of this risk?
- [32:34]
C
There's certainly a time and a place for automation, but you want to automate something when you can very specifically say what it is that you're automating and you have very good reason to believe that the information needed for the output is in the input. When you can evaluate how well it works in your current use case, when you have sufficient recourse for someone who's been harmed by the automation? Because one of the things about automation is that it scales whatever you're doing. And if it's getting it wrong, and that getting it wrong is harmful to people, and you do it more and more and faster and faster. You've got to be prepared to make things right or decide, no, that's too harmful. It's not the kind of thing that can be made right, we're not going to go down that path. So, you know, advice to leaders who are making decisions here. I would say, first of all, you know, think about values. And I know that for many people in business, the only value that matters is stakeholder value. And the way to get stakeholder value right now is to promise AI. So setting that aside to say, okay, well what are our other values and to what extent does this automation actually speak to them? And especially thinking about, you know, what could go wrong when you put, if you're talking about something like ChatGPT, a synthetic text extruding machine, into the middle of a sensitive process, how might that impact what your company's reputation or what it is that you say that you stand for and think about sort of like the long term durability of the systems you're putting in place? Is this still going to work? Would this still work if OpenAI went belly up and didn't have access to ChatGPT anymore? Anymore, would this still work if it turned out that large language models aren't all that? And so on.
- [34:11]
B
Just a few things, I mean, there's just a few data points I want to add to that. So there was a survey that was done by an org planning platform in the UK called Org View. They found that 55% of companies that replace workers with AI regret the decision. So already you have some buyer's remorse happening. And I'm assuming that there's a notion that there would be workers would be able to, that that still were there, could be using AI tools to shore up the difference. But that is case. That is not the case. The largest study that's been done, or one of the largest studies, I'm assuming, but I don't know if there's been a larger study that's been done in Denmark of 25,000 workers across 7,000 organizations, suggested that there were very modest productivity gains by workers using artificial intelligence tools at the rate of 3%. But those gains were offset by the new labor displacing tasks that they had to deal with. There was also no increase in earnings for those workers. And so these are not proving to be the you know, the kind of huge and amazing kind of product productivity gain that the companies are making, making them out to be. So the message to business leaders, I would say is really invest in your people and really thinking about what are your people doing? How can you better support people who are already doing the work that you need to do if people are having an issue with being productive enough? I mean, what are ways in which they can be supported organizationally? I'm thinking about this kind of with an organizational sociologist hat on and thinking about what are ways in which those, you know, a product can be made better by that. The reason why large language models are very attractive is because if their values, as Emily suggests, are very much only for shareholders, shareholders often like to see layoffs because that means you can be making more profit per headcount. But if your product is not keeping up the task and staying as high quality as it was, then that's going to be a huge issue if you're not. And so in some cases, there's some really interesting cases in which some companies are leaning into not using AI and saying that no, we're actually giving you a high touch experience. And that is very important to signify because I think a lot of people are culturally saying that AI is shoddy and jank and what trustee McMillan Cotton said it is mid. The outputs are not really verifiable or if you can verify them, they are very labor intensive to do so. The outputs of images are very shoddy and take a lot of labor to correct, especially down in the supply chain. And it's just overall a bad value proposition.
- [37:27]
A
So when you paint that picture, Alex, I was thinking back to what you were talking about earlier in EdTech and our conversation earlier about this sense that this technology is going to open up these whole new audiences for what's going on here. And you also mentioned, and I've seen firsthand by the way, that oh, there's companies not using AI and you get this hands on approach. My concern when I read all of this together is that sure, there are companies that will say yes, there's no AI, it's human only. But of course they'll look at that with, you know, money bags in their eyes and say, well now of course there's this hefty premium for anything with no AI or for human, you know, for human touch. And so we end up with this world that's, you know, even more extreme where yeah, it's, it's AI slop for the masses that we know is mid or low quality. And then you have to pay a premium for, you know, the quality of service that you're either getting today or certainly were getting a few years ago. We were talking about ed Tech and this is, you know, as I look across the spectrum of not just corporations but also social services, it feels like there's this like this erosion of quality or you know, some people call it like an inshitification. And so, you know, to what degree are the two of you seeing this in your research across some of these different sectors and you know, in your minds, is there a way that we can reverse this?
- [38:57]
B
Yeah, I mean it's a good question and I think we're seeing a little bit of kind of a rush. You know, there's definitely this intensification process, this term that Cory doctor coined. The thing about it is that thinking about what we could do to reverse it. I mean, one thing is that what we hope, and I think one of the hopes of the book is to suggest that the kind of LLM output is just not up to snuff for any critical task, right? And so, you know, if there's synthetic output, it is just seen as either scammy or spammy. And because many of the times where it is successful is in scams and in spam. And that's because it's, it's sort of this thing where output is where you don't really care about the quality of the synthetic text and you don't care whether any of the things within that text has any verifiable truth, has any truth claims in it. You don't really care about the truth values. And there's a paper called Chat GPT is bullshit, right? And it uses the Henry Frankfurt definition of bullshit. The idea that the bullshitter doesn't care about the truth values of their claims. They're just trying to reach their goal. They're, you know, therefore, like Trump is like that bullshit or par excellence, because it's just like get to get, get the deal, right? And so to some degree there is, you know, like we could see that any kind of synthetic output is just that kind of, you know, Detrius is the second waste. And so I don't know if humans as being the value differentiator. It's more like, well, in the best of cases, anybody that's doing anything that's worth its salt should be done with humans in the loop in a meaningful way.
- [40:55]
C
I just want to add there that you said we might end up in a situation where the current status quo becomes the luxury tier where you actually have people involved and then everybody else gets pushed down to this. Well, you've got to deal with the, the crappy synthetic system. And there's no magic bullet here. We just have to resist it at every turn. Right? If this is coming into your school system, say no. Right. If this is coming into your workplace, say no. And that's part of what we're trying to do in this book is to empower people to use no at every turn and to recognize it when it's happening. And also, you know, the, the no can be firm, it can be angry, but it can also be humorous. And this is where we recommend Ridiculous Praxis and saying this is, this is low value, it's fake, it's bad, bad, it's mid all of these things because sometimes the people who are making the decisions, it's not money bags they have in their eyes, but stars and in particular the sparkle emoji that got appropriated by the tech companies for this stuff. It looks like magic. And so I want the magic. And so to empower people to educate those around us so that we collectively make better decisions is really important.
- [42:03]
A
It makes complete sense to me. I wanted to ask the two of you a slightly different question. So one of the things I normally ask guests that I speak to here is I ask them what they think is bullshit. And I'm not going to ask the two of you that because I think we've spent quite enough time talking about what is bullshit and I know we've got some strong and well supported views here. I wanted to flip the question around and ask in this fear, what isn't bullshit? What are you excited about? What's a good use of LLM or generative AI or some of these newer or modern technologies? And is it really a case of say no to everything, just shut it off, full hands over your eyes and ears or are there very specific, very targeted use cases where there's potential for, you know, excitement and value?
- [42:56]
B
So I'm very excited about the uses of language technology that are for community empowerment. And so, and I'm being very specific here, I'm not saying LLMs and I'm not saying diffusion models and or generative AI, but the cases in which there are things that empower communities to do things which serve that community. So an example that we talk about in the book is the example Teheku media in which they provide machine translation and automatic speech recognition tools for the Te Rea Maori language. So focusing on and the thing that's exciting about that is that they have the people in the community have control over which data gets used for training models. They ask their community elders on what data that can be used. Certain data can't be used. And those are tools that are in which data is owned by the community as well as the computing power itself. So this is kind of like the anti OpenAI. The idea instead of building this big everything machine that works for every language everywhere, you have a very narrowly scoped task that works for the specific community. And building more Teheku medias is amazing. It's one thing that DARE is trying to do in powering a network of individuals through a federation that we call the Hooniki Federation that is just getting off the ground now. And those things were really focused on building the imagination of folks and basically doing it in a way that is not environmentally ruinous, that does not rely on data and is really providing for a specific need.
- [44:42]
C
So I would like to add there, I'm a technologist, just like Alex is. I run a professional master's program in computational linguistics training people how to build language technologies. So I definitely think there are good use cases for things like language technology. And the Tahiku media example is wonderful. But I see no beneficial use case of synthetic text. And I actually looked into this from a research perspective. I have a talk called when if ever is synthetic text safe, desirable and appropriate or those adjectives in some order? I don't remember the exact title. And basically it has to be a situation where first of all, you have created the synthetic text extruding machine ethically. So without environmental ruin, without labor exploitation, without data theft. We don't have that. But assuming that we did, you would still need to meet further criteria. So it has to be a situation where you either don't care about the veracity of the output or it's one where you can check it more efficiently than just writing the thing in the first place yourself. It has to be a situation where you don't care about originality because the way these systems are set up, you are not linked back to the source where an idea came from. And then thirdly, it has to be a situation where you can effectively and efficiently identify and mitigate any of the biases that are coming out. And I tried to find something that would fit those categories and I don't. So certainly language technology is useful. Other kinds of well scoped technology where it makes sense to go from X input to Y output and you've evaluated it in your local situation, great. But you know, the giant random eight ball why?
- [46:15]
A
Well, it's interesting, right? And the why question is very interesting to me, right, Because I think most of most of what you said there is not controversial, right? It's, it's hugely energy intensive. It's probably a lower quality than what people would come up with if it weren't synthetic. And yet you have to juxtapose that with the fact that there is an awful lot of adoption and people seem to be getting value out of that. Maybe just what that says about us as people, that we're willing to settle for something worse because it's, because it's easier and not worry about the, you know, any of the details behind the scenes. So it's. Yeah, I'm just still kind of thinking about that.
- [46:58]
C
I mean, every time someone says, well, I'm using it because, you know, I don't have time, or I'm using it because it's easier, I think it's always worth asking, well, why don't you have time? And if it's easier, what are you. What's the opportunity cost there? What are you missing out on for not actually connecting with a person to have a conversation or thinking through something yourself? And, you know, oftentimes the source of the problem isn't the person themselves who made that decision, but the structures that put them into the corner where it felt like this was the best way out. But what I got to tell deans of universities from the west coast of the US and Canada last year was the only use of ChatGPT for a university is as a contrast eye test to see where resources are missing, where students and staff turn to this. It means something is lacking in terms of what they would need to actually fully engage in the educational project. And that information is of value to administrators. But, like, that's it, that's the end of it.
- [47:56]
B
I will also say the cases in which people say that, well, I find this very helpful. I would also note that I think that's a pretty even term in terms of the people who are using it. It's kind of a limited number of workers who are using it. So in a survey Pew did, they found that 17% of workers were using LLMs at least some of the time. And I think 1% were using them all the time. We're finding cases which I think there's a mismatch reality in which especially business leaders find them to be more useful than many other people who are more junior. In the job letter, there was another story that came out by Noel Shriver in the New York Times that talked about the uses of LLMs at Amazon, effectively, how the deployment of LLMs has become almost mandatory at Amazon in the programming work, and how even though more senior devs appreciated that more junior devs weren't really being forced to use it, and their work was looking much more like factory work than the kind of creative work that often goes with software engineering and programming. And so I think that kind of notion that it is kind of useful is probably happening only for a very narrow set of workers and people. Maybe it's happening more for students because they're being constrained and pushed for time to not engage in classes to the pain of their instructors. And talked to a lot of instructors and our instructors ourselves. And so that itself, I think is another area where a lot of this is happening. But I think that narrative and what's happening across most workers, there's a big mismatch there.
- [49:51]
A
So just following that thread for a minute, Alex, on the workers side, again, one of the narratives we hear is that there's kind of two sides to adopting these technologies. There's organizations or enterprises that can adopt them to try and drive productivity organizationally. And maybe that displaces workers. Maybe that's the do more with less mandate. But then there's this other narrative of as an individual worker, you should adopt some of these synthetic language tools or some of these automation tools because it's empowering for you. It helps you take back a modicum of control from your employer by making you more efficient. Maybe it takes less time to do what you're doing, or you can do better quality work in the same amount of time that you're doing worse quality work. Now, do you think there's merit to that argument or do you think that's still worth resisting?
- [50:46]
B
Yeah, I don't think there's really merit to that. I mean, I think the cases in which these tools have been deployed have been in cases which there may be some marginalization gains, but you have to check the output of this very meticulously. There's been many, many different cases in which people like lawyers have been using tools like this in legal briefs. And those legal briefs have may have a lot of made up case law in them. There's cases in which journalists have used them. And it is made up a bunch of books that don't exist. And these are. I think there was a case with the Chicago Sun Times, which I think they had someone that was in their pipeline. I don't think it was a journalist at Chicago sometimes, but they had made a list of fake books. And these are kind of coming up over and over. So if you have cases in which you're in. You're using these tools with the intent of being more productive, often it is doing the opposite. It's slowing workers down, making them less productive. There's also the case that it makes you less collaborative because you sort of produce something, but you are not then passing something on to a coworker to use it in any kind of veri, verifiable or useful way. There's a good example when in talking with animators, as I was talking with somebody from the Animation Guild who reported that they were under pressure to use midjourney to produce like an image and then fix all the artifacts that come out in an image. And what you get out of mid journey or stable diffusion or whatever is you get a PNG or a JPEG file and then you might have to fix the artifacts that come out of it. But really what you want to work with is something like an Illustrator file that has many different layers to it. Right. And that's not what it produces. And so if you're actually going to fix the artifacts, you actually have to reproduce the image with all, all its, all its layers. And so that's not actually helpful as a tool of collaboration. It's actually breaking the collaborative pipeline there. And so there may be solo folks and individual contributors that may find them very useful. But once you get into a larger organization, once you have to actually verify that information, it falls apart pretty quickly.
- [53:07]
C
I would just add briefly to that. I would ask workers who are using something because they feel like it helps them speed things up to think about how long they actually get to maintain the benefits of that before they're just asked to do more in the same amount of time. I think that these kinds of so called efficiencies generally are not going to accrue to the workers.
- [53:26]
A
Right, right. And there's, and there's certainly a risk of that. So, you know, just kind of tying a bunch of this conversation together, we're seeing all these trends happening right now around the hype, around adoption, around pushback. When you look out over the, your outlook for the years to come, to what degree are you kind of optimistic or pessimistic about the trajectory we're on right now and our ability to get to bend the curve in a way that's actually net positive for us as people.
- [54:01]
C
So I am an optimist at heart. I also don't make predictions. So I can tell you sort of what gives me hope though, is watching people standing up and saying no and watching people adopting ridiculous praxis. Also, taking a page from Karen Howe's amazing book Empire of AI, she tells amazing stories of people in Chile and Uruguay who organized to resist the imposition of data centers. And she makes the point that people who have had more power taken from them nonetheless maintain agency and nonetheless push back. And I think that it is really important to resist narratives of inevitability, even the ones that say, oh well, it's here to stay, so we have to learn to live with it. That is still a narrative of inevitability and therefore still a bid to Steal our agency. But we all have agency and we can continue to claim. Claim it.
- [54:54]
B
I'm less of an optimist, I think, than Emily is. The pessimistic part of me is just that there's more and more investment that's being put in this. I mean, we have one of, I think, the largest tech investment round we've seen with OpenAI and an investment round led by SoftBank that was to the tune of $40 billion. And so more. More money is going after that. But I think, you know, an optimistic reading of that is that this is kind of the last gasp. It is. We are tossing this much money at it. This is the big bet. If you don't come out of this, this is, you know, this is. This is going to be your last chance. And it's not like Masa is known for good investments. And wework is indicative of that. And so it might be the case that we are seeing a lot of that. That bubble really reaching its. Its peak in size. The things that are optimistic are the kind of ways in which workers in particular are pushing back. We're seeing efforts from the Writers Guild of America, of course, that had strong protections around generative AI in the workplace. We've seen some of that work from public workers in Pennsylvania and from seiu, the Writers Guild and the Authors Guild also, or, sorry, the Authors Guild and the Animators Guild as also being organizations, have taken a strong line of that line against generative AI. And that really does give me hope. And I think we're seeing a really nice confluence here of people trying to understand what's behind all of this. Is this all hype and what can we do about it? And I think to that end, I think our book is very helpful, and I hope there's a tool for folks seeking that out.
- [56:43]
A
Amazing. I appreciate the thorough answer and I wanted to say a big thank you to each of you, Alex and Emily, for joining today. I thought this was a really fascinating conversation and appreciate your time.
- [56:52]
C
Thank you so much.
- [56:53]
B
It was great to talk to you today. Jeff.