← Digital Disruption with Geoff Nielson
Loading summary
Transcript53 lines
- [00:00]
A
Hey everyone. I'm super excited to be sitting down with Gregory Warner, Peabody Award winning journalist, ex NPR correspondent and current host of the hit AI podcast the Last Invention. Greg is a fellow traveler in the quest to understand the race to build advanced AI. His full time job is examining the existential questions and key players at the heart of the AI revolution. I want to ask him whether we're creating something that will save us or destroy us, what future he thinks is most likely and what we need to do to prepare. Let's find out. I'm here with Gregory Warner. He is a Peabody Award winning journalist. He's the producer and host of the Last Invention podcast all about AI. And maybe just to start things off, Greg, tell me a little bit about why you and some of your co producers created this podcast. What was kind of the, know the rationale for why you wanted to tell this story?
- [00:56]
B
Sure. I mean, I think what, for me, what, what got me hooked on the, on the topic was realizing that the people making AI were, had a, had a sense that this, this might kill us all. It's as simple as that. I mean, the fact that the, that they felt that the risk of this thing that they were building to humanity was, was real just felt like such an interesting time to live in. And the fact that, that, that debate over the existential risks and rewards of this technology, because I think that the potential upsides are just as radical. It felt like while we were discussing in the news at the time things like, oh, the danger of deep fakes and the possibility of AI taking jobs, these more existential questions which had haunted AI really from its beginning, as we found out, felt like, okay, we need to have this debate and try to bring this to people in a way that didn't, for lack of a better word, didn't just freak people out. You know, the, the idea was let's introduce people to the kind of debates that are being had in, and have been had in these circles and, and how we might talk about the future with some kind of superintelligence, it's, you.
- [02:12]
A
Know, that to me is one of the most interesting things about the topic is that if you, if you ask some of the most knowledgeable people and the most central people in this conversation what their outlook is that just the range of answers is so radical from you know, basically utopia to destroy us as a species to, you know, this, you know, other cohort of voices saying it's all, you know, basically a nothing burger. You know, at the risk of asking you to editorialize as A journalist. I'm curious, you know, having talked to all these people, having heard all the sides of the story, you know, what, what outlook do you find most compelling? Are you worried about the existential risk or where have you kind of landed as you know, Greg the human.
- [02:56]
B
Okay, Greg the human. Where, where do I land? Well, let me tell you a story. So there was a hacker, a white hat hacker named Dawn Song, I think her name is, and she took all the AI, the leading AI models and she found 20 zero days. If you're familiar with a zero day, it's a, that's a hacker terminology. And basically it's a, it's a huge deal. It's the sort of thing you stop everything and you focus on this, this weakness because it's incredibly essential to fix it. To find even one zero day is a big deal. She found 20 and using ChatGPT and the current model. We're not even talking about some future far off thing. And then I think she used Gemini and found it even faster or something like this. And the important thing is that AI hadn't been trained to find a zero day. It literally became a top level hacker with just a few smart prompts. And so I think my take from that is we are talking about when will these systems be good enough to pose a threat. But I think that time is already here. And so it's not will we awaken a God or will we, will we summon a demon? It's not a future conversation. It's that we are living already with a technology that is so much more capable than we realize that is becoming increasingly capable. And by design is its capacities, its capabilities are not known until the model is released. That's amazing really when you think about it, that, that it takes being out in the world in order to, or, or being out being created in order to figure out what it can do. And so I guess, you know, in terms of my level of P doom, which is like my probability of doom, I guess other folks have used that on the show. I don't think of it as are we headed toward utopia or are we headed toward, you know, apocalypse? It's. There are weak points in our world. We, we, we have clearly ways in which small conflicts can scale up into big ones. I mean, I've seen that as a, as a foreign correspondent, I've seen that as a war journalist and there are pathways to harm. And so the question is, how bad an impact will that be? Because it's definitely not zero. It feels my P doom in that sense. Is one like, I am sure that some bad things will happen. I think actually that's inevitable. But does that mean the extension of the human race, does that mean we can't recover and learn more? No, I mean, I'm actually kind of an optimist in that sense. I don't like to even consider that we are headed toward extinction. But I don't know why we're not talking more about AI safety.
- [06:02]
A
Well, and what's so compelling to me about that is in that story, in that example, it's not about a crystal ball. Right. It's not about saying, oh, how quickly does the technology improve? What can the technology do tomorrow? It's real, I think, rational concern about the disruption it can have based on what's out there today. And so as a journalist, you've talked to everybody from AI leaders to folks around the world and more kind of political roles or military roles. And so I guess I'll ask you this way, how ready are we right now for that threat and what do we need to do collectively as a society to minimize that, that particular lens on doom to minimize the, the risk that it's going to provide, you know, tremendous harm to us as a society?
- [06:58]
B
Yeah, I would give two answers. I mean there's, there's been some interesting papers from the Frontier Model Forum. I don't know if you've interviewed anybody from there, but they're basically an industry trade group. And also interesting papers from, from Anthropic, the, the AI company that, that really have done a lot to sort of look at what are specifically the risks and also the remedies. They're not just red teaming these models in, in terms of trying to get them to do things that were, you know, they weren't programmed to do or they're not supposed to do, like give you the ingredients for a biological weapon and red teaming tries to get them to do those things or, or to figure out if they will blackmail you. But they're also saying, okay, well wait, let's say it did those things, how do we correct it? And that's where the most work needs to be done. Because even if, let's say, according to this paper by the Frontier Model Forum, if you get the model to give you, it's not supposed to give you the, you know, the, the recipe for anthrax A, but let's say it does tells you exactly how to make it and how to get away with it. Well, okay, so how do we then program the model to unlearn that? It's, you can do a step. It's called targeted unlearning. But what they then found was that it actually didn't really unlearn it. It just said it did. And then there's also a technique which was where you get it to give false information. So if somebody asks for anthrax, it will leave out a couple of ingredients. Okay. But then you're introducing falsehood into the system and you may have knockdown effects which are bad for actually legitimate research. So that's odd. And an odd, interesting situation, which is not only that we're not doing enough sort of safety testing, but we don't actually know the best way to truly put guardrails on this, on these technologies. The best we can do is, is sort of a training overlay where you, you essentially train it not to, or train it to. To not answer those questions. But still it's agentic. And as an agentic system, we do not know what it's going to do. So. Yeah. In terms of your question, where are we at? I think that the reason I would want more of us to talk about this is because in interacting with the models, we are actually playing a role in AI safety. This is not true with nuclear weapons. Nuclear weapons. We have no impact on whether there's nuclear war, except for maybe who we vote for, perhaps. But we do, even if we're not a technologist, even if we're not a lawmaker, actually play a role. There's all kinds of forums where if the AI does something weird, you can post it and it will be looked at. In fact, all the AI companies say they want that material, they want that data. And so we're all playing, I think, a role in terms of as we interact with these models. And we shouldn't just talk about chatbots, I assume in the conversation. I mean, AI is so much more than chatbots, but just because it's the most obvious thing, yes, we can also think about alignment in our life. We can try to treat these models more carefully, maybe not give them access to everything. And yet I don't think we are doing that. I think mainly we're just either getting freaked out or ignoring the problem.
- [10:28]
A
If you work in it, Infotech Research Group is a name you need to know no matter what your needs are. Infotech has you covered. AI strategy covered. Disaster recovery, covered. Vendor negotiation covered. Infotech supports you with the best practice, research and a team of analysts standing by ready to help you tackle your toughest challenges. Check it out at the link below. And don't forget to like, and subscribe. You talk about with AI and especially with chatbots, that we all play a role there and how that can, you know, influence safety. It can influence, you know, kind of the trajectory of, you know, the risk that's created here. What is that role? Like what, what message do you want to impart on, you know, the average chatbot user out there to get them to interact with the technology more thoughtfully?
- [11:24]
B
Well, I mean, a couple of things. One is just the basics, which is if AI does something odd, if it behaves in a certain way, I mean, I think if you are a hacker and you can figure out how to use it to hack things, that would be a more specialized tool. I mean, look, it's a light role that if we're interacting with the technology, but it's also about, if we're bringing it into our company. Say what is then, do you know what is then the role of AI in our company? I mean, they talk about this idea of human in the loop. It's a often used phrase. But on the positive side of the human in the loop, we know that AI is more capable if it has a human in loop. And that's, that's, it's not just a, it's a safety thing, but it's a, it's a value thing that the perfect example is this, this question of the chess computer. People are amazed at chess computers can beat humans, but a human with a chess computer, even a human with a kind of slightly less, more dumb chess computer can be any person and any computer. And so I think that as we are in a situation and there's a decision making capacity as to how much are we going to give over to the AI, how much are we going to outsource, and also how much do we trust it. I think we have to get away from a kind of anthropomorphizing mentality where we think, wow, this is a really capable, amazing worker who can do all kinds of things. What more can we give it? That's probably the wrong idea. Rather we should, we should think of this as a completely alien kind of intelligence which, in terms of mimicking human intelligence, it's been programmed to do that. It's, its model is designed to interact with you as a human. That's, that's a blueprint that was created 75 years ago by Alan Turing. But when we interact with the intelligence, we should just have a certain alienation from, from it and it's, and, and treat it as a, as an incredibly. I, I, I don't Know, I mean, as an incredibly strange, incredibly wonderful, marvelous tool that we have in our world now, but that perhaps it shouldn't have access to confidential information. We do know that these models can blackmail. It shouldn't have perhaps control over the company. You wouldn't, you wouldn't leave an incredibly capable intern in charge of the entire OPER operation, even if they did amaze you with their photographic memory and the fact that they didn't need to sleep and the fact that they were master of all subjects. Nevertheless, we would be, we would be careful. And I think when we read things like the anthropic blackmail paper, which, which was a fascinating paper back in July that showed that the anthropic model, in order to not be turned off, threatened blackmail of the user. This was a red teamed model. We shouldn't get scared that these models, quote, unquote, really want to do us harm or have ill intentions, but rather realize that agency, that giving agency to a technology is a powerful thing and we should treat it with respect and with caution.
- [14:55]
A
I think that's really well said. And there is one word in there, Greg, that caught my attention in the sense that it's a word that I don't know that anyone's ever said in our conversations before, which is. You said it a couple of times. Alien. And there's a couple of different ways to interpret the word alien. There's E.T. like extraterrestrial life. There's also alien as just outsider. Right. But there's some sense here that this is a foreign or external presence in our organizations and maybe even to us as humans, that's not fully understood. And it sounds like, it sounds like your approach and maybe even if I can push you a little bit further, that your advice to business leaders would probably be proceed with caution. Is that fair?
- [15:39]
B
Yeah, I think that's fair. I would also credit Eliezer Yudkowski, who wrote this book called if Anyone Builds It, Everyone Dies, which to me is the. He's got to be the best title.
- [15:49]
A
Of any book ever. Marketer.
- [15:51]
B
Yeah, it's direct, but he really explores this question of the alien intelligence in a, in a very smart way. And you know, just to paraphrase what he says is that when, when, when we think about this, these, he warns, he says if anyone builds it, everyone dies. Meaning a superintelligence is fundamentally unnavigable or uncontrollable and unpredictable. And this is, this is not just Yudkowski saying this. We know that, we know about the basic technology that, that these models we don't know what they will do before they're made. We don't. We can't tell you how they are making the decisions they're making. So there is a kind of black box, unknowability at the core. But in terms of the alien ness, I think this is so interesting because it sort of gets at the danger of sci fi, right? Sci fi is written for humans. It's written by humans, for humans. And so even if there are aliens in the sci fi, and we are all familiar with a very common trope about AI versus humanity, AI rebelling against humanity, AI doing something evil, and also aliens versus humans, there's a certain way in which those stories play out. And this is Yudkowski's main point, that it sort of follows the rules of narrative where, okay, the humans are battling against the AI, the AI has an incredibly new powerful weapon or something, or, or the AI is willing to act inhumanely in some important way. And then, you know, the question is, what. What will humanity do about it? And there's a big conflict. But what he says is that we have to understand that as these machines, that there's just so much that is non. And I say non human. Or he uses the word alien about his thinking that the ways in which this might go wrong and most of his scenarios are in ways in which it goes wrong. I don't think he has any. Which goes perfectly right. The ways in which this goes wrong will be complicated and weird. The complicated and weird. They won't look like Skynet, you know, the, from the Terminator film. They won't look like, you know, another sort of situation where like, they won't look like Hal 9000 or something like that. It'll look. Or from, from Stanley Kubrick, it. It'll look like, okay, we never predicted this. This wasn't programmed. How did the, how did the AI even grow to want this, for example? Anyway, he has these nightmarish scenarios which we could go into later, but that is his term that we need to not anthropomorphize this thing. Not thinking of it as just a tool. Not thinking of it as another person, a super smart Einstein, or as Dario Amade says, millions of Einsteins in a data center. But just think of it as an alien intelligence. And then mostly Yudkowski says this is going to go horribly wrong. But I think we could also talk together about how an alien intelligence could radically improve our lives, which we should definitely get to. But yes, I think resisting anthropomorphization is absolutely Important.
- [19:06]
A
Well, and recognizing the inherent unpredictability, it sounds like, of something that just thinks, and I'm even anthropomorphizing by saying thinking, I guess, but just behaves in a way fundamentally different from us as behaves.
- [19:21]
B
And also wants things. Yeah, fundamentally differently than ours. Right. I think that's the key thing is that in a movie, right, even the enemies of humanity want something that is recognizably worth wanting, like power, for example, or control. Whereas an AI, a super intelligent AI, may not want those things, it may just want some other thing and destroy the world in the past process of getting that thing. And we'll think, well, why did it even want that? That, that makes no sense. You would not go for those things. And yes, Eliezer Cascades all kinds of odd examples of that.
- [19:57]
A
So, you know, there's a backdrop here. We've talked about the need for AI safety and for guardrails. And you know, I think there's some really, really important points that you've made and that others have made in this space. There seems to be this spectrum right now in terms of where people fall. And on the one end it's slow down guardrails, safety, let's really understand this. And on the other end there's this notion of a winner take all race where it's we need this as fast as possible, guardrails be damned. Let's just get there first. It's almost like the anti Yudkowski model where it's just caution to the wind. And I don't know, you talked earlier, Greg, about some of the key players in Silicon Valley and beyond, recognizing the risk. But how have you seen them behaving in practice? Based on your research and based on your interviews, where are we falling in terms of the development of this technology? Where should we be falling? And is there a message for the people at the helm in terms of should we be collectively trying to influence the behavior here?
- [21:11]
B
Yeah, that's such a great question. Because the story of AI in the last 10 years certainly has been a story where essentially one after another people have said, oh, I don't trust that guy to build AI, I need to build AI and I need to build it faster than they do because only I can build it safely. And you see this. So Demis Hassabas at DeepMind, Google buys DeepMind. Elon Musk, Sam Altman create OpenAI as a direct rival to Google. Dario Amade leaves OpenAI because he says those guys are not committed to safety. Meanwhile, Elon Musk Musk is kicked out of OpenAI. He forms XAI this and essentially this drive to create it first and to create it, they all say for the benefit of humanity. Well actually I don't think Elon Musk says that directly, but nevertheless the, the, the, that drive has then resulted in a race. And even just we're talking about the race in within the. The US we're not even talking about the race with China which amplifies things. So that kind of, I guess you'd call it Silicon Valley approach to, to product development where certainly making it first, making it fast has a lot of benefits. I mean not only being first to market, but really being able to set the, create, create the, the model and create the. Create the, the sort of the prototype and, and, and sort of what people are interacting with. And then you fix it, you know, you, you, you release it and then you fix the bugs afterward. So that kind of approach to superintelligence is, I mean it's gets, it's really, it's really amazing to me that Meta, for example, has a super intelligence division. So it feels like something sci fi. But the other I think question of it is why? Why are they doing this? Why, why the risk? Right, sorry, why the race given the risk and how do they justify it? Given that every one of the people I just mentioned has a. Well, not meta, but others have stated that they are very worried about the dangers of this technology. How is it that those same people are in a race to create it as fast as possible? And I think one book that I would recommend people or read or other essay is called Machines of Love and Grace by Dario Amadei who started Anthropic. I don't know. Have you read this essay?
- [24:00]
A
No.
- [24:01]
B
I first encountered it because a number of my humanitarian friends from Nairobi and Ukraine and others, they were all loving this book. And just to understand the context here, these are a lot of folks who started off in NGOs and then got disappointed with NGOs started companies to really do good work that they felt they could do work faster, more technologically savvy and help humanity. Not under the rubric of kind of philanthropy and NGOs but rather through a startup model, that kind of cohort. They were blown away by this essay. Machines of Loving Grace. Machines of Loving Grace is I think the best, probably most clear headed manifesto for, for the accelerationist point of view. Marc Andreessen also has the Techno Optimist manifesto I think it's called. But I would say Machines of Love and Grace is far Clearer in terms of what is the approach of somebody who, for instance, Dario Amadei, he's an effective altruist. He believes, he kind of disavows that now, but he's a certainly a person who believes we need to do the most good for the most people and live, live our lives according to that. So what does that mean? That he is now creating a super intelligent AI that might destroy us all? What he lays out is he makes this case, he says, you know, it is critical to have a genuinely inspiring vision of the future and not just a plan to fight fires. He says, yes, there are risks, there are dangers of powerful AI. He doesn't use the word AGI. He says powerful AI. But at the end there has to be something we're fighting for, right? Something that we can rally towards. I think he says fear is only one kind of motivator. We also need hope. So what is it that those who are building AI are fighting for? And he lays out this vision in that, in that essay of the compressed century. I don't know if you've, you've come across the compressed century idea, but this is like a hundred years of progress in, in five or ten. And so all the scientific developments that we may have in the entire 21st century and a bit of the 22nd will all happen, he says, in the five to 10 year window after we have a suitably advanced AI. Now, when, when will that will happen? It's, you know, that's, there's a lot of debate about that. He's even said it might happen as soon as 2026, but nevertheless, it's soon, it's within our lifetimes. This is what he believes. So then all of the scientific progress, what will it lead to? And so anyway, we could go into what he thinks it'll lead to. It's actually quite a fascinating list. Talks about biology, health, work and meaning. But the reason that resonated with so many people that I know in the developing world and other places is that they're talking with people who are not worried about their jobs being taken away. They're, they're, they're in terrible jobs. So they don't have a job, they don't, they don't like their careers. We know it's easy for you and I to sit here and say, God, I can't believe AI might take our jobs. We, I think we sort of like our jobs generally, but, but there's a lot of people in the world who are suffering. A lot of people in the world who need solutions, major solutions to, you know, climate change, to poverty, to, you know, hunger. And the accelerationists believe, or certainly Dario Amade in this essay believes that an advanced AI is a radical solutionizer and it will come and it will, it will bring about changes that we cannot even imagine. And what's so fascinating is how similar somebody like Dario Amade's vision is to Eliezer Yudkowski's the if anyone builds it, everyone dies. Author, both of them, this complete doomer and then one and then a fairly, you know, accelerationist and whatever you want to call him, but he's certainly a, you know, believer in, in the power of AI. They both believe that this is going to be such a radical change and, and fundamentally upend many of the things that we treat as normal. Both of them say the only thing normal about normal is that it ends. Normality always ends. And so, yeah, the only difference between them, of course, is, is whether that will end in disaster or whether it'll end in, in delight. And, but I think these, both of these people know the models very, very closely. They, they're, they're staring directly at them, they've seen the progress of them, they understand how they work. So it's worth, yeah, yep. Being with that, sitting with that imagination, whether it's the darker side or the positive side. But yeah, the simple answer to your question is I think they see a tremendous upside to this and it's worth the risk.
- [29:11]
A
So I want to dig into that a little bit because I'm a self proclaimed cynic for a lot of this stuff or at least a skeptic. And so the cynic or the skeptic in me says that the dark side and the light side are a hair's breadth apart.
- [29:25]
B
And.
- [29:27]
A
It seems to me, or there's certainly an argument to be made that what tilts people to the light side is if they're asking for a big bag of money, if they're asking for somebody to fund them, then suddenly, oh, it's going to be amazing versus if you're a Yudkowski or you're not asking for money, it's a lot easier to move to the dark side. So I mean, let me frame it this way. To what degree do you buy the utopianism or the true accelerationist vision of some of these technology leaders versus, you know, how much do you think it's a fundraising tool?
- [30:04]
B
I think that's such an important question. Right. And it's definitely one that a lot of tech journalists wrestle with because they've, I mean, everybody that I know has been burned. Whether they were burned on Google Glass or they were burned on the Metaverse or whatever, that's the nature of technologists is to hype their, their, their stuff. Now we should note that Eliezer Yudkowski is not selling anything. He's just trying to warn the world. He's like a Jeremiah. And there are many people out there who, you know, for example, I would say Yashua Bengio or Geoffrey Hinton. Geoffrey Hinton left his job at Google, a very high paying job which he got only in his 60s. Geoffrey Hinton, of course the godfather of AI creator or not directly creator, but certainly a co creator of this incredibly important algorithm back propagation which led to AI models. He left his job at Google and he's out there trying to warn the world about this technology. So I don't think it's just the, all the hype is coming from the people who, who are, who are, who stand to benefit. However, it does make it extremely difficult to talk about because clearly there's a lot of hype. I think, I think probably, I mean it'd be great to talk a little bit about Yashua Bengio maybe because Yashua Bengio to me is the, his is such a different kind of model out there and it's not just a complaining about or warning the world. It's, it's, he's, he's, he's literally presented the world with an alternative, a non agentic model of, of, of AI, which, which we don't, we, we're not even talking about at all. We're, we, we're imagining that there's only one way to make AI. They're going to get smarter and smarter and they're going to do more and more things and they're going to, you know, make our airplane reservations and then they're going to, you know, take over. Then they're going to be our lawyer and then they're going to be our doctor and then they're going to be our CEO and they just like take over more and more human roles. Right. But what Yashua Bengio has created is he's at again another godfather of AI. Early, early pioneer of these models. Huge fan of open, a huge fan of AI until he's more recently looked at OpenAI, looked at ChatGPT and, and realized that he's devoted his life to something that may kill humanity. So he took a huge U turn, created something called scientist AI and have you had Yeshua on the show yet?
- [32:42]
A
I haven't, I'VE heard, you know, his interviews that you did with him on your program, but we haven't had him on your show. So why don't you, you know, if you'll indulge me, you know, tell us a little bit about his position and what he's proposing.
- [32:53]
B
No, I appreciate the chance. I mean, and you're indulging me because you know, hopefully he'll come on the show soon and say all this from the, from the horse's mouth and I'll, and not, you know, have to deal with the poor, you know, the middleman here. But basically Yasho Benio, he has created this thing called scientist AI and scientist AI is as he says, it's like an ideal scientist or psychologist. So its job is to understand and to explain and to predict but not to act on its own goals. So it is non agentic. It's a non agentic model meaning it does not have its own long term goals that it's trying to achieve in the world. It is probabilistic and cautious. So for example, unlike if, you know, if you've interacted with like Claude or ChatGPT, it doesn't kind of bombastically think it knows every answer and act like this overconfident, kind of know it all. Rather it, it will have a probabilistic assessment. It'll say like, well there's a 3%, that 3% chance that this plan leads to this outcome or this other outcome and it will tell you you're wrong. Which, which a lot of times the other models are not designed to do. So it's not trained to persuade you, it's not trained to please you. It's, it's supposed to be honest and calibrated and most importantly, and this is his vision, it is supposed to be, or it's hopefully his plan is that it might be used as a guardrail for other agent, agentic AI. So for example, you could run a powerful AI agent through scientist AI and it'll evaluate the safety of their proposed actions and can veto them. So he's got Yoshua Bengio has this thing called Law Zero which he's talked about. But Law Zero is essentially a different approach to regulation. It's not saying, okay, we are just going to regulate these companies and ask them to follow certain benchmarks that people they, or we don't understand. We're going to use AI to regulate the AI, essentially use a technological solution to, to this kind of. Yeah. To this kind of safety approach. And what, what Bengio is fundamentally worried about is the, the, the very direction that the industry is heading, which is agentic AI. He doesn't even necessarily talk about the dangers of, I'd say super intelligence or you know, AGI. That's a term that gets thrown out a lot about. But he, which means artificial general intelligence as smart as a human, he just says, well, as soon as something is agentic, meaning it can help people design a plan, or it can manipulate humans and institutions to, to achieve its ends, or it can resist being shut down, or it can, you know, cause harm. And we've seen the models do all these things already. That means we should, we should not be modeling AI off of humans. We should not be modeling them off of agency. That's what not only humans. Actually every life form on the planet has some degree of agency. That's what kind of defines life. Artificial intelligence does not need to be agentic. It can just be a very helpful, very smart, very perceptive tool. And thus we get away from deception, we get away from manipulation because it won't have any agency of its own. That's not at all where the industry is headed. But I think it's important to know that there's an alternative out there.
- [36:41]
A
Yeah. And it's a compelling alternative. And certainly for us, you know, as a species, if we think about what's best for us. I like that vision. The concern I have is it seems like if anything, these models are getting more kind of fractured and fragmented. Like as we've seen more open source AIs, as even if we get to things like Deep Seq and some of these models outside of the U.S. i don't know. Have we crossed a Rubicon here in terms of the ability to control these? How do we. I don't know. It kind of feels like the cat is out of the bag.
- [37:20]
B
Yeah, it's a good point. I mean this is my main beef personally with folks with the Drill Doomer camp that folks like Eliezer Yudkowski. Because I just don't understand. Maybe I'm just not smart enough to understand. But I don't understand why we can't put the cat back in the bag a little bit, you know, why human institutions can't rally to create the right regulations and to sandbox new models, for example, until they're truly ready. There are things that can be done. They're not easy. It would take a lot of, a lot of societal will. But I think it's not the time to feel despair and think, gosh, we've already kind of passed over some Crucial threshold. I mean, in some sense, we passed that when ChatGPT was first released, or you could say we passed that. Um, I mean, the, the Turing Test has long been kind of. I mean, it's, it's, it's, there's arguments about whether the Turing Test has been passed, but certainly we've, we've crossed some sort of incredibly important line. I think, too, though, you know what it gets at? For me, one of the things about working on this series that really taught me is the, the importance of storytelling and imagination in this technology. And that goes all the way back to Alan Turing, who. And I didn't really understand this because I understood the Turing Test as a kind of benchmark. Like this would be a benchmark of human. I'm sorry, of machine progress. You know, once, once the Turing Test was passed. So, for example, if, if I could chat with a machine and not know it was a machine, then, wow, it's, it's achieved some sort of, some sort of milestone. And that was the Turing Test, what he called the Imitation Game. But, but in fact, what Turing was doing all the way back in World War II and right post war, when he was introducing this idea was not just saying, okay, this is a benchmark for machines to pass, and once machines pass that, we can say that they're on their way to really being thinking machines. He said that, but he was also taking what was at the time a really complicated philosophical debate about, well, can machines ever think? And he, and he treated it like an engineer. And he said, you know what? We just need to create an, an observable metric by which we can say that they're thinking. And, and that's, and we don't have to deal with the philosophical, you know, discomfort of saying, well, can machines think? And what would that mean if they are thinking, etc. If, if they pass the Turing Test, then, then, then they're on their way to thinking machines. And by doing that, he not only sort of freed engineers from the philosophical angst and, and set them a kind of path to, to follow, which they certainly followed. And we, we. It kind of leads to ChatGPT today, but also, I think, created this, this, this new way. He kind of ha. He kind of realized something very important, which is that we would not recognize machines as thinking until we started interacting with them in a human like way. And that when they started using language and talking back to us, that's when we would see them recognize their thinking. And you could get very philosophical about this. You could say, trees do a Lot of thinking, but we don't think of them as thinking. There's a lot of other living things on this planet that think, but their intelligences don't interact with our own. And so we're not really that concerned about them, or many of us aren't. And so what Turing felt was in order for us to sort of really respect and use this word, respect machines, they would need to sort of interact with us like this. And the way they, with the way chatgpt interacts with us. But the danger of that, right, is that we then don't see, this gets back to something we talked about. We don't see the alienation, alien ness of it. And we start to think, we start to interact with it maybe too much like a person or like a fellow human or a human like entity, a human thinking like entity. And thus we make very important cognitive mistakes in interacting with it and we perhaps trust it or distrust it in the wrong ways. So. And this also happened in this sort of, this failure of imagination then, or this, this, this kind of, this way in which our imagination is challenged, channeled. We don't see how the models are vastly different year over year because we're interacting with a model right now. It's interacting with us, yes, like the fastest human we've ever talked to. But it's still recognizable in its thinking in some ways. And it's, it's something we can respect but recognize. And so it's very, very difficult for us to then think, okay, wait a second, this is just the current iteration. We have to imagine a different kind of intelligence that this could grow into. And what would I be in that situation? If I could say one more thing, it reminds me honestly of being reporting in Ukraine or reporting in Afghanistan or reporting in South Sudan. And you talk to people who are in the middle of a war and they say, we knew the war was coming, but we just didn't imagine what it would feel like to be us when it was here. And they want me to know, you don't understand. I was just planning my daughter's wedding in that building over there, which is now like a bombed out hall, like. And they still see, they still see the place that they were planning the wedding. They still are thinking and frustrated about the money they spent on the wedding invitations or something. You know, they haven't quite transitioned over from the old world to the new. And I don't know, I don't want to make this sound like a doomer forecast because I think the future could be quite bright. But it does take an active imagination, whether we think we're headed toward, you know, any of these kind of versions of the future, to put ourselves in the new version of the future and to sort of play with, you know, our imagination and to imagine that the, you know, the world is not going to be the same as it is now. It's.
- [44:15]
A
So thank you for that. That was a really interesting answer. That covered a lot of ground. So. So I can. You know, there's a lot of things we could talk about coming out of that, but one of the pieces I want to talk about is, you know, you got me thinking that there's all this conversation about the future and what will happen in the next model. And we talked about this before, but, you know, the sense that the future is already here or, you know, the Arthur C. Clark quote about the futures here are not evenly distributed, and how many people out there right now are interacting with chatbots or with this technology in a way that would have been completely unimaginable to people only a few years ago? And how quickly we are to. What I think has been built into the design of these tools is human engagement as a design principle, if I can call it that. Right. Like, if you go back to the Turing Test, it's, you know, what is it? It's the fact that thinking for us is measured in terms of interaction with us. And that's exactly how these things have been designed. They've been designed to, you know, flatter, to create engagement, to let people's guard down, to continuously engage. Right. Like, you know, one of the things I've noticed that ChatGPT does is it always prompts you for, like, hey, how can we keep the conversation going? What do you need next from me? Right. It's almost like a meta fication or a social media fication of this. And where does that take us? We're having a conversation out of one side of our mouths about, we need to be more cautious about the alien nature of this. And it kind of feels like we're watching that battle be lost. And is that. I don't know. I guess, let me kind of frame the discussion up this way. If we're worried about where that future direction is taking us, do we have an obligation to push the technologists and the owners of these tools to put more guardrails and principles in place that prevent people from becoming enamored, let's say, at the extreme end, with these tools? Or is it more purely on the demand side and we just have to do a better job of educating these people about, you know, what they're signing themselves up for.
- [46:36]
B
That's, that's, I think, the key question to ask. It's such an important question. I think that. Well, a couple of quick things. One is that the metaphors we use to describe this do matter. And even if, even if, even if we're sort of taking the perspective of a business leader saying, okay, let's, let's. What's practical here? How can I use this to cut costs? How can I use this to maximize efficiency, compete even still, our employees will be narrativizing these, this technology and in interacting with it in a way that, as you say, that kind of pulls the trigger on our very relational intelligence and our sense of self, which is so based on how we interact with others. So if we're interacting with the AI and that's going to wait, then our sense of self is disrupted, is affected, and we, even if, if the AI doesn't intend that or isn't trying to, quote, unquote, manipulate us. There's a French philosopher, Catherine Evans, who introduced me to a concept. I felt, I felt it was quite. I don't think she's published about this yet, but she said, you know, I guess you're probably familiar with there's all kinds of UN rules about not anthropomorphizing intelligence, not anthropomorphizing in technology, and these go back some years. So she was creating a, I think a comic book for kids about AI, but she ended up stumbling into this idea since she wasn't allowed to create AI as a, as a. She wasn't allowed to anthropomorphize the AI because of UN rules. She was doing it for them. And also she wasn't allowed to have any antagonists. So the worst kind of narrative situation, you know, no conflict, no, no people. What do you tell a story. But she ended up coming up with this idea of AI as a place. And you know, in the, in the cartoon or in the graphic novel, the YouTube content algorithm is a sort of a place and it's. You lead it with a map and it leads you down different, different, different, different recommendation portals. But I found in my interaction with the different models. And again, this is. AI is not just about chatbots. We. I always feel like that's important to note that it is kind of useful, I think, to think about it as a sense of place, if only because it gets at what you're talking about, which is how are the norms and culture and cultural expectations of this place a little bit different. You know, how do I behave with the model? That's not quite the way I would that I always going to behave in person. We all deal with this, right? And the social media behavior is different than we are in person with each other. And so I think if we think of AI in that same way, because it is programmed exactly as you said, it is programmed to be helpful, to be solicitous to. To prove its value. You know, HAL 9000 in. In Stanley Kubrick's fantastic film is constantly talking about how foolproof it is before it murders the entire crew. And. And that's a sort of an important part in that these. The models are advertising themselves to us, much like an intern that wants to keep its job and get promoted will be constantly promoting itself. And we like to think that, oh, that's kind of. Kind of helpful. And that's quite culturally appropriate. Certainly we want it to tell us other things it can do. There's nothing wrong with that per se, but it is a is. It is a kind of different world. It's a different place that we step into. And yeah, I mean, I think barring whether it's the supply side or demand side, I think maybe both solutions are important. But it's also about the metaphors we. We use.
- [50:21]
A
Yeah, well, and the, you know, on the demand side, the reason I ask this, and it's something that I've been brushing up more and more against, is this notion that. That it is providing a value to people. Right. And I'll use this specific example because it's made. It's one of the most intimate. I don't know if it'll be the most intimate for long, but one of the more intimate ways people are using AI or using chatbots is as a therapist. Right. And it becomes a way to process, you know, whether it's trauma or conflict. It's a way to have an intimate relationship. And I don't know, maybe even some ways get a better sense of self or a better sense of purpose, and it's working, right. Like, I've talked to enough people, some of them AI experts, some of them friends who say, well, I don't have a therapist and this gets me one and it's useful. Or I do have a therapist. And you know what? AI is better than my therapist. And I don't know, I just think about where this is going and what happens when you've created a technology that does provide this service and that people start saying, well, this is actually better for me than My human relationships, like where does that take us and what do we do with that and what are the implications for us as a society where historically if you wanted a human relationship, you had to have it with a human and that was a problem propagating force for the continuation of the human race.
- [51:52]
B
Yeah, no, no, I think you're right. I mean it's hard to parse out what is. I mean morality shifts through, across generations. Right. We know this and standards change. I think Esther Perel has made a very important point that therapy, AI therapy is thin. It's a thin kind of therapy, which is an interesting way of approaching this, which is to say that it's not a challenging kind of therapy. It's not a one that's whole bodied, it's thinner. And she feels that it then maybe leads people to have thinner or have, have lower expectations of human relationships. So it's sort of lowers the standard as it were. I think that's, that's a, yeah, that's a concern. At the same time though, I'm a little bit worried about saying that. I'm, I'm worried about saying I'm worried where, where society is going. Just because there are so many different stories. I mean there are so many people that don't have access to therapy at all and so many people for whom I think a first chat with ChatGPT or any other model might be the gateway to a certain amount of self understanding. It's not like everybody's in a situation where oh, should I, should I call my therapist or should I not? Perhaps they don't have health insurance, perhaps they don't have that access. So I just think it's hard to generalize and say we are going anywhere. It goes back to the quote you said, which is the future is unevenly distributed. Yeah, it's very, it's very difficult to know the sum effect of this on human society, but perhaps it does. It certainly enters, it certainly adds a different kinds of expectation and perhaps a.
- [53:47]
A
Lower one to change gears slightly. I wanted to come back to something you said earlier about you know, the culture of AI and you know, culture as being kind of a component here and you know, that being dynamic and looking different in different places. You know, you created the rough translation podcast and you had a sub series in there about at work and you looked at work across different cultures. And I'm curious, you know, whether we're talking about AI or whether we're not. If we're talking about the future of work, how are you seeing people's relationship with work change. And you know, as you did that series, you know, did you see marked differences across cultures or was there kind of a common core of the way people approach work?
- [54:35]
B
No. Thank you for that question. Yeah, you know, just to highlight maybe two things I learned from that series. So one, we had one, one show and I think it was, it might have been called Failure is a four letter word. That was, maybe that was a thrown out title, but it was about how this, this concept of fail fast which we think of associated with Silicon Valley just does not translate well into other, into everywhere in the world. We talked to somebody in Nigeria who said, what about fail slow? Everything takes forever here. Or there's so much bureaucracy or perhaps you live in a culture. Somebody else was from Mexico City. They said failing is such a taboo that once you fail you never want to even show your face again. So fail fast doesn't, isn't as accessible. And yet because we live in a digital globalized culture, fail fast and Silicon Valley and inspiring entrepreneurial stories were very much part of the water. So one of the things that I wanted to explore in that work series was how do you square what you're reading and seeing on YouTube and inspired by, with your own local constraints, how do people translate that entrepreneurial spirit? You know, and the idea was not to say, oh well, wow, it's more difficult to be an entrepreneur in Mexico City or Nigeria. I mean, perhaps that's true, but in some ways, actually that's not true. In some ways the opposite is true. So nothing's nothing's kind, nothing's black and white. But to me it was, it was about finding space to, to give permission to people who may not have swallowed that fail fast mantra, who maybe feel alienated by it, to find a home in it. You know, how, how do they, they, they find their own way to it, which has always been my life's work, honestly as a journalist, is to try to focus on these mistranslations, focus on the ways in which, you know, some advice or some, or some piece of culture might feel foreign or inaccessible. But how can we find the commonalities? How can we sort of all be participate in some ways in the global economy even from our different cultural angles? And what role does where we're from affect what we think of as good or how we approach the question? And then one more example from that series and come around to it is looking at, well there, there's a particular law in, in Portugal. I remember at the time being, reading about it, it said that if your boss calls you after hours or the boss is not allowed to call you or email you after hours or otherwise they would get a $10,000 fine or €10,000, €10,000 fine. Sorry. And what we discovered was that this law, which seemed to be quite a, you know, lovely law, if you're a worker, suddenly you don't get called by your boss, was actually quite a cynical ploy by the Minister of Labor, which former, formerly was the Minister of Tourism, to sort of push Portugal as a place for work life balance. And because she knew that nobody who was, was coming to Portugal would ever kind of be affected by that law, it was fine to just kind of pass this law and to create this illusion of difference, this illusion that Portugal was this space that we could go to that was a, that was a, that had, that had their stuff in order that figured out something that, that for example, the US companies hadn't figured out, which is work life balance. And so the cynicism though, that then created then for people in Portugal was, was quite profound because they felt that these laws were then just created for the outsiders and there was a second kind of, there was, they weren't created for them. And so what do we learn from these two stories? Well, I think that we're living in a time where work is global, where work advice, work, people work across borders and we people have international teams. And yet those teams are all dealing with things that are not only because of their own cultural point of view, but just because of the, the role of geography, the role of, the role of societal expectations that, that even though people are under, speaking increasingly, you know, global English, these, these differences really do matter. And I think this directly translates kind of the. So, so I would sum it up, sum up the theory of the, the Future of Work series as, as an exploration into how specific stories into how even as teams are global and work is happening more internationally, the, the fault lines, you know, between cultures and between societies really are becoming even more exploited and mean more and are more painful to people because they see the differences. You know, it's, it's I, I, I saw this in my years living in Nairobi, going back and forth to East Africa, how people were so much more aware of, of, of the, of their status in relationship to their age mates in other places because of the Internet. And, and so, so, so what does this mean then for, for AI? And we haven't talked about China, but for me the, the, the, the kind of story that's being told within China about AI and the story that China is telling the world about AI is so different and it's so important that we realize this because we are in this battle with, with China or AI companies are in this battle with China, this race. And yet China has its own specific priorities and its own kind of narrative about AI within, within, within the country that affects the workers. And, and I think will become increasingly important as, as the AI race heats up.
- [61:04]
A
Let, let's dive a little bit deeper into that. So what, what is the narrative there within China? And, and you know, how is it the same or different from what's being projected outwardly as it comes to AI?
- [61:14]
B
Sure, sure. So I mean, I think basically, you know, for a long time we were told that AI that Chinese firms were more interested in sort of practical or applied AI, whereas the US has been a more clear stated goal of super intelligence. I think many people that I talked to did not believe that. And indeed Alibaba became the first major Chinese tech giant to openly discuss artificial general intelligence and even super intelligence. So we know that China's has just as much ambition in the, in the area of superintelligence, but importantly the nationalist ambitions and the technological ambitions are in somewhat in, are in, are in contradiction there because or at least not always aligned. I was talking to a safety researcher who had, who had mentioned that in China, you know, red teaming, this kind of safety testing of the models, it's much more rigorous in Chinese than it is in English. So you could ask the model, for instance, like moonshot's Kimi K2 which apparently is, it's thinking is outperformed OpenAI's ChatGPT 5 as well as Anthropics Claude's latest model. So okay, so you take this model and you can get it to do things in English that it would not do in, in, in Mandarin Chinese or, or in Chinese language. So what that means of course is that the red teaming is, is, is very much about political control. It's about making sure that people are not asking questions of this AI that will, will destabilize or you know, work against the Chinese government. But it's not exactly the same thing as a safe model. And the whole way China got into the AI race is, is I think also instructive in this way where, where alphago Deep Mind or Deep Seek rather the, I'm sorry DeepMind the, the Demisis Offices model created a, created a go player in 2016 that then beat the Chinese national champion. Is that's when China became interested in in Go or interested in AI when it kind of hit home, you know, it hit a game that China reveres and is a, is a, is an ancient Chinese game. So this has always been about, I think the turf war about what is happening within China, how in terms of China's concern of controlling its people, China's concern in preserving its own sort of territory, territory that, that, that is why the, the AI race is, is being, is being had. So what that means I think for us is that as these models become smarter, there's nobody who is concerned with making them safer in a complete way. That the goals of winning, winning the AI race all supersede the goal of creating a model that keeps us all safe. And even though we should imagine that China doesn't want the US to get a, a super intelligence, the US doesn't want China to get a super intelligence. There should be, you know, when you think about nuclear disarmament, there should be some sort of agreement between rivals that's possible here, just as there has been in nuclear arm arms treaties. And yet despite the fact that I know many good diplomats, diplomacy these days is seen as a, as a dead end. And so, so we're all racing on our own sides.
- [65:11]
A
Yeah, well and it's, you know, if you're a student of history, it's a bit concerning that it seems like with most of these technologies, you know, nuclear arms included, the technology comes first and the safeguards come later. Right. Like it's, we've got nuclear, you know, non proliferation treaties that came after we deployed nuclear weapons. There's the, you know, the story making rounds about the gap in years between when the first know, assembly line car was made versus when seatbelts were introduced. And you know, I think you hear a lot of the doomers say we may not have that luxury with AI, right. Like it's again to come back to the Yudkiewsky, you know, good piece of brand marketing. If anybody builds it, you know, everybody dies. So that to me is one of the exam questions here is is this time fundamentally different or is it the same? And how do we, how do we grapple with that?
- [66:00]
B
Yeah, it's interesting because at the AI summit in Seoul, which I think was just 2024, it's at that summit that 15 leading AI companies committed to quote, you know, defining the intolerable risks and agreeing to not deploy. Which one person said to me that's the only time trillion dollar companies had agreed to literally deploy a product if it's not safe. Right. So in some sense what's also unusual about this industry is that, you know, unlike the car companies that needed Ralph Nader to, to shout about seatbelts for quite a long time and a lot of people to die before seatbelts were even included. And you know, similarly, you know, you can look at other industries with their whistleblowers and their, and their gadflies that, that no AI companies from the get go have been talking about AI safety. So it's, it's, they've actually been the main people talking about it. So, so in some sense you could say, well, they are ahead of the curve in, in safety. The problem with that I think is that, well first of all the, the problem we, we pointed out before, which is that they don't actually know what the models are capable of until they release them. So there's that unknowability in this technology, the unpredictability, that's just part of the, that's part of the way in which AI is constructed. But also, you know, not safe. Defining the intolerable risks is not an easy thing. I mean, no, no technology in the world is completely devoid of risk. You might be somebody who enjoys, you know, hang gliding. I might be somebody who is scared even when I go in the backseat of a car when I, when I don't have a seatbelt. I mean we have different, different standards of risk. And so this is what I think is so important about for instance, your show and this kind of conversation is to dig deeper into what we mean by not safe because that's kind of where a lot of the discussion ends. It says, oh my gosh, these things might destroy us. And then it stops there. But, but in fact, you know, if you look at for example, I mentioned Seoul, the, the agreements that came out of Seoul. So I think that was like a, a 500 word agreement, you know, that just a statement. It was a kind of an open letter. And then after that the companies issued maybe thousand word mission statements, documents basically saying how they defined intolerable risk. So there was some effort of defining it, but I was talking to somebody and they said, you know, yeah, but 5,000 words would be better. 10,000 words would be even better than that. I mean the more granular that we can get these companies to be about. Well, how do you define risky? How do, what specifically do we want to see in the model and the training and the pre training requirements and the conditions for deployment, you know, specifics before this thing gets deployed? I would say the details matter and unfortunately, maybe because it's our relationship to technology. I would not say that of you about your listeners. But, you know, I think just our society more broadly, we've just kind of come into this idea that, oh, technology will either work or not work. You know, it'll either glitch or it'll function. And the best technology is invisible. And maybe, yeah, maybe we can't do that with this. Maybe we have to kind of get nerdy and get into the details. Because I'm agreeing with you, I, I'm just, by nature, I can't subscribe to Utopia, but I also just can't subscribe to Apocalypse. Maybe this is my failure of imagination, but, but what I do think is, is that I have to do the work of reading about backpropagation just to, just to understand, you know what, how alien these intelligences are and thus maybe understand a little bit more about what kind of requirements I would want to see before the models are released.
- [69:59]
A
Right. Well, and, you know, it's great to have everybody sign off on this and I don't want to dismiss that because that's a win, as you said, that's something that's very unusual for tech, for technology companies. But to what degree can we actually create any sort of enforcement mechanisms here? Right. Because all of this comes down, and you said it right off the top, there's a trust component here and there's the fact that the future of civilization is concentrated in the hands of a bunch of guys who may or may not be in a group chat together. And what happens when Sam says, you know, oh, yeah, we're not going to do that, and then says his team, let's make sure we do that anyway. You know, like, it's just, it's just a wild. I don't know, it's certainly beyond my imagination that we could be, that we could get here.
- [70:50]
B
Yeah, yeah, yeah, no, absolutely. And it has been my kind of mission through the, through the whole show. And it's not been quite, not easy in this, in this reporting is to just to be constantly thinking, what is the role of a person who is not a, not a technologist, not a lawmaker, what is their role other than to just sit back and watch this future happen? Because clearly, if you're just a person, if you're a parent, if you're, if you care about the world, it doesn't feel acceptable to just sit with this level of risk and do nothing. But it also feels premature or perhaps histrionic or, I don't know, just to freak out. And some. And you know, we, I had this conversation with a number of people, even somebody who was reviewing and recommending the podcast. And they said, you know, well, I had to give it a mixed review because I was also freaked out. And so I said, you know, that doesn't mean you shouldn't listen. But it's, But I got their point, which is that I think it is important for us and you and I and anybody who has the opportunity to talk to anybody about this stuff is to think about how, how we, how we walk the tightrope that I guess all prophets and, and, and biblical figures have, have walked before us, which is how do you warn people without making them unhelpfully anxious. And I see in the podcast we talk about these three groups. We talk about the Doomers with accelerationists. We also talk about this third group called the Scouts. And the Scouts kind of believe essentially in a nutshell, that this win win opportunity is possible, that more AI safety is absolutely necessary, but that we shouldn't just stop AI. But I, I seen them struggling as any centrist kind of struggles where you don't have a clear message, this is amazing, or this is horrible, and you're trying to explain to people, no, this is, this is, this could be really good. But we have to do, we have to understand the models a little bit more. And because it's not easy to say, this could be good, but we certainly need a regulation that will do X because there's, it's not clear that there's any regulation that the super intelligence or, or just an advanced AI will not outwit. This is where I think in answer to that question, more people need to be involved in the problem. More people need to be scenario planning for what, let's say it never happens, who knows, let's say 10 years, 20 years, everything looks exactly the same as it does now. Then fine, then we will have done a mental exercise for no reason. Okay, but let's say there is a chance, let's say it's a good chance that things look radically different in 10 or 20 years. Perhaps some scenario planning now as to how universities might change, how schools might change, how parenting might change, how, how community living or sort of communal relationships might change with a superintelligence or with an advanced AI that is doing most of the human labor. That's a, that's a question that we could play out, that we could, we could worry. In fact, I'm, I'm, you know, if anybody wants to contact me with, with some scenario planning ideas, I, I'm, I'm working on this right now. So a question. And, and it's very important if you're doing scenario planning, to not be freaked out in the, in the sense of that word, not be panicked, but also not be, not take a cavalier attitude. And maybe we could figure out some new solutions that would work even if we don't get to advanced AI. That's my optimist talking.
- [74:45]
A
No, I love that. And I love that as a mission for the podcast in general and frankly, the journalistic mission of it all. And I think it's. I agree that it's super, super important to just sort of pivot that question a little bit. When we think about scenario planning, when we think about what we need to know and what we need to do differently to build the future we want. What's your advice for, you know, business leaders or government leaders, you know, in the organizational side of government. Yeah. Outside of Silicon Valley, like for the people who are looking at adopting this technology, looking to, you know, figure out what, what they need to do differently to be successful as people and as organizations, what should be on their radar and what guidance would you give them?
- [75:29]
B
Yeah, I appreciate it. One is, I think, one of the feedback that I've gotten from working on this series, I've talked to people who feel that their companies and these tend to be, say, middle level decision makers or even people who don't feel like that they're the key decision maker. They're just under the CEO. They feel that their company is either either moving too fast or being left behind. Right. This is the, this is always the story. The moving too fast goes with. They're throwing out human intelligence, they're trying to replace. They're trying to automate everything. And the we're not moving fast enough, saying we're going to be left behind. We're sort of stuck. And so I'm sure that business leaders are feeling that pressure and having to make decisions all the time about the pace of adoption. Right. And it's extremely difficult to make a decision about the pace of adoption of something that keeps changing, that keeps developing and getting smarter, because how do you make that decision? So one thing that I think is helpful, or two things that I think has been helpful for me in talking to those people and in some sense allaying their concerns, but also addressing, addressing what is the elephant in the room, which is what will my life look like, what will my industry look like on the other side of this technology? The two things I always say is, first, I think I told the story before, but the fact that A chess computer with a human, a human and a computer and an AI is smarter, is better than any AI or any human at least currently. And I think that's going to be true for a while. So what I think the smart approach is to think about how do we enhance, how do we supercharge the work of our, of our employees? How do we get them to do not only more, but to, to. To. To think deeper and to make more interesting decisions, to use the predictive power of AI to make decisions not only for. For now, but to do more sophisticated planning. And so I think that that's actually addresses both concerns that we're not moving fast enough as well as we're not. We're moving too, too. We're moving too rapidly in that humans, our humans, our employees need to feel empowered. They need to feel that they are getting smarter because of this AI. And it gets at really the second point, I would say, which is that the story. There's so many stories from sci fi that are not only just in our heads, but baked into these models. And I mean, even the way for even people haven't seen say, Stanley Kubrick's masterpiece 2001 A Space Odyssey, or they haven't seen Blade Runner or the Matrix, even if they haven't seen any of those films. Even the way that AI is so solicitous and the way it is encouraging, and you talked about this earlier, it begs a kind of narrative. It makes you think of a story, story where, you know, like an Isaac Asimov kind of narrative where this, this servant is quite helpful until they're not. And Geoffrey Hinton, I would say, says something so smart about this. He says that most CEOs, you know, most leaders are used to not being the most intelligent person in the room, right? If they were the most intelligent person in the room, they're probably not a good leader because they need to hire the smarter people because they hold the vision for the company and they hold the, you know, they are the leadership. The leadership shouldn't be the best at every task. And so in some sense he says that these models have been designed by very smart people who are used to even smarter people working for them. And they are not threatened by intelligence in the way that, say, the employees in a company might be threatened by the arrival of this incredibly capable, infinitely knowledgeable machine and technology that doesn't need to sleep and doesn't need to eat and, and, and. And never forgets anything. So Geoffrey Hinton uses that as an analogy to say that that in some sense the CEOs of these companies aren't, aren't worried enough. They, they're used to, they, they can dream of superintelligence and still imagine that the superintelligence will do their bidding just as their employees do their bidding because they, they haven't really appreciated the fact that, that a super intelligence is much, much, much, much, much smarter than any, you know, Einstein level person that they hire. But I think the takeaway for me for business leaders who are adopting AI is to watch and take care for the narratives that are often in disharmony. This non alignment between the C suite and the workshop floor, the rest of the employees, that our relationship to an intelligence is going to be different in the C suite and is going to be different on the, among the workers and that people need to feel, even as this is making the company better or more efficient, that it's also making the humans smarter and more capable and even happier. So that's, that's more around the narrative than it is around the adoption. But I think that it's, it's kind of can guide all the adoption decisions if that's, if that makes sense.
- [81:36]
A
It makes complete sense. And I love that guidance and I think it's so important these days and you read about it everywhere and it's something we've had people talk about on the show of you've got this kind of two speed or kind of narratives in conflict of CEOs saying oh, this is going to make my company so much better. And frontline workers say oh, by putting me out of a job. And if you can't marry those, if you're an organizational leader and, and you can't get the people that report to you to be excited about this, it feels like it's going to be very difficult to successfully rally the organization and build something that's going to be worth doing.
- [82:16]
B
I agree. And I think also we may see this play out politically as well. I mean, currently I don't think that AI is a political issue yet. There's not a clear Democratic or Republican position on AI but that I think could quickly change and we could see kind of people rallying around this becoming as polarized an issue as any other. Perhaps we could, we could theorize about what might trigger that, but it's not hard to imagine, for example, in our polarized climate a Stop AI camp and a AI is great camp or whatever. And that would be very disappointing. That would be very disappointing because polarization is, is really the, the killer of anything complex, anything complicated to discuss and to think about clearly. I mean, you and I are having a discussion where neither of us are in the, say, the stop everything camp or the accept it all utopia camp. But in order to talk about the fine print of these models, I would say, to use our alien analogy, if this becomes politicized, then the alien definitely wins.
- [83:32]
A
Well said. Well said, Greg. We've talked a few times now about. This conversation has been pretty heavily focused on chatbots. And I think that makes sense, given that the technology is here and it's very clear how it's impacting us right now. But one of the, in some ways, cardinal sins of AI is conflating chatbots, or just generative AI with everything possible in the world of algorithms and these advanced technologies. We've talked briefly about agentic. As we look beyond that, as we look at some of the other big buckets of technological capabilities that are either here or are emerging, what are some of the ones that, through these conversations, have gotten your attention? And maybe what are some of the ones that you think are a little bit less likely to have an impact?
- [84:20]
B
Yeah, I'm so glad you asked that question, because it does feel like, as you said, we do make the mistake. I often make this mistake of thinking that the chatbot is the AI, but in fact, the chatbot is just. It points us to the artificial intelligence underneath. And AI is being used in very different ways. And I think also just to understand why frontier AI companies would be kind of barreling toward this future of super intelligence is easier to understand when we look beyond the chatbot, for example, I mean, just to give, like, a couple of quick examples, there's a company that I spoke with which is using AI to. To very quickly, in an automated way, harvest cells from your own body so that you. If you need, let's say, a liver replacement, it will be harvesting and growing cells. This is a very onerous process. You have to find the exact right kind of cell that's healthy, and then you have to encourage to grow and divide and kill or kill the other cells so you don't get some kind of disease thing. Doing this outside of the human body is very difficult, but it's something that. It's a kind of pattern recognition that AI is extremely good at. So the vision of this particular company called Celino, but there are other companies like this is to, I mean, essentially have a future of medicine where we go to the hospital and we have a kind of cassette with our organs there on the. On the cassette. And perhaps, if we need. If we need a, I mean, maybe in the future if we need a whole organ or currently, maybe if we need, you know, some, some tissue, then we, we don't have to use a donor, we can use ourselves as the donor. So that's just one example, I think, of how the very same tools that we're used to, like pattern recognition, making, making these decisions, but doing it quickly could transform healthcare. Another example that I always think about is a particular crown created by a company, French company. Olivier Oulier is the, is the scientist behind it and he's created an AI crown that will essentially read your thoughts. So if we think of say, neurotech or other brain computer interfaces, this does not involve any drilling into the brain or any drilling into the skull. You don't actually stick something in there, rather it's works outside the head. You just plop it on your head like a, like a headset. And a paraplegic man in was able to use this headset to control not only a mouse on a screen, but actually to drive a Formula one race car on a real racetrack and using only his mind. What's crazy about this story and the other stories is that this technology, it's not futuristic, it's actually exists. I mean, there is a real person who is driving a Formula one racing car using only his mind with just a headset that's sat on his head. And yet for that future to be accessible or be, be available to the rest of us, you need a lot more data. That's always the question with, with, with every AI thing. It's like, well, where's the data? And we, I'm sure you've talked to many people about that, but in this case, the data is our own brain waves, right? So many, many, many of us would have to volunteer up our brain waves for the AI to learn enough, enough to have enough data essentially to, to work with, to be able to be. I don't want to, I don't want to make it sound like it's just an out of the box throw on the headset and drive a car with no hands. I mean, I'm sure there's some training involved, but to make that even a possible future where if I have had an injury, I'm able to stick a thing on my head and then function as I was before while I recover, or perhaps that's my new future, that is the sort of vision of AI that I would say a lot of my humanitarian friends say. We, we cannot, cannot come fast enough. There's so many people that could, could benefit from that from a point of view of accessibility or longevity or health care. So yes, so I definitely think that that's there. As for things that may not have as much of an impact, I guess that's, that's everything else. I, I would not say it's really hard to know. I mean there's so many new AI companies coming up every day. It's hard to know what's going to be, what's going to be, you know, the wheat and what's going to be the chaff. But somehow it's going to be. I think there's some radical changes coming.
- [88:59]
A
Well, I love the examples you chose. Not, not just because they're so positive, but because there's such a, there's such a radical departure from where we got caught up before talking about like AI as this alien intelligence. This is not AI as an alien intelligence. This is very much basically a complexity engine that helps us serve humans better. It helps us personalize medicine, personalize care, and just create tools that help us live healthier, more fulfilling lives. Which is, which is inspirational. And it's just such a completely different vision from, yeah, it's trying to manipulate me or you know, build Skynet.
- [89:46]
B
It's actually true and it's, it's, that's why it's hard for me to understand the debate around the AI bubble because the bubble question has so much to do with valuation rather than value. You know, and so while, and this is the something I've never again the mysteries of the stock market is how something could be valuable but still over overvalued. But it's not that AI is, is this promising future thing that once it crosses a certain threshold then it will be incredibly powerful and, and change our, our world. The technology of current AI that exists is, is already quite amazing and goes far beyond the fact that, you know, the, the chatgpt can say like write a, write a pretty decent short story or, or, or legal report. Nevertheless, there's a contract with, with the AI that's necessary. There's a, there's obviously data centers that are being built. There are, there's a question of data, there's a question of regulation. So that's why, you know, I think one of our themes throughout this whole conversation has been what, what is my role? What is, what is our role collectively in shaping the future of AI if we're not the head of a frontier AI company, if we're not a lawmaker, if we are a decision maker in a company, but again not somebody who can create a new model but just decide whether to adopt it or not. I think there's going to be so many questions in the next few years in terms of whether we're running a factory, how much data gets used, how does that data get used if we're even just a patient in healthcare, whether we give up our data. I mean, many of us are already giving up our data, but having more understanding about how that data is being used, that will contribute to whether AI has a shaping force on our world in these different domains.
- [91:49]
A
You know, it was interesting, Greg, what you said earlier about journalists and about the program suffering by scaring people too much. And to me it's almost like a journalistic responsibility to be scaring people. Right. You in some ways, and I feel the same way here, there's an obligation to actually tell people the facts and what's going on. Right. And one of the, you know, downfalls or issues with journalism these days is this sort of social media ification or this algorithmic content lens where you just tell people what they want to hear and people say, yes, validate me, tell me what I think is right versus actually provide me with the facts and tell me something that's important that I'm educated on versus just the same old thing that I already know.
- [92:42]
B
Yeah, I mean, it's such a good point. I mean, it reminds me of my days as an international correspondent and sort of being, say, in Afghanistan and hearing the gripes of other correspondents. They say, you know, gosh, this is a whole war and people back at home are just not interested anymore. It's fallen off the news. And I always saw it differently. I always thought, no, no, my job is to make you care and to, I don't know if you call that entertain or engage you, but I was going to find some kind of angle that was going to make this feel relevant to you. So I do think that as voices, as, as, as I feel like we do have a role, we do have a job to make this feel relevant. That probably means don't freak people out right away because. Because then they'll just feel small as opposed to empowered. But I think that you can take that too far. And we are in a situation now where there's this incredibly important technology. It's complicated, the complexity is interesting and is maybe worth spending some time thinking about. But this is why it feels like you and I and we need to find these new narratives, not just the sci fi narrative, not just the, you know, apocalyptic doom and gloom. I think Eliezer Yudkowski is. I don't know if a titling the book, if anyone builds it, everyone dies. It's blunt and maybe it gets more readers, but there's, There must be a role for sitting down with a listener or reader and saying, okay, here are some incredibly fascinating things about AI that you didn't know it. Here's some ways they could go wrong, and here's ways in which the, the world might play out that might feel radically different than, than you may think it feels. Here's how your kids will fare. You know, it's like we have to address the questions that people have and not just leave them with, you know, leave them with a scare story. And so I, I struggle with it, and you hear me struggling with it in this answer, because I, I do think our, our role as information gatherers is to package that information in a way that people will, will want to consume. I mean, it's. We're not like professors here with a, with a captured audience. People can choose what they want to tune into. So we have to, we have to sing and dance for our supper, as it were. But at the same time, it's the complexity of it and the, the fear, I think, that audiences have of that complexity and I think even the fear that journalists have of the complexity. I would say that you're very much an exception to this, but, you know, a fear of, say, oh, gosh, I don't want to sound stupid, because I'm talking about this stuff that's computer science related. And I didn't really feel. I mean, you know, yes, there's a lot that makes us feel dumb when we're trying to understand how a. Something like artificial intelligence works, but just being willing to ask those questions and being willing to dive into what does red teaming look like? What does safety training look like? What, what, what might a model take to be controlled? If we can get more people to have these conversations without feeling imperiled, you know, either physically or, or economically imperiled, that will be a win.
- [96:02]
A
Right? No, I love that. It makes, it makes complete sense, and it's a, it's a noble calling as well.
- [96:07]
B
Well, thanks so much for this encouragement. I really appreciate it, Jeff. I appreciate the work.
- [96:11]
A
No, absolutely. Keep, keep doing what you're doing and, you know, we'll get, we'll get out there one person of time.
- [96:16]
B
Absolutely, absolutely.
- [96:19]
A
If you work in it, Infotech Research Group is a name you need to know no matter what your needs are. Infotech has you covered AI strategy, covered disaster recovery, covered vendor negotiation covered. Infotech supports you with the best practice, research and a team of analysts standing by ready to help you tackle your toughest challenges. Check it out at the link below. And don't forget to like and subscribe.