A (2:47)

Yeah, it made me feel a little foolish because I, you know, like many of you, I have a group chat with a handful of friends, and there's a lot of AI in there, you know, of us turning each other into various things, various squids, creatures, you know, all sorts of stuff. And I did, after finishing the piece, feel kind of like I was, you know, just like kind of playing in a sandbox with, you know, a shovel.

C (3:09)

And bucket There's a lot of AI everywhere these days. Yeah, of course. I mean, but that's. I mean, it's funny that you talk about playing. I mean, so much of what we've seen is just a lot of fun. Like the sort of the gimmicky stuff we've seen, which, I mean, maybe we'll get into this, but, you know, the vision that we're sold of Utopia and solving the world's problems, and what are we getting? We're getting sort of, you know, cute little studio Ghibli generators and, you know, erotic chatbots.

D (3:33)

Yeah.

A (3:33)

It's like the new wave of. Remember when the Snapchat filters came out? And at least for people in my, you know, in my age group as elder. Elder. Aging millennials, you know, we thought the Snapchat filters were so fun and like, wow, it can put a face right on top of yours and, like, it mimics your expressions. And, oh, look, now it's on grandma, and grandma's erect. You know, I remember that we were looking at that in the same. With the same whimsy. I feel like that is accompanying these, these little AI apps nowadays.

C (4:04)

Yeah, yeah. And now we have Sam Altman barbecuing Pikachu. Yeah.

B (4:10)

I want to get into, like, the, like, really interesting sort of, like, conversations you've had with researchers for this. But before we do, like, could you help me understand, like, what broadly you think is the difference between, like, the kind of, like, AI tools that consumers might be familiar with might do, like, you know, research or Ghibli or that kind of stuff, and this hypothetical AGI?

C (4:30)

Sure. So look, I'll do my best here because, I mean, there are a lot of good faith people who genuinely think they're sort of. They're building this technology. And I think the difference between what they're aiming towards and what we have today, I mean, the clues in the word. Right. So it's the generality. So even the best sort of tools we have today are really, really good at one thing. They're really good at generating images or generating video. Chatbots are sort of getting towards being more general. And I think that's the sort of. The excitement about AGI has ramped up a lot in the last few years. You can talk to them and they can talk back at you about anything, but it's not hard to push a chatbot and break it, make it say something really dumb. I mean, I don't think anybody would seriously trust them to do something really serious. You wouldn't trust it with Your health or your money. But what we're aiming for is an AI that you would, that you really could just ask it to do anything. You would sort of ask a reasonably capable person of doing. Do your taxes, help run your family, logistics, run a business. And these are real examples a lot of people in the field. Imagine building an AI that can go out and earn your company billions of dollars. So it's the idea of an AI that can basically do what a smart person can across the board, not just in these niches.

A (5:56)

And I see these advertisements all the time, along with, I'm in, I guess the ex. Right age group, where, you know, I've joked about on the show before, it's like they're sending me the balding medications, the shoes that make you look taller. And they're. And the other suite of ads that I get are from these kind of like Rise and Grind bros or, or bras. You know, it seems to be men and women are very interested in pushing this and they basically are like, are you over 35 and not using AI to optimize your life? Sign up for this course and we'll take you through these 30 different AIs to help you, like, become whatever it is, you know, and it' usually has to do with, you know, making your business successful, giving you the body that you want. You know, all these things that we see online that we, that we really crave. And this seems like a new grift that it's like, hey, if you're not using all of these, the suite of tools, you're left behind, you know, join me, join my seminar.

C (6:51)

Yeah, no, no, I'm nodding along to that. Yeah. The grift side of this is enormous. You know, rewind a few years and, you know, these same people were shilling for NFTs or whatever.

A (7:02)

Right, right.

C (7:03)

You know, the industry pivot. Yeah, I speak to a lot of people, I speak to a lot of founders of startups. And it's the same company that was doing crypto stuff a few years ago, but now, oh, everything's swung and everything is now chasing AI.

B (7:15)

Yeah. I read an article in Science that collected statements from tech leaders related to AGI, and I want to read some of them here because I think they're interesting. So OpenAI states that its mission is to ensure that artificial general intelligence benefits all of humanity. Google DeepMind's company vision statement notes that artificial general intelligence has potential to drive one of the greatest transformations in history. AGI is mentioned prominently in the UK government's national AI Strategy. See, the US Department of Commerce's National Artificial Intelligence Advisory Committee's charter says that it should advise the President on progress towards artificial general intelligence. Microsoft researchers claim that evidence of sparks of AGI were present in GPT4. And Sam Altman, CEO of OpenAI, called GPT5 a significant step along the path to AGI. So it's like this is, this is quite a collection, like. Yeah, yeah, it's like major world governments, like, you know, tech CEOs of like multi billion dollar companies and you know, researchers, they're all working on the presumption that this AGI thing is real and is definitely 100% coming. I mean, why should we doubt all these people, you know?

C (8:31)

Yeah, why should we doubt these people? I mean, yeah, maybe, maybe we're wrong. Maybe we should pack up and go home. That is exactly what has been bugging me for all these years, covering this industry and what sent me down the rabbit hole. Because it never used to be like that. Ten years ago, the idea of AGI, the idea that you could make an AI that really could do everything that a human could, was ridiculous. And even when OpenAI was founded just less than a decade, the sort of the swagger and the ambition of this new company, that was its mission statement from the start, was to build AGI. It really made them stood out because no one else was actually saying that seriously. And there's no accident to that.

A (9:12)

Right.

C (9:12)

This is a new company that was coming out and we're going to take a big swing at this in this concept. But yeah, over the years, I think a couple of things at least have happened. One, if one company is saying they're going to make AGI, then you've got to say you're making it too. Otherwise, what's the point of view? Just became the thing over the horizon that the best AI companies in the world were chasing.

A (9:37)

And this was just a name that one guy made up that they were like, he was like, oh, you should call it AGI. It doesn't really come from anything other than that guy who worked beneath. I can't remember his name, but you talk about him in the piece.

C (9:49)

Yeah. So I think you're talking about Shane Legg.

A (9:52)

Yes, Shane Legg.

C (9:53)

I mean the origin of the term is kind of fun. I mean, there's this guy, Ben Goertzel, who's a really lovely, sweet guy, but he will present himself as being sort of on the edge of things. He's just what he's drawn to. He's drawn to these fringe ideas he's been in the AI field for ages. So back in the mid 2000s, he was sort of influential figure in this fringe community that was interested in making an AI that could do these sort of human like things. And it's important to say that even though this modern concept of AGI is maybe at most 20 years old, the ideas that that is built on go way back, back to the 1950s when people first started talking about art intelligence. Those early pioneers wanted to build a machine that could do the things that people could. So those ideas have been bubbling around for a while. But it was only the mid 2000s with this guy Ben Goertzel, who wanted to put a name to the stuff that he and his colleagues in his fringe community were working on. And he turned to a former colleague of his called Shane Legg, who put forward this term AGI, let's call it artificial and general intelligence. It's like AI, but it's broader, it's.

A (11:03)

Bigger, it's more general, it's like my generalized ang anxiety. Broader, bigger, not specific, 100%.

C (11:09)

And the amazing thing is like Shane Legg went on to Co found DeepMind, now Google DeepMind, one of the biggest AI companies in the world. So Shane Leg took this term AGI and all the concepts behind it into DeepMind. It's probably important to, I mean some of your listeners may know this, but whether or not they do, just as a sort of a point of fact footnote, after these guys, Ben Goertzel and Shane Legg came up with the term AGI to sort of name this ambitious set of ideas of emerged that there was another figure who had used the term AGI in a book back in the 90s. And so this guy's often, you know, he's sort of given credit for first coming up with the term, but it died and disappeared. And it wasn't until the mid 2000s that AGI as a label for all of this sort of really took off.

B (11:55)

We were talking a little bit before the show and before we were recording, and I mentioned that was like one of the things I really liked about your piece. I'm really interested in conspiracy theory. Especially when we're talking about conspiracy theories. It kind of like less conventional sense when it's sort of being promoted by people who are otherwise very respected and credentialed. And I also like contrarian takes. And this is this one definitely has both those elements.

A (12:15)

You have this great quote in the piece that I screenshotted to read because I thought it was so just like tight and Easy to understand. You write, every age has its believers. People with an unshakable faith that something huge is about to happen, a before and an after that they are privileged, parentheses or doomed to live through. And I think this captures the conspiracy mind so well because there are two sides of it, you know, especially as we've seen over the, you know, five or so years, is that there are people who believe in conspiracies from both angles, Right? That it's either this amazing, it's gonna usher in this amazing golden age of prosperity and wealth, or it's going to be an apocalypse. Both are a conspiracy, but it's just kind of like, pick your pill, pick your flavor. And I thought you presented that in a really, like, easy to understand way in the piece.

C (13:03)

Yeah, no. Yes, thank you. But, yeah, you're totally right. This, I mean, I should say, like, the idea of even treating AGI as a conspiracy theory at first. I was only half serious when I first started thinking about it, right. Because obviously AGI isn't a conspiracy. Like you were saying earlier, Travis, this is what the biggest, richest companies in the world tell us, sincerely, they're going to build. But when you start to look at it, things like this just pop out, these parallels. Like what we're being told is that this is a sort of a savior, like technology that's going to get rid of all the world's ills, it's going to make us more prosperous, it's going to cure disease, it's going to help us solve climate change. Or maybe not, right? Because that's where it flips on a dime, right? Because if you have a technology that really could be that powerful, then, of course, are us feeble humans going to be able to control it? And if we can't, then that's the end of us. It's all part of the same belief system, the flip between boom and doom.

A (14:02)

And it's been presented to us like that in popular culture for decades and decades. I think to Hal from 2001, right, this malevolent artificial intelligence, or even better, Skynet from the Terminator, you know, this evil artificial intelligence. These ideas are out there. And what's far less common, actually, if I'm just kind of digging into my video library, in the back of my head is AIs that are benevolent, that actually are going to help, that are going. I mean, the only one I can really think of off the top of my head, and I think this is a bad comp. And you guys might not even know what I'm talking about, but the old 80s Disney film Flight of the Navigator where you have an artificial intelligence who is running the ship, who ends up being a good character and getting David back home. So like. But that's a very, you know, I can't think of too many examples where the AI in the movies or the book is something that is good and something that is going to actually bring about this thing that all of these Silicon Valley guys are saying that it's going to this new age of, you know, this golden age.

C (15:03)

Yeah, I hadn't thought that through, but yeah, as you're speaking, I mean, some positive examples, if I had to think of some. You've got Wally. Right.

A (15:12)

And Right. Okay, yeah, Wally's good. Yeah.

C (15:14)

Johnny five, I mean maybe that's dating.

A (15:16)

Yes, Johnny five is good.

C (15:18)

But these guys are what they're like sort of, they're played for comic relief. Right, Right.

A (15:23)

And they're weapons. Right. Johnny 5 was a weapon that went rogue. You know, it was the opposite. It was built to be a weapon, but actually it became this kind of goofy guy.

C (15:32)

Yeah, I think. Yes. It doesn't make good drama, does it? Like the idea of a genuine, beneficial, beneficent, all powerful AI that just basically solves all our problems is. That's pretty boring.

A (15:43)

It's boring. And does it get people to spend. I don't want to go and watch a society get it better than I've got it here. I want to go to the movies and see somebody who's got it worse than I do so I can leave the movies feeling good.

C (15:55)

Yeah, I think, I mean obviously we've seen since ChatGPT came out like three years ago now and sort of the world woke up to what present day AI could do. We've seen that sort of that wave of doomerism really take off. I think that just like really that grabbed the imagination because it's exciting. It's exciting to be scared, right?

A (16:16)

Yeah. And it's exciting at least, you know, personally, it's exciting to think about living in an apocalypse where all of a sudden your credit cards don't matter anymore, your job doesn't matter anymore. All the video games that you've played. It turned out to be training. I think a lot of people, and I hear my friends, you know, are being like, oh, preparing for the end of the world. Oh yeah, you know, it's my apocalypse mobile or whatever. Myself included, like Travis included. We both have like kind of like off road vehicles that he prob. Probably needs, but I don't. And it's like that's fun to think about. It's fun to think about this system collapsing because all of our current problems are now solved. Which oddly enough is what they're saying the AI is going to do.

B (16:55)

Yeah, you know, it's pretty normal for like, I guess like tech companies to talk about like their product in very grand world changing terms. Like Facebook talks about community and connectivity and like they're bringing the world together. And these were like, I think overblown promises. But like, you know, it's the general rhetoric of like, you know, startups generally. It's like in order to convince investors to like, you know, put up their hard earned money and convince people that you're a worthy bet, you have to make these big promises of like why, why you're so significant and stuff. But it feels like the AI is like these companies, they are taking it to a whole new level. Like they're making like extraordinary promises. They almost like they're saying this is going to be the last invention. You know, this is something, you know, more profound than the printing press in terms of how it's going to change. Humanity is just, it is, is worrying how like just incredibly overblo rhetoric is on the potential impact of what they're.

C (17:49)

Building and the stock market's buying it. Right.

A (17:51)

I mean big time.

C (17:51)

What we're seeing now is utterly unprecedented. Like the money flowing into these companies, the valuations we're seeing, and it's nearly all riding on this promise that is really quite vague and hand wavy. There's something that stuck in my head like when I don't know if you guys watched it, but when Sam Altman came out and they were announcing that deal with Nvidia for just nuclear power stations worth of computer chips and energy and Altman said something like, now we don't have to choose between curing disease and giving everyone free education. We can do it all. And this is what they're sort of telling us, that one this near future all powerful AI can do those things, just educate the world for free and cure disease. But if we try and stop him, then we would have to choose, it's on us. If we sort of Somehow don't support OpenAI in this mission that oh, we tried to do both but we couldn't, we could only cure disease. We can't help you kids. There's something like really turns me off about this. Really. I was gonna say subtle rhetoric. It's not subtle at all. But the way we're sold this technology is laughable and yet, as I said, the stock market is buying it.

A (19:02)

And what's so funny and interesting to me as I'm thinking about, as you're talking about this is like they're saying, hey, these Nvidia chips, they're gonna, yeah, do nuclear power plants are gonna do all this. But all I'm actually seeing in my own life is that they can give you a couple ext on your computer games because, like, you have a kind of a, you know, shitty processor or whatever, and like, the AI is basically adding in, you know, where your game would be choppy otherwise, like, that's what I'm seeing on my screen. They're like, yes, this is going to solve world hunger and nuclear power plants. They'll be able to talk to one another, everything. But all I'm really seeing is like, I'm getting 10 to 15 extra frames per second in like, Battlefield or something. Like that's what it feels like the application is. And there's nothing wrong with that.

C (19:49)

But yeah, I was gonna say, what's wrong with that? I mean, I often think. I mean, don't get me wrong, like, this is an amazing subject to cover because it is so wild and like, the stuff that's going on and people are saying is really off the charts. But a lot of me think it's like, what's wrong with just, like, having better frame rate? Let's all sort of come back down to earth and sort of, you know, treat this technology as if it were a normal technology that just made lots of little things a little bit better?

A (20:16)

Yeah, that's a really, really good point. Why does it have. Have to be the thing that saves the world? And I think that gets into. And I know Travis has got some questions about this, about the people who are pushing these kinds of narratives, but. Yeah, why isn't it just enough? Why can't the stock market rally on the fact that, like, somebody with, like, a slightly older GPU can play the latest games at higher frame rates? That's gonna sell cards. I mean, I got the 4000 series card so that I could use the DLSS technology. That. That's a really good question. I think speaks to an illness amongst the wealthy in Silicon Valley. And all of us maybe, is that it's not good enough to solve small problems. We need to convince people to invest in something that's much, much bigger. Like you said, a way better movie. Something that's not just kind of boring and utilitarian, I guess.

E (21:12)

Right.

C (21:12)

And we want to believe, like Fox Mulder says.

B (21:16)

Yeah, you know, another way that sort of like that AI companies sort of talk about the product in a way that's different from I guess previous sort of like Silicon Valley giants is that they keep talking about AGI as this goal, this endpoint. There's something that we are working towards is going to happen. We don't know when. And this is different than like, I guess like to return to Facebook. They talk about generality. So it's like we're going to make the world more connected and more, more we're going to build communities. These are sort of like vague goals I suppose that like they aren't talking about we're building ultimate community one and they'll, it'll, it'll change the world then. I mean, what consequences that is that they keep pushing back when the CGI is going to happen. And you discuss how this is sort of like this, this mirrors conspiracy theory talk in your piece. And like the big prophecy, the big event is always happening soon, always in the perpetual near future. I mean we see this a lot in QAnon where like the big storm of like arrests is always going to happen very the first predicted 2017 that didn't happen. So they pushed it back and back and back. They always have an excuse about why isn't going to happen. I mean, but this is a little strange behavior coming from these like, well, credentialed AI researchers and these big money tech firms. I mean, how does this talk kind of manifest this belief that this AGI thing is we're just on the cusp, we're getting a little closer, it's going to happen in a year now. I mean, how does this manifest in the space?

A (22:34)

If I can just interject really quick and you say it perfectly in this passage from the piece that I screenshot, it'll be a perfect way to tee this up. You're right. You have to admit it all sounds a bit tinfoil hat. If you're building a conspiracy theory, you need a few things in the mix. A scheme that's flexible enough to sustain belief even when things don't work out as planned. The promise of a better future that can be realized only if believers uncover hidden truths and a hope for salvation from the horrors of this world.

C (23:04)

Right? Yeah. I mean there's a lot there that I would guess that the people building this technology on the inside, maybe you think about it one way because these are the scientists and the engineers that, you know, know exactly what they're building and yet, you know, they will still believe that, you know, given this thing in front of me, we're two years away, three years away, 10 years away, whatever from, from building AGI. You probably have to separate them from, you know, just the rest of us who are just, you know, following along sort of the, the AGI stands that. That want to believe. So I have sympathy for people who are just following along and are told these amazing things are going to happen. You know, why wouldn't you sort of get excited by that and sort of. And not think about it too critically? And then when it doesn't happen, you think, oh, well, maybe next time I'm going to sort of keep my faith. But the people actually building it. Yeah, I scratched my head. I mean, I talk to these guys a lot and there's a very large spectrum of opinion. We're talking about the AGI believers here. But I don't want to give the impression that that's sort of the majority of the field. I mean, there are people who really, really push back against this sort of this overhyped talk, but the people building this technology who genuinely believe AGI is coming. Like you ment in the intro, there was a Microsoft paper a little while back called Sparks of AGI where they played around, the scientists played around with an earlier version of GPT4 and were just blown away by what it could do and really, I think, just got overexcited and wrote this academic paper and put it out there saying that what they'd seen within this model was Sparks of AGI. And I think what was going on there is that even the insiders, the scientists and engineers building this technology were not prepared for it getting as good as it did. So we're laughing about the whole notion of AGI, but just take a pause and think ChatGPT and all the models that have come since are incredible. This is stuff that people didn't think we'd see so quickly five years ago. And that's true even for the people building it. And I think they were just blown away by how good this tech had got and so thought, wow, if it's got this good, this fast, then then just sort of project that on a few years and we are going to have this awesome human, like, intelligence. But the other crucial thing that I think has happened in the last few years is that the AI we now have, we interact with by talking to it, by typing to it in natural language. And I think even if you try really hard not to, it's difficult to not get that sort of hair on the back of your neck feeling that you're talking to something, right? That I just think we're so hardwired to see some kind of intention, some kind of intelligence behind the language that has been spat back at us, that even if we know better, that we just feel there's something more there than there actually is. And I think that plays into all this a lot, that we give these systems the benefit of the doubt that they're maybe smarter than they are. Related to that. There's a massive problem with how we evaluate these models. It's now a bit of a joke. A new model comes out and you, you know, there's a leaderboard of, you know, my model can do this better than your model. And it's. It's sort of. It's almost like a new release of an iPhone every few months where, you know, this iPhone has slightly got a slightly better camera, it's got a shinier case and stuff. So what all these evaluations do is sort of, you know, they make the model do a bunch of tests. You know, maybe it's like, how good is it at generating code? How good is it, Is it answering sort of math problems? And they're trained to do those things. So when they do very well at them, you think, oh, my model is broadly intelligent because it can solve a math exam. But again, I think that's confusing the models for you're treating them as if they were people. If I sat a math exam and I did really well in it, then you'd probably think that, oh, he's a smart guy. It's a proxy for my broader intelligence. But with these models, if it passes that particular math test, all it tells you is that it's passed that particular math test. You shouldn't then project more onto it. So I think there's a real mess with how we evaluate these models, how we think about them. And all of that allows this AGI myth to sort of take hold and be more persuasive than it ought to be.

B (27:14)

I want to mention. Yeah, recently I read that there was a team led by University of Oxford that carried out a Systemic review of 445 benchmarks for LLMs across major machine learning conferences. And they assessed how well the benchmarks adhere to the concept of, I guess, construct validity as whether a benchmark truly measures the absence abstract phenomenon like reasoning, robustness and safety. It purports the measure. And they found that basically. So about only about 16% of the benchmarks reported uncertainty estimates or statistical tests. I guess the point review is that like the majority of like the benchmarks that people are using to evaluate the, I guess real abilities of these things aren't really, they're measuring proxies that don't actually evaluate the core thing that they're trying to measure. So even when we talk about how, how impressive and powerful these things are, we still don't even have something really concrete that we can use to evaluate how much these things are improving or how much they are, I guess, you know, to say being taught to the test, you know, being able to pass the test without having a really kind of like more impressive sort of like abstract reasoning ability.

C (28:22)

Yeah, yeah, that's it. That's it exactly. And because we don't have, I say we, you know, the sort of, the industry, the, the academic field does not have a good grip on exactly what today's AI can do, it leaves the floor wide open for how good they're therefore going to get.

A (28:40)

Yeah. And it's a way better story that you invent this thing and it goes, and it goes out of control and it's up to you to figure out the way to safeguard society from it or use it for good. It's not nearly as interesting a story for a rich guy anyways who sold a couple companies and doesn't have to worry about money. Instead of being like, I invented this thing and get better frame rates and you know, it's integrated into all these other apps and it makes it easier to edit and audio stuff. Yeah. Clean up your. Yeah, you want to make little videos. It's fun. You know, you can kind of make a little video. You know, you want to make your own Tom and Jerry cartoon, but put your friends on Faces on the Animals. Like you could do that. And it's fun. It's like, it's boring. I feel like for a guy like Sam Altman or any of these guys.

B (29:26)

Well, yeah, I mean, yeah, if you talk about more modest terms, you can't build a trillion dollar company.

A (29:31)

Right? Exactly. But why. But this goes great. Back to Will's point is like, why not? Why can't you build a trillion dollar company on the fact that like, hey, you got a shitty processor. Well, guess what? Our new tech is going to get you 20 more frames or you know what, you're editing something. Well, our new tech is going to be able to pull all the transcripts for you already so you can put it on top. Like. Yeah. Why isn't the little convenience stuff enough? I think that speaks to something bigger about our society and about how it's more fun for Us to think, even if it's a doomer scenari, they're like, oh man, like, here I am like in the early days of Skynet, like, am I going to fight for Kyle Reese or am I going to like become like a capo for the Terminators?

C (30:12)

You know, you're not going to go down to history as somebody who changed the world if you make a better frame rate, no matter how many people might want that.

B (30:19)

Yeah, the really interesting thing is like just the massive stakes, like, of the way they talk about it, like this is, this is the most consequential kind of like, you know, invention of our time is going to change the world and sort of unpredictable ways. And this is, this is why they have like a range of predictions about the ultimate consequences that range from like total tech utopia or, or absolute destruction of humanity. And like you mentioned, like, even the concept of AGI isn't, isn't universally held. There's increasing a lot of pushback on the belief that this is a sensible goal for companies. But there are some big names who talk about it and talk about in these big existential terms. For example, there's the British Canadian computer scientist Geoffrey Hinton, called the godfather of AI as credentialed and as decorated as, as anyone in the field is. And he predicts that the coming super intelligence, which he believes is a certainty, will replace humans.

A (31:16)

Fantastic. Yeah, this is the good, the good stuff.

D (31:19)

Nearly all the leading researchers believe that we will get super intelligence. We will make things smarter than ourselves. There's a very good chance it'll happen in the next 20 years, maybe 50%. It's coming quicker than I thought. After a while, the super intelligence is just get fed up with the fact that we're so incompetent and just replace us. They may keep us around for a bit and they'll certainly keep us around to keep the power stations running for a while. They would take over. They would run things. I've talked to Elon Musk about this. He thinks they'll keep us around as pets because the world will be more interesting with people in.

C (31:48)

I mean, that's, that's the plot of the Matrix, right?

B (31:50)

Yes.

A (31:51)

Yes.

B (31:51)

Yeah, that's. Yes, that's like. Yeah. So we create super intelligence. Super intelligence surpasses humans and then the superintelligence enslaves humans. Essentially.

A (31:59)

That's.

B (31:59)

Yes, that's the Matrix.

C (32:00)

I interviewed Geoffrey Hinton a couple of years ago. So he was retiring from Google and he wanted to make an announcement as he retired, like you know, he's an honorable guy. He didn't want to shittle over Google while he was still an employee. But the week he stepped down, you know, he went public with these, these fears. And he told me at the time that he was just going to spend a couple of weeks, you know, putting it out there. And it kind of amuses me that he's basically been on a non stop press junket ever since. I think he absolutely, absolutely loves the new role he has as this doomsayer of the industry. I think it's fascinating that he has come out and had a sort of a career in his sunset years as this guy going out and doing all this fear mongering. He told me that he was essentially surprised at how good AI had got in such a short time. And I think this goes back to what we were saying about there's something weird going on with language models that when you talk to this stuff it gives people, I think even like Geoffrey Hinton, who knows how the tech works inside out, the sense that there's, there's more there than there is. But also it's the conclusion, you know, the sort of the logic to his argument that, you know, even, even if you accept like he does, that this technology is going to become far smarter than, than we are for him, that automatically means that it's going to turn on us, that it's going to, you know, keep us, keep us as pets. But says who? This is, this, this is just, this is just the stuff of science fiction.

B (33:27)

Yeah, you know, it sounds like, yeah, they're like assigning, I don't know, like more kind of like, I don't know, feelings of affection or feelings of hostility, which are like something beyond mere cognition to these AI systems. And there was a paper that I read while researching, it was, talk about, I think it was called like why AI is harder than we thought or something to that effect. And it talks about. So one of the fallacies that I think causes people to overestimate how easy AI is going to be is the fact that I guess like intelligence or knowing in humans is embodied. It's like something part of our more complex nervous system rather than mere sort of like disembodied cognition and thinking and sort of like a brain in a box kind of thing. But it's very strange that people like Hinton are sort of like assigning embodied kind of like knowing and feeling to these AI systems like already, which I don't think people haven't really developed a path to that as far as I know, at least.

C (34:20)

No, no. And I mean, even people are split on that. I mean, we've said already, already that there's no firm definition of what AGI is. But you know, even, even among people that are convinced it's coming, some people think it is just going to be like a brain in a vat and, well, brain in a laptop. Other people think that you're going to have to have like you say, like a body and a robot because intelligence doesn't exist without the sort of the interaction with the world.

A (34:44)

Part of me wonders if like us humans are just very impressed by video still, because it seems like they, this belief that AI is on the kind of course that its creators believe it's headed towards, whether that's a utopia or a dystopia. It's one thing that we've all seen, right, is like the Will Smith spaghetti video from the early days. And then you see what it looks like now generating video of human beings. And I think that just the fact that the AI has gotten better at generating video, it's like any conspiracy theory where you take, take one small piece that's real and true and impressive and then you use that to say, well, if this is possible, then this is possible, and if that's possible, then this is possible. And then you get to your super grand conspiracy theory. But to me, so much of this, because if you go to the LLMs, even on the latest ChatGPT, on the off chance that I'll try to use it for research, I end up doing more work because the ChatGPT will spit out something. And I'm like, that doesn't sound real. That sounds like it made up like a Reddit community that doesn't exist. I have to, now I have to go and like fact check the AI and now I'm doing even more research and it's like, that still isn't great. But what is great is the video generation. And part of me wonders if, like, we're all just so, like screen bound that, like, that's enough for us, that we're like, oh, wow, well, look at Will Smith five years ago and look at him eating spaghetti today. Like this thing's going to take over the world. World.

C (36:17)

Yeah. I mean, video has, video has got a lot better really quickly.

A (36:21)

A lot.

C (36:21)

Yeah. And it's, yeah, it's amazing. Most purposes it's, you know, it's near perfect. I know video, Video as a medium is just, it's just extremely popular. We're so familiar with it, so conversant. With it. So seeing a machine sort of turn our thoughts into a near perfect video is truly awesome. And like, you know, the technology is, is truly awesome. But it's a video generator. It's not going to sell. Save the world.

A (36:46)

Yeah, exactly. It's gonna destroy the visual effects industry most likely as studios just are greedy and they wanna spend less. I don't think that AI actors will make as big a splash as they're saying it is until we have a generation of kids who grows up with only AI actors so that they don't really have anything to compare it to and they're perfectly happy to watch those characters. I think that could be potentially the future. But most immediately is, I think, at least from my peers in entertainment, is that this is going to decimate the visual art industry.

B (37:21)

It's funny. Yeah, like they can't even like sell it in terms of like saying, well this is a new revolution in entertainment technology. The same way that like I guess like sound was for film.

A (37:31)

Right, right, right.

B (37:32)

It's a new leap in entertainment is going to change how it's produced and it's going to change what consumers expect from their entertainment. But like that's not, that's, that's huge. That's like a multibillion dollar industry. You're dis. That's not enough. They need this spiritual element, this sort of like this history defining element in order to sell their product.

A (37:51)

Like they could go to the effects houses and say we have this new tool that's going to make it so much easier for your artists to create even more what they want on a smaller scale budget. Like it could have been a multi billion dollar industry as something as a tool for, for artists to use as opposed to this thing that's going to inevitably replace them. Because people are, you know, I, I, me personally, I'm just that cynical and I think that the studio systems are that greedy. But like even then, why wasn't that enough? And I think it speaks to this culture of like Silicon Valley. They want to be gods. Because if you create the super and you, you talk about this in the, in, in your piece, Will, is that like if you can be the guy who created the super intelligence, if you can create Hal, right Then like you're a God in a weird way, you know, in, in a way. And I think that's nothing less is good enough for these guys who have achieved every kind of, kind of material. I've said this before, I'm on this kick that I think these guys have conquered what they believe is the material world and they want to conquer the spiritual world through technology.

C (38:54)

Yeah, I like that. There's something weird going on as well. I mentioned this in the piece as well. There's a lot of parallels with sort of New Age thinking. It sort of peaked around the 80s and 90s, that if we could only sort of access our inner powers, you know, humanity can transcend itself and, you know, we're all sort of float up into the sky with great smiles on our faces. There's some aspects in that too, in the sort of the stories that we're told about, you know, what AGI is going to do to us. But the kind of sad thing, in a way is that it's no longer, you know, humans that are going to, you know, save themselves. We have to look to a machine to do it for us. And I don't know, I feel like there's something. There's a lot to unblock pack there.

A (39:38)

Yeah, very cynical. It's a very cynical outlook. And when you have these kind of like massive, you know, egos at the top of this, this sort of ladder, you gotta, you gotta wonder, like, what happens when it doesn't pan out? I mean, I think then you're, you know, I mean, whatever, they'll find a way to grift. You know, all of these bubbles burst at some point and Yeah, I don't know. Yeah, we'll. We'll be catapulted into the next thing as, you know, you talk about. It's like at first it was the computer, then it was the Internet, now it's AI. What's the next, you know, what's the next big dot com bubble going to be?

C (40:12)

Yeah, I think we're primed to, you know, when we're told, you know, well, the next big thing is going to be AGI, we just sort of, you know, nod along and say, okay, cool, when's it coming?

A (40:20)

Yeah, I get roped into this every year with the video games. It's like ray tracing now. It's like Nvidia Remix. They're like, oh, you can play GTA 4 but with like real sun pathed God Ray, I'm totally messing that up. And you're Computer Science Guy. I'm embarrassing myself. But that's like, you know, that's how I experience it as a consumer is. It's just. It's always like, hey, there's like one piece, there's a new piece. But then you're like, okay, sweet, like, I'm going to enable ray tracing and then Your frames go all the way down to zero. But wait, there's an AI that can bring your frame rates back and you're just, you're held hostage by these technologies. Technologies when all you want to do is like, you know, you know, get spawn camped for 45 minutes.

B (41:02)

I was thinking about the broad economic implications of like, like what you're saying and how you're describing, you know, the AGI kind of like talk that's coming out of these industries. And like, it's like you're not alone. Like, Margaret Mitchell, who is a pioneer of AI ethics. She works, she's a scientist and researcher with the AI platform hugging face has ascribed artificial general intelligence as just vibes and snake oil, which is, you know, I think sounds right to me, but I think it's concern in light of the amount of money that's being poured into the AI space. I looked up as like, according to the State Science and Technology Institute, 40% of venture capital dollars for deals under $100 million are going to AI startups. I mean, and presumably a lot of that money is going on the bet or the promise that these AGI, like dreams could be made real. That just speaks to, I mean, to get into the bubble talk that's just very, very concerning the idea that like all of our hopes and dreams, all the capital is flowing directly into the hope that this thing is going to be real when, you know, some very astute people are calling it just snake oil.

C (42:11)

Yeah, Meg Mitchell is, is great. I'm very much with her on the vibes in snake oil. But I mean, again, we're back to the cynicism. That's all you need these days, right? That's, that's all you need in politics. That's all you need to sell your company.

B (42:24)

Yeah, maybe there's, there's less of a, of an indictment than she intended it to be. If it's like, well, if it's vibes, if it's good vibes, if people are feeling the vibes, then, you know, you could pour billions and billions and billions of dollars to it and like, and see a return. You know, it may be, I think.

A (42:38)

This is just part of something that human beings do. Maybe that's why we're so intent on the robots saving us or destroying us, is because, like, you go back to like, you know, Aesop's fables. You got Jack and the Beanstalk, right? You know, this guy, he gets, he gets fucking housed on the road. He's going to sell the family's cow or whatever. I can't remember what he's going to do, but he's got to save the. The family. He's got the cow. And this, like, guy appears on the road, and he's basically like, hey, man. For, like, four magic beans, like, I'll take your cow, and who knows what the beans will do? Plant them. You'll see. And he gets home, right? And he gets home and his mom is like, what the fuck, dude?

C (43:14)

You can.

A (43:15)

You know, that's real life. The real life is you get home and the beans aren't magic, right? And your wife's mad at you, your mom's mad at you, your partner's mad at you because you fucked up and you believed something that was ridiculous. You believed in a conspiracy theory. But the story is that he plants the beans in the ground and it grows. And I think that is the American dream, is that you can plant some. Some regular, ordinary beans in the ground, and it can take you up to the. To the giant. You know, to the giant's castle. And I don't see anything different between what these. What these guys are doing. And it's just like, they're telling you what this beanstalk is. You know, you're planting the seeds and then they're like, well, the beanstalk is the video getting better. The beanstalk is. Your frame rate's getting better. The beanstalk is. It's going to manage your finances. I just feel like we're doing.

B (44:00)

We.

A (44:00)

We're. We're stuck doing this, especially in America.

C (44:03)

Well, I mean, what I think is going on, I mean, Jake, you clearly, you clearly want better video games.

A (44:08)

Clearly, clearly. I have two. I run on two issues.

C (44:12)

You want to know, where's. Why are the AI companies not giving me that? Like, I mean, there's a lot of reasons, right? But, like, one is if they did that and you didn't think it was very good, then you'd say, like, no, you haven't given me. You've failed. But if they can. Can keep selling you something that they don't have to deliver on, then they can just do that all the time, right? AGI is always going to be something that's just around the corner. And the technology they're making, I mean, people are finding, like, amazing uses for it. But like you were saying earlier, they're not generate. They're not building video generators to solve a problem in Hollywood. They didn't build chatbots to solve a problem that any business had or any of us actually had. They built this stuff because it was cool. And they could. And then once it's out there, the rest of us have to sort of figure out, figure out what, what to do with it. And I think, you know, that's what's, that's what's powering this industry. They're making stuff and throwing it out there and you know, they're just saying, you know, the stuff's going to get better and better and better until one day, boom AGI. But, you know, that's not a deliverable product. There's not like a spec sheet that they are building a product to, towards and then, you know, selling it to Jake and then have Jake go on the Internet and rant about it, you know, because that, that would be failure.

A (45:20)

And they know so well that that is failure. That if, if the people turn on you online, you know, forget about it, you might have to dis a couple years and come back with something that's like, totally. Yeah. I mean, you bring up a great point, is that so much of the tech world that now is so, you know, just so enveloped in artificial intelligence. We're never asking for any of this shit. We didn't ask for the Internet. Honestly. Nobody was going, ah, I'd love to be able to like click on people's profiles and see what they ate for lunch and see who, I really want to know what my Aunt Barbara ate for lunch. You know, it's like, I want to know what color Uncle Vic is, what color he thinks he is after he took this, this Facebook quiz. We didn't ask for email or any of that stuff. They just invented and then go, you need it?

B (46:11)

Well, it was like, well, the Internet was originally invented by governments to share data more rapidly. And then there was, it was sort of developed by scientists and researchers who had academics who had a desire to share information more frequently. And then eventually in the 90s, people got the idea that if you turn it to a consumer product, you can, you can make a lot of money.

A (46:30)

And it feels like they want you to think of it as an intelligence, right? Because you could easily just come out with like, I don't know, Adobe could be like, we have a new update and the update allows you to drag your, your video a little bit further and we'll, you know, we'll sort of like auto generate some frames to help you out. Like, it could have just been kind of marketed as like a feature as opposed to an intelligence. And I think that's, you know, that's really what this whole crux is, is how far will this intelligence grow and, and what are we to do about it?

C (46:59)

It, I guess OpenAI had no idea what it had on its hands when it released ChatGPT. I mean, it took the company by surprises as much as it did everybody else. I mean, they'd been tinkering away at this technology for a while. ChatGPT was the slickest version of it. The key thing that ChatGPT did that previous language models hadn't was this sort of back and forth dialogue. So you could chat to it, because models beforehand you could sort of, hey, here's the first line of a story about unicorns prancing on clouds. Finish the story and you'd get, you'd get the story. It was the back and forth chatting that ChatGPT brought that really sort of gave people chills. But OpenAI, I've spoken to people there a bunch of times. They had no idea how this was going to take off. And they've been sort of scrambling to capitalize on that success ever since. What they do is astonishingly expensive. And so any way they can spin their tech into products, they obviously can.

B (47:58)

I wonder if you could speak about, in light of all this, what would be, you think would be a better, more productive way, I guess for people and maybe especially like tech journalists to talk about AI advances and the goals of these AI companies. Because like you mentioned, these products are very, are they sometimes they are very spectacular and impressive and they're doing things that were unimaginable still five years ago. I mean, I could, I, you could count me among the people who like, sort of scoffed at sort of the first generation of AI image generators. And I thought, but boy, they got to, they got a lot better much faster than I thought they would. But, you know, at the same time, it's like you don't. Well, how do you, I guess, how do you balance the acknowledgment of the, the advances and its potential impacts on consumers and the economy and all these things without buying into this AI hype? You know, I think it's, I don't know, it's a very, I think it's a, it's a very serious challenge for people who cover the sphere to, you know, walk that fine line in between the acknowledgment of the advances and the acknowledgement of the impacts it might have and the, the many use cases that they could have without, you know, talking like these founders, you know, talking about how AGI is upon us and is either utopia or, like you, the apocalypse.

C (49:10)

Yeah, this is something that we think about and talk about all the time within the broader media. I mean, there's, there's a lot of people that are just boosters who just get excited and sort of, you know, re up the hype that the companies themselves are putting out. But there's also a lot of people who are just the cynics, you know, who are just sort of the opposition to that. And I think neither gets it right because you need to recognize that the technology that has been developed is amazing and the genuine applications that it's potentially going to have are amazing. And hopefully we're going to see more good ones than bad ones. But since this is in the hands of basically Internet companies, who knows? But walking that path between the hype and the cynicism is difficult. And so you always got to make sure that you're chasing cheerleading when it's justified and you're really pushing back when you think something is overhyped. But I mean, in general and specifically on AGI, it's just taken for granted now that the industry is on a path to AGI, whatever that is. I mean, even thinking of it as a destination is nonsense because it's not a thing, right? There's not going to be one day when it's low. We've made AGI, here it is. But I think really stop taking that for granted. Like the sense that this near future technology is inevitable, I think that really needs to be pushed back on. Like, says who? It's only the people building it are telling us it's inevitable. And there are enormous costs involved in building. Not just the obvious sort of financial and sort of environmental costs, but the potential harms on increasing inequality and the massive upheaval for jobs and education. Something that gives me some personal dread. The idea of being an educator now gives me shivers.

A (50:56)

Not to mention the environmental costs and all of that stuff.

C (50:59)

Yeah, nobody is going to support a company like OpenAI building all these data centers and the power stations to power them if all they're making is slightly better technology than we have already. I wish that we could just think AI is going to be as big as the Internet again. The Internet has done amazing, but I think even that is not enough. You have to sell this idea that we are going to change the world and therefore any costs along the way are worth it.

A (51:29)

Taking advantage of a shitty world is really unfair, you know, because if everything was awesome and, you know, some snake oil salesman came up and he was like, I'm gonna change the world, you know, and most people were like, no, we like it the way it is.

B (51:43)

Yeah. The other thing is that I think the reason why a lot of people buy into it is just because I feel like for a lot of people, like, including myself, AI research is such black box. And that is something people who are much better at math than me go in and do. And they come out of the black box and they tell us scary things are on the horizon, and, gosh, I have no reason to doubt them. And so you just kind of like, go along with it.

C (52:04)

Yeah. And even to the people building it, it's still, still largely a black box. I mean, the. The engineers making these models do not fully understand how they work, how they do the amazing things that they do. And whilst there's still that sort of mystery to them, you can sort of choose to believe that there's more potential inside this tech than there maybe is.

A (52:25)

Yes. As long as there's a mystery, there's somebody who's got, like a narrative that solves it. And people are going. People who are uncomfortable with a mystery are going to gravitate towards one narrative or another.

B (52:38)

Yes. We've been speaking to Will Douglas. Heaven. Yeah. Go read his article and we'll put the link in the show notes.

A (52:45)

Read that.

B (52:45)

Yeah. MIT Technology Review helps. It helps. It helps color a lot of coverage of AI, I'll say that. So, yeah. Will, thank you so much for joining us today.

C (52:54)

No, thank you. I had fun.

A (52:56)

Where can. Is there anywhere you can direct our listeners to find more of your stuff, more of your writing?

C (53:01)

Yeah, just go to technology review.com we have a bunch of stuff that goes up every day. I mean, we also do really cool biotech tech stories and energy stories. So, yeah, it's a good place to get all your tech news. Cool.

A (53:15)

Go check it out, folks. We'll put the link in the show notes. Thanks for listening to another episode of the QAA podcast. You can go to patreon.comqaa and subscribe for $5 a month to get a second episode every week plus access to our entire archive of premium episodes. We've also got a website that's qaapodcast.com listener until next week. May the super Intelligence bless you and keep you as pets.

E (53:47)

We have auto keyed content based on your preferences right now. You know, I know. Presumably everybody knows no great secret. Musk and Bezos and Ellison and Altman and others are putting hundreds and hundreds and hundreds of billions of dollars into AI and robotics. Correct?

A (54:07)

Correct.

E (54:07)

Okay, now, does anybody really believe that these guys are doing it in order to improve life for the average American.

C (54:17)

Zero people believe statistically.

E (54:21)

It's funny that you say that. Yeah, I was at Davenport.

B (54:24)

Those four guys don't even believe.

E (54:27)

I was at Davenport, Iowa a couple of months ago and we had a few thousand people out at a rally. So I said, you know, I said what I just said. Now they're putting all this money. Raise your hand, thousands of people, raise your hand if you think AI and robotics is going to help the working class of this country. In a room of several thousand people, two hands went up. So people understand and what are their goals? What are they trying to do? And this is where becomes kind of creepy in my view. And I don't claim to be the world's greatest expert, but what you're going to see with AI and robotics is the displacement of millions and millions of people from the jobs that they have. You know, I want to see manufacturing rebuilt in America, but for a worker, it's not going to mean anything if robots are doing the work. And we want to see young people start their own small business businesses, etc. But it's going to be incredibly hard when we see more concentration of ownership and when entry level jobs are going to be done by AI. So you're looking at a revolution, a huge transformation, economic transformation, cultural transformation of our society. Who is determining what's happening? You have much say in it?

A (55:40)

No, I got a handful of people.

E (55:43)

Who are really determining the future of the world. World. That's scary stuff.