The Computer Scientist

Summary

Podcast Summary: Autocracy in America – "The Computer Scientist"

Host: Garry Kasparov
Guest: Gary Marcus
Date: September 5, 2025
Podcast by: The Atlantic

---

Overview

This episode explores the current and potential impact of artificial intelligence (AI) on American society, democracy, and politics. Host Garry Kasparov, world chess champion and democracy activist, is joined by cognitive scientist and AI critic Gary Marcus. Their discussion weaves personal chess history, AI’s technological evolution, the dangers of both the technology and those controlling it, and how democracy can resist or succumb to technological autocracy.

---

Key Discussion Points & Insights

1. Human vs. Machine: From Chessboards to Chatbots

• Kasparov’s Chess vs. Deep Blue: Kasparov recounts his battles against IBM’s Deep Blue, framing the human-machine dynamic.
  • Quote: “In 1985, ... I beat all 32 [chess computers] ... But just 12 years later, in 1997, I was ... fighting for my chess life against just one machine, a $10 million IBM supercomputer ... Newsweek’s cover called it the Brain’s Last Stand.” ([01:08])
• AI: Hype and Reality: AI, now far beyond chess, inspires both utopian dreams and dystopian fears.
  • Quote: “AI is still just a tool ... it is not a promise of dystopia or utopia. ... it is how we use it for good or bad.” ([01:55])

---

2. The Illusion of Machine Intelligence

• Brute Force vs. Understanding:
  • Early chess computers relied on brute force computation; today’s large language models (LLMs) still depend on data aggregation, not understanding.
  • “Large language models can perform complex tasks ... but are these machines intelligent? ... It is still brute force.” ([05:31])
  • LLMs can recite rules but don’t understand them—shown in their inability to follow basic chess rules consistently.
    • Quote: “If you ask a large language model, even a recent one, to play chess, it will often make illegal moves. That’s something a six year old child won’t do.” ([07:18])
  • Gary Marcus: “...when it actually comes to playing the game, it doesn’t have an internal model of what’s going on.” ([08:14])

---

3. Skepticism, Realism, and the Alignment Problem

• AI Realist, Not Skeptic:
  • Both guests reject hysteria, focusing on honest assessment.
    • Quote: "I love that you called me an AI realist rather than a skeptic." – Gary Marcus ([09:52])
• AI as Dual-use Technology: Useful or dangerous depending on who wields it.
  • Kasparov: “Humans still have monopoly for evil ... every technology can be used for good or bad, depending on who uses it.” ([10:59])
  • Accidents and misuse—rather than intentional machine malice—are the present concern.
    • Marcus: “It will just do really bad things by accident because it’s so poorly connected to the world.” ([11:22])

---

4. On ‘Alignment’ and the Limits of Scale

• Alignment Problem: Machines still can’t reliably do what we want; feeding more data isn’t fixing this.
  • Quote: “Not even close. ... We have nothing like a real solution to alignment.” ([14:37])
  • Quantity (more data) does not transform into quality (true understanding or ‘superintelligence’): “We will get to superintelligence eventually, but not by just feeding the beast with more data.” ([15:45])
• Field Critique: The current AI field is rife with hype, lacking the intellectual honesty of its early days.
  • “...now you just have people hyping stuff, praying...” ([15:45])

---

5. AI for Good: What’s Possible and What Isn’t

• Best AI Applications: Highly specialized systems like AlphaFold (for protein folding)—not general chatbots—offer real, concrete advances.

  • Quote: “The best AI for helping people ... is not the chatbot. The best piece of AI right now ... is AlphaFold.” ([18:03])

• Responsible Progress: The US is currently failing to implement regulatory or legal frameworks for safe AI development.

  • Marcus: “If we want AI to be net benefit to society, we have to figure out how to use it safely and justly. ... when we’re doing nothing, then... negative consequences.” ([18:46])

---

6. AI, Politics, and Propaganda

• AI and Disinformation:
  • AI’s most alarming social effect: supercharging propaganda, fake news, and “information wars”—directly threatening democracy.
  • Kasparov: “That’s where AI plays a massive role ... the sheer power behind them could at one point decide the results of any elections.” ([19:36])
• Path to Fact-Checking AI: Marcus is (cautiously) optimistic that future AI could fact-check at scale, but currently political will is absent.
  • Quote: “We can build AI that could do fact checking automatically, faster than people ... But ... the far right has so politicized the notion of truth that it is hard to get people to even talk about it.” ([20:17])
  • Historical lesson: The US once overcame a similar crisis with the rise of fact-checking in response to yellow journalism.
• Current Climate: Political leaders on all sides have little interest in defending truth, preferring expedient misinformation.
  • “No political ... force... interested of defending the truth ... [because] it may ... interfere with their political agenda.” ([21:52])

---

7. Orwell’s Warnings and Techno-fascism

• SCI-FI Warnings Realized: The dystopian fears of Orwell and other sci-fi writers—in which technology enables an elite to control minds and outcomes—are now manifest.
  • Quote: “We are exactly where Orwell warned us about, but with technology that makes it worse. Large language models ... can persuade people ... without people even realizing...” ([24:00])
• Techno-oligarchy in America:
  • Marcus argues the US already exhibits “techno-fascism,” with intent to surveil citizens, consolidate power, and even replace federal workers with AI.
  • Quote: “That’s exactly what’s happening in the United States right now is techno-fascism.” ([25:21])

---

8. Resistance and the Power of the People

• Apathy as the Default:
  • Most people prioritize convenience and have relinquished privacy and power to tech companies.
  • Gary Marcus: “IPhones are the opiates of the people.” ([26:31])
• Potential Tools for Resistance:
  • The same levers as always—strikes, boycotts, mass action—could rein in tech, but few are motivated.
  • Marcus: “We could all say, look, we’re not going to use generative AI unless you solve some of these problems ... We could boycott it.” ([28:58])
• Structural Problem with Mass Action:
  • Kasparov’s skepticism: It’s unlikely students and others will willingly give up convenience (e.g., ChatGPT for homework), even if the consequences harm them long-term.
    • Marcus: “It’s very unlikely. ... The students ... have given birth to this monster because they drive the subscriptions up.” ([30:08])

---

9. Outlook: Knife’s Edge or Call to Action?

• Not Hopeful by Default: Marcus describes himself as agnostic: “It’s important to realize that we still have choice. It’s not all over yet. We still have some power ... but it is not the default. ... unless we really do stand up for our rights.” ([31:29])
  • Quote: “We are America, and we still could. And we should. Our fate rests 100% on political will.” ([32:22])
  • The episode ends with a call for civic activism over resignation.

---

Notable Quotes and Memorable Moments (with Timestamps):

• Brute force vs. intuition:
  “If you ask a large language model, even a recent one, to play chess, it will often make illegal moves. That’s something a six year old child won’t do."
  – Gary Marcus ([07:18])

• Meaningless Data Mountain:
  “We will get to superintelligence eventually, but not by just feeding the beast with more data.”
  – Gary Marcus ([15:45])

• Caution over Hype:
  “Now you just have people hyping stuff, praying. There’s actually a great phrase I heard, pray and prompt ... the whole field is built on that these days.”
  – Gary Marcus ([15:45])

• On Fact-Checking AI:
  "I genuinely believe that in principle we can build AI that could do fact checking automatically, faster than people. ... Not current AI, but future AI could actually do that at scale. ... Part of it is political will and right now we lack it."
  – Gary Marcus ([20:17])

• Modern Techno-Fascism:
  "That’s exactly what’s happening in the United States right now is techno-fascism ... The intent appears to be to replace most federal workers with AI ... surveil people, ... accessible to a small oligarchy."
  – Gary Marcus ([25:21])

• On Civil Resistance:
  "We could all say, look, we’re not going to use generative AI unless you solve some of these problems ... right now, the people making generative AI are sticking the public with all of the costs ..."
  – Gary Marcus ([28:58])

---

Timestamps for Key Segments

• Chess and AI Background ([01:08]–[04:05])
• Brute Force and AI Limitations ([05:16]–[09:35])
• Skepticism vs. Realism ([09:35]–[11:22])
• Alignment and Superintelligence Debate ([14:21]–[16:23])
• AI for Good – AlphaFold Example ([18:03]–[19:36])
• AI, Propaganda, and Politics ([19:36]–[24:31])
• Tech Oligarchy & Techno-fascism ([24:31]–[28:58])
• The Challenge of Resistance ([28:58]–[32:29])

---

Tone and Language

The conversation is candid, incisive, and direct, with both Kasparov and Marcus blending humor, skepticism, and urgency. They cut through technological hype, focusing on risks, ethical dilemmas, and the need for civic action. Kasparov is pragmatic and sometimes wry; Marcus is passionate but realistic, with flashes of optimism grounded in historical memory.

---

Conclusion

The Computer Scientist presents a bracing assessment of AI’s real influence on democracy, warning against both panic and complacency. The episode closes with the imperative that America’s fate—autocratic or democratic—depends not on the machines, but on the political will of its citizens.

---

Most Memorable Call to Action:

"We are America, and we still could. And we should. Our fate rests 100% on political will."
— Gary Marcus ([32:22])

Transcript

[Narrator]

From Joe Wright, director of Atonement and darkest hour comes Son of the Century, an eight episode series streaming on mubi. Luca Marinelli stars in what critics are calling a towering performance of puffed up vanity, bringing Mussolini's rise to life with bold cinematic storytelling, magnetic performances and an unforgettable score by Tom Rowlands of the Chemical Brothers. Already hailed as remarkable and gripping, Mussolini's Son of the Century is a daring new vision of the past. Streaming September 10th on MUBI.

[Mint Mobile Advertiser]

Mint is still $15 a month for premium wireless. And if you haven't made the switch yet, here are 15 reasons why you should 1. It's $15 a month.

[Gary Marcus]

2.

[Mint Mobile Advertiser]

Seriously, it's $15 a month.

[Garry Kasparov]

3.

[Mint Mobile Advertiser]

No big contracts.

[Gary Marcus]

4.

[Mint Mobile Advertiser]

I use it.

[Blinds.com Advertiser]

5.

[Mint Mobile Advertiser]

My mom uses it. Are you playing me off? That's what's happening, right? Okay, give it a try. @mintmobile.com Switch upfront payment of $45 for.

[Narrator]

3 month plan $15 per month equivalent required New customer offer first 3 months only then full price plan options available, taxes and fees extra. See Mint Mob.

[Garry Kasparov]

In 1985, at the tender age of 22, I played against 32 chess computers at the same time in Hamburg, West Germany. Believe it or not, I beat all 32 of them. Those were the golden days for me. Computers were weak and my hair was strong. But just 12 years later, in 1997, I was in New York City fighting for my chess life against just one machine, a $10 million IBM supercomputer nicknamed Deep Blue. It was actually a rematch. I like to remind people that I beat the machine the year before in Philadelphia, and this battle became the most famous human machine competition in history. Newsweek's cover called it the Brain's Last Stand. No pressure. It was my own John Henry moment, but I lived to tell the tale. A flurry of books compared the computer's victory to the Wright brothers first flight and the moon landing. Hyperbole, of course, but not out of place at all in the history of our love hate relationship with so called intelligent machines. So are we repeating that cycle of hype and hysteria? Of course, artificial intelligence is far more intelligent than all chess machines. Large language models like ChatGPT can perform complex tasks in areas as diverse as law, art, and of course helping our kids cheat on their homework. But are these machines intelligent? Are they approaching so called AGI or artificial general intelligence that matches or surpasses humans? And what will happen when they do? If they do? The most important thing is to remember that AI is still just a tool, as powerful and fascinating as it is, it is not a promise of dystopia or utopia. It is not good or evil. No, more than any tech, it is how we use it for good or bad. From the Atlantic, this is Autocracy in America. I'm Garry Kasparov. My guest is Gary Marcus. He is a cognitive scientist whose work in artificial intelligence goes back many decades. He is not a cheerleader for AI Anything but. In fact, his most recent book is called Taming Silicon Valley. How we Can Ensure that AI Works for Us. He and I agree that humans, not machines, hold the monopoly on evil. And we talk about what humans must do to make sure that the power of artificial intelligence doesn't do harm to our already fragile democratic systems. Gary Marcus, welcome to our show.

[Gary Marcus]

This is the Gary Show.

[Garry Kasparov]

You are an expert on artificial intelligence, and you have worked on it for many decades, starting at a very young age. So before we talk about AI, I have to ask you, back then in 1997, who you were rooting for?

[Gary Marcus]

Who was I rooting for?

[Garry Kasparov]

Me or Deep Blue? But be honest, please. No, no bad blood.

[Gary Marcus]

You know, in 1997, I had become disenchanted with AI and I don't think I had. I really cared that much. I knew that eventually a chess machine was going to win. I had actually played Deep Blue's predecessor, Deep Thought, and it had kicked my ass, even, I think, with its opening book turned off or some humiliating thing like that. Not that I'm a great chess player, but I saw the writing on the wall, so I wasn't really rooting. I was just watching as a scientist to see, like, okay, when do we sort this out? And at the same time, I was like, yeah, but that's chess, and you can brute force it, and that's not really what human intelligence is about. So I honestly didn't care that much.

[Garry Kasparov]

You said brute force. With all the progress being made, would you say that machines are still relying almost exclusively on brute force? Or we see some, you know, transformation from simple quantity into some quality factors?

[Gary Marcus]

I mean, I hate to say it's a complicated answer, but. It's a complicated answer.

[Garry Kasparov]

It is complicated answer. I wouldn't ask you a simple question.

[Gary Marcus]

I figured, Don, in some ways, we've made real progress since then, and then some not. The kind of brute force that Deep Blue used is different from the kind of brute force that we're using now. The brute force, the beat. You was able to look at an insane number of positions essentially simultaneously and go several moves deep and so forth, and large language Models don't actually look ahead at all. Large language models can't play chess at all. They make illegal moves. They're not very good. But what they do do is they have a vast amount of data. If you have more data, you have a more representative something or other. So if you take a poll of voters, the more voters you have, the more accurate the poll is. So they have a very large sample of human writing, in fact the entire Internet, and they have a whole bunch of data that they've transcribed from video and so forth. So they have more than all of the written text in the Internet. That's an insane amount of data. And what they're doing every time they answer a question is they're trying to approximate what was said in this context before. They don't have a deep understanding of the context, they just have the words. They don't really understand what's going on. But that deep pile of data allows them to present an illusion of intelligence. I wouldn't actually call it intelligence. It does depend on what your definition of the term is. But what I would say is it's still brute force. So let me come back to chess for a second. If you ask a large language model, even a recent one, to play chess, it will often make illegal moves. That's something that a six year old child won't do. And I don't know when you learned chess, I can't remember, but you were probably quite young, so I'm guessing you were four or something like that.

[Garry Kasparov]

Five and a half.

[Gary Marcus]

Five. When you were five and a half, pretty much immediately you understood the rules. So basically you probably never made illegal moves in your chess career, starting when you were a little child and 03 was making them. This weekend I asked a friend to go try it out. And when you were five and a half, you'd only seen whatever, one game, two games, now 10 and whatever. There are millions of games, maybe tens of millions or hundreds of millions that are available in the training data. And Lord knows they use any training data they can get. So there's a massive amount of data. The rules are there. Wikipedia has a Rules of Chess entry that's in there. All of that stuff's in there. And yet still it will make illegal moves, like have a queen jump over a knight to take the other queen.

[Garry Kasparov]

Making mistakes, not mistakes, actually violating the rules. So again, just tell us how come. Why? Because the rules are written and technically they can extract all the information that is available. And they're still making illegal moves.

[Gary Marcus]

Yeah. And in fact, if you ask them verbally, they will report the rules, they will repeat the rules, because in the way that they create text based on other texts, they'll be there. So I actually tried this. I asked it, I said, can a queen jump over a knight? And it says no. In chess, a queen cannot jump over any piece, including a knight. So it can verbalize that. But when it actually comes to playing the game, it doesn't have an internal model of what's going on. So even though it's. It has enough training data that it can actually repeat what the rules are, it can't use those in the service of the game because it doesn't have the right abstract representation of what happens dynamically over time in the game.

[Garry Kasparov]

Yes, it's very interesting because it seems to me that what you are telling us is that machines know the rules because rules are written, but it still doesn't know what can be done or cannot be done unless it's explicitly written, correct?

[Gary Marcus]

Well, I mean, it's worse than that. I mean, the rules are explicitly written, but there's another sense of knowing the rules which we actually understand what a queen is, what a knight is, what a rook is, what a piece is, and it never understands anything. It's one of the most profound illusions of our time that most people witness these things and attribute an understanding to them that they don't really have.

[Garry Kasparov]

Okay, so now I think our audience understands why you're often called an AI skeptic. But I believe AI realist is better because I share just your overall view of the future of AI and human machine collaboration.

[Gary Marcus]

Let me just drop in. I love that you called me an AI realist rather than a skeptic.

[Garry Kasparov]

I share that. And I always say AI is not a magic wand, but it's not a terminator. It's not a harbinger of utopia or dystopia. It's a technology. It doesn't buy you a ticket to heaven, but it doesn't open the gates of hell. So let's be realistic.

[Gary Marcus]

Yeah, so let me talk about the realism first and then the gates of hell. So on the realism side, I think you and I have a lot in common. We're both realists, both politically and scientifically. We both just want to understand what the truth is and how that's going to affect society and so forth. I mean, the fact is, I would like AI to work really well. I actually love AI People call me an AI hater. I don't hate AI but at the same time, to make something good you have to look at the limitations realistically. So that's the first part. Is it going to open the gates to heaven or hell? That's actually an open question. Right. AI is a dual use technology like nuclear weapons, right? Can be used for good, can be used for evil. And when you have a dual use technology on the table, you have to do your best to try to channel it to good.

[Garry Kasparov]

But look, I also keep repeating that humans still have monopoly for evil. I think we can disregard the fact that every technology can be used for good or bad, depending on who is going to use it. And I think that the greatest threat coming from the AI's world is potentially this technology being controlled and used by those who want to do us harm.

[Gary Marcus]

Mostly agree with you there. First of all, neither of us are that worried about the machines becoming deliberately malicious. I don't think the chance of that is zero, but I don't think it's very high. Agree. We should be worrying about malicious humans and what they might do with AI, which I think is a huge, huge concern. We also have to worry because of the kind of AI that we have now that it will just do really bad things by accident because it's so poorly connected to the world. It doesn't understand what truth is, it can't follow the rules of chess, et cetera. It can just accidentally do really bad things. And so we have to worry about, I think the accidents and the misuse, maybe less about the malice.

[Garry Kasparov]

Now let me ask very primitive or just a question that has no scientific background. So by analyzing our chess decisions, we always say, okay, this part is being just made through calculation, this one through recognition of patterns. Now in your view, what percentage of these decisions or suggestions made by AI based based on calculations and what percentage is attained to understanding? I mean, I don't want to use word intuition, but recognition of patterns. So let's say strategy versus simple tactical calculation.

[Gary Marcus]

So first thing is I should clarify something which is there are different kinds of AI out in the world. So for example, a GPS navigation system is all what I would call calculation and no intuition. It simply has a vast table of different locations and the routes that you can take between those places for different segments of it and the times that are typical and so forth. All calculation, nothing. I would describe as pattern recognition. I would still call it AI. It's not a sexy piece of AI and it's not what most people talk about when they talk about AI. Right now most people are talking about chatbots like ChatGPT. When Deep Blue beat You that was all calculation. Maybe you could argue there was a tiny bit of pattern recognition. Stockfish is now kind of a merger of the two. It's kind of a hybrid system, which I think is the right way to go. The things that are popular mostly aren't hybrids, although they're increasingly kind of sneaking some hybrid stuff in the back door. I would say they're not doing any calculation at all. I would say that they're all pattern recognition. A pure large language model is all pattern recognition with no deep conceptual understanding and no deep representations at all. There's no deep understanding even of what it means to jump a piece or illegal move. None of that is really there. So everything it does is really pattern recognition. When it does play chess, it's recognizing other games. There's an asterisk around this, which is they can do a little bit of analogy in certain contexts. So it's not pure memorization, it's not pure regurgitation, but it comes close to that. And it's never kind of deep and conceptual.

[Garry Kasparov]

So before we move into politics, and I will just give you some statements and you tell me if I'm right or maybe they have to be corrected. So this infrastructure and this whole industry has not solved the alignment problem.

[Gary Marcus]

Not even close. The alignment problem means making machines do what you want them to do or things, things that are compatible with humans. And already we saw a great example, which is chess. You tell it, I want to play chess. Here are the rules of chess, and it can't even stick to that. Now you get to something harder, like don't cause harm to humans, which is much more complicated to even define what harm means and so forth. They can't do that at all. There is no real progress, I would say, in the alignment problem. Adding more data doesn't help that much. With the alignment problem, there's another thing called reinforcement learning. It helps a little, but we have nothing like a real solution to alignment.

[Garry Kasparov]

Okay? So the bottom line is that simply adding information or just cleaning this human data and just building the skyscrapers of this data doesn't help very much. So we reached a plateau. So the idea that is we simply keep piling more and more data and we'll transform this quantity into a quality to move to the next level. It doesn't work because again, there's no evidence this kind of super intelligence going to happen tomorrow or just in the.

[Gary Marcus]

Foreseeable future, it's not going to work. We will get to superintelligence eventually, but not by just feeding the beast with more data. You know, I thought what you were going to ask me was, is this field intellectually honest? And my answer is not anymore. I used to be an intellectually honest field, at least for the most part, and now you just have people hyping stuff, praying. There's actually a great phrase I heard, pray and prompt. Like you pray and prompt and hope you get the right answer. And it's just, it's not reasonable to suppose that these things are actually going to get us to AGI, but the whole field is built on that these days.

[Garry Kasparov]

We'll be right back.

[Gary Marcus]

Foreign.

[Progressive Insurance Advertiser]

This episode is brought to you by Progressive Insurance. Do you ever find yourself playing the budgeting game? Well, with the name your price tool from Progressive, you can find options that fit your budget and potentially lower your bills. Try [e-mail] Progressive Casualty Insurance Company and affiliates Price and coverage match limited by state law. Not available in all states.

[Blinds.com Advertiser]

@Blinds.Com it's not just about window treatments. It's about you. Your style, your space, your way. Whether you DIY or want the pros to handle it all, you'll have the confidence of knowing it's done right. From free expert design help to our 100% satisfaction guarantee, everything we do is made to fit your life and your windows. [e-mail], the only thing we treat better than Windows is is you. Visit blinds.com now for up to 45% off with minimum purchase plus a professional measure at no cost. Rules and restrictions apply.

[Garry Kasparov]

Okay, so to support your reputation as AI realist and not be just on a negative side because you already said enough and I couldn't agree more with everything you just said. We have to support our reputation of those who believe that AI still brings something good into this world. So how do we benefit from AI's interference or infusion of virtually every aspect of our life?

[Gary Marcus]

So I think it's a multi part answer because AI affects so many parts of our life right now. The best AI for helping people in my opinion, is not the chatbot. The best piece of AI right now, I think is AlphaFold, which is a very specialized system that does one thing and only one thing, which is to take nucleotides in a protein and figure out what their 3D structure is likely to be. It may help with drug discovery. Lots of people are trying that out. That seems like a genuinely useful piece of AI and should be a model. But I would say of the big AI companies, DeepMind is the only one seriously pursuing AI for science. @ scale, most people are just like, well, can I throw a chatbot at something? And mostly that's not going to lead that much advance as opposed to creating special purpose solutions. I think we have to be intellectually honest about the limitations of this generation of AI and build better versions of AI and introduce new ideas and foster them. And right now we're in this place where the oxygen is being sucked out of the room, as Emily Bender once said, and nobody else can really pursue anything else. All the venture funding is to do large language models and so forth. So there's the research side of it. There's like finding the right tools for the job side of it. There's also a legal side of it, which is if we want AI to be net benefit to society, we have to figure out how to use it safely and how to use it fairly and justly. If we don't, which is what's happening right now in the United States, when we're doing nothing, then of course there's going to be lots of negative consequences.

[Garry Kasparov]

Negative consequences. I think the one place where we all feel these negative consequences is politics or things related to politics like propaganda and simply just sharing information. That's where AI plays a massive role. Because again, we saw the influence of these various forms of AI being used to influence the elections and it seems unstoppable now. So just briefly, so what do you think? Anything can be done or we are just. We have entered the era of these information wars that will be run by these chatbots and the sheer power behind them could at one point decide the results of any elections.

[Gary Marcus]

This is a place where I may be an AI optimist, although not short term. So I genuinely believe that in principle we can build AI that could do fact checking automatically, faster than people. And I think we need that right now. It's sort of politically hot, so nobody even wants to touch it. But I think in the long run that's what we need to do. Think about like the 1890s with all the yellow journalism of people like Hearst and so forth, all bullshit. Some people think it led to war based on false facts and that led to fact checking being a thing. And we may return to that because I think people are going to get disgusted by how much bullshit they are immersed in. And I think in principle, not current AI, but future AI could actually do that at scale, faster than people. And I think that could be part of the solution eventually. Part of it is political will and right now we lack it. The far right has so politicized the notion of truth that it is hard to get people to even talk about it. But I think that there will be a swing back in the pendulum of that someday. Whether that happens in the United States is a very complicated situation right now. But I think the world at large is not going to be satisfied with this state of affairs where you can't trust anything. Dictators love it, it's great for them. That's why it was the Russian propaganda model. Putin loves the idea that nobody knows what to believe and so you just kind of have to go along with what he makes you do.

[Garry Kasparov]

But it seems to me that the political moment definitely in this country also in Europe now is not very friendly to this notion.

[Gary Marcus]

So fact checking, very unfriendly.

[Garry Kasparov]

People believe what they want to believe. And unfortunately fake news have this element of sensationalism that always attracts the attention. And I think lies become weapons on both sides. There's some blatant lies, there's some more, you know, just covert lies. But at the end of the day, I think no political, meaningful political force in this country now is interested of defending the truth, defending the pure, correct fact checked data, because it may and most likely will interfere with their political agenda. And also the facts, they always lose in the battle of public opinion these days against fake news.

[Gary Marcus]

I mean, my one moment of optimism is we saw this before in the 1890s and eventually people got fed up. It's not gonna happen soon though. Right now people are complacent and apathetic and they have given up on truth. I could also be wrong in my rare moment of optimism. I think that things are going to get so bad that people will resist. But I mean that's an open question. At least once in history people did get fed up with that state of affairs. It is also true what you're saying. Lies do tend to travel faster than truth. And that's part of what happened in the social media era is that whole thing got accelerated. The social media companies don't care about truth and they realized they would make more money by trafficking in fake narratives. And that's part of why we are where we are now.

[Garry Kasparov]

Yeah, you mentioned a couple of times, 1890s and early the 20th century. So as this one of the moments of transition. So what about the, let's say mid 20th century with the booming sci fi book industry, had many, many stories about the future influence of technology, technology to dominate our society, technology interfering with democracy. The great writers, you know, just they predicted that at one point we would deal with this direct challenge of technology in the hands of few to influence the opinion of many. Are we now at this point, I.

[Gary Marcus]

Keep thinking about rewriting word for word remake of 1984, which I think was written in the late 40s. We are exactly where Orwell warned us about, but with technology that makes it worse. Large language models can be, I don't know, we call them super persuaders. They can persuade people of stuff without people even realizing they're being influenced. And the more data you collect on someone, the easier that job becomes. And so we are exactly living in the world that Orwell warned us about.

[Garry Kasparov]

Okay, so let's talk about tech bras. So they believe that all powerful technology could actually help to improve the society. Because society has too many problems. They cannot be resolved any other way but to lead the public, to educate the public, to control the public mind, to cure these problems. Is this threat real and is it doable? Because some people even say that, oh, it may lead us to something called techno fascism, where while preserving all the elements of representative democracy, we will end up in some kind of dystopian society where the few in charge of massive data will make election results predictable and bend them to their favor.

[Gary Marcus]

I mean, that's exactly what's happening in the United States right now is techno fascism. The intent appears to be to replace most federal workers with AI, which is going to have all the problems that we talked about. The intent is to surveil them, to surveil people, to get massive amounts of data, put it all together in one place, accessible to a small oligarchy. That's just what they're doing. This is not science fiction. That could happen in 10 years. This is essentially the active thing that is happening right now that has been happening for the last few months.

[Garry Kasparov]

Question. So is it inevitable? So how the society at large can resist this pressure from this new techno oligarchy that has all the money, that has control of technology? And again, let's be honest, most of the public, they care more about convenience rather than about the security of their devices. For instance, that's known that people want these devices, this new technology, to bring some short term benefits.

[Gary Marcus]

IPhones are the opiates of the people.

[Garry Kasparov]

Exactly. So because our reliance on these new devices, because we are willing to use the simplest passwords, because to do complicated password is too time consuming. So again, ignoring even threats to our personal data. So can we rally enough people to make this threat?

[Gary Marcus]

I think the default path is what you described. I would add privacy to it. So people have given up on privacy. They won't do the basic things on security, and they have given up an enormous amount of power. And that power hasn't even just gone to the government. Power has really gone to the tech companies who have enormous influence over the government. And unless people get out of their apathy, that's certainly where the United States is likely to stay. It's only if there is mass action and if people realize what has happened to them. There were huge protests specifically directed towards Elon Musk, and he was kind of, as far as I can tell, pushed aside. Those protests were somewhat effective in mitigating some of the more egregious things that he tried to do. So he's at least kind of not at center stage anymore. But short of that, I think the default is the sort of dark world that we're talking about. That reminds me a lot of contemporary Russia, where few people have most of the power. Most people have essentially no power. And to a surprisingly large degree, people just consent to that. Giving up their freedom, giving up their privacy, maybe giving up their independence of thought as these systems start to shape their thoughts. To me, that's extremely dark. But not everybody seems to understand what's going on, and less more people understand what's going on. This is where we're going to stay.

[Garry Kasparov]

Yes. So to wrap it off, can you give us just some glimpse of hope? Any idea how can we fight back? By using enormous power that AI and all these devices give to us? Because we are many, we are millions, and they are few, though they're very powerful few. So what's the best bet for us to take back our future back in our hands, and also to make sure that the political institutions of the United States, of the great republic will survive its 250th anniversary that will be celebrated next year.

[Gary Marcus]

I think our powers are the same as they always were, but we're not using them. So we have powers like striking. We could have a general strike, you know, strikes, boycotts. You know, we could all say, look, we're not going to use generative AI Unless you solve some of these problems. Right now, the people making generative AI are sticking the public with all of the costs, cost to the information ecosphere, all the enormous climate costs of these systems, like they're just sticking everything to the public. And we could say, that's not cool. We would love to have AI but make it better. Make it so it's reliable. It's not destroying the environment, massacring the environment, you know, and then we'll use your stuff right now. We'll boycott it. So we could say, hey, we're not going to do this anymore. You know, we'll come back to your tools later. They're nice, but I think we could live without them. You know, they save some time, and that's cool.

[Garry Kasparov]

But are you sure, Gary? I mean, let's be realistic. So I hate pouring cold water in your concept of our hot resistance, but do you seriously think that people today, I mean, starts with students, will stop using ChatGPTs?

[Gary Marcus]

I think it's very unlikely. But the reality is that the students, by adding to the revenue streams and user numbers massively, students are a huge part of it. They're adding to the valuations of the companies. They're giving companies power. And what the companies are trying to do is to keep those students from ever getting jobs. And the companies probably are going to succeed in that, right? The people who are losing their jobs first are students. The students graduating are entering this world where junior workers aren't getting hired as much, and probably in part because of AI. In some ways, they're the most screwed by all of this. And they have given birth to this monster because they drive the descriptions up. So OpenAI can raise all of this money because a lot of people are using it. A large fraction. I don't know the exact numbers. Are students using them to write their term papers? If students just stop doing that, it would actually undermine OpenAI. It might lead to the whole thing collapsing. And, you know, that would actually change what their employment prospects are like.

[Garry Kasparov]

Yeah, I'm. I'm very skeptical about them. Just.

[Gary Marcus]

I'm skeptical about it, too.

[Garry Kasparov]

So is it fair to say that regarding AI, short term you are pessimistic, You. You have very uneasy feelings. Mid term you're optimistic, and long term you're bullish?

[Gary Marcus]

Um, no, it's more agnostic. It's like, I think this could work out, but we have to get off of our asses if we want it to work out. We may reach some point where people in the US do fight back. We have more of an expectation historically of having certain kinds of freedoms than I think the Russian people do. And so it could turn around. And to the extent that it makes me an optimist to think it could turn around. Yeah. But generally, I like the metaphor that we're kind of on a knife's edge and we have choice. It's important to realize that we still have choice. It's not all over yet. We still have some power to get ourselves on a positive AI track, but it is not the default. It is not where we're likely to go unless we really do stand up for our rights.

[Garry Kasparov]

So it's not the most optimistic forecast, but at least it's a call for action.

[Gary Marcus]

But we could. We could take action.

[Garry Kasparov]

Exactly.

[Gary Marcus]

We are America, and we still could. And we should. Our fate rests 100% on political will.

[Garry Kasparov]

Gary Marcus, thank you very much for this most enlightening conversation.

[Gary Marcus]

Thank you so much for the conversation.

[Garry Kasparov]

This episode of Photography in America was produced by Arlene Orevolo. Our editor is Dave Shaw. Original music and mix by Rob Smirciak. Fact checking by Ino Alvarado. Special thanks to Polina Kasparov and Mick Ringgold. Claudian Abay is executive producer of Atlantic Audio. Andre Walnuts is our managing editor. I'm Garry Kasparov. See you back here next week.