“The dangers are human, not AI. What’s dangerous is what a human does with AI, not what the AI does itself. In fact, even the idea that there is such a thing as the AI in itself is a mistake.” — Keith Teare

I’m in Korea this week. So rather than doing a traditional one-on-one That Was the Week tech summary, Keith Teare and I are trying something different. We invited Jonathan Rauch — Brookings Institution senior fellow, serial author and one of the most rigorous minds in Washington — onto the show to discuss AI.

Rauch had a simple mission. He wanted to find out why Keith Teare is just about the only person in the universe who believes that AI is benign. Jon had five buckets of doom to dump on Keith: labour market disruption, political upheaval, mental health and cognition, malicious actors, and the biggest daddy of all — AI developing consciousness, setting its own agenda, and killing everyone (even Keith).

But Keith maintained his Yorkshire stoicism under intense scrutiny from the analogue Rauch machine. AI is a word-counting machine, he explained. Large language models train on words, not experience. They split words into a probabilistic graph of correlations. When you ask a question, a large statistical engine fires, word by word. In that sense, he says, AI is no cleverer than a calculator. The idea that it has awareness, consciousness, or a plan is mythological. What’s dangerous is what a human does with AI, not what AI does itself. The dangers, he says, are human.

Jon wasn’t entirely reassured (his Brookings brand is scepticism, after all). What worries him most is that humans will handle these technologies irresponsibly. On that, he and Keith agree. The short-term labour disruption will be significant. White-collar service provision — legal, accounting, junior consulting — is already going. Jobs will go too. Work, Keith insists, will not. But nobody in politics is having the conversation about what comes next. Not JD. Not AOC. Only Keith and Jon.

Five Takeaways

• AI Is a Word-Counting Machine: Keith’s Core Argument: Large language models train on words and only words. They split those words into a probabilistic graph — how close is word A to word B? When you ask a question, a large statistical engine fires, producing output word by word. There is no awareness. There is no consciousness. There is no plan. The idea that such a system could develop its own agenda is mythological. It’s no cleverer than a calculator. It’s just a very big, very fast calculator. Rauch’s counter: the brain is also just dumb neurons. We get emergence from dumb neurons. Keith’s reply: what the AI can do is constrained by what humans allow it to do. The agency is human.

• Doomerism as Business Model: Before engaging with any specific AI doom argument, Keith signals a prior: whenever there is ambiguity in a major technological change, a business model emerges to monetize doubt. It was true of nuclear power. It was true of climate change. It is true of AI. This doesn’t mean the fears are groundless — they wouldn’t sell if they weren’t reasonable. But it means they should be approached with prior scepticism. The doom argument works precisely because AI genuinely contains possible negative outcomes. The business model packages and amplifies those possibilities beyond their actual probability.

• The Guardrails Are Human: Keith’s metaphor: AI sits in a prison where humans decide what the doors are. If you give it access to email, it can email. If you don’t, it can’t. It cannot take actions it has not been permitted to take. The word “guardrails” is commonly used, and it’s apt: the constraints on what AI can do are entirely under human control. The word output is the statistical engine — that’s not controllable. But its ability to act on words is highly constrained. The danger is not what AI does. It is what humans choose to allow AI to do.

• Jobs vs Work: The Labour Disruption Argument: Rauch’s young friends in junior consulting are watching their jobs go in real time. Keith distinguishes between jobs — paid labour — and work, which is closer to effort and creative agency. Jobs can go. Work, he argues, will not — humans will always be reinterpreting the future they want and working to make it happen. But the short-term disruption will be significant: white-collar service provision (legal, accounting, consulting), teaching, driving. The wealth creation AI enables could supplement the end of paid labour. But no one in government is having that conversation.

• Rauch’s Verdict: Clarified, Not Reassured: After fifty minutes with Keith Teare, Jonathan Rauch reaches a considered position: his worst fear — that AI becomes an autonomous engine of anti-human malfeasance — is unlikely to happen unless humans make it happen. His residual concern: that humans will not handle these technologies as maturely as one could wish. He’s not optimistic about political systems that are already too rigid, too partisan, and too dysfunctional to adjust as they did to the industrialization of the late nineteenth century. On that, he and Keith agree. Nobody knows. Not Keith. Not Andrew. And, despite his brilliance, not Jonathan Rauch.

About the Guests

Keith Teare is a British-American entrepreneur, investor, and publisher of the That Was the Week newsletter. He is a co-founder of TechCrunch.

Jonathan Rauch is a Senior Fellow at the Brookings Institution and a contributing writer at The Atlantic. He is the author of The Constitution of Knowledge: A Defense of Truth, The Happiness Curve, Kindly Inquisitors, Gay Marriage: Why It Is Good for Gays, Good for Straights, and Good for America, and many other books. He is based in Washington, D.C.

References:

• That Was the Week by Keith Teare.

• The Constitution of Knowledge: A Defense of Truth by Jonathan Rauch.

• Eliezer Yudkowsky and Nate Soares, If Anyone Builds It, Everyone Dies — the AI doom book referenced in the conversation.

• Sam Harris and Tristan Harris podcast on AI risk — referenced by Rauch as the catalyst for his questions.

• Episode 2902: Keith Teare on his jobless AI future vision — the preceding TWTW episode directly referenced.

About Keen On America

Nobody asks more awkward questions than the Anglo-American writer and filmmaker Andrew Keen. In Keen On America, Andrew brings his pointed Transatlantic wit to making sense of the United States — hosting daily interviews about the history and future of this now venerable Republic. With nearly 2,900 episodes since the show launched on TechCrunch in 2010, Keen On America is the most prolific intellectual interview show in the history of podcasting.