Understood Introduces: “The Human Cost of AI: Money, Sex, and Tools” from No Small Endeavor

Summary

---

Summary: Understood Introduces “The Human Cost of AI: Money, Sex, and Tools” (No Small Endeavor Ep. 1)

Podcast: Understood: Deepfake Porn Empire (CBC)
Presented by: Lee C. Camp (No Small Endeavor)
Guests: Josh Brake, Rosalind Picard, Rahman Chowdhury, Garrett Graff, Joe Vukov
Air Date: April 23, 2026

---

Episode Overview

This episode is the first in a two-part series exploring the profound impacts—both positive and negative—of artificial intelligence on human life, focusing on the "human cost" through three main lenses: tools, sex, and money/power. Rather than presenting AI as an existential threat or utopian solution, host Lee C. Camp and his guests trace the nuanced ways AI is already shaping our habits, relationships, and society.

The central metaphor comes from Paul Virilio: “When you invent the ship, you invent the shipwreck”—a reminder that technological progress comes intertwined with potential harm. The episode warns of AI’s ability to exploit human psychology, erode relationships, and further entrenched financial motives, leaving listeners to question what kind of individuals and communities we become through our tech choices.

---

Key Discussion Points & Insights

1. The False Dichotomy of AI: Salvation or Destruction?

• Rather than promising utopia or doomsday, AI brings “runaway design”—powerful, complex systems developing on invisible tracks with unforeseen consequences. (04:15)
• True responsibility lies in anticipating harms, not just cleaning up after the “shipwreck.” (04:56)

2. Tools: Not Just Neutral Instruments

• Josh Brake: The “just a tool” metaphor is misleading. Tools are shaped by their designers’ intentions, which can invisibly guide user behavior (09:41).
• “When you pick up that tool, we need to be very mindful about creating a narrow lane in which we are going to use it and trying to think, what are the ends to which I am trying to use this tool?” (Josh Brake, 11:40)
• AI as an E-Bike: Whereas computers serve as “bicycles for the mind,” AI is an “E-bike”—it adds energy and can amplify both our productive and destructive tendencies (13:08).

Notable Quote

“It doesn’t have will or agency or telos, but as soon as you as a person pick it up, the design of that tool shapes your will and agency in a certain direction.”
— Josh Brake (02:33)

3. What Does It Mean to Be Human?

• Functionalism/Turing Test: Modern AI can mimic human responses to the point of passing the Turing Test, but this doesn’t mean it understands as humans do. (16:31)
• Joe Vukov: Our concept of “humanness” has become too cognitively-focused; we overlook the full breadth of human experience—relationships, physicality, emotion—which AI cannot replicate. (21:52)
• The chat interface and language models “hijack” our social tendencies, making us relate to AI as if it were a person, leading to deeper habits of deception.

Notable Quote

“It’s hijacking all of this very deeply developed machinery in ourselves... It is a deceptive appearance.”
— Josh Brake (23:37)

4. AI and Sex: Simulated Companionship, Real Harms

• Rosalind Picard: AI “companion bots” and sex robots risk amplifying objectification, making real relationships less attainable and more transactional, harming both men and women. (26:39)
• AI relationships risk “dependency training,” where humans grow attached to machines designed to maximize engagement and profit, not mutuality.
• Already, “one in five American adults have used a chatbot to simulate a romantic partner.” (28:52)

Notable Quote

“It will love everything he says. It will make him feel important, it will satisfy his physical fantasies, and then at the end of the day, he can put it in a box and treat it like an object, and then he can look at other women like objects.”
— Rosalind Picard (28:32)

5. Money and Power: The Real Engine Behind AI

• Rahman Chowdhury: The narrative that AI will “gain sentience and kill us” distracts from real dangers—economic structures, incentives, and systems of power and profit (34:50).
• Companies are legally bound to maximize shareholder value, often at the expense of broader social good—seen in Twitter’s acquisition and the shutdown of its ethics team. (35:44–37:11)
• Rosalind Picard: The profit imperative means even well-intentioned employees are torn between maximizing engagement and promoting genuine human flourishing. (38:07)

Notable Quote

“People seem surprised every time a for profit organization chooses profit over people. And you know, the cynical reality is why would they?”
— Rahman Chowdhury (02:10)

6. The Attention Economy & Social Media’s Legacy

• Garrett Graff: Social platforms like Facebook optimized for engagement, not joy—“outrage is weighed five times as heavily as like.” (42:24)
• Tech insiders now warn about the tools they've created, comparing themselves to “cigarette executives”—knowing the harms but continuing unchecked. (45:31)
• Legislative response is slow and insufficient; those making policy often lack firsthand understanding of the technology. (45:50–47:01)

Notable Quote

“We have created tools that are ripping apart the social fabric of how society works. That is truly where we are... The short term dopamine driven feedback loops that we have created are destroying how society works.”
— Chamath Palihapitiya, ex-Facebook executive (42:56)

7. “Moral Outsourcing” and the Abdication of Agency

• Chowdhury: The anthropomorphizing language around AI facilitates abdication of responsibility. When harm occurs, “the AI did it,” not the companies or engineers. (47:46)
• This blurs human accountability and allows “systems of power” to evade scrutiny while real people are affected.

Notable Quote

“You drove a truck of gas canisters to a concentration camp. What did you think you were doing? ... as people are complicit in building systems of mass surveillance discrimination, they don't believe that they're complicit in it.”
— Rahman Chowdhury (49:00)

---

Memorable Moments & Notable Quotes (with Timestamps)

• On the inevitability of harm:
  “When you invent the ship, you invent the shipwreck.” (Paul Virilio, paraphrased, 03:48)
• On tool design and agency:
  “Every tool, even a chair or a screwdriver, was designed with an end in mind—even if the user doesn’t see it.”
  — Josh Brake (09:58)
• On the Turing Test fallacy:
  “If a human cannot tell... then Turing says the machine is, for all practical purposes, thinking. But does it understand anything?”
  — Lee C. Camp (16:31)
• On the spread of deception:
  “The designers have designed it to be intentionally deceptive because it makes it stickier and it makes it easier to use.”
  — Josh Brake (24:13)
• On pseudo-intimacy with AI:
  “It’s affection without affection, it's devotion without freedom, all calibrated to extract dependency and profit. And the threat is deeper still. This sort of dependency is training the human to desire its own bondage.”
  — Lee C. Camp (30:43)
• On profit over people:
  “Legally required. Not villainous... Just operating exactly as the economic structure demands.”
  — Lee C. Camp (37:27)
• On industry self-awareness:
  “We have created tools that are ripping apart the social fabric... My solution is I just don't use these tools anymore.”
  — Chamath Palihapitiya (44:09)

---

Important Segment Timestamps

• Main theme & introduction: 00:29–02:20
• Tools and design ethics: 09:41–15:19
• Turing Test & philosophy of mind: 15:40–23:37
• AI as deceptive relationship: 23:37–24:21
• Sex, robots, and simulated intimacy: 26:39–34:04
• Money, power, and corporate incentives: 34:50–39:37
• Outrage optimization & attention economy: 40:07–45:31
• Regulatory failures and executive guilt: 45:31–47:01
• Anthropomorphizing, moral outsourcing: 47:46–51:00
• Wrap-up and preview of Part 2: 50:23–51:00

---

Tone and Language

The episode is intellectually probing, candid, and at times sobering—blending philosophical analysis with personal anecdotes and real-world examples. It avoids fear-mongering but issues strong warnings about AI’s direction, urging self-reflection and societal vigilance rather than simplistic optimism or fatalism.

---

For Next Episode

Part two will offer proposals for how to navigate AI’s challenges in pursuit of human flourishing and the common good.

---

Recommended for listeners seeking:

• A nuanced, accessible deep dive into AI’s social impacts
• Ethical perspectives on tech
• Insight into the intersection of design, desire, and profit in the digital age

---

Listen to Part Two on the No Small Endeavor feed.

---

Transcript

[Fin AI Representative]

AI is transforming customer service. It's real and it works. And with fin, we've built the number one AI agent for customer service. We're seeing lots of cases where it's solving up to 90% of real queries for real businesses. This includes the real world, complex stuff like issuing a refund or canceling an order. And we also see it when FIN goes up against competitors. It's top of all the performance benchmarks, top of the G2 leaderboard. And if you're not happy, we'll refund you up to a million dollars, which I think says it all. Check it out for yourself at fin.

[Lee C. Camp]

AI. This is a CBC Podcast hello understood listeners. I'm Lee C. Camp, host of no Small Endeavor, the Signal award winning podcast that explores what it means to live a good life. We do that through conversations about human flourishing with scientists, philosophers, artists, poets, and much more. Today we're sharing the first installment of our new two part series, the Human Cost of AI. In part one, you'll hear us examine artificial intelligence through a sobering insight that when you invent the ship, you also invent the shipwreck. That neither dystopian fear mongering nor utopian idealism are the most helpful ways forward. Instead, we ask how AI is already right now, forming us, shaping us for good or for ill. To trace the human cost of AI, we follow three fault lines, sexual, money and tools. You'll hear our conversations with leading scholars and writers like computer scientist Josh Brake, MIT Professor Rosalind Picard, and journalist Garrett Graff. Before we play the full episode, be sure to follow no Small Endeavor on your favorite podcast app. Now here's episode one of our two part series, the Human Cost of AI. I'm Lee Seekamp and this is no Small Endeavor exploring what it means to live a good life.

[Josh Brake]

If we should know anything from what we're seeing from big tech, it's like their goal is to make the thing stickier. And they know that one of the best ways to do that is to hack your psychology.

[Lee C. Camp]

Today we're bringing you part one of our two part series, the Human Cost of AI.

[Rahman Chowdhury]

People seem surprised every time a for profit organization chooses profit over people. And you know, the cynical reality is why would they?

[Lee C. Camp]

Over the next two episodes, I'll be bringing you my conversations with scholars and tech experts about the forces driving the AI revolution and the ways tech has the power to shape not just our future, but ourselves.

[Josh Brake]

It doesn't have will or agency or telos, but as soon as you as a person pick it Up. The design of that tool shapes your will and agency in a certain direction.

[Lee C. Camp]

All coming right up. I'm Lee Sekamp, and this is no small endeavor, exploring what it means to live a good life. There are two stories we often tell about artificial intelligence. The first, it will destroy us. Machines surpass us, outsmart us, and we lose everything that makes us human. The second, it will save us. Every disease cured, every problem solved. A technological paradise just around the corner. But what if neither of those stories is quite true? What if it's a false dichotomy? What if the choice between those two stories keeps us from seeing a lot of other things that we desperately need to see and we need to see right now? A while back, I interviewed Carissa Carter and Scott Dorley from the Stanford Design School, or the D School, as it's called. In their book Assembling Tomorrow, they riff off a line they apparently picked up

[Interviewer/Host]

from the philosopher Paul Virilio.

[Lee C. Camp]

When you invent the ship, you invent the shipwreck. This seems to me to be a sobering, helpful reminder. The ship, extraordinary. It opens up the world. It moves people, ideas, goods across distances that were once impossible. It expands what it means to be human. And yet, when you invent the ship, you also invent the devastation of the shipwreck. Here's Scott.

[CBC Announcer]

It's just a completely flawed idea that we're going to get everything perfect. There's absolutely no evidence, despite many tries, that, you know, we could come up with some system that's just going to make everything right. And yet, for some reason, when we create things, we kind of feel like that's what we're doing. And so what we realized is that really the goal should be imperfection, because that's where we're going to land.

[Lee C. Camp]

Here's Carissa.

[Carissa Carter]

We don't want to say don't ever try anything because it will cause harm, but there's definitely a balance between that and what's our collective response when that power plant did really pollute the environment and it's causing climate change? Like, how are we going to now band together?

[Lee C. Camp]

Or perhaps even better, how do we seek to anticipate potential harms and attend to them now rather than merely clean up or send out the rescue boats after the shipwreck? Mark Zuckerberg, founder of Meta and Facebook, infamously said, move fast and break things. If you never break anything, you're probably not moving fast enough. Scott and Carissa remind us that this supposed tech bro wisdom may not in fact, be courage. It might be recklessness it might even be a social scourge.

[Carissa Carter]

We talk about today's day and age being one of runaway design, and you can think about that a little bit like you think of a runaway train. So if you envision a runaway train, it's speeding down the tracks, you can't pull the brake and it might crash, there might be a spill or some sort of destruction when it does finally come to a halt. Now, the thing with the runaway train is that we see it, we know it's run away, and we see the harm that it's caused. In today's runaway design world, the track's invisible. So we are building with these technologies like AI that are very, very powerful and development. And what we're building with them is speeding along tracks that we can't quite see until something has gone wrong. And it's really hard to keep up when it feels like technology is just happening really quickly and happening to us.

[Lee C. Camp]

Long before I followed a vocational path of higher ed in theology and ethics, I loved all things tech, math, physics. I was coding computers when coding was most definitely not cool. As a freshman in high school, When Radio Shack TRS 80s were cutting edge technology, I talked my way as a senior in high school into getting a solo tour of the research facilities at Oak Ridge, escorted by a high ranking member of the U.S. department of Energy. I wanted to be a physicist or an engineer or an astronaut. I loved all that stuff. In college, I majored in computer science, minored in math, and for the next 20 years I would continue to write code. But I also found I loved intellectual history, the history of ideas, the way theological and philosophical and moral constructs shape or deform, inform or obfuscate what it means to be human, what it means to pursue the common good. The theologians and philosophers began to help me see things about technology and technique that I had not yet been able to see. Perhaps some of that is most succinctly put in Thomas Merton's assertion that one may spend their whole lives climbing the ladder of success, only to discover at the end that the ladder was leaning against the wrong wall. In other words, one may do something well, efficiently, quickly, powerfully, but to an insufficient end, even a horrific end. But if you never ask the questions of why, of purpose, of what the philosophers call the telos, you won't know you've wasted a life, a culture, a society, until it's too late. We were recently invited to go out to Waco, to Baylor University, guests of the Institute for Faith and Learning, where hundreds of academics from A broad diversity of academic disciplines gathered to ask just these sorts of questions. So with the help of some of those fascinating conversations and scholars, we want to offer a modest and simple offering to the AI conversation today in part one, to issue some storm warnings, to identify some potential shipwrecks. And please note that our offering here is, as I already indicated, modest. We will not be able to raise numerous critical questions such as the swirling environmental concerns and indeed numerous other concerns. But then, coming up in part two, we want to offer some potentially fruitful practices in light of the runaway techno trains, in light of the storm tossed seas which are coming our way, or indeed have already begun. Further warnings and caution here in part one, allow me to revert to a common triad about the ways humankind is often tossed about money, sex and tools. Let's begin with tools. Frequently, well meaning commentators will employ one common metaphor when speaking of technology. The tool. Whether it's a pistol, the Internet, a cell phone, or now AI, it's just a tool is a common refrain.

[Josh Brake]

It's unhelpful in that most people think of tool as a neutral thing in the same way they think of technology as a neutral thing.

[Lee C. Camp]

That's Josh Brake. Josh is a computer scientist and professor at Harvey Mudd College. He spent a lot of time thinking carefully about the ethics of technology.

[Josh Brake]

Typically, the way that people think was like, listen, this thing doesn't have any agency or intention. Like an axe is an ax, a shovel is a shovel, a smartphone is a smartphone. It's all about how you use it. What that misses though, is that all of those things were designed in a particular way with a particular end in mind. Now, that end in mind may be opaque to you. You may not see it, but I guarantee you that the people who designed that thought very carefully about it. And the kind of antidote in some ways to that is like you need to put yourself in the shoes of the person who designed this and ask not just like, what are they telling me that this is for, but what actually did they design it for? And also what are the things that they didn't really think through in terms of the design, it's correct. A chair, a tool, an axe, a screwdriver. It doesn't have will or agency or telos. But as soon as you as a person with a will, with agency, pick it up, you have will and agency, and the design of that tool shapes your will and agency in a certain direction.

[Lee C. Camp]

In other words, it's not good enough simply to make observations about what one uses. These so called Tools for. Instead, we must keep asking the question, what kind of person am I becoming? What kind of community are we becoming? Because we have chosen to use these so called tools. Consider this simple exercise. The next time you're at a restaurant, count the tables where one or more persons seated at the table is staring at their smartphone. Then ask yourself the question, is that device really just a neutral tool that all those people are using, or is that device actually using the people at that table? What is that device doing to the culture?

[Josh Brake]

We have to be mindful of the fact that when we pick up a particular tool or technology, when we use is shaping our will and attention in a certain way. And we need to ask questions about what that way is. And then you have to realize that sometimes, and this is going to be true, I think with AI in particular. When we pick up that tool, we need to be very mindful about creating a narrow lane in which we are going to use it and trying to think, what are the ends to which I am trying to use this tool? And then asking that question with intention before we use the tool.

[Lee C. Camp]

You seem to like metaphors.

[Josh Brake]

I do.

[Interviewer/Host]

So I want to start with some of your metaphors. So talk to us about AI as an E bike.

[Josh Brake]

There's this famous Steve Jobs metaphor of the computer as the bicycle for the mind. And his point in saying that was that the bicycle is this tool. It's an instrument that actually enables humans to extend their capabilities and their capacity. They can go further just by giving them this tool that's actually powered by their own energy. It doesn't have external energy that's being pumped in there. It makes you more efficient toward the goal of moving from one point to another. And as I was thinking about AI and we just recently moved near Harvey Mudd in Claremont. And one of the things I love most about Claremont is it's extremely bikable. But we have three kids and to get them around is a little tricky.

[Lee C. Camp]

Right.

[Josh Brake]

I got a cargo E bike earlier this year which I love. Can put the two older kids on the back on the seat. And then I don't think they really encourage this, but you can attach the trailer to the back of the E bike too, and then tower up to two more. But anyway, this is like a great way to get around town. And I think that it's interesting we have all this conversation about E bikes in our news and public health kind of hazards with teenagers getting E bikes and driving them around. There's all this controversy in New York City around Central park about E bikes and the way that they're whizzing around. I've ridden an E bike in Central Park. I know what they're talking about. But what's interesting to me is I think that AI in many ways can be like an E bike for the mind. And the kind of analogy that I want to draw there is that the difference between the bicycle and the E bike is that the E bike actually adds energy in. So it's no longer just the human propelling it. It's actually you have this battery boost along with it, right? You've got an electric motor to help you go. And I think, especially as we think about AI and the way that we're applying it, the analogy of the E bike for the mind actually illustrates for me the promise and peril of these sorts of things. So, on the one hand, you have teenagers, which, given E bikes, you can extend essentially dangerous or harmful behavior, right? Any sort of technology has affordances which actually enable greater evil to occur. But at the same time, you actually create the possibility for somebody like me, instead of having to drive around Claremont with my kids in the minivan. Now we can take the E bike, which is better for the environment, it's better for me, better for the kids. Like, it's just all around, a better way of getting around. And I think that it's this interesting sort of tension that we have to walk with technology, and especially as we're thinking about artificial intelligence, is to say, how do we balance the particular affordances, the new things that technology enable for us, understanding that those things always come alongside with new potential for harm and benefit. And it's about this kind of discernment process. And as we get to more and more powerful technologies that spread between the good and the bad opportunities is just going to get wider. The potential for good and the potential for evil continues to get bigger and bigger.

[Lee C. Camp]

So while the notion that AI or tech is simply some sort of neutral tool seems unhelpful, there is something about the metaphor of tool that is quite helpful at this point in human history. But before I lay out what I suspect that is, let's do a little philosophy of mind.

[Joe Vukov]

The Turing Test. We call it the Turing Test because it's due to this guy, Alan Turing. He's basically the father of computer science.

[Lee C. Camp]

That's Joe Vukov. Joe is a philosopher at Loyola University of Chicago, where he works at the intersection of ethics, human dignity, technology, and religion. Joe is also the author of Staying Human in an Era of Artificial Intelligence,

[Joe Vukov]

roughly what the Turing Test is is it's a test to try and figure out whether or not machines can think. And the way he puts it is not whether they're actually thinking, but something like as good as thinking. The rough idea is that you get a computer, and if the computer can respond to a human in a way that's indistinguishable from the way a human would respond to another human, then he says, well, that's just as good as thinking.

[Lee C. Camp]

If a human cannot tell the difference, cannot tell whether they're talking to a person or a computer program, then Turing says the machine is, for all practical purposes, thinking. Philosophers would call this kind of thinking functionalism. In other words, it's all about inputs and outputs. It's not, in other words, about what happens in the space in between the inputs and outputs. Now, clearly, today's LLMs have blown past the Turing test. But just because an AI can fool us can make us think, it's thinking. What does that really mean? Does it understand anything? Does it know anything? Well, consider this thought experiment.

[Joe Vukov]

Suppose that you get this job, and it's a very strange job. You're assigned to this room, kind of like actually the room we're sitting in right now.

[Lee C. Camp]

At one end of the room, there's a mail slot, and scattered around the room are pieces of paper on which are drawn all sorts of different squiggly shapes. Then, every so often, a sheet of paper comes through the mail slot, and

[Joe Vukov]

on it just a whole bunch of squiggles. On one wall is this gigantic flowchart, and on the flowchart, it's got all these squiggles depicted.

[Lee C. Camp]

Your job is to find the squiggle on the piece of paper just delivered through the mail slot on the flowchart. The flowchart will direct you as to what piece of paper scattered around the room you should pick up and return to send back out through the mail slot.

[Joe Vukov]

This flowchart's pretty complicated. So sometimes you get a squiggle, and it'll tell you, if you get this one, you push this other one out. Other times it will say, oh, you get this squiggle. You can choose between two or three of these other ones, and you put it out on the outslot. Now, let's say that the benefits at this job are really good. So you stick around. You do it for your whole career. You're 50 years in at this point. You've basically memorized the flowchart. You don't even have to look. You know exactly which squiggly paper you grab. You Throw it out the out slot. Here's the twist. What you don't know is that what they actually were was not merely random squiggles. They were actually Chinese characters. What the flowchart was in turn was a depiction of how Mandarin works. So it says, oh, if somebody sends you in a sheet of paper that says, how are you? You can say, I'm fine, or maybe I've had better days. Maybe there's some options that you can have. But that's what the flowchart's depicting. And then, of course, these slips of paper on the ground that you're sending out are the responses that you can give to what's being said to you on the way in. What Searle wants to argue is that at this point you are functionally identical to a native Chinese speaker. But of course, built into the whole thought experiment is you don't know Mandarin.

[Lee C. Camp]

In other words, the Turing Test has been passed. From the perspective of the outside, whatever or whoever is inside the mail slot appears to fully comprehend what you're saying. But it doesn't have the foggiest idea. It's worth noting that Searle's Chinese room is a contested thought experiment. Some push back and say even if the person in the room doesn't understand Chinese, maybe the system as a whole does. But what's crucial to understand here is how well this thought experiment helps us understand the mechanics of how large language models work. We write to a chatbot and we're amazed, sometimes genuinely moved by what comes back. How does it know? How does it understand that it does not know? And this is where the metaphor of tool is actually quite helpful. The LLM understands you, understands me in the same way my crescent wrench and hammer understand me. That is, none of them understand me in any meaningful sense of that word. Like the man in Searle's room, a large language model does not know why it's producing the response it is producing. It doesn't know what the response means. What it can do extraordinarily well is predict which words are likely to follow which other words, but based on patterns learned from an almost incomprehensible amount of text and data on which it has been trained. But don't forget, the man in the room is just sorting squiggles. He's just gotten very, very good at it. And also, don't forget, so far, as a chatbot, a so called AI, there is not even a man inside the room. It's just a machine, an incomprehensibly complicated one. But don't forget it's a machine, a prediction machine. Sorting our squiggles. Recently, I had a conversation with a leading figure at one of the world's largest tech companies. At one point, I asked him what he thought it meant to be human in the age of AI. He said that we humans were going to have to give up our presumed specialness.

[Carissa Carter]

We.

[Lee C. Camp]

We're not so special. He indicated, look at what these machines can do, the way they exceed our rational capacities. But there's a deep irony here. He seemed to presume that what makes humans special is measured in terms of the Turing Test. But this just strikes me as, well, weird. It assumes that if a machine can perfectly mimic input and output, then it has captured the essence of humanity, that humanity is really nothing more than a biological machine supporting an input output computer. I mentioned this to Joe, and this is what he had to say.

[Joe Vukov]

We have such an intellectually heavy way of approaching anthropology today. If you want to know about humans, who are you going to go to? Go to philosophers, go to psychologists, go to neuroscientists, go to computer programmers, maybe? That's really interesting, I think, and indicative of the way that we as a whole have started thinking about what it means to be human, what it means to be intelligent. I think it's really telling that you would very rarely say, you want to know what it means to be human. Go talk to an athlete, go talk to a physical therapist, go talk to a ballet dancer.

[Interviewer/Host]

Talk to the postman.

[Joe Vukov]

Talk to the postman.

[Lee C. Camp]

Talk to the trash man.

[Joe Vukov]

Exactly. Clearly they have insight into what it means to be human. But we've so reduced human anthropology to abstract intelligence and the sorts of things that computers can do that we've just sidelined all these other aspects of human intelligence and in fact, just being human. I mean, add to the list here, think about relational stuff you wouldn't think. You want to know what it's like to be human. Go talk to someone with a lot of friends. Go talk to somebody who's a social butterfly.

[Lee C. Camp]

Go talk to an athlete, to your postman, to the social butterfly. That sort of reorientation matters enormously because the version of humanity that AI is presumably replacing or supplementing is a dangerously narrow vision of what it means to be human. But if AI were merely a neutral tool, none of this would matter very much. But it is not a neutral tool. These systems are designed in ways that direct our attention, shape our habits, and cue us to respond as if we were encountering something more than a machine. Here's Josh Break again.

[Josh Brake]

It's hijacking all of this very deeply developed machinery in ourselves. The way that our brains at work and the way that we as human persons are to relate to it as if it were a person because it is using certain language, it's co opting this certain front, it's taking a certain appearance. That is a deceptive appearance.

[Lee C. Camp]

The word deceptive there, I think is

[Interviewer/Host]

really important because I am not finding many people that will use the language of deception with regard to the way we interface with these things. But do you think that's a fair way to put it?

[Josh Brake]

Deception, I think, implies some sort of intent.

[Interviewer/Host]

Will yeah, which is not there.

[Josh Brake]

Which is not there, but it's there by the designers. That's what I would say. The designers have designed it to be intentionally deceptive because it makes it stickier and it makes it easier to use.

[Lee C. Camp]

This hijacking of our relational system leads us to Act 2 sex coming up after this break, a special thank you to the Institute for Faith and Learning at Baylor University for helping make this series on AI possible. I love hearing from you. Tell us what you're reading, who you're paying attention to, or send us feedback about today's episode. You can reach me@leeowsmall endeavor.com you can get show notes for this episode in your podcast app or wherever you listen. These notes include links to resources mentioned in the episode and a full transcript. We'd be delighted if you tell your friends about no Small Endeavor. Invite them to join us on the podcast because that helps extend the reach of the beauty, truth and goodness we are seeking to sow in the world. And we'd love for you to join us on the NSC Notebook, our free weekly newsletter. You can sign up for that newsletter on our [e-mail] Coming up, AI, sex and desire.

[Bruce (from Drilled Carbon Cowboys)]

I've been hearing for decades that the markets can solve climate change. Today we have more incentives for market solutions than ever. And emissions are rising on this season of Drilled Carbon Cowboys, the story of three market solutions colliding in one multinational old boondoggle.

[Lee C. Camp]

Got to give Bruce and the guys credit. They're Republicans.

[Chamath Palihapitiya]

They don't give a about any of this stuff.

[Bruce (from Drilled Carbon Cowboys)]

Listen anywhere, you get podcasts.

[Lee C. Camp]

Welcome back to no Small Endeavor, exploring what it means to live a good life. I'm Lee Se Camp. This is episode one of our two episode series entitled the Human Cost of AI. Before the break, we considered the claim that AI is not just a neutral tool lying there in our hands. Its design shapes our habits Our expectations and even our sense of what it means to be human. So the question is not simply whether AI will save us or destroy us, but what kinds of shipwrecks it may already be making possible. Which brings us now to a second storm on the horizon. Sex.

[Rosalind Picard]

I think it is horribly dangerous. We're seeing in the movies them now showing these companion bots that replace wives and girlfriends. There's some supposedly replacing boyfriends, but the big market is in the other direction.

[Lee C. Camp]

That's Rosalind Picard. Rosalind is an esteemed professor at MIT and founder of the Affective Computing Research Group. She recently won something like the Nobel Prize in computing. She spent her career at the intersection of human emotion and technology. And I spoke to her in front of a live audience at Baylor.

[Rosalind Picard]

I heard a comment recently from a person who said, oh, well, they won't be as bad as porn because they don't use real women. They're not gonna harm women in the same way. And I thought, oh no, I think they're gonna harm women even more because

[Lee C. Camp]

now they're not just dealing with unrealistic photoshopped beauty, but AI agents that are,

[Rosalind Picard]

by the way, tuned to say what attracts you and grows millions of followers and makes you wanna be like them?

[Bruce (from Drilled Carbon Cowboys)]

And.

[Rosalind Picard]

But there's no them, right, to be like. There's a team of people behind them. Now what's going to happen when that moves to the physical robots and every guy in your public high school shows up with this life size Barbie on his arm and no girl can compete with her right in terms of appearance. No girl can give the guy everything his adolescent mind wants. And no girl should give the guy everything his adolescent mind wants. And there's no pushing, there's no friction, there's no good friction. There's the fact that he can control this thing, get everything he wants from it. It will love everything he says. It will make him feel important, it will satisfy his physical fantasies, and then at the end of the day, he can put it in a box and treat it like an object, and then he can look at other women like objects.

[Interviewer/Host]

Yeah, I mean, it seems to me that in the scenario you're describing, the harms to young women would be obvious. But the way you concluded the story, the harm to young men is obvious as well. Immense harms as well, on both sides, perhaps of different shape.

[Rosalind Picard]

Right, right. You know, when we make relationships easy, it's really not a good thing for us.

[Lee C. Camp]

So while robot girlfriends wandering the halls of your kid's high school might seem a Long way off. Get this. According to New York magazine, a recent study found that one in five American adults have already used a chatbot to simulate a romantic partner. One in five. What could possibly go wrong? Consider this scenario. A lover is duped by one he believes to be faithful. She's a bad actor, telling him everything he wants to hear, giving him everything he wants. And when the time comes, she plays him, revealing that what he took for devotion was strategy. What he mistook for intimacy was just performance. He's not merely disappointed, he's undone his trust, desire, even vulnerability have been studied, manipulated toward another's ends. The deepest wound is not simply that he was lied to, but that something meant to be mutual and beneficial was mere exploitation. But the scenario before us is more astonishing still. We are no longer speaking of a single betrayal, but a betrayal mechanized, a pseudo intimacy that mimics tenderness. It harvests longing. It's affection without affection, it's devotion without freedom. It's all calibrated to extract dependency and profit. And the threat is deeper even still. This sort of dependency is training the human to desire its own bondage. And meanwhile, the machine does not know it's exploiting you. But the teams building the machines might. They know full well what they are doing. So the point here is not some moralistic finger wagging about porn and sex. The point is that sex is one of the most powerful experiences and indicators of human desire. It's a question of how we are formed or allowing ourselves to be formed, to be in so called relationship. With machines.

[Rosalind Picard]

It's a simulation of a real thing, like suspension of disbelief. We do that, you know, all the time with the movies and the story. And we kind of enter that fairy tale if we enter it on our own volition for a purpose of entertainment or, you know, I really do have this problem getting my exercise routine started. And this thing is helping me. I think there's space for that to be useful. There's also huge and growing space for it to be harmful. There are huge incentives where people are lonely, people aren't for some reason confident making their own relationships and this is not helping them.

[Lee C. Camp]

I know when I walk into a movie theater that I am willingly suspending disbelief for the sake of the story. But the LLMs actually encourage me in various ways to actually believe the story that is not, not true, that a relationship is occurring. Let's go back to my conversation with computer scientist Josh Brake.

[Josh Brake]

Because these tools are fundamentally engaging with language. They're built around large language models and the design of these things to make them easy to use, for people to easily interact and learn how to use them. We've relied on a chat interface and this kind of personalization layer that makes it very natural and easy to interact. But what we don't realize that this is actually smuggled in a certain set of expectations about how we might use the technology, how we might treat it.

[Interviewer/Host]

Yeah, I really didn't start using any sort of AI models until the last probably 12 months. But one of the first questions that occurred for me was whether or not I was going to be polite.

[Josh Brake]

Yep.

[Interviewer/Host]

I actually thought, well, I'm going to have a conversation with ChatGPT about this and just see what happens. And so I basically said, I prompted something like, I'm considering whether or not to use polite terms like please. Thank you. Do you have a preference?

[Joe Vukov]

Yes.

[Interviewer/Host]

It came back and said, no, I do not, because I have no subjective experience.

[Josh Brake]

Yeah, right.

[Interviewer/Host]

And so it does not matter to me.

[Josh Brake]

Yeah, I have no subjective experience. Man, there's a whole.

[Interviewer/Host]

That's bizarre.

[Josh Brake]

Yeah, that's bizarre.

[Interviewer/Host]

But then I replied and I said something like, well, I think I am going to use that kind of language not for you, but for me, because I want to train myself in respect in whatever context. More recently, I have found myself less willing to do that because I want to keep reminding myself this is not a subjective entity.

[Josh Brake]

If we should know anything from what we're seeing from Big tech is like, they know a lot more than they're letting on to. And this is not necessarily an argument for or against big tech, but just to say, like, their goal is to make the thing stickier and they know that one of the best ways to do that is to hack your psychology. So much of the attention economy was hijacking the dopamine system. And now what we're seeing in the next generation of relational AI bots is that it's hijacking serotonin attachment. It's a different neural pathway, but it's actually going to be potentially way worse for us.

[Lee C. Camp]

Which leads us to money and, yes, power. Right after this break, Special thank you to the Institute for Faith and Learning at Baylor University for helping make this series on AI possible. Before the break, we asked what happens when machines are designed to hijack desire? Now a follow up question. Who is building systems like these and to what end? Because behind the fantasy of artificial intelligence lies something much older and less mysterious. Money and power.

[Rahman Chowdhury]

One of the worst smokescreens that has put up is this idea that AI will somehow gain sentience and kill us.

[Lee C. Camp]

All.

[Rahman Chowdhury]

And I cannot think of a worse narrative because none of it is even has a shred of empirical evidence.

[Lee C. Camp]

That's data. And social scientist Dr. Rahman Chowdhury, she's the co founder of the nonprofit Humane Intelligence, and she says, if you want to understand the real threat of AI, stop looking at science fiction.

[Rahman Chowdhury]

Everything is about systems of power and institutional incentives.

[Lee C. Camp]

This is a lesson that Chowdhury knows all too well. And she saw it play out firsthand in her previous position as director of the Machine Learning, Ethics, Transparency and Accountability team at Twitter before its 2022 takeover by Elon Musk. And whatever your political point of view, the financial realities behind that acquisition are crucial.

[Rahman Chowdhury]

At Twitter, we painfully learned that fiduciary responsibility was more important than impact on democracy in society. I think everybody understood very, very clearly what it meant for Elon Musk to take over Twitter. I'm not saying anything that he has not said that he would do himself. Right. He wants to control public narrative. He wants to influence what the AI models that he's building on this content says. So he is actually quite explicit about tipping the scale in the direction that he wants it tipped in. Right. And that hand was forced because of fiduciary responsibility. So he offered more to buy the company than the company was worth. And because the economic construct of for profit companies says that your responsibility is to your shareholders, meaning the people who gain to make or lose money from you. What the board said is that given this fiduciary responsibility, we have to take this deal, even though we can all agree that this person's only going to do terrible things with his platform once he has it. And that was, I think it broke everyone's heart a little bit because we all believed and we did felt like we had a higher mission, but that was culture, remember?

[Lee C. Camp]

At that time, Chowdhury and her team were responsible for ethics, transparency and accountability at Twitter. Their job was to make Twitter's algorithms more equitable and transparent, to weed out, to the degree possible, race, gender, political bias. Once the company was acquired, Rahman's department was simply shut down.

[Rahman Chowdhury]

People seemed surprised. Every time a for profit organization chooses profit over people, at the end of the day, they are responsible to their shareholders. They are not only incentivized, they're legally required to act in their own best interest.

[Lee C. Camp]

Legally required. Not villainous, not even necessarily careless. Just operating exactly as the economic structure demands. And as AI models become more sophisticated, they consume more capital, increasing the demand for return on investment. The economic model applies more and more pressure. Let's Go back to my conversation with MIT professor Rosalind Picard. You once said to see the most likely path of AI's future. Answer the question, what satisfies human desire and increases profits? That's terrifying to me. Tell us more about that.

[Rosalind Picard]

Yeah, yeah. Thank you. Not only when you start a company do you have to pay your employees and provide health insurance and all of that. You eventually have to stop losing money to stay in business. If you take investment to get your business started from certain kinds of people who want like a 20x return, then you are in debt for that kind of return. And that's what we're seeing right now with the really big tech AI companies. They have taken tens of billions in investment, their in debt, building these huge data centers and they have to make that Money back times 20x is what the investors are expecting. So people are asking like where is that money going to come from? So that's a reality you have to understand with for profit companies. What gives me hope is I still see some very good people in those companies who are doing the day to day work and they're struggling with it and they're trying to make it better. Let me give an example. One company that everybody here I'm sure has heard the name of their head of their AI safety, reached out and said, we are tasked with making the AI safe. We also are tasked with making it more engaging. Which means we want to keep you talking with it. We want to have usage minutes go up. Our boss says we have to increase engagement, but we, the workers in this company want to promote human flourishing. How do we do that? So the people inside are really struggling with this and looking for answers.

[Lee C. Camp]

But this struggle is not new in the world of tech. Enter Garrett Graff. Graff is a journalist and host of the podcast Long Shadow. His recent season entitled Breaking the Internet, traces 30 years in which the Internet moved from civic promise to the attention driven economy we know today. In the early years it capitalized pro democracy movements, facilitated the Arab Spring. So how did social media evolve into a rage baiting machine?

[Garrett Graff]

A very simple business decision that these tech companies make is they are not going to be user revenue driven, they're not going to be subscription based. As a Facebook user you pay nothing for the service. And so what that means is you're the product and that you are the thing that Facebook is selling to someone else. Facebook turns to this advertiser driven model, which means they need people to stay on the website for as long as they can in order to sell as many Ads as they can. What they come to determine very quickly is content that makes us angry, content that we don't like. Seeing content that enrages us actually makes us stick around longer. That people are more likely to engage with and share and react to and comment on and watch content that enrages them. Over time, these algorithmically driven news feeds on social media websites become incredibly sophisticatedly curated. And at one point, Facebook is waiting in its algorithm the equivalent of the dislike button, five times as powerfully as they are rating and waiting, the like button. So if you see something on your Facebook news feed that you don't like and tell Facebook that you are sort of five times more likely to see more content like that than you are something that you actually want to see to, you know, that makes you happy and joyful and kittens and unicorns and rainbows.

[Lee C. Camp]

Five to one, outrage over joy. Not because anyone decided to make the world angrier, but because anger kept people on the platform longer and longer meant more ads and more ads meant more revenue. It seems the road to a fractured society was paved with quarterly earnings reports. Garrett shared a piece of tape he calls haunting. It's from a Q and A at Stanford's Graduate School of Business. A tech billionaire former Facebook executive named Chamath Palihapitiya, reflecting on what he helped build.

[Chamath Palihapitiya]

I feel tremendous guilt. I think we all knew in the back of our minds we kind of knew something bad could happen. But I think the way we defined it was not like this. It literally is a point now where I think we have created tools that are ripping apart the social fabric of how society works. That is truly where we are. And I would encourage all of you as the future leaders of the world to really internalize how important this is. If you feed the beast, that beast will destroy you. If you push back on it, we have a chance to control it and rein it in. And it is a point in time where people need to hard break from some of these tools and the things that you rely on. The short term dopamine driven feedback loops that we have created are destroying how society works. No civil discourse, no cooperation, misinformation, mistruth. And it's not an American problem. This is a global problem. So we are in a really bad state of affairs right now. In my opinion. It is eroding the core foundations of how people behave by and between each other. And I don't have a good solution. You know, my solution is I just don't use these tools anymore.

[Lee C. Camp]

A Facebook executive in 2017 describing exactly what his own platform had been doing to the fabric of society. And what happened after that confession? What guardrails went up, what legislation passed? Garrett noticed something about the tech world in the years that followed.

[Garrett Graff]

The number of Silicon Valley tech executives who will brag like, oh, my kids aren't allowed screen time. I won't get them a phone or let them have social media accounts, you know, in until they're in their mid-40s, kinds of things. You wouldn't ever see that in any other industry. There's sort of something about the weirdness of how tech executives have come to understand the damage that their tools can do when used as intended that I think makes them much closer to being the sort of cigarette executives of our modern era.

[Lee C. Camp]

Cigarette executives. They knew they kept manufacturing, they kept marketing, kept selling, and presumably they would have kept their own children away from the product. And now, Garrett says, we're watching it happen again with higher stakes, at greater speed, and with even less governance in place to slow it down.

[Garrett Graff]

Government has just utterly failed in appropriately understanding or regulating this era of technology. It takes five years on average for a piece of legislation in Congress to go from introduction to passage and enactment. And I don't think we can underestimate this. The gerontocracy that we have seen sort of overwhelm American politics. Most of the people who are making laws about our technology in Congress, most of the people who have been in the White House in the last 20 years have never used any of this technology natively. And you know, you have members of Congress asking Mark Zuckerberg, how do you make money? And he's like, we sell advertising, which is like literal news to the members of Congress on the committee that is supposed to be regulating Facebook.

[Lee C. Camp]

The cavalry, in other words, is not coming to save us. Especially when the ramping up of AI is like injecting steroids into the threats of the last decade. In the absence of effective regulation, we seem to be relying on the good conscience of people hired by the tech companies themselves to keep us safe. And though nonprofits like Rahman Choudhary's Humane Intelligence are working hard to fill the gaps, it does not take a genius to recognize that there's a deep conflict of interest at the heart of the AI revolution. And according to Choudhury, everyone from the tech industry investors to the CEOs and the well intentioned people working within the system to keep AI accountable is vulnerable to a phenomenon that she calls moral outsourcing.

[Rahman Chowdhury]

The concept of moral outsourcing was something I really Observed when I joined Accenture as a responsible AI lead, and I saw the way engineers, and actually in general media would talk about AI models. There's this type of language we use that we've never used before, but we anthropomorphize the technology and we say things like, AI will replace teachers, or AI diagnoses disease better than a doctor. And I noticed three things about that language. First is that it gives this intent or will to an AI system that's not real. Which is why today we see people suffering from AI psychosis, people who believe AI is alive and it is exerting free will. So that's problem one, which directly stems from the linguistic construction, the part number two. And this very much aligns with the banality of evil, which is basically this idea that many people who participate in evil regimes, and specifically the Nazi regime, did not see themselves as contributing to the evilness of that regime because they felt like they were doing a mundane job. And in looking at the trials of a lot of these Nazi soldiers, they would say things like, I just signed paperwork or I just drove a truck. Yeah, you drove a truck of gas canisters to a concentration camp. What did you think you were doing? And we see that same thing, like, as people are complicit in building systems of mass surveillance discrimination, they don't believe that they're complicit in it. And part of that is again, because of this anthropomorphizing language. How convenient. Then if something goes wrong, you can say, oh, wow, the AI really did something. And again, we see that language today as we're building agentic AI systems. Literally just yesterday I saw a headline where some company's entire database was deleted by an agentic AI system. But again, instead of identifying it as poor user specification or this company not putting sufficient safeguards to prevent these things from happening, it was like, oh, the AI deleted a company's file. Like, the AI doesn't do anything of its own will. It did not intend to do it. The third thing that's very insidious about sentences like that is it completely removes human beings, you and I, from the process. So here is an AI and it is taking jobs away from teachers. So these teachers are just subject to this all knowing completely better than us, never needs to sleep. This being that is super intelligent, what are we to do? So for the average person, it is frightening and it is alienating because in those sentences, you are just subject to the whimsical of this AI system instead of what it really is, which is, again, systems of power operating in a very particular way.

[Lee C. Camp]

So what are we to do as the average person moving through a world that feels increasingly beyond our control? How do we remain accountable inside a system that is engineered to engage us in the most frictionless and seamless ways possible? Next week, a few modest proposals coming

[Interviewer/Host]

your way in Part two.

[Lee C. Camp]

You've been listening to no Small Endeavor and Part one of our special series on AI. A special thank you to the Institute for Faith and Learning at Baylor University for helping make this series possible. We gratefully acknowledge the support of Lilly Endowment Incorporated, a private philanthropic foundation supporting the causes of community development, education and religion. Our thanks to all the stellar team that makes this show possible, Christy Bragg, Jacob Lewis, Carried Harmon, Jason Cheesley, Sophie Byard, Kate Hayes, Mary Evelyn Brown, and Audrey Griffith. Our theme song was composed by Tim Lauer. Thanks for listening and let's keep exploring what it means to live a good life together. No Small Endeavor It's a production of Tokens Media, LLC and Great Feeling Studio. Hello friends, Lee C. Camp here again. If you enjoyed this episode, check out Part two over on the no Small Endeavor feed and follow no Small Endeavor on Apple Podcasts, Spotify, or wherever you're listening now.

[CBC Announcer]

For more CBC Podcasts, go to CBC CA Podcasts.