The Trust Engineers

Summary

Radiolab: "The Trust Engineers" (February 10, 2015) – Episode Summary

Main Theme & Purpose

This episode of Radiolab dives into the inner workings of Facebook’s “Trust Engineering” team (later renamed “Protect and Care”), focusing on their attempts to make conflict resolution and emotional communication smoother on the platform. The story traces how Facebook responded to the ever-growing number of reported photos by building new tools—based on behavioral science and language analysis—to nudge users toward more understanding, empathetic digital interactions. The episode explores both the promise and the profound ethical questions of using “trust engineering” at unprecedented social scales.

---

Key Discussion Points & Insights

1. Facebook’s Scale and Problem of Reports

• Facebook’s Massive Presence: As of March 2014, 1.3 billion active monthly users (03:07), more than the number of Catholics globally, leading Jad to say, “That can’t be true!” (03:07)
• Photo Reports Explosion: Post-Christmas 2011 saw more images uploaded to Facebook in a week than had ever been uploaded to Flickr in total (04:11). With photos came millions of “reports” for flagged content.

2. The Mismatch Between User Complaints and Actual Violations

• 97% Misreported: Most photo reports were mismatches, e.g., family photos flagged as harassment, puppy pictures flagged as hate speech (05:58), leading to confusion:

  “Pictures of puppies reported for hate speech.” – Jad Abumrad (06:20)

• Emotional, Not Policy-Driven, Complaints: Most reporters were people appearing in the photos who simply disliked the image—exes, embarrassing party pics, even sadness over shared pets:

  “It was maybe a shared puppy…maybe it’s your ex wife’s puppy.” – Arturo Bejar (07:10)

3. Investigating Emotion and Language in Reports

• Adding Emotional Context: A new “How does this photo make you feel?” box yielded more accurate information. Users selected pre-written emotional states like “embarrassing,” yet even when “embarrassing” was listed, many wrote “it’s embarrassing” (08:30).
  • Changing the phrasing to “it’s embarrassing” increased usage by 28%—a subtle linguistic cue shifting responsibility from self to subject:

    “It’s always good to mirror the way people talk… when you just say ‘embarrassing’… it’s silently implied that you are embarrassing. But if you say ‘it’s embarrassing’, well, then that shifts the sort of emotional energy…” – Jad Abumrad (09:18)

4. Engineering Conflict Resolution: Prompts and Humanization

• Initiating Dialogue: Facebook wanted conflicted users to talk, but when prompted via a blank message box, only 20% actually sent a message (11:03).
  • By providing a default, pre-written message (“Hey, I didn’t like this photo. Take it down.”), the response rate soared to 50% (11:39).

    “We weren’t expecting to see that big of a shift.” – Arturo Bejar (11:44)

  • Including the recipient’s name (“Hey Robert, I didn’t like this photo…”) added another 7% effect (12:22).
  • Softening language (“Would you please take it down?”) outperformed blunter or more apologetic approaches (“Would you mind…”, “Sorry to bring this up…”). Use of “sorry” reduced positive outcomes:

    “Turns out the ‘I’m sorry’ doesn’t actually help. It makes the numbers go down.” – Arturo Bejar (15:03)

5. The Science of Nudging and Ethical Dilemmas

• Massive Social Experiments: Facebook routinely runs experiments on users – “any given person is probably currently involved in… 10 different experiments.” (17:14)

  “I’ve been a research subject and I had no idea.” – Andrew Zolli (17:25)

• The “Emotional Contagion” Controversy: A public outcry followed news that Facebook tweaked news feeds to research emotional spread online (impacting 700,000 users) (19:16). The criticism raised issues of consent and manipulation.

  “There is a power imbalance at work…highly centralized power and highly opaque power at work.” – Kate Crawford (20:50)

6. Facebook’s Power to Shape Social and Civic Behavior

• 2010 “Get Out the Vote” Experiment: Facebook increased voter turnout by an estimated 340,000 people via social nudges (21:50), showing real-world consequences and prompting concerns about democratic influence:

  “That is a profound democratic power that you have.” – Kate Crawford (22:58)

7. Debate: Engineering Compassion or Overstepping?

• Critics’ Concerns: Some, like Kate Crawford, argue against the “hubris” of engineering emotional or moral outcomes at massive scale (23:28), warning about undemocratic influence.
• Defensive Perspective: Arturo Bejar argues that these interventions are a response to real requests for help (“all of the work…begins with a person asking us for help” [25:30]), not arbitrary manipulation.

8. Digital Communication, Social Norms, and Analogy to Other Technologies

• Missing Non-Verbal Cues: Online, lacking body language or tone creates new challenges in conflict and empathy (26:03).
• “Automobile Age” Analogy: Like traffic systems, digital interaction needs new “tools” and scripts to avoid crashes and enable coexistence (30:22).

  “We created turn signals so we can coexist in this great flow without crashing into each other.” – Andrew Zolli (30:43)

---

Notable Quotes & Memorable Moments

• “Puppies reported as hate speech.” – Jad Abumrad (06:20)
• “It’s always good to mirror the way people talk.” – Jad Abumrad (09:18)
• “Turns out the ‘I’m sorry’ doesn’t actually help. It makes the numbers go down.” – Arturo Bejar (15:03)
• “I’ve been a research subject and I had no idea.” – Andrew Zolli (17:25)
• “There is a power imbalance at work. Highly centralized power and highly opaque power at work.” – Kate Crawford (20:50)
• “That is a profound democratic power that you have.” – Kate Crawford (22:58)
• “Who are we to decide whether we can make somebody more compassionate or not?” – Kate Crawford (23:28)
• “We really care about the people who use Facebook…you really have to respect people’s response and emotions, no matter what they are.” – Arturo Bejar (24:11)
• “In the absence of [nonverbal] feedback, how do we communicate? What does communication turn into?” – Arturo Bejar (26:03)
• “We created turn signals…so we can coexist in this great flow without crashing into each other.” – Andrew Zolli (30:43)
• “Maybe they’re conversation starters. Maybe that would be a beginning.” – Jad Abumrad (30:13)

---

Timestamps for Important Segments

• 01:39: Beginning the story; introducing Facebook’s dilemma
• 04:11: Data: More post-Christmas photos on Facebook than all of Flickr ever
• 05:58: Discovery of mass miscategorization in photo reports
• 07:10: User motives: relationship and emotional fallout drives reports
• 08:30: Nuances of emotional language—“It’s embarrassing” effect
• 09:42: Linguistic psychology in interface design
• 11:03: Blank text box doesn’t work; prewritten messages do
• 12:22: Impact of personalization (“Hey Robert…”)
• 14:00: Weekly “trust engineer” meetings and the power of data-scale
• 17:14: Most users are simultaneously part of 10 Facebook experiments
• 18:59: Public backlash to Facebook’s “emotional contagion” experiment
• 21:50: Facebook shifts 340,000 votes with a simple UI tweak
• 23:28: Critics challenge the idea of engineering compassion
• 25:30: Arturo Bejar defends user-request-oriented interventions
• 26:03: Communication without nonverbal cues online
• 30:22: The analogy: Building social “road rules” for digital life

---

Overall Flow and Tone

• The hosts blend curiosity, skepticism, and humor.
• Arturo Bejar is depicted as earnest, thoughtful, and nuanced in his mission to make digital communication healthier.
• Robert Krulwich frequently acts as the skeptic, questioning whether “engineering compassion” is possible or desirable.
• Kate Crawford provides a sharp critical perspective, warning of the risks in private companies quietly shaping democracy and emotional life.
• The conversation moves with pace and clarity, using real world analogies and tangible data for illustration, while retaining Radiolab's signature storytelling vibrancy.

---

Usefulness for Non-listeners

If you’ve never heard this episode, this summary captures:

• The fascinating “invisible” work done behind the scenes of social platforms,
• The unexpected psychological subtleties of online communication,
• The potent, sometimes unsettling reach of private tech companies into the fabric of everyday emotions, relationships, and even democracy.

You’ll get both a look at how Facebook engineers nudge user behavior, and a sharp discussion about the broader implications of those nudges—ranging from the language of kindness to the ethics of massive-scale A/B testing on a digital population.

---

For more context and related research, visit radiolab.org.

Transcript

[Jad Abumrad]

Your new home is now ready. Dr. Horton, America's builder, has new homes that are ready today with new construction communities in Ellensburg and throughout the Greater Seattle area. Dr. Horton has the right home for you. At Dr. Horton, we're still building with flexible living spaces, smart home technology, and two and three car garages. More communities and more homes available every day. Find your new home in Ellensburg now ready@dr. Horton.com Dr. Horton, America's builder and equal housing opportunity builder. Horticultural when the flu is keeping you.

[Andrew Zolli]

Up at night, don't try to tough it out.

[Jad Abumrad]

Knock out your flu symptoms with NYQUIL Intense Flu. You got this. It provides powerful relief of your flu symptoms so you can sleep well through the night. Nyquil Intense Flu.

[Arturo Bejar]

The nighttime Sniffling, aching, aching fever.

[Jad Abumrad]

Best sleep with a flu medicine. Use as directed.

[Arturo Bejar]

Keep out of reach of children.

[Andrew Zolli]

Lowe's early Black Friday deals are going fast. Don't miss. Up to 50% off.

[Jad Abumrad]

Select major appliances.

[Andrew Zolli]

Plus up to extra 25% off when you bundle. Select major appliances. And with Christmas around the corner, you're.

[Jad Abumrad]

Gonna need more string lights, right?

[Andrew Zolli]

Save $4 on GE LED 100 count string lights. Now just 5.98 Lowes.

[Jad Abumrad]

We help you save. Valid through 12.

[Andrew Zolli]

3. Selection varies by location. Select locations only while supplies last. See Lowes.com for more details.

[Jad Abumrad]

Wait, you're listening.

[Kate Crawford]

Okay.

[Jad Abumrad]

All right. Okay. All right.

[Arturo Bejar]

You're listening to Radio Radio Lab from WNYC.

[Jad Abumrad]

And npr. All right. Hey, I'm Jad Abumrad.

[Robert Krulwich]

I'm Robert Krulwich.

[Jad Abumrad]

This is Radiolab, the podcast. So here's a story we've been following for a while.

[Robert Krulwich]

Yes.

[Jad Abumrad]

Comes from a friend of mine, Andrew Zolli, who is a great thinker and writer. He wrote a book called why Things Bounce Back, and he's a guy who thinks a lot about technology.

[Andrew Zolli]

I have been interested for a long time in the relationship between technology, technology and emotion because, well, I've thrown more than one cell phone to the ground.

[Jad Abumrad]

Andrew and I were having breakfast one day, and he pitched me on this idea of doing a story about Facebook. I'm not a huge believer in doing stories about Facebook, but this story was wickedly interesting and profound in its way. So he and I have been following it for a couple years, up and down through this roller coaster of events. It really begins in 2011.

[Andrew Zolli]

Well, let me back up for a minute. One of the challenges talking about Facebook is just the scale of the thing. So, you know, there's 1.3 billion people on Earth as of March 2014. Those are active monthly users. There's a billion people who access the site through mobile devices. Just to put that in perspective, there's more Facebook users than there are Catholics.

[Jad Abumrad]

That can't be true. Yeah, no, yeah, it turns out it is true, but they're neck and neck anyhow. The overall point is that when you have one out of every seven people on the planet in the same space trying to connect across time and geography.

[Andrew Zolli]

You are bound to create problems sometimes.

[Jad Abumrad]

Facebook making headlines again tonight. The issue this time, privacy. Before we go there, we should introduce you to the guy in our story who is the problem solver.

[Arturo Bejar]

My name is Arturo Bejar, and I'm a director of engineering at Facebook.

[Jad Abumrad]

Story begins Christmas 2011.

[Andrew Zolli]

People are doing what they do every holiday season. They're getting back together with their families, and they're going to family parties, and they're taking lots and lots of pictures, and they're all uploading them to Facebook.

[Arturo Bejar]

And at the time, the number of photos that were getting uploaded was going pretty crazy.

[Jad Abumrad]

In fact, in just those few days between Christmas and New Year's, there are.

[Andrew Zolli]

More images uploaded to Facebook than there.

[Arturo Bejar]

Were the entirety of Flickr.

[Jad Abumrad]

Wait, you're saying more images were uploaded in a week to Facebook than all of Flickr? All time, yeah.

[Arturo Bejar]

Whoa.

[Jad Abumrad]

Which created a situation.

[Arturo Bejar]

The number of photos was going up, and along with the number of photos going up, the number of reports was going up.

[Andrew Zolli]

What he means by reports is this.

[Jad Abumrad]

Back in 2011, if you saw something.

[Andrew Zolli]

On Facebook that really upset you, you could click a button to report it.

[Jad Abumrad]

You could tell Facebook to take it down, which, from their perspective, is a really important mechanism, because if you're Facebook, you don't want certain kinds of content on your site.

[Arturo Bejar]

You don't want nudity. You don't want, like, drug use, hate speech, things like that.

[Andrew Zolli]

So a day or so after Christmas, thereabout, engineers come back to work, and they find waiting for them, literally millions of photo reports.

[Arturo Bejar]

Yes. The number of people that would be necessary to review everything that was coming in. It kind of boggled the mind.

[Jad Abumrad]

How many people would you have needed?

[Arturo Bejar]

I think at the time we were looking at it, which is two years ago. And again, all this has grown much since then. We're looking at, like, thousands.

[Jad Abumrad]

Thousands. Like some giant facility in Nevada filled with nothing but humans looking at Christmas porn. We were actually joking about this, but we found out later there actually are thousands of people across the world who do this for Internet companies all day long, which clearly warrants its own show. But for Our purposes just know that when a photo is reported, a human being has to look at it exactly right.

[Arturo Bejar]

Because there needs to be a judgment on the image. And humans are the best at that.

[Andrew Zolli]

So Arturo decided, before we do do anything, let's just figure out what we're dealing with.

[Arturo Bejar]

And so we sat down with a team of people and we started going through the photos that people were reporting.

[Jad Abumrad]

And what they found was that about 97% of these million or so photo reports were drastically miscategorized. They were seeing moms holding little babies.

[Arturo Bejar]

Reported for harassment, Pictures of families in matching Christmas sweaters reported for nudity, pictures of puppies reported for hate speech.

[Jad Abumrad]

Puppies reported as hate speech.

[Arturo Bejar]

Yes. And we're like, what's going on? Right.

[Andrew Zolli]

Hmm.

[Jad Abumrad]

So they decide, let's investigate.

[Andrew Zolli]

Okay, so step one for Facebook. Just ask a few of these people.

[Arturo Bejar]

Why don't you like this photo?

[Andrew Zolli]

Why did you report this?

[Jad Abumrad]

Responses come back and the first thing they realize is that almost always the person complaining about the image was in.

[Andrew Zolli]

The image they were complaining about, and.

[Jad Abumrad]

They just hate the picture. Like, maybe they were doing a goofy dance. Someone snapped a photo and they're like, why did you post that?

[Andrew Zolli]

Take it down.

[Jad Abumrad]

Maybe they were at a party, they.

[Andrew Zolli]

Got a little too drunk, they hooked up with their ex, Somebody took a picture and that person says, oh, you know, that's a one time thing that's never happening again. Take it down.

[Jad Abumrad]

Arturo said, there were definitely a lot of reports from people who used to.

[Arturo Bejar]

Be couples, and then they broke up and then they're asking to take the photos down.

[Jad Abumrad]

And the puppy, what would be the reason for that?

[Arturo Bejar]

Oh, because it was maybe a shared puppy.

[Jad Abumrad]

You know, maybe it's your ex wife's puppy. You see, it makes you sad.

[Andrew Zolli]

Take it down.

[Arturo Bejar]

So once we've begun investigating, you find that there's all of this relationship things that happen that are, like, really complicated.

[Andrew Zolli]

You're talking about stuff that's the kind of natural detritus of human dramas.

[Jad Abumrad]

And the only reason that the person reporting it flagged it as like, hate speech is because that was one of the only options.

[Andrew Zolli]

They were just picking because they needed to get to the next screen to submit the report.

[Arturo Bejar]

So we added a step.

[Andrew Zolli]

Artur and his team set it up so that when people were choosing that.

[Arturo Bejar]

Option, I want this photo to be removed from Facebook, some of them would.

[Andrew Zolli]

See a little box on the screen.

[Arturo Bejar]

That said, how does a photo make you feel?

[Andrew Zolli]

And the box gave several choices.

[Arturo Bejar]

The options were embarrassing, embarrassing, saddening, upsetting. Bad photo. And then we always put in an.

[Andrew Zolli]

Other where you could write in whatever you wanted about the image.

[Arturo Bejar]

And it worked incredibly well. I mean, like, 50% of people would.

[Jad Abumrad]

Select an emotion, like, for instance, embarrassing.

[Arturo Bejar]

And then 34% of people would select other. And we read those. We sit down, and we're reading the other. And what was the most frequent thing that people were typing into other? It was, it's embarrassing.

[Jad Abumrad]

It's embar. But you had embarrassing on the list.

[Arturo Bejar]

I know.

[Jad Abumrad]

That's weird.

[Arturo Bejar]

I know.

[Jad Abumrad]

Arturo was like, okay, maybe we should just put its in front of the choices.

[Andrew Zolli]

As in, please describe this piece of content. It's embarrassing.

[Arturo Bejar]

It's a bad photo of me, it makes me sad, et cetera.

[Andrew Zolli]

And when they wrote out the choices.

[Jad Abumrad]

That way, with that extra word, we.

[Arturo Bejar]

Went from 50% of people selecting an emotion to 78% people selecting an emotion.

[Jad Abumrad]

In other words, the word emotion, it's.

[Andrew Zolli]

All by itself, boosted the response by.

[Arturo Bejar]

28% from 50 to 78.

[Jad Abumrad]

And in Facebook land, that means thousands and thousands of people.

[Robert Krulwich]

Let me just slow down for a second. I'm trying to think of. What could that be? It's. Do people like full sentences?

[Jad Abumrad]

Here's thinking. It's always good to mirror the way people talk. Arturo's idea, though, which I find kind of interesting, is that when you just say embarrassing and there's no subject, it's silently implied that you are embarrassing. But if you say it's embarrassing, well, then that shifts the sort of emotional energy to this photograph thing, and so then it's less hot and it's easier to deal with.

[Robert Krulwich]

Oh, how interesting. That thing is embarrassing. I'm fine. It's embarrassing.

[Jad Abumrad]

It is responsible, not me.

[Robert Krulwich]

Good for Arturo. That's like, a subtle thought.

[Jad Abumrad]

It's very subtle, but it still doesn't solve their basic problem. Because even if Facebook now knows why the person flagged the photo that it was embarrassing and not actually hate speech, they still can't take it down.

[Andrew Zolli]

I mean, there's nothing in the policy, the terms of service, that says you can't put up embarrassing photos.

[Jad Abumrad]

And in fact, if they took it down, they'd be violating the rights of the person who posted it.

[Arturo Bejar]

Like, there's nothing we can do. I'm sorry.

[Robert Krulwich]

Oh, so they'd actually fence themselves in a little bit.

[Jad Abumrad]

Yeah.

[Robert Krulwich]

Huh. For me, I'd always put in another.

[Jad Abumrad]

I would just be like, go deal with it yourself, is what I would say. Talk to the person. No, honestly, that's the solution. He wouldn't put it that way. But what he needed to have happen was for the person who posted the picture and the person who was pissed.

[Arturo Bejar]

About it to talk to each other.

[Andrew Zolli]

To work it out themselves.

[Jad Abumrad]

So Arturo and his team made a tweak where if you said this photo was embarrassing or whatever, a new screen would pop up and it would ask.

[Arturo Bejar]

Do you want your friend to take the photo down?

[Jad Abumrad]

And if you said, yes, I would like my stupid friend to take the photo down.

[Arturo Bejar]

We put up an empty message box.

[Andrew Zolli]

Just an empty box that said, we think it's a good idea for you to tell the person who upset you that they upset you.

[Arturo Bejar]

And only 20% of people would type something in and send that message.

[Andrew Zolli]

They just didn't do it. They just said, I'd rather you deal with this.

[Jad Abumrad]

So Artur and his team were like, okay, let's take it one step further. When that message box popped up, we.

[Arturo Bejar]

Gave people a default message that we.

[Andrew Zolli]

Crafted to start that conversation.

[Arturo Bejar]

Just get the conversation going. And it's kind of funny. The first version of the message that.

[Andrew Zolli]

We did was like, hey, I didn't like this photo. Take it down.

[Robert Krulwich]

Hey, I don't like that photo. That's a little aggressive.

[Jad Abumrad]

It is. But when they started presenting people with a message box with that sentence pre written in, almost immediately we went from.

[Arturo Bejar]

20% of people sending a message to 50% of people sending a message.

[Jad Abumrad]

Really?

[Arturo Bejar]

It's been surprising to all of us. Like, we weren't expecting to see that big of a shift.

[Robert Krulwich]

So this means that people just don't want to write, they'll sign up for pretty much anything?

[Jad Abumrad]

No, not necessarily.

[Robert Krulwich]

No.

[Jad Abumrad]

Maybe it's just that it's so easy to shirk the responsibility of confronting another person that you need every little stupid nudge you can get. I see.

[Arturo Bejar]

Okay.

[Jad Abumrad]

That's how I see it. Okay. So they put out this pre written message. It seems to really have an effect. So they're like, okay, if that worked so well, why don't we try some.

[Andrew Zolli]

Different wordings instead of, hey, I didn't like this photo. Take it down.

[Jad Abumrad]

Why don't we try, hey, Robert, I.

[Andrew Zolli]

Didn'T like this photo. Take it down.

[Jad Abumrad]

Just putting in your name works about.

[Andrew Zolli]

7% better than leaving it out.

[Robert Krulwich]

Meaning what?

[Andrew Zolli]

It means that you're 7% more likely either to get the person to do what you ask them to do, take.

[Jad Abumrad]

Down the photo, or to start a.

[Andrew Zolli]

Conversation about how to resolve your feelings about it.

[Jad Abumrad]

Oh.

[Robert Krulwich]

We're now measuring the effectiveness of the message. So if I'm objecting Will the other party pull it off?

[Jad Abumrad]

The computer, pull it off, or just talk to you about it?

[Robert Krulwich]

Okay.

[Jad Abumrad]

They also tried variations like, hey, Robert, would you please take it down? Throwing in the word please or would you mind taking it down?

[Arturo Bejar]

And it turns out that would you please? Performs 4% better than would you mind?

[Jad Abumrad]

They're not totally sure why, but they tried dozens of phrases like, would you please mind? Would you mind? I'm sorry to bring this up, but would you please take it down? I'm sorry to bring this up, but would you mind taking it down? And at a certain point, Andrew and I got.

[Andrew Zolli]

We're here to see Arturo.

[Jad Abumrad]

We just wanted to see this whole process they're going through up close. So we took a trip out to Facebook headquarters. Menlo Park, California. This was about a year ago.

[Arturo Bejar]

Had you been here before?

[Jad Abumrad]

No, I have not. So it was before the hubbub. We met up with Arturo, who sort of walked us through the campus.

[Arturo Bejar]

Yeah, like the hammock. This is kind of a little like chair hanging out.

[Jad Abumrad]

It's one of these sort of like socialist, utopic Silicon Valley campuses where people are like in hammocks and there's volleyball happening.

[Arturo Bejar]

We actually had baby foxes here.

[Jad Abumrad]

Really? They had foxes running around at one point. So we were there on a Friday, because every Friday afternoon, Arturo assembles this really big group. Welcome to the meeting to review all the data. You got about 15 people crammed into a conference room like technical folks.

[Arturo Bejar]

Mojtaba Software engineering. Trust engineering at Facebook.

[Jad Abumrad]

Dan Farrell.

[Arturo Bejar]

I'm a data scientist.

[Jad Abumrad]

Paul.

[Arturo Bejar]

I'm also an engineer.

[Jad Abumrad]

A lot of these guys call themselves trust engineers. And every Friday, the trust engineers are joined by a bunch of outside scientists.

[Arturo Bejar]

Dacher Keltner, professor of psychology, UC Berkeley.

[Jad Abumrad]

Matt Gillingsworth. I study the causes and nature of human happiness. Miliana Simon Thomas.

[Kate Crawford]

And my background is neuroscience.

[Jad Abumrad]

This is the meeting where the team was reviewing all the data about these phrases. And so everybody was looking at a giant graph projected on the wall. It's kind of supporting your slightly U shaped curve there in that, especially in the deletion numbers. The hey, I don't like this photo. Take it down. And the hey, I don't like this photo. Would you please take it down? Are kind of the winners here.

[Kate Crawford]

It's kind of interesting that you see the person that's receiving a more direct Message is higher. 11% versus 4%.

[Jad Abumrad]

One of the things they notice is that anytime they use the word sorry in a phrase like, hey, Robert, sorry to bring this up, but would you please take it down.

[Arturo Bejar]

Turns out the I'm sorry doesn't actually help. It makes the numbers go down, really.

[Jad Abumrad]

Seven and nine are the low. Some of the low points, and those.

[Kate Crawford]

Are the ones that say sorry.

[Jad Abumrad]

So, like, just don't apologize. Just don't apologize because, like, it shifts the responsibility back to you, I guess.

[Robert Krulwich]

No, it doesn't. It's just. It's just.

[Jad Abumrad]

No, man. It's like. It's a linguistic psychology, subtle thing.

[Robert Krulwich]

You're making that up.

[Jad Abumrad]

I am, kind of. But one of the things that really struck me at this meeting on a different subject is that the scientists in the room, as they were looking at the graph, taking in the numbers, a lot of them had this look on their face of like, holy. I'm just stunned and humbled at the numbers that we generally get in these studies. That's Amelia Simon Thomas from Berkeley. My background is in neuroscience, and I'm used to studies where we look at 20 people, and that's sufficient to say something general about how brains work. Like, in general, at Facebook, people would scoff at sample sizes that small. That's Rob Boyle, who's a project manager at Facebook. The magnitudes that we're used to working with are in the hundreds of thousands to millions. It's kind of an interesting moment because there's been a lot of criticism recently, especially in social science, about the sample sizes, how they're too small and how they're too often filled with white undergraduate college kids. And how can you generalize from that? So you could tell that some of the scientists in the room, like, for example, Dacher Keltner, who's a psychologist at UC Berkeley, they were like, oh, my God, look at what we can do now. We can get to all these different people of different class backgrounds, different countries. To him, this kind of work with Facebook, this could be the future of social science right here. There has never been a human community like this in human history. Somewhere in the middle of all the excitement about the data and the speed at which they can now test things, the bottleneck is no longer how fast we can test how things work. It's coming up with the right things to test. Andrew threw out a question.

[Andrew Zolli]

What is the statistical likelihood that I have been a guinea pig in one of your experiments?

[Jad Abumrad]

I believe 100%.

[Arturo Bejar]

But if you look at the data.

[Jad Abumrad]

Any given person, that's Stan Farrell, data scientist. And when we look at the data, any given person is probably currently involved.

[Arturo Bejar]

In, what, 10 different experiments, and they've.

[Jad Abumrad]

Been exposed to 10 different experimental things. Yep.

[Andrew Zolli]

That kind of blew me back a little bit. I was like, I've been a research subject and I had no idea.

[Jad Abumrad]

Coming up, everybody gets the idea, and the lab rats revolt. Stay with us. Hi there. This is Pedro Suarez calling from London, England. Radiolab is supported in part by the National Science foundation and by the Alfred P. Sloan foundation, enhancing public understanding of science and technology in the modern world. More information about [e-mail]. oh, that's great. Bit of a tongue twister. This is Radiolab, and we'll pick up the story with Andrew Zolli and I sitting in a meeting at Facebook headquarters. This was about a year and a half ago. We had just learned that at any given moment, any given Facebook user is part of 10 experiments at once without really their knowledge. And sitting there in that meeting, you know, this was a while ago, we both were like, did we just hear that correctly?

[Andrew Zolli]

That kind of blew me back a little bit. I was like, I've been a research subject and I had no idea. And I had that moment of discovery on a Friday, and literally the next day, Saturday. This is scary. The world had that experience. Facebook using you and me as lab rats for a Facebook experiment on emotions.

[Jad Abumrad]

Barely a day after we'd gotten off the plane from Facebook headquarters, the kerfuffle occurred. Facebook exposed for using us as lab rats. As lab rats. Lab rats, shall we say? Facebook messing with your emotions. You might remember this story because for a hot second, it was everywhere Facebook altered.

[Andrew Zolli]

The amount of.

[Jad Abumrad]

It was all over Facebook story was. An academic paper had come out that showed that with some scientists, the company had intentionally manipulated user news feeds to study a person's emotional response. Seriously. They wanted to see how emotions spread on social media. They basically tinkered with the news feeds of about 700,000 people, 700,000 users, to test how they'd react if they saw more positive versus negative posts and vice versa. And they found an effect that when people saw more positive stuff in their news feeds, they would post more positive things themselves and vice versa. It was a tiny effect. Tiny effect, but the results weren't real. The real story was that Facebook was messing with us. Gives you pause and scares me when you think that they were just doing an experiment to manipulate how people were feeling and how they then reacted on Facebook.

[Andrew Zolli]

People went apoplectic.

[Jad Abumrad]

It has this big Brother element to.

[Kate Crawford]

It that I think people are gonna.

[Jad Abumrad]

Be very uncomfortable with.

[Andrew Zolli]

And some people went so far as to argue.

[Jad Abumrad]

I wonder if Facebook killed anyone with their emotional manipulation.

[Andrew Zolli]

So stunt if A person had a psychological or psychiatric disorder, manipulating their social world could cause them real harm.

[Kate Crawford]

Make sure you read those terms and conditions, my friends.

[Jad Abumrad]

Always. That's the big takeaway.

[Kate Crawford]

What you hear is a sense of betrayal that I really wasn't aware that this space of mine was being treated in these ways and that I was part of your psychological experimentation.

[Jad Abumrad]

That's Kate Crawford.

[Kate Crawford]

I'm a principal researcher at Microsoft Research.

[Jad Abumrad]

Visiting professor at mit, strong critic of Facebook throughout the kerfuffle.

[Kate Crawford]

There is a power imbalance at work. I think when we look at the way that that experiment was done, it's an example of highly centralized power and highly opaque power at work. And I don't want to see us in a situation where we just have to blindly trust that platforms are looking out for us here. I'm thinking of an earlier Facebook study, actually, back in 2010, where they did a study looking at whether they could increase voter turnout. They had this quite simple design. They came up with a little box that would pop up and show you where your nearest voting booth was. And then they said, oh, well, in addition to that, when you voted, here's a button you can press that says, I voted. And then you'll also see the pictures of six of your friends who'd also voted that day. Would this change the number of people who went out to vote that day?

[Jad Abumrad]

And Facebook found that it did that if you saw a bunch of pictures of your friends who had voted and you saw those pictures on election day, you were then 2% more likely to click the I voted button yourself, presumably because you, too had gone out and voted. Now, 2% might not sound like a.

[Kate Crawford]

Lot, but it was not insignificant. Again, I think, by the order of 340,000 votes, the votes that they estimate, they actually shifted by getting people to go.

[Jad Abumrad]

These are people who wouldn't have voted.

[Kate Crawford]

Who wouldn't have voted, and who. They have said in their own paper and published paper that they increased the number of votes that day by 340,000.

[Jad Abumrad]

Simply by saying that your neighbors did it, too.

[Kate Crawford]

Yeah. By your friends.

[Jad Abumrad]

Now, my first reaction to this, I must admit, was okay. I mean, we're at historic lows when it comes to voter turnout. This sounds like a good thing.

[Kate Crawford]

Yes. But what happens if someone's running a platform that a lot of people are on and they say, hey, you know, I'm. I'm really interested in this candidate. This candidate is going to look out not just for my interests, but the interests of the technology sector. And I think they're you know, they're a great candidate. Why don't we just show that get out to vote message and that. That little system design that we have to the people who clearly. Because we already have their political preferences, the ones who kind of agree with us and the people who disagree with that candidate, they won't get those little nudges. Now, that is a profound democratic power that you have.

[Jad Abumrad]

Kate's basic position is that when it comes to social engineering, which is what this is, companies and the people that use them need to be really, really careful. In fact, when Andrew mentioned to her that Arturo had this group and the group had a name, he actually runs.

[Andrew Zolli]

A group called the Trust Engineering Group. His job is to engineer trust.

[Jad Abumrad]

When Andrew told her that Facebook users.

[Arturo Bejar]

That's his job, you're smacking your forehead.

[Kate Crawford]

I think we call that a facepalm.

[Jad Abumrad]

She facepalmed really hard.

[Kate Crawford]

These ideas that we could somehow engineer, compassion, I think, to some degree, have a kind of hubris in them. Who are we to decide whether we can make somebody more compassionate or not?

[Arturo Bejar]

So I do want to set this up.

[Jad Abumrad]

Let's see. How do we do this? A couple months after our first interview, we spoke to Arturo Behar again. At this point, the kerfuffle was dying down, and we asked him about all the uproar. I know this is not your work. This. The emotional contagion stuff. But literally, like, hours after we got back from that meeting, that thing erupted. Do you understand the backlash?

[Arturo Bejar]

No. I mean, I think that. I mean, we really care about the people who use Facebook. I don't think that there's such a thing as. I mean, if anything I've learned in this work is that you really have to respect people's response and emotions, no matter what they are.

[Jad Abumrad]

He says the whole thing definitely made them take stock.

[Arturo Bejar]

There was a moment of concern of what it would mean to the work. There was like, is this going to mean that we can't do this? Part of me, like, being honest, coming here is I actually want to reclaim back the word emotion and reclaim back the ability to do very thoughtful and careful experiments. I want to reclaim back the word experiment.

[Jad Abumrad]

You want to reclaim it from what?

[Arturo Bejar]

Well, suddenly, like, the word emotion and the word experiment, all these things became really charged.

[Jad Abumrad]

Well, yeah, because people thought that Facebook was manipulating emotion, and they were like.

[Arturo Bejar]

Yes, but in our case. Right. And in the work that we're talking about right now, all of the work that we do begins with a person asking us for help.

[Jad Abumrad]

This was Arturo's most emphatic point. He said it over and over that, you know, Facebook isn't just doing this for fun. People are asking for help. They need help. Which points to one of the biggest challenges of living online, which is that, you know, offline, you know, when we try and engineer trust offline, or at least just read one another, we do it in these super subtle ways, using eye contact and facial expressions and posture and tone of voice, all this nonverbal stuff. And of course, when we go online, we don't have access to any of that.

[Arturo Bejar]

In the absence of that feedback, how do we communicate? What does communication turn into.

[Jad Abumrad]

Just because.

[Andrew Zolli]

The other stupid kids.

[Jad Abumrad]

The best riff on this, I gotta say, is Louis CK On Conan.

[Arturo Bejar]

It's just this thing.

[Jad Abumrad]

It's bad. He did this great bit about technology and kids.

[Andrew Zolli]

You know, kids are mean, and it's because they're trying it out.

[Jad Abumrad]

They look at a kid and they go, you're fat.

[Andrew Zolli]

And then they see the kid's face scrunch up and they go, ooh, that doesn't feel good to make a person do that. But they got to start with doing the mean thing. But when they write you're fat, then they just go, mmm, that was fun. I like that.

[Jad Abumrad]

Anyhow, back to Arturo.

[Arturo Bejar]

I mean, I think about, like, what it means to be in the presence of a friend or a loved one, and how can you build experiences that facilitate that when you cannot be physically together?

[Jad Abumrad]

Arturo says that's really all he's up to. He's just trying to nudge people a tiny bit so that their online selves are a little bit closer to how they are offline. And I gotta say, if he can do that by engineering a couple of phrases like, hey, Robert, would you mind, et cetera, et cetera, well, then I'm all for it.

[Robert Krulwich]

Why not take the position that to create a company that stands between two people who are interacting and then giving them boxes and statuses and little advertising and so forth, this is not doing a service. This is just. This is a way to wedge yourself into the ordinary business of social intercourse and make money on it.

[Jad Abumrad]

No.

[Robert Krulwich]

And you're acting like this group of people now is going to try to create the moral equivalent of an actual conversation. First of all, it's probably not engineerable. And second of all, I don't believe that for a moment. All I'm thinking is they're gonna just go and figure out other ways in which to make a revenue enhancer.

[Jad Abumrad]

No, I don't think it's one or the other. I think they're in it for the money. In fact, if they can figure this out and make the Internet universe more inducive, to trust, less annoying, it could mean trillions of dollars. So, yeah, it's the money, but still, that doesn't negate the fact that we have to build these systems, right? That we have to make the Internet a little bit better.

[Robert Krulwich]

That's fine. This idea, however, that you're going to have to coach people into the subtleties of the relationship. Tell him you're sorry. Tell him this. Here's the formula for this. He doesn't want. He did something. You need to repair that. Here are the seven ways you might repair that. To do all that, it's as if the Hallmark card company, instead of living only on Mother's Day, Father's Day and birthdays, just spread its evil wings out into the whole rest of your life. And I don't think that's a wonderful thing.

[Jad Abumrad]

I think, you know, I have a slightly different opinion of it. I mean, you gotta keep in mind how this thing came about. I mean, they tried to get people to talk to each other. They gave them the blank text box, but nobody used it, right? So they're like, okay, let's come up with some stock phrases that, yes, are generic. But think about the next step. After you send the message saying, you know, jad, I don't like the photo. Please take it down. Presumably, then you and I get into a conversation. Maybe I explain myself. I say, oh, my God, I'm so sorry. I didn't realize that you didn't like that photo. I just thought that that was an amazing night. I just thought that was a great night. I didn't realize you thought. You looked. So sorry. I'll take it down. It's cool. See, now, presumably, we're having that conversation as a next step.

[Robert Krulwich]

Why do you presume that? How many of the birthday cards that you've sent to first cousins have resulted in a conversation?

[Jad Abumrad]

Maybe not.

[Robert Krulwich]

But that's the thing. Sometimes these things are actually not. They're really the opposite of what you're saying. They're conversation substitutes.

[Jad Abumrad]

Maybe. Maybe they're conversation starters.

[Robert Krulwich]

Maybe that's the deep experiment.

[Jad Abumrad]

Are they conversation starters or substitutes? Well, I hope they're conversation starters.

[Robert Krulwich]

Yeah.

[Jad Abumrad]

Because maybe that would be a beginning.

[Andrew Zolli]

It kind of in my mind goes back to, like, the beginning of the automobile age.

[Jad Abumrad]

This is how Andrew puts it.

[Andrew Zolli]

There was a time when automobiles were new and, you know, they didn't have turn signals, the tools. They did have, like, the horn didn't necessarily indicate all the things that we use it to indicate. It wasn't clear what the horn was actually there to do. Was it there to say hello? Or is it there to say get.

[Robert Krulwich]

Out of the way?

[Andrew Zolli]

And over time, we created norms. We created roads with lanes, we created turn signals that are primarily there for other people so that we can coexist in this great flow without crashing into each other.

[Jad Abumrad]

And we still have road rage.

[Andrew Zolli]

And we still have road rage. We still have places where those tools are incomplete.

[Jad Abumrad]

Thanks to Andrew Zolli. Many, many, many, many thanks. Yes, definitely, for bringing us that story and for reporting it with me for so long. And to Arturo, who you kept bringing.

[Robert Krulwich]

Back into the studio to keep.

[Jad Abumrad]

Yes, thank you very much to Arturo and the whole team over there. And by the way, they have changed their name. It's no longer Trust Engineering. It is the Facebook Protect and Care team. Really? Yeah. We had some original music this hour from mooninite. Thanks to them. Props to Andy Mills for production support. And also Andrew Zolli put together a blog post. If you go to Radiolab.org, you can see it, which covers some really interesting research that we didn't get a chance to talk about. And if you've ever sent an email with a little smiley face, you're definitely going to want to read this. Radiolab.org, i'm Jan Abumrad.

[Robert Krulwich]

I'm Robert Krulich.

[Jad Abumrad]

Thanks for listening. Your new home is now ready. Dr. Horton, America's Builder has new homes that are ready today with new construction communities in Ellensburg and throughout the Greater Seattle area. Dr. Horton has the right home for you. @ Dr. Horton. We're still building with flexible living spaces, smart home technology and two and three car garages. More communities and more homes available every day. Find your new home in Ellensburg now [e-mail] Dr. Horton, America's builder, an equal housing opportunity builder. Hey Fidelity, what's it cost to invest with the Fidelity app? Start with as little as $1 with no account fees or trade commissions on.

[Kate Crawford]

US stocks and ETFs.

[Jad Abumrad]

Hmm, that's music to my ears. I can only talk.

[Andrew Zolli]

Investing involved risk including risk of loss. Zero Account fees apply to retail brokerage accounts only. Sell order assessment fee not included. A limited number of ETFs are subject to a transaction based service fee of $100.

[Jad Abumrad]

See full list at fidelity.

[Andrew Zolli]

Com Commissions Fidelity Brokerage Services LLC Member nyse, SIPC.