How Meta Is Making Billions From Scam Advertising

Summary

Podcast Summary: "How Meta Is Making Billions From Scam Advertising"

Podcast: Question Everything
Host: Brian Reed
Guest: Jeff Horwitz (Investigative Reporter, Reuters)
Release Date: November 20, 2025

---

Overview: Main Theme & Purpose

This episode explores how Meta (Facebook’s parent company) is knowingly earning billions of dollars in advertising revenue from fraudulent ads and scam operations on its platforms. Investigative reporter Jeff Horwitz, who broke the recent story based on internal Meta leaks, sheds light on alarming internal data, the real-world impact of scam advertising, and the legal loopholes—primarily Section 230—that enable Meta to avoid accountability. The episode examines the moral and systemic dilemmas posed by the current structure of internet regulation.

---

Key Discussion Points and Insights

1. Section 230: The Law Behind Internet Platforms

• Brian Reed introduces Section 230 as the 1996 law making internet platforms largely immune from liability for user-generated content, safeguarding vibrant online debate — but also facilitating neglect and exploitation.
• Section 230 has protected platforms even when harrowing consequences result from user activity (examples: harassment via fake Grindr profiles, recruitment by extremist groups on Facebook, spread of illegal sexual images on Reddit).
  • Quote:

    “For better and for worse, Section 230 has made it so that companies are largely not responsible for what happens on their online platforms, even if they're making huge profits from that activity.”
    — Brian Reed [04:44]

2. The Meta Leak: Inside Knowledge of Fraud

• Jeff Horwitz obtained a large leak of Meta’s internal documents, revealing how fraud and scam ads are rampant on Facebook, Instagram, and WhatsApp.
  • These include policy docs, engineering info, and screenshots of internal management discussions.
  • Jeff explains:

    “They believe they do not owe any user a duty to remove fraudulent ads from the platform. Like that's just not a thing that they are responsible for.”
    — Jeff Horwitz [05:26]

• Scale of Problem:
  • Meta made ~$160B in advertising revenue in the last year; Meta’s own estimate is that ~10% ($16B) came from scam or banned ads.
  • 15 billion scam ad impressions served daily.
  • Meta’s platforms implicated in about a third of all successful scams in the US.
  • Quote:

    “Meta's own research suggests its products have become a pillar of the global fraud economy.”
    — Brian Reed [09:24]

3. Scam Ad Examples and Victims

• Real ad example: Fake "McCormick spices" ad scam; hundreds reported getting duped.
  • Meta only removed ads after being publicly alerted; systemic solutions lag behind.
  • “If I fell for the McCormick spices and I gave my money, like, who can I go after?”
    – Brian Reed [12:00]
  • Jeff: “Functionally, no one.” [12:07]
• Victim story: Canadian Air Force recruiter’s Facebook account hijacked for weeks; her contacts lost thousands of dollars to crypto scams carried out in her name. Meta only responded after massive escalation.
  • Quote:

    "...the recruiter just kind of, like, broke down in tears because people were being defrauded, quite literally because they trusted her."
    — Jeff Horwitz [14:43]

4. Organized Crime & Human Trafficking

• Behind many scam operations: syndicates engaging in human trafficking, forcing victims into “scam farms” on the Myanmar border.
  • Quote:

    “These are like basically prison camps where people who've been mostly human trafficked are forced to work in what are essentially scam call centers. This is an industry that is like a few hundred thousand people working in it.”
    — Jeff Horwitz [16:15]

5. Meta’s Approach: Profits Over Enforcement

• Meta knows about the scams but dismisses or ignores 96% of valid user complaints about scam ads ([20:05]).
• In 2023, company targets for improvement still aimed to dismiss 75% of scam ad complaints.
• Even billionaire victims (like Andrew “Twiggy” Forrest) have little recourse.
  • Forrest sued Meta over AI-enabled scam ads using his likeness. For once, a court allowed his case to go forward, threatening Meta’s reliance on Section 230 ([22:10]).

6. How Meta “Fights” Fraud

• Limited mitigation: High-risk advertisers subjected to extra verification – but with a hard limit on financial impact (only up to 0.15% of Meta’s total revenue, about $100M).
  • When financial cost exceeded that, they'd stop enforcement ([23:49]).
• Perverse incentives: Meta sometimes charges suspected scammers extra fees (“penalty bid”) instead of banning them, making scam ads more lucrative.
  • Quote:

    “The more likely an ad buyer is to defraud Meta’s users, the more money Meta makes off them.”
    — Brian Reed [25:58]

• Meta only blocks an advertiser if its system has at least 95% confidence they’re a scammer; less than that and their ads stay ([29:10]).
  • Quote:

    “For an advertiser that Meta's automated system thinks is dirty... the level of certainty that Meta's prediction system needs to have to cut that advertiser off is at least 95%.”
    — Jeff Horwitz [29:12]

7. Wider Implications & Accountability

• Other countries with stricter regulations see better enforcement; in the US, Meta only acts when forced by law.
• Leaks from whistleblowers (not public oversight) remain the primary way the world learns about these practices.
  • Quote:

    “Leaks are a primary source of information about, like, how the hell products that are so deeply ingrained in our lives actually work... This is a broken system.”
    — Jeff Horwitz [33:41]

• Systemic failures are not inevitable: platform design choices could prioritize safety and integrity, but profit wins under the current system.
  • Quote:

    “Social media platforms… how people behave on them is heavily influenced by the design choices of the owners of those platforms.”
    — Jeff Horwitz [35:02]

---

Notable Quotes & Memorable Moments

• “Meta did $160 billion worth of revenue last year from advertising... around 10% of that was coming from banned ads.”
  — Jeff Horwitz [08:46]
• “If a fellow billionaire can’t appeal to Mark Zuckerberg personally and get a scam ad taken down, what hope do any of us have?”
  — Brian Reed [21:59]
• “Imagine if Section 230 were changed so users could easily sue Meta for serving them scam ads. ...Meta prioritized cleaning up its platforms of fraud in countries where there are regulations about it.”
  — Brian Reed [33:15]

---

Important Timestamps

• [01:20] – Brian introduces Section 230 and its impact on free speech and platform accountability
• [06:28] – Jeff Horwitz joins; description of what the leaked Meta files contain
• [08:46] – Scale & impact: 10% of Meta’s ad revenue from banned/scam ads
• [12:00] – Who’s legally responsible when you fall for a scam ad? (Answer: No one, due to Section 230)
• [14:03] – Victim’s ordeal fighting to take down a scam; Meta’s inaction despite law enforcement involvement
• [16:15] – Organized crime and human trafficking behind scam operations
• [20:02] – Meta's own stats: dismissing 96% of scam/fraud complaints
• [23:49] – Revenue-first approach: fraud enforcement capped to protect Meta’s bottom line
• [25:58] – Meta profits more from likely scam ads (“penalty bids”)
• [29:12] – Meta’s threshold for banning scam advertisers: at least 95% certainty
• [33:41] – The broken system: whistleblower leaks as only window into internal workings
• [35:02] – Design choices impact online behavior; accountability is possible

---

Conclusions & Final Thoughts

• Meta’s business model, protected by Section 230, allows it to profit massively from scam activity while evading responsibility.
• Current US regulatory frameworks only incentivize change under external pressure; platform self-policing is both minimal and financially constrained.
• Real-world harms from scam ads are enormous, touching victims and enabling global criminal enterprises including human trafficking.
• Transparency is achieved not through regulation but dangerous whistleblowing and leaks.
• Design decisions, not inevitability, shape the internet’s moral landscape—change is possible if priorities shift.

---

For More:
Read Jeff Horwitz’s full reporting at Reuters.
Join ongoing debate about Section 230 on the Question Everything Substack.

Transcript

[Brian Reed]

We've been working with this great new sponsor here at Question Everything. It's called Plaud. That's P as in pancake P L A U D Plod. It's this nifty tiny AI assistant and notetaker device that you can turn on and it listens to any conversation you're having and then uses AI to create to do lists or action items or to synthesize different things. We've actually turned the word plodding into a verb around the office. So we'll start a meeting and I'll say I'm going to Plaud. This really just frees your brain from note taking, remembering menial things. If you're interested, go check out plaudplaud AI and if you get one, use the special code Question. It helps out the show.

[KCRW Sponsor Announcer]

I just got my new phone and the KCRW app is the best way to get the music and shows you love from kcrw. And it's been totally redone to be cleaner, faster and more reliable. And there's two new music Dance 24 and Vintage 24 and they're only in the app plus real time now playing so you never miss a track ID. Look up KCRW in the App Store and be sure to make a free account to use all the new features.

[Brian Reed]

There's a conflict I'm grappling with at the heart of Section 230, the law from 1996 that's responsible for the Internet as we know it.

On the one hand, section 230, lots of people argue, is the reason we have free speech on the Internet, which that is something I definitely want. Free speech on the Internet. The law is so fundamental to online life, like the foundation of your house, it's easy to take for granted. But ever since I started paying attention to it, I haven't been able to stop seeing section 230 everywhere. Like last week, when the new dump of Jeffrey Epstein documents came out with Donald Trump's name littered all over them. Millions of us could go onto X and TikTok and Substack and Blue sky and talk about them. We could theorize about what Epstein meant when he wrote that Trump was the dog that hasn't barked. Or whether we thought Epstein was telling the truth when he claimed that he had photos of Trump with scantily clad young women in his kitchen. And we could do that, arguably thanks to section 230, because it makes it so that Internet sites and providers can't get sued for what users post on their platforms. And the same for users who repost something from someone else. And that means when people share damning information about the President, X and TikTok and Substack and BlueSky don't have to worry about Trump suing them the way he sued news organizations for many billions of dollars because of section 230. Platforms are comfortable providing a space for the public to debate this important information quickly, raucously. I don't want to lose that. And if Section 230 were repealed or reformed, it is possible that we could. But at the same time, in the last year or so especially, I find myself more and more noticing the negative effects of section 230. Because this same law, this same legal foundation that the modern Internet rests on, it's also propping up some pretty gnarly, rotting, and extremely unsafe structures. If you look at a list of cases where Section 230 has helped Internet companies escape accountability, it's like a carnival of horrors. There's a guy who begged Grindr to get rid of a fake profile made by an ex that said he was looking for violent sex and led to him getting harassed. In real Life, Grindr used Section 230 to get that case dismissed. There's the family members of victims of an attack by Hamas who sued Facebook saying the platform not only allowed known Hamas members to have accounts, but helped them recruit by recommending their content to people. Facebook used section230 to get that case dismissed. Minors who sued Reddit for letting illegal sexual images of them spread. Reddit got the case dismissed with section 230. There are so many cases like this over the years. For better and for worse, Section 230 has made it so that companies are largely not responsible for what happens on their online platforms, even if they're making huge profits from that activity. As a journalist for a long time, my reflex has been to say, you know what? This is a necessary evil. We need to have a hands off approach to the Internet in order to protect free speech. And I get that point of view. I know many of you hold it too. I've been hearing from some of you. I don't deny that there are risks to changing section 230, but also I think it's important that as journalists especially, we fairly acknowledge the costs of this law. I know for a while I was reluctant to do that. And once we do, maybe we can imagine more productive, fairer ways to approach the Internet. There's a story that just broke that is yet another revelation of how Section 230 is invisibly shaping our world. How it allows big tech companies to do really shitty things with impunity. A new leak of sensitive files in just the last few weeks from inside Facebook's parent company, Meta. That's what our show's about. Today on a week when you may have heard Meta just had a big win in court, defeating the federal government's attempt to break it up as a monopoly. The company emerges even stronger and it makes what's revealed in these recently leaked files even more maddening.

[Jeff Horwitz]

They believe they do not owe any user a duty to remove fraudulent ads from the platform. Like that's just not a thing that they are responsible for.

[Brian Reed]

From placement theory and kcrw, this is question Everything. I'm Brian Reed. Stick around.

So you've been hearing me talk a bit on the show about this device, the Plod, which is this small, AI powered, note taking, personal assistant thinking device, I guess is what I call it. I actually reached out to them. I thought it might be able to help us. And there's one idea in particular, one kind of weird idea I had that I was having trouble figuring out a tech solution for that when I saw this device called Plod. I thought, oh, maybe this can help us. It's a little bit of an experiment, question everything, type experiment. All right, Plod, I want you to help me track my media consumption and the PLOD is helping us do it. I'm going to tell you more about that a little bit later.

The newly leaked information from inside Meta. It came to investigative reporter Jeff Horowitz, based in California covering Meta.

[Jeff Horwitz]

I've had a lot of luck in people wanting to give me documents.

[Brian Reed]

Jeff works for Reuters. That's where he published this recent story. But he's been investigating tech companies for a while. He was on the team at the Wall Street Journal that got documents from former Facebook engineer Francis Haugen when she wanted to blow the whistle on disturbing practices she saw inside the company. These became known as the Facebook files. Jeff also wrote a book based on employee leaks called Broken Inside Facebook and the Fight to Expose its Harmful Secrets.

[Jeff Horwitz]

It's never anybody's first choice. Nobody's like, hi, my well paid tech job, I would like to put that at risk. But people sort of have some qualms about things. They raise things internally. Maybe the internal concerns don't get handled the way they think they should. And I think at that point somebody might be open to talking to an outsider.

[Brian Reed]

Jeff can't say a ton about when or how he got the recent leak to protect his sources. But he describes it as a collection of material from inside Meta, including documentation about policy decisions the company's made, engineering information, internal data about Facebook, Instagram and WhatsApp, and screenshots of discussions, including with management on Meta's internal Facebook system, which is called Facebook Workplace. And what these records are about is fraud. Fraudulent advertisers on Facebook, Instagram and WhatsApp, along with fraudulent accounts and pages used to scam people.

[Jeff Horwitz]

So I don't think you would have invited me on here if the breaking news was that there are scams on the Internet, Right? Like, we all understand that there is fraud in the ecosystem. People get swindled to E commerce, they get pig butchered, which is the fancy name for winning someone's trust and then draining their life savings.

[Brian Reed]

Pig butchered. That's what it's called. I've never heard that before.

[Jeff Horwitz]

The idea is that you are forming a relationship with the animal and fattening them up so that when it's time to butcher them, it's all very easy.

[Brian Reed]

Wow. Very graphic. Okay. What's staggering about this recent leak is the scale of fraud on Meta's platforms.

[Jeff Horwitz]

So Meta did $160 billion worth of revenue last year from advertising, more or less, and it determined that around 10% of that was coming from banned ads.

[Brian Reed]

10% of all the money Meta made last year came from ads that really should never be on the platform in the first place. Per Metta's own estimate, These are fake businesses selling fake products that steal your money or your credit card details, bunk investment schemes you might transfer cash to. And never see. Crackpot medical products, 10% of all their revenue, $16 billion last year alone. Just to give you two other crazy numbers from the Reuters story here, Meta was feeding its users 15 billion scam ad impressions every day. And Meta, this one company, estimated that its platforms were involved in a third of all successful scams in the US As Jeff puts it in his report, Meta's own research suggests its products have become a pillar of the global fraud economy.

[Jeff Horwitz]

This is a significant piece of their actual business, and Meta knows it.

[Brian Reed]

That's the other thing that's damning about the records Jeff was given. They make it clear that Meta is aware of the scale of fraud and scams on its platforms. And they give a view into how Meta operates behind this shield of section 230, where the company knows it's going to be really hard for people to successfully sue them. We get to see what Meta does or doesn't do when it knows its users are being Ripped off. Can you give me some examples of the scam ads that you came across in your reporting on this? Like, I know in your story you have a picture of an ad. It seems to be posted from an account called Spice Flavor. It has a picture of a box of spices with the name McCormick on the box. And I guess McCormick's a spice brand. I have some other spices in my drawer at home. And then the post says, McCormick celebrates 136 years of presence in the USA. A special offer in honor of this event, a coaster with a set of your favorite McCormick spices for your dishes. And then you can like click on Facebook, shop now and I guess buy this spice coaster.

[Jeff Horwitz]

Yeah. If you do click on that, you are going to give someone your credit card and that credit card is going to go see the world. McCormick has had to issue statements online being like, for the love of God, people, please don't try to buy these things. Hundreds and hundreds of people in the comments were like, oh man, they got me. So this has been going on for a long time.

[Brian Reed]

People really like McCormick spices. I didn't know there was such like a kind of an emotional connection. I mean, they're good, but yeah, it's.

[Jeff Horwitz]

It's kind of grandma coded, I think, a little bit. Yeah, I wrote about it first. The McCormick thing at the Wall Street Journal.

[Brian Reed]

When you reported on it for the Journal for the Wall Street Journal, you alerted Facebook about it?

[Jeff Horwitz]

Oh, yeah, no, no, they were very aw McCormick spice scam.

[Brian Reed]

And did they take it down at that point or no?

[Jeff Horwitz]

I mean, yeah, that particular ad came down immediately, as did the other ones that I flagged. But obviously this is not a system that is going to work for a platform that's running 15 billion scam ad impressions a day.

[Brian Reed]

You're the janitor of meta.

[Jeff Horwitz]

Yes. Yeah, a little bit.

[Brian Reed]

And just to clarify, if I fell for the McCormick spices and I gave my money, like, who can I go after?

[Jeff Horwitz]

I mean, functionally, no one.

[Brian Reed]

And that's because of Section 230.

[Jeff Horwitz]

That's certainly a portion of it. A big portion of it is that there's just an expectation that platforms aren't liable for the content that runs on them.

[Brian Reed]

Did you end up interviewing anybody who'd been taken in by one of these scams?

[Jeff Horwitz]

Oh, yeah, I've talked to a lot of people in this story. Was a Canadian Air Force recruiter who got hacked and account got taken over by a scammer. Like, basically she woke up one morning and was not in control of her account. And her account was suddenly posting about how, you know, great news, she's been crypto certified, whatever the hell that means. Posted an ID badge of, like, the crypto trading firm that she was now working at, supposedly. This was an interesting one. I think it really goes to Meta's overall enforcement, because the scam in question was like a slow burn.

[Brian Reed]

A pig butchering of sorts. Though in this case, the butcher is pretending to be someone the pig knows. The scammer wants to get the Air Force recruiter's followers to send them money under the guise of selling them crypto. And whoever was behind this really took their time with it.

[Jeff Horwitz]

The scammer hacked the account, and then they were just literally posting about their success crypto trading and how great this was. And, like, they even posted a picture of this plot of land they. That supposedly the recruiter had purchased to build their dream home. You know, weeks and weeks of posting. Meanwhile, this Canadian Air Force recruiter is trying to get this stuff taken down, first through reporting, and then by getting people that she knows to report, and then by getting her bosses to get in touch with the Mounties, Canada's police force.

[Brian Reed]

But even trying to use her connections at the Canadian military and police to stop this identity theft, that didn't work.

[Jeff Horwitz]

What came back is, like, look, you know, Meta doesn't frequently respond to them either. Meta has a lot of law enforcement requests come in, and most of them don't get processed. Right.

[Brian Reed]

And she wants to get it taken down because it's still her name.

[Jeff Horwitz]

It's her name, and it's, like, very clearly trying to defraud everyone she knows. And, like, this is someone who has thousands of Facebook friends because, you know, they just. They're a recruiter, they're a social person. They know people, right? And it was about a month in that the first person got in touch. Being like, that wasn't you, was it? Tens of thousands of dollars got lost by four or five different people in this particular scam. The recruiter just kind of, like, broke down in tears because people were being defrauded, quite literally because they trusted her. Meta actually didn't shut down the account for weeks more until she basically organized a brigade of everyone she knew to just try to, like, repeatedly report her account over and over for scam activ and for impersonation and for being hacked and, like, just any report they could file. Like, she estimates that there were more than 100 reports filed before Meta actually took action on this thing.

[Brian Reed]

Yeah, you write about this one guy, Mike Lavery. He's someone she had worked with as a recruiter, basically, who called her and said he'd lost a bunch of money.

[Jeff Horwitz]

Yeah, I think he lost around $40,000. And, like, Mike Lavery, really good guy. And by the way, like, hats off to people who are willing to talk about this stuff. I think sometimes there's, like, a sense of stigma that, like, oh, my God, I got ripped off on the Internet. What a dummy. Mike definitely did not strike me as a dummy. Mike is a smart guy who let his guard down because he was talking to someone who he had the highest personal and professional respect for, that he knew.

[Brian Reed]

How did she feel about it? What had happened when she learned that people had actually given money to these scammers that were using her face and account and her credibility?

[Jeff Horwitz]

Just horror. I mean, that people were being defrauded precisely because they trusted her.

[Brian Reed]

The people being scammed aren't the only victims here. Organized crime is behind a lot of these schemes. And reporters, including some of Jeff's colleagues at Reuters, have documented how these organizations have set up a vast system of human trafficking where people, oftentimes in Asia are lured to a meeting with the promise of a job and then drawn into indentured servitude, put in a compound, and forced to scam people around the world.

[Jeff Horwitz]

These are like basically prison camps where people who've been mostly human trafficked are forced to work in what are essentially scam call centers. This is an industry that is like few hundred thousand people working in it. Myanmar border, basically in the jungle. This is sort of one of the more lawless areas of the world, and.

[Brian Reed]

People are being held against their will and not being paid to do this or how's that work?

[Jeff Horwitz]

Yeah, you can see these compounds, satellite photos, like, basically towns, like, tens of thousands of people, and they're surrounded by barbed wire and guards, and inside, they are busy attempting to scam people via text, via social media, via, you know, any method they can find. So it's kind of a really abusive ecosystem on both sides. Right. Like, the people committing the fraud are, like, in an even worse position than the people they're defrauding quite often while.

[Brian Reed]

Facebook is making money in the process.

Up next, meta's way of fighting fraud. Spoiler. It involves charging even more money.

So that weird idea I told you about earlier, it was a reporting experiment I wanted to try. I'm interested in people's media diets. Like, we all take in so much media content every day. How can you track all the little pieces of content you're taking in? There actually isn't a really good tech solution to do this. We thought about having people take diaries, but that seemed kind of hard for the digital era. It was suggested to us to try parental monitoring apps, but that actually wasn't right. And then I learned about the plod, which is this little device. I have it right here and you push it and it starts recording. I turned it on and I'm just using my phone. Like I use my phone going on social media, seeing what's coming across my feeds, scrolling through stuff, letting it play.

[Jeff Horwitz]

We have some big news right now. Corporations are announcing mass layoffs.

[Brian Reed]

The only practical way to fix your phone addiction in 2020. Humor you see in Duron's videos or even like Community. When I go, I don't mind rich and cheap.

[Jeff Horwitz]

I'll take rich and cheap.

[Brian Reed]

Really. Sometimes if I was just reading something that wasn't audible to the plot, I would just note it, you know, and tell it what I was looking at. I'm on Instagram. The Free Press. It's time for the Supreme Court to confront Trump. Dr. Becky, my top tips for parenting a defiant kid. When I was done, there's an app on your phone. You go in there, it makes a transcript of it, gives you a summary, and then you can interact with it. You can ask it questions. So I said, this is a recording of me going through my social media. Make a ledger of every piece of media you hear me consume, or note a bullet pointed list. And it did it in seconds. It did the thing that, honestly, like I've been trying to have producers figure out how to do for months. Anyway, it's Plaud P. L A U D. Plaud with a P as in pancake. If you're interested, go check it out. It also really helps support our show. Go to Plaud AI and use offer code question.

Welcome back. I'm talking to Reuters reporter Jeff Horwitz about his recent scoop from Inside Meta showing the massive amount of fraud and scams happening on its platforms and the role that Section 230 has played in allowing this to fester. It's worth mentioning that while reporting this story, Jeff reached out to Meta and a spokesman told him the files he was looking at, quote, present a selective view that distorts Meta's approach to fraud and scams. While those internal records showed the company projected about 10% of its 2024 revenue would be from scam ads, the spokesman said the number was actually lower. But he wouldn't tell Jeff what that supposedly lower figure was. One of the things I found most enraging about your reporting is that Meta knows this is happening. Right, like you found this information from their own internal research and reports and documentation?

[Jeff Horwitz]

Very definitely, yes.

[Brian Reed]

These records Jeff was given from inside Meta show that in 2023, the company dismissed or ignored 96% of all valid complaints people sent in about scam ads. In a typical week, Facebook and Instagram and WhatsApp were getting about 100,000 complaints from users saying scammers were targeting them and dismissing all but about 4,000. That's pretty damn close to dismissing all of the complaints. So when that Canadian Air Force recruiter was making hundreds of requests to Facebook asking him to do something about her page being hacked, it's not unusual that Facebook was doing nothing because that seems to be their M.O. jeff saw in the files that Meta did resolve to do better with this. According to another 2023 document, instead of dismissing nearly all of the complaints users send in about scam ads, their target for the future was to dismiss three quarters of them. That was their goal. Just to give you a sense of how few fucks Meta seems to give about this, there's this guy, Andrew Forrest. He's an Australian billionaire with a cute nickname, Twiggy. Twiggy. His nickname is Twiggy.

[Jeff Horwitz]

That's right. You gotta love the Australians for this stuff.

[Brian Reed]

Scammers started using pictures of Twiggy in ads, trying to get people to send money for investment schemes and things like that. It was really bad. There were hundreds of thousands of these scams.

[Jeff Horwitz]

Dr. Forrest was very incensed about the sheer volume of ads ripping off Australians using his likeness and, you know, in some cases, voice, because they were using generative AI impersonations to say, Twiggy thinks you should invest in crypto with this, you know, altcoin type stuff. He got really incensed. He tried to talk to Meta about it, I think, like, tried to sort this out with Mark Zuckerberg personally. Billionaire to billionaire did not receive the attention that I think he was hoping in terms of getting this to be taken down, and so he sued.

[Brian Reed]

If a fellow billionaire can't appeal to Mark Zuckerberg personally and get a scam ad taken down, what hope do any of us have?

Twiggie's lawsuit is pretty interesting. Meta does what it normally does in these situations. It filed a motion to dismiss the lawsuit saying Meta is not responsible for these scam ads at Twiggy because they come from other users. And so section 230 shields the company. But the court, in a very rare move, did not rule In Meta's favor here, the court said it's possible in Andrew Forrest's case that section 230 may not apply. And so for now, Twiggy has been able to puncture section230 just a little bit. The lawsuit's moving forward and the next phase could get juicy because Meta has to disclose how its ad system works. This could become a landmark section 230 case. If Twiggy is successful, though, there's a lot further to go and we'll see where it ends up. In the meantime, Jeff says the contents of the leak do give a sense of what Meta has tried to do to combat scams and frauds, and it's telling about how the company tends to approach problems like this.

[Jeff Horwitz]

So, for example, there was an effort last year, I mean, I think it's still ongoing, to put extremely high risk advertisers, like the ones that Meta is like, virtually certain are bad news from, like, previous behavior.

[Brian Reed]

Like they can just tell.

[Jeff Horwitz]

Yeah. From previous behavior. Or let's say, you know, like, I don't know if they're using a credit card that was used in three separate scam attempts. You know, that might be an indication of something like that.

[Brian Reed]

Jeff says Meta has models it can use to predict the likelihood based on various factors that an account or a post or user is a scammer. And it started this effort where it would put higher risk accounts through extra steps before it allowed them to buy advertising. Extra verification, for instance. Sounds reasonable. But here's the thing.

[Jeff Horwitz]

That program was given a limit for how much revenue it could cost Meta, and that limit was 0.15% of all Meta revenue. Don't get me wrong, that's. That's like well over $100 million. But it's also not much, if that makes sense.

[Brian Reed]

According to Jeff's reporting, Meta's position is basically, there are all these accounts we know are probably bad actors, but once going after them costs us 100 or so million dollars, we're going to stop there. We're not going to sacrifice any more money. Everybody else can just skate by. They could have just said, if we're virtually certain that an account or advertiser is a scam, we just don't do business with them ever, Right? Yeah.

[Jeff Horwitz]

Or we're going to put high risk advertisers through their paces a little more and it'll cost us what it costs us. That's like kind of not a way that they tend to approach things. It tends to be like, okay, well.

[Brian Reed]

Here'S the budget When Jeff reached out to Meta for his story, the company told him this revenue percentage he saw in the leak. It wasn't meant to be a hard.

[Jeff Horwitz]

And fast number, but like internally, people are talking about this as a number that they can't exceed, which is 0.15%. They were going to do it, but it was limited by revenue impact. And so that's one approach. Another approach that they I think have leaned into a little more heavily and which actually is effective, even if it's a little like queasy making, is they charge extra to advertisers that they suspect are scammers. If there's an entity that seems likely to be a scammer but isn't certain to be a scammer, they'll actually charge them a penalty bid. So let's say scammer Brian is assumed to be 80% likely to be a scammer by Meta. And honest Jeff, who's another advertiser and selling legitimate goods, has no risk Meta is going to charge a penalty on whatever you bid, depending on how likely it is that you're a scammer. This is something that does reduce how many scam ads get sold, but they make more on every ad they do sell.

[Brian Reed]

Dicey, huh? The more likely an ad buyer is to defraud Meta's users, the more money Meta makes off them. What is the argument that that is not Meta being complicit in the fraud?

[Jeff Horwitz]

I think the argument is that Meta's goal is to reduce the rate of scam activity on its platforms overall, and that this does that because penalizing scammers does cost the money. They make less revenue overall per the company.

[Brian Reed]

Okay.

[Jeff Horwitz]

And I think that's credible. So you are hurting their business disproportionately.

[Brian Reed]

Right. That's like the kind of a thousand foot view, but like the ten foot view on, you know, one account where they're, they have a high level of certainty that it's a scammer and they're going to make more money on it, rather than just saying we're not going to do business with you. What's the argument from that perspective? That that one instance, which is replicated many times over, is not Meta being part of fraud.

[Jeff Horwitz]

I would say that some of the documentation I've seen did express similar concerns to what you've raised about exactly how Meta was going to apply this and under what circumstances. And all sorts of questions like that just, they were. It was awkward, right? Like, I think it's just a feels wrong tool. I think To a lot of people, like, if you think about it in an offline sense.

[Brian Reed]

Yeah.

[Jeff Horwitz]

Like, before you recommend someone does business with someone or, you know, like, you show them something, like, if you have questions about whether they're legit, you check them out first. Right?

[Brian Reed]

Yeah.

[Jeff Horwitz]

You don't just, like, be like 15% scammer, let them go. Like, you're like, okay, no, I'm going to need to see your id. I'm going to need to verify that you're like, to the best of my knowledge, clean. It's. I think this is a place where just like, the business of the Internet just works in a way that like, is kind of fundamentally uncomfortable to an offline mindset.

[Brian Reed]

This is the business model that section 230 plus the lack of significant regulation of social media companies in the US has allowed to develop.

[Jeff Horwitz]

You know, the company is the. Where that fraud is uncomfortably high on its platforms and is, you know, working to bring that down in future years, but has chosen to not take what it referred to as like, the aggressive.

[Brian Reed]

Fast train because in the meantime, it makes them a lot of money.

[Jeff Horwitz]

I have not seen any document that says we love scams because they make us money. I have seen a lot of documents that say that we can't do things because the revenue impact would be too high or we are very concerned about hitting legitimate advertisers. So I think the threat, this is something that I think is just important to understand about how the company approaches these things. For an advertiser that Meta's automated system thinks is dirty, suspects is dirty, and is likely attempting to defraud people, the level of certainty that Meta's prediction system needs to have to cut that advertiser off is at least 95%. So if it's like 85% likely that an advertiser is a scammer, they will not stop taking their money.

[Brian Reed]

And this is like a formal metric they have.

[Jeff Horwitz]

Yeah, yeah, absolutely. Look, this is kind of not wildly surprising in context of the history of the Facebook files and the level of certainty that you had to have to take down, say, violence, inciting content was in that same ballpark, the level of certainty you had to have before you'd remove an account that you thought was likely, sort of, shall we say, interested in children in the wrong way. Also, in that same ballpark, it seems.

[Brian Reed]

I mean, just as kind of a layperson to this, who's new to this relatively, it seems high to me. But how should I think about that benchmark?

[Jeff Horwitz]

I mean.

I think you should think about it any way you want. Brian.

[Brian Reed]

Jeff's a reporter's reporter. He's not going to tell me or you what to think. He's going to give us the facts, let us make up our minds. As he and I were Talking about Section 230, he seemed reluctant to stake out a position on it. He told me he doesn't know what the right solution is there, whether reforming 230 is the way to go or not, given the trade offs. What he does know, though, the legal framework we've set up for the online world in the US has allowed Meta to act the way it does.

[Jeff Horwitz]

One of the things that the Meta documents I've seen make very clear is that the company does have the capacity to reduce the rate of scams. And in fact, it could do so very dramatically and abruptly if it so chose. But that's just sort of not the balance it struck. Let's say I ran a pawn shop and 10 guys walked through the door, each with some gold jewelry, and I knew that nine of them had probably stolen that and the tenth one was clean. I probably wouldn't do business with them. Right? Yeah, I would be worried about my legal exposure, but that's kind of not the model that we've taken in the online world. It just seems strange to think that you could have no responsibility for targeting fraudulent ads to your user base. I mean, that is what Meta has said, right? In the Andrew Forrest case, they explicitly state in their effort to get this thing dismissed that they believe they do not owe Dr. Forrest or any user a duty to remove fraudulent ads from the platform. That's just not a thing that they are responsible for. They may do so, but that's up to them. And it's not a thing that legally they are responsible for. It's just weird reading that, right?

[Brian Reed]

Yeah. I've encountered this too, as I've kind of dug into this world more, where we've, I think, kind of accepted as a society this too big to fail kind of mindset. Like just the scale and volume of activity on these platforms and on the Internet means the rules have to be different. But do they? Like, is that like a fundamental truth that we have to accept, or could it be that these businesses are fundamentally flawed and maybe shouldn't exist as they do?

[Jeff Horwitz]

I mean, something that I try to make very clear in my work is that there are other ways this could be. There are other ways that things could be prioritized. I think there's still the sense of like, well, you know, it's the Internet what you going to do? People do crazy stuff on the Internet and they do, but it is a platform choice to figure out what sorts of behavior are rewarded by a platform. Right? Like, do you reward people who, you know, share moderately things that don't later get reported as misinformation? Or do you reward the people who share a ton and post the most inflammatory stuff possible? Like, I remember during the pandemic, significant amount of like, anti vaccine content was coming from users who were commenting more than 200 times per hour on posts.

[Brian Reed]

Oh my God.

[Jeff Horwitz]

Which is like an indication of like, what behavior is rewarding. And I don't know what the appropriate limit on how many times a comment per hour can be made that affects what people see on the platform is. But like, maybe it's not 200. Those are decisions that can be mathematically embedded in a platform and one of them leads to a very different environment than the other.

[Brian Reed]

This is exactly what I'm saying. And this is why you need more like, external accountability. Basically.

Imagine if section 230 were changed so users could easily sue Meta for serving them scam ads. One thing I found noteworthy from Jeff's reporting, internal records showed that Meta prioritized cleaning up its platforms of fraud in countries where there are regulations about it. I guess outside accountability works. In the US we have very few examples of that. And so we're left in large part to rely on reporters like Jeff to fill in the gaps.

[Jeff Horwitz]

Leaks are a primary source of information about, like, how the hell products that are so deeply ingrained in our lives actually work. Like, this is not an ideal way for information about such societally important platforms to make it into the public domain. It's periodic. It requires on, you know, individual employees to take risks. This is a broken system that, you know, perhaps my career has been the beneficiary of.

I think social media is something that we are still trying to get used to, right? Like, it feels like it's old hat. We've had it forever now. But in the, you know, terms of like, relative novelty of forms of communication and media, it is brand spanking new. You think of the printing press and how that changed Europe and made the world better in many places and also led to like, literally decades upon decades of warfare that took a lot of time before society adjusted to that.

[Brian Reed]

And the thing I'm learning to keep in mind, we can shape it, our online experience, the state of our information ecosystem. It is not inevitable, I think, if.

[Jeff Horwitz]

The entire impact of my career were to make clear that social media platforms that how people behave on them is heavily influenced by the design choices of the owners of those platforms. If that was the only thing that people took away from my career, I would be wildly pleased with myself.

[Brian Reed]

Jeff Horowitz at Reuters.

Today's show was produced and edited by Managing Editor Kevin Sullivan. Check out our substack question everything.substack.com where there's debate a brewing over my section 230 stance and keep tuning in here on the podcast as well.

Before we go, you may have seen in the news this week that Meta had a huge, huge victory in court. The federal government was trying to break the company up, claiming it was a monopoly. This was a real potential threat to the company, but the judge disagreed. He sided with Meta. They get to keep operating as they are and I saw this one statement from a Meta spokesperson after the decision that jumped out at me. Given the reporting you just heard on our show, the spokesperson said meta's social media platforms are, quote, beneficial for people and businesses and exemplify American innovation and economic growth.

The executive producers of Question Everything are Robin Simeon and me, Brian Reed. Our team here at Question Everything also includes producers Sophie Kazis and Zach St. Louis, contributing editors Neil Drumming and Jen Kinney, associate producer Kevin Shepard and contributing producer Sam Egan. This episode was fact checked by Annika Robbins, mixing and sound design by Brendan Baker. Our music is by Matt McGinley. Special thanks to Blake Morrison. Our partners at KCRW include Arnie Seiple, Tejal Algemera, Natalie Hill and Jennifer Farrow. We'll see you next week. Before we go, just one more thing to say about our sponsor Plaud. One question I definitely had. I wanted to know what their privacy practices are and so I asked the company and here's some things they told me that I just want to share with you. I asked if you can control your data. You do. You can choose to store the data locally on your device or securely in the encrypted cloud. You can export the data. You can permanently delete it anytime the company tells me they don't sell share or market user data. All recordings, transcripts and notes belong entirely to you as the user and aren't used for advertising or AI model training without explicit consent. And they also point out they adhere to a bunch of international privacy regulations like GDPR and others. You can read all about this on their website Plaud Plaud AI. If you do decide to buy one, please use our offer code. Question.

[KCRW Sponsor Announcer]

I just got my new phone and the KCRW app is the best way to get the music and shows you love from kcrw. And it's been totally redone to be cleaner, faster and more reliable. And there's two new music Dance 24 and Vintage 24 and they're only in the app plus real time now playing so so you never miss a track id look up KCRW in the App Store and be sure to make a free account to use all the new features.