← The AI Policy Podcast
The AI Policy Podcast
Trump Signs EO Targeting State AI Laws While Meta Showcases Risks of Weak Tech Regulation
00:56:33
We talk Trump's new EO on state AI law and Meta's reliance on scam-driven revenue.
Loading summary
Transcript47 lines
- [00:00]
A
Foreign.
- [00:11]
B
Hi, everyone. Welcome back to the AI Policy Podcast. It's great to be back with Greg. It's been a while since we've done one of these together, but today we're going to start by discussing Trump's recent executive order that targets state AI laws. And then later, we're going to break down the recent headlines about Meta's fraud revenues. So I'm Sadie McCullough, and I'm joined by Greg, as always. So welcome back.
- [00:32]
A
Great to be speaking with you, Sadie. And let me just say to our listeners, if you have not been reading the reporting coming out of Reuters about Meta and fraud, boy, oh, boy, should you stick around, because I can't believe I'm saying this, but it is such bombshell reporting. It's one of those things where, like, how can you claim to have a credible opinion about AI regulation unless you know about what Reuters has found reviewing internal documents at Meta? It's that much of a bombshell kind of reporting, and there's a lot of them, and they're still coming out. So we're going to dive through a lot of that material as well as the congressional request for an inquiry around that and how it does and does not tie to debates on AI regulation.
- [01:19]
B
All right, well, I have a lot of questions for you, so let's not waste any time and jump right in on the executive order. So, on December 11, President Trump signed an executive order targeting state AI laws. We've done sections on this EO in past episodes, first on the leaked draft and then after Trump officially announced that he'd be signing it. So now that we have the final text, I think it's definitely worth unpacking what's changed from the first draft and what actions we can expect for the administration going forward. So, Greg, let's start with the former. How does this final version compare to the initial draft?
- [01:53]
A
Yeah, so I think in. In broad strokes, the leaked draft was accurate. There were changes between the earlier leaked draft and this final one. But the basic mechanisms of the executive order are ret. So let's talk about that. I mean, the goal here is to fight state AI regulatory laws to make them less burdensome, to make them less likely, and, of course, to move the AI regulatory debate to a federal standard on the grounds that interstate commerce is at risk when you're talking about AI, and so therefore, it should be a federal prerogative. So what are the mechanisms that this EO creates for fighting state AI regulations? Well, the first is it directs the Department of Commerce to review existing laws that it identifies as overly burdensome regulation. And it also says that the AI czar in the White House, David Sachs, and the director of White House ostp, Michael Kratzios, can specifically suggest laws that are worthy of investigation here. Note that that is laws that have passed, not laws that are, you know, being actively debated in state legislatures. The second thing is this Department of Justice task force, so it's being called the AI Litigation Task Force, whose sole responsibility shall be to challenge state AI laws that are considered overly burdensome. So they're going to file lawsuits and basically say, you know, you can't do that. Then there are some additional hooks, like withholding federal funds. So restrictions include withholding state funding on broadband equity access and deployment funds, and perhaps federal research type funds. And that is similar to what you've seen in the past. There have been times when Congress would sort of say, you cannot get access to, for example, state highway repair funding unless you adopt our position on this other issue. And it's a way that the federal government has in the past tried to federalize certain types of legal regimes when states are not going the way it wants. Those have not always been found to be constitutional by the courts. But these ones, they're saying, are sort of tied to AI, which I think is meant to make it more likely to survive court review. Then there is Federal Trade Commission preemption. And here I'll quote from the actual executive order itself, which is an explanation of, quote, the circumstances under which state laws that require alterations to the truthful outputs of AI models are preempted by the Federal Trade Commission Act. So this is something where they're really focused on the truthful outputs. And in the mind of the administration, AI systems have been forced to generate false outputs, often for woke ideology reasons or DEI things. I think the example of the launch of Gemini, where it depicted, you know, historical figures from historical contexts such as, like, African American Nazis, you know, when asked to generate a picture of a soldier in Germany in 1940, and that's just because they had certain safeguards that were designed to show proportionate representation of different ethnicities. And the way that interacted with requests for content really did not rub Republicans the right way. So that example looms large. And just anything that is coming to ideologically motivated outputs, they're interested in going after that. And the Federal Trade Commission act gives them sort of an interesting legal hook for going after that. So that's the mechanisms for fighting state laws. Then there is also mechanisms for introducing federal standards and legislation, which is to Say, you know, something is going to happen at the federal level. It's not as though nothing is going to be done. So within 90 days, quote, the chairman of the Federal Communications Commission shall, in consultation with the special advisor for AI and crypto, initiate a proceeding to determine whether to adopt federal reporting and disclosure standard for AI models. And that preempts state laws. So we're talking about a federal reporting and disclosure standard. And keep in mind, right, there are disclosure and reporting standards that apply in some of these state laws. So it is, it is trying, at least in this instance, to replace something with something, to replace something at the state level with something at the federal level. And then David Sachs and Michael Kratzios are also directed to jointly prepare a legislative recommendation establishing a uniform federal policy framework for AI that preempts state laws. So they've got this FCC hook that they can do within existing legislative authority. And then they're also asking these two White House officials to engage and prepare legislative recommendations that the White House can take to Congress and say, please pass this. This is what should be the federal standard.
- [06:52]
B
Sure. So we're seeing like a two pronged approach here.
- [06:54]
A
Exactly. And there's a lot of that in here. So that's, that's sort of what is maintained, that sort of overarching architecture. Broadly the same. They added in a few things that think reflect the debate as it's been happening in Congress and in the media about this executive order. So for example, in the purpose section, they added a line that says a federal framework, quote, should also ensure that children are protected, censorship is prevented, copyrights are respected, and communities are safeguarded. So essentially they're pointing directly at many of the harms that are talked about in the AI regulatory debate and saying whatever our federal legislative proposal is going to be, it needs something to say on these key issues in the AI regulatory debate. And then they also added a section in the legislation section. This is part B, quote, the legislative recommendation called for in subsection A of this section shall not propose preempting otherwise lawful state AI laws relating to one, child safety protections. So that's really interesting. That's a noteworthy concession. I think they're recognizing that with the report reporting about AI leading to increases in select cases of teen suicide or depression or exposures to sexual content, there's a lot of very politically hot button issues where they're saying states, if you're going after this stuff, we're not going to try and stop you. And I think that's a, that's a reflection of the fact that, you know, protect our children is such a salient political message, it's such an important topic that they don't want to say that that's part of this executive order is, is reduc protections that are going to exist at the state level for, for that.
- [08:39]
B
That's a great breakdown. And what, what can you tell us about the people who are supporting this executive order? Who are they and what are they saying?
- [08:47]
A
Yeah, well, standing behind President Trump when he signed this executive order, among that group was David Sachs, the Aisar, Sriram Krishnan, who is sort of David Sachs's number two, and Senator Ted Cruz, who had been very prominent in the debate on the, the legislative side of the federal moratorium on AI regulation. So Sachs has said on X, quote, this is not AI amnesty or AI moratorium. It is an attempt to settle a question of jurisdiction. So this gets back to that argument about the interstate commerce clause and how this is an appropriate federal matter. And then Senator Cruz wrote on X, quote, president Trump took an important step today to promote American leadership in AI. We don't want China's values of surveillance and communist control governing AI. We want American values of free speech, individual liberty and respect for the individual. So I think that's noteworthy. Keep in mind here, right, he's saying we're, we're doing this because we're competing against China and AI. And I have to mention, right, that in the context of the decision to sell H200 chips to China and strengthen their AI ecosystem, you know, these kind of arguments just land a little bit differently than they might have six months ago in this administration. And then finally, I want to bring up a quote from Dean Ball, who previously served in the White House Office of Science Technology Policy and is now at a think tank. And he said, quote, on X, I am happy to see the administration pushing forward on a national AI law, which America needs to maintain the dynamism of its AI industry and to prevent over regulation by the EU and state governments. Nearly all major technologies are governed by a combination of federal and state laws. However, he provides this caveat, quote, on its own, the EO cannot create a nationwide preemptive framework for that. We will need a new law passed by Congress. And so I think that sort of highlights the strategy of the White House here. They're trying to take immediate steps by threatening court action by setting up this Department of Justice task force, by, you know, involving the FCCC and the FTC to just try and slow states down, to complicate states efforts to regulate AI. But Ultimately, it comes back to what we've been saying on this podcast for years now. Something has to be done by Congress. Something has to be done at the federal level. Executive action alone is not enough. And it's so weird that almost all sides agree that that is true and yet so little progress in that direction has been made.
- [11:27]
B
Absolutely. And such different approaches to getting there. That being said, who, who's in opposition of this now? Let's switch gears here. And what are they talking about?
- [11:40]
A
Well, I think it's really interesting that there is pretty broad bipartisan opposition to this move. It's one of those areas in which while there are Republican supporters, of course, Senator Ted Cruz, for example, there's also quite prominent Republican opponents, and some of them are attacking it on a states rights kind of argument that really the federal government needs to know its under Article 10 of the Constitution and other kinds of opposition is focusing on the fact of the AI harms and the need to prevent them and the fact that this is a selling out to big tech companies. So let's just think about, you know, who's out there publicly opposing it. Governor Gavin Newsom of California, a very prominent Democrat that just passed an AI safety law that he signed. Unsurprising, right, that he would be in opposition. But, but Ron DeSantis, the governor of Florida, has also come out and as a prominent Republican said on X quote, an executive order doesn't can't preempt state legislative action. Congress could theoretically preempt states through legislation. The problem is that Congress hasn't proposed any coherent regulatory scheme, but instead just wanted to block states from doing anything for 10 years, which would be an AI amnesty. And this AI amnesty phrase is showing up repeatedly in criticisms of this executive order as sort of a letting the tech companies off easy while there are these harms coming in. So Republican lawmakers in Congress similarly promoting and opposing this. So Utah Representative Doug, and I might say his name wrong here, but FIFA, F I, E, F I A fifi, maybe. Trump's new order, quote, is an overreaching act that fundamentally disregards the 10th Amendment and the necessary role roles of the states in technology governance. While I support a singular national framework, it must be developed through the proper legislative process in Congress to ensure full debate and transparency, not through unilateral action that threatens state funding with no federal standard to replace it with. So I think what you're seeing here, and by the way, plenty of Democratic lawmakers have come off in opposition to this, not just state ones, like, for example, California State Senator Scott Weiner, who was heavily involved in drafting SB 53, 3, but also those in Congress. So you really are seeing pretty broad opposition to this. But if you look at the specific arguments that different constituencies are making, many of them are not necessarily opposing federal action over state action. They're supporting congressional action over executive action. And that, I think is really kind of where we are in this sort of muddled debate, where different people disagree for different reasons. And it's very hard to reach consensus on anything, even though there are many people who agree with parts of each other's arguments. And so into that muddled debate, the White House has taken this pretty decisive action. But given, you know, that we, we have seen folks already threatening lawsuits, I expect this will head to the courts in not too long.
- [14:46]
B
Sure. So forgive me if I'm jumping the gun to a question I wanted to ask you in a second, but you sparked my interest when you started talking about Gavin Newsom and his recently signed bill. So what can you tell me about what happens now to that bill, in particular, other bills like it that have already been passed now that this executive order has also been signed?
- [15:09]
A
Sure. Well, Gavin Newsom in California, as is State Senator Scott wiener, as is SB53. And so I think Scott Wiener actually said it best in terms of what is the future of SB53. He told the New York Post, quote, if the Trump administration tries to enforce this ridiculous order, we will see them in court, end quote. And I think that's exactly where this is headed. An earlier version of the executive order, the leaked version of the executive order, very clearly targeted California's SB 53. That that explicit reference was removed from this updated version, the final version of the executive order. But it's crystal clear that the White House does not like SB53 and views a key role of this Department of Justice task force, squash. So it's very difficult for me to see how this does not head to court in the not too distant future. I think it's very clear that they're on the, they're on the talking list. And then what's interesting is not everybody who is affiliated with the Trump administration or previously affiliated with the Trump administration expects the Trump administration to win in court. Dean Ball, who we talked about a moment ago, he told Reuters, quote, I think the administration has a 30 to 35% chance of this working legally, which I think is so interesting. And now keep in mind, when you say of this working legally, you're talking about, you know, one what is right legally, but you're also talking about what courts are going to decide legally. And that depends on, you know, which court, which judge. There's a lot of uncertainty. Even folks who think the law is on their side are not always sure that they're going to win in court. And so here I think you have Dean Ball, who in general is sympathetic to the idea of federal preemption, and he's not so sure that this is going to turn out in the administration's favor in court. But as I've said on this podcast before, the Trump administration can get some of what they want just by raising the specter of court, just by raising the specter of losing federal funding for AI or for other things. Because that can, you know, be in the background of state legislatures minds as they're debating, well, is this really something that we want to go to the mattresses over? I think some people will be delighted at the opportunity to fight with Trump Senator Scott Weiner. It's probably going to raise his national profile. Right, if there's a big lawsuit about his bill and the Trump administration. But other folks who are just trying to get some stuff done, perhaps they will view the calculus differently and will change their behavior, will change the wording of their laws, to take this example, determine for that. Exactly.
- [17:50]
B
So on that note, what has the Trump administration done to mitigate concerns of their Republican allies?
- [17:57]
A
Yeah, and I think, I think the way you phrase the question is, right, I haven't seen a lot of evidence that they've done things to mitigate concerns from Democrats. But given that, you know, Senator Blackburn, a Republican of Tennessee, had been staunchly in opposition to the earlier legislative attempt at an AI regulatory moratorium, folks in the White House, notably Sachs and Krishnan, have been going, you know, to Congress, trying to meet with leaders, talking to them, their staffs, on, you know, this is not an assault on states rights. This is something that we can, we can be comfortable with. Now, I think one thing that's noteworthy is according to reporting by the Washington Post, Trump actually called Senator Marsha Blackburn because her pulling her support for the effort to include the legislative moratorium was so influential. And I think, I think here I'm sympathetic to the Trump administration because some of the criticisms that Senator Blackburn articulated on that legislative proposal actually probably would have been allowed under the legislative proposal because it didn't ban all types of AI regulation. It was, as I said before, focused on those sort of preemptory requirements before a product reaches the market, not regulating the harms that AI causes. So, so that type of clarification what they're trying to achieve, under what circumstances regulation would be perhaps even supported by this administration. And that call out that whatever this federal framework is going to be is going to target those specific harms. Those are a lot of harms that Senator Blackburn had talked about when she was opposing the federal one. So I think we're coming back to the where does Congress go next? What does this legislative proposal look like? And as our colleague Matt Mand pointed out, there's no deadline for when the legislative proposal for this is going to go into effect. So the fight in the courts could start almost immediately. How long it's going to take the White House to come up with a appropriate legislative proposal and much less get it passed into law, who knows? And so that's kind of where we are right now.
- [20:13]
B
Yeah. And to go off of that, you already started touching on this, but what impact can we expect this executive order to have on state and federal legislation moving forward?
- [20:24]
A
Yeah, well, we already talked about, you know, how this is going to end up in the courts. But there's Shaquille Hashim, who's the editor of the AI newsletter Transformer. He wrote something that I thought was actually kind of insightful here. Quote, it's legally messy, possibly unconstitutional, and will probably just be ignored by blue states like California and New York while court battles rage on. That will have the effect of creating less regulatory certainty rather than more. And it's that final part of what he wrote that I thought was so interesting. You know, part of the argument for why this is a pro innovation, pro economic growth move is about regulatory certainty. And the reality here is all this is headed to the courts. That's a long period of time to not have certainty. And so really, the part where we could drive towards certainty is getting something passed through Congress, which is not easy in the year 2025. But I think it's pretty clear that that is what it would take.
- [21:29]
B
Great. So what I'm hearing you say overall, and to summarize, is that this executive order is going to end up in a lot of court cases and it's having a lot of opposition and support, opposition from a bipartisan group. But we'll have to see what kind of legislation can potentially come through and end up in a uncertain regulatory landscape.
- [21:54]
A
Yeah. And when has the Trump administration ever shied away from a fight in court?
- [21:58]
B
Exactly. So, as always, we'll be tracking this. Our program director, Olock Mehta, published a commentary on this last Friday, which you can find on our website@csis.org Greg, let's move on to the second topic, which is Meta. So the administration is betting that fewer rules at the state level will lead to better outcomes. But before we accept that bet, it's worth looking at how these firms behave when regulation is already very light. Which brings us to Meta. In the weeks before Trump signed this executive order, there were a slew of headlines about Meta's questionable internal decisions. So I know you've been reading a lot about this. Can you unpack some of what has been reported on recently?
- [22:40]
A
Yeah. So, first, let me just give a shout out to Reuters, which has clearly gotten access to a remarkable trove of internal documents from Meta that Meta spokespeople have said are genuine documents that were leaked to Reuters. And they're quoting from these documents. And the claims in these documents are just astonishing. One of. One of my colleagues basically said, okay, this is tobacco companies saying cigarettes cure cancer bad. Right? I mean, like, it is a remarkable string of pretty bad stuff. And again, it's coming from Meta's internal documents, their own assessments of themselves and their behavior. So it's. There's not a lot of reason to doubt what's being said here. That said, Meta did push back on some of these claims. So I'll just say, you know, what I'm saying, based on this podcast, is assuming the Reuters reporting is true, Senators Blumenthal and Hawley are calling for investigations by the Federal Trade Commission and the securities and Exchange Commission. And that sort of going into the nitty gritty detail will presumably bring out the truth in its full light. But given that we're talking about a lot of internal documents, I think we just have to go forward and talk about these because the claims are so explosive. Now, why do I want to talk about these? These are primarily talking about Meta's social media stuff. Well, it's for the reason you said, Sadie, which is, you know, these. Meta is a extremely powerful social media company. They are an extremely powerful AI company, and social media is often in Washington, D.C. explicitly talked about as the failure to regulate, as in, we should have done something, and we did not. That was actually the frame that multiple senators from both parties used. When OpenAI CEO Sam Altman testified before Congress in, I want to say, the summer of 2023, they literally said, let's not repeat the mistake of failing to regulate social media. So Meta is not an especially heavily regulated company. There is not a lot of mandatory reporting and disclosures from Meta to the government about what goes on in their platforms. And now looking at some of these Documents, we have learned what's going on in their platforms and what Meta is doing about it, and it's just very, very, very bad. Okay, so let's, let's just talk about some of what's on here. Number one, according to internal documents from Meta, as captured by Reuters, quote, Meta internally projected late last year that it would earn about 10% of its overall annual revenue, or $16 billion, from running advertising for scams and banned goods, internal company documents show. So one out of every one out of every $10 that the company earns from advertising is coming from scams and banned goods. And when we talk about banned goods, I think it's, it's worth just pointing out, like some of the banned goods are also probably scams. So, for example, advertisements for pornography or prostitution services happen on Meta, and then when you click on those ads, you don't get pornography or prostitution services. What you get is a financial scam. Right.
- [26:31]
B
So when you say on their platforms, are you just talking Facebook and Instagram.
- [26:35]
A
Or Facebook, Instagram and WhatsApp and, and just so, just so folks know, you know, the way that WhatsApp is commonly used in the United States is as a messaging service, really like a replacement for text messag. But in foreign countries, WhatsApp groups can reach hundreds. I'm not sure if it's still true, but at one point it was definitely thousands of people. And so the WhatsApp experience in foreign countries looks a lot closer to the way social media works sometimes, the way Instagram and Facebook work here in the United States. So it's certainly relevant there. Okay, let's just go down the list of bad things. Actually, I have a different idea. So on this podcast, I've already read a lot of quotes, and maybe some listeners out there love it, maybe some listeners out there hate it. But I'm going to read the longest quote ever.
- [27:32]
B
This is definitely going to be the top of the tally for our team.
- [27:36]
A
It's the letter that Senators Blumenthal and Hawley sent to the chairman of the Federal Trade Commission and the chairman of the Security and Exchange Commissions. And for those of you who are watching on YouTube, I'm holding up the letter here, and I highlighted everything that I thought was important in the letter. And as you can see, I basically highlighted every word. And that's why I feel it's so important to read this thing out loud. This letter is a banger and it goes through everything in the Reuters reporting. So this was have me on the.
- [28:09]
B
Edge of my seat.
- [28:09]
A
Greg, you gotta, you gotta Listen to this folks. So this was sent on November 22, 2025 Dear Chairman Ferguson and Chairman Atkins, we write to urge the Federal Trade Commission and securities and Exchange Commission to investigate and if appropriate, bring enforcement actions against Meta for its facilitation of and profiting from criminal investment scams, fake government benefit schemes, deepfake pornography and other fraudulent activities. According to recent reporting from Reuters, internal documents show that Meta has earned earned billions of dollars from fraudulent and scam advertising while internally stifling measures to prevent fraud. By its own assessment, 10% of Meta's annual revenue for last year was generated from scam advertisements, a windfall of $16 billion in one year alone. Meta's central facilitating role in scams against consumers is unprecedented. By its own employees assessment, Meta was involved in 1 1/3 of all successful scams in the United States and was unmatched by other big tech platforms. Using the Federal Trade Commission estimate of $153 billion in overall fraud per year, this would suggest that Meta was responsible for more than $50 billion in consumer loss last year. This internal finding supports a December 2024 report by a UK financial regulator that found Meta's platforms Facebook, Instagram and WhatsApp were linked to 54% of payment scams and a JP Morgan Chase investigation that found that Meta accounted for nearly half of all reported scams on Zelle, such as romance scams that feign a love interest to deceive a victim, and fake investment schemes. Meta's ill gotten gains appear to be no accident. It has made conscious choices based on business considerations that turned a blind eye and enabled it to profit from illicit advertisements. According to Reuters, Meta had specific revenue guardrails that blocked its anti fraud team from taking action that would cost Meta more than Even a mere 0.15% of the company's total revenue. In line with that approach, Meta only bans advertisers if its automated system predict the marketers are all but certain to be committing fraud, and by its own estimates, it has ignored or incorrectly rejected 96% of user reports of fraud. Moreover, perversely, Meta reportedly charges higher rates for ads that it suspects might be fraudulent, in effect imposing a scam tax that provides an additional lucrative revenue stream that it knows is tied to fraud.
- [31:04]
B
So they're enabling.
- [31:06]
A
Further, because Mesa's advertising algorithm targets individuals based on their past clicks and interests, a person that falls for a scam is more likely to be targeted by even more scams or fraudulent advertising, making Facebook and Instagram a fraud tap that can repeatedly exploit vulnerable individuals. Meanwhile, these scams have been allowed to take over Facebook and Instagram as Meta has drastically cut its safety staff, including for Federal Trade Commission mandated reviews, even as it dumps unimaginable sums into its generative AI projects, as it does every time its abuses and failures are discovered. Meta has repeatedly claimed it is cleaning up its act, but there is overwhelming evidence that it continues to knowingly profit from fraud. Even a short review of Meta's ad library at the time of this letter shows clearly identifiable advertisements for illicit gambling, payment scams, crypto scams, AI deepfake sex services, and fake offers of federal benefits advertisements that often falsely appear to link to legitimate sites. Further, despite the company's rules against nudity, advertisements for pornographic and sexually explicit material, and which often lead to malware, are widespread on Meta's products. While Meta has been warned about advertisement deepfakes impersonating politicians, it still continues to run fraudulent clips, including President Trump claiming to offer a, quote, $1,000 SNAP replacement program preying on the most vulnerable in America. The beneficiaries of these scams are often cybercrime groups based in China, Sri Lanka, Vietnam and the Philippines. The reporting makes clear that Meta will not act on this rampant fraud and criminality unless the FTC and SEC put an end to it. It According to Reuters, Meta executives actually limited their anti fraud enforcement efforts to focus only on, quote, countries where it feared near term regulatory action, end quote. In line with these efforts, in anticipation of enforcement, Meta had budgeted for penalties of up to $1 billion, which were a mere fraction of the $7 billion in annual revenue from high risk advertisement alone. The FTC and SEC should immediately open investigations and if the reporting is accurate, pursue vigorous enforcement action where appropriate to force Meta is full to fully disgorge all profits from fraudulent advertisements, impose steep civil penalties, hold individual executives personally accountable, and seek binding terms to end this scourge on consumers, our economy and our national security. Thank you for your attention to this matter. Sincerely, Richard Blumenthal, United States Senator Josh Hawley, United States Senator wow, that makes.
- [34:01]
B
Me want to delete my Instagram.
- [34:04]
A
Well, it might help your bank account because who knows when you're going to be targeted for one of these scams. I mean, I read the whole letter because I think the letter is explosive. The reporting is also explosive. So. So everything that Reuters has been putting out for the past several months on Meta, if you don't have a Reuters subscription, get a Reuters subscription. Like this is the most important body of evidence that we have right now in the regulating technology debate. It is evidence of extraordinary bad behavior by Meta. And what's even just more appalling is the degree to which they know that this is going on. They have means of going after it, and they disabled those means anytime they appear to be effective, right? So if, if 10% of the revenue is coming from scam fraud, sales of illegal goods, et cetera, and you only give a budget to your fraud prevention team of affecting 0.15% of revenue, you're saying, okay, 10% of our revenue comes from bad stuff, and we want you to get rid of that. And when we say get rid of that, you're not allowed to affect more than 0.15% of revenue. So they're explicitly telling these people, do not succeed in getting rid of these scams because we want them to continue. And gosh, here's the other thing. You know, when you think about tech regulation, the social media regulations, the Internet regulations, a lot of the mental models for this stuff goes back to the phones, the phone regulations, right? So think, Think about, like, regulating telephones in the 1930s in the United States. Well, you know, if you're at and T and you're selling phone service, you're not wiretapping every conversation. So, yes, the mafia is probably facilitating crimes on your phone network, but should we hold AT&T, you know, to be punished for what the mafia does using their phone lines? No, obviously not. Right? But here's how it's different. In the case of Meta, they are wiretapping every single conversation that's going on. They know what all of these ads look like. They know the user reports, and that's some of the most incredible stuff. So this is from the November 6 Reuters article. Meta was also ignoring the vast majority of user reports of scams. A document from 2023 indicates by that year, safety staffers estimated that Facebook and Instagram users each week were filing about 100,000 valid reports of fraudsters messaging them, the document says. But Meta ignored or incorrectly rejected 96% of them. Okay, wow, you think that's bad? Guess what gets worse. Okay, here's, here's another quote. But despite the surge in online fraud, another 2022 document notes the company's, quote, lack of investment, end quote, in automated scam detection. Back then, Meta classified scam ads as a, quote, low severity problem, end quote, viewing them as bad, quote, user experience, end quote. The document says. Internal documents show that Meta directed staffers then to focus mainly on fraudsters masquerading as celebrities and usurping Major brands. Such impersonation scams risked upsetting advertisers and public figures, one 2022 document notes, and thus threatened to reduce user engagement and revenue. But ongoing layoffs at Meta were hindering enforcement. A planning document for the first half of 2023 notes that everyone who worked on the team handling advertiser concerns about brand rights issues had been laid off. The company was also devoting resources so heavily to virtual reality and AI that safety staffers were ordered to restrict their use of Meta's computing resources. They were instructed to quote me. Sorry. They were instructed merely to, quote, keep the lights on, end quote. I mean, my goodness, it's like you are staring at a burning fire going on your platforms and your orders is for everybody to cover their eyes. That's the solution here.
- [38:39]
B
And Meta must be like seeing all of these reports coming out in Reuters. Like, what are they? Are they saying anything? Are they refuting these claims? Are they making any comments?
- [38:50]
A
So again, there has been no claim that the documents are not legitimate. The rebuttals are stuff like, oh, you know. Stone said that while layoffs had occurred, the company had substantially expanded the number of staff addressing scam advertising in recent years, but doesn't say how many, doesn't say what they're doing, doesn't say how they've been effective. It's pretty crazy. Okay, we gotta read another quote. Okay. According to a December 2024 presentation, Meta's user base is exposed to 22 billion organic scam attempts every day. That's on top of the 15 billion scam ads presented to users daily. So, 15 billion from ads, 22 billion that don't involve advertising. Right. And then now, now here comes another quote that is again, incredible quote. Even when advertisers are caught red handed, the rules can be lenient. The documents indicate a small advertiser would have to get flagged for promoting financial fraud at least eight times before Meta blocked it. A 2024 document states some bigger spenders, known as high value accounts, could accrue more than 500 strikes without meta shutting them down. Other documents say, that is incredible. I mean, literally, you can commit fraud 499 times, that's fine, but 500? My goodness, we're going to have to do something about this. And look, it's just because it's so incredibly lucrative. This is 10% of Meta's advertising revenue. Okay, here's the craziest thing of all. Sadie, I'm ready. It gets worse. So the letter that I read, I don't believe you.
- [40:43]
B
Anymore.
- [40:44]
A
The letter that I read was from November 22nd. It was based on the Reuters reporting to date on November 22. But on December 15, another incredible bombshell coming from Reuters authored by Jeff Horowitz and Engin Tom say that the link to China in this story is also extraordinary, that Meta knows that a huge share of fraudulent scam advertising is coming from China, that they're making a lot of money off it. And when they introduced safeguards that were effective at reducing the amount of fraud and scams coming from China, Mark Zuckerberg noticed its effect on Meta's revenue and shut down the anti scam and fraud task force. Wow.
- [41:39]
B
Wow.
- [41:40]
A
So. So yeah, this is why it's not just the folks who are, you know, calling for tech regulation, I'm sure are going to be activated by this. The China hawks, I'm sure are going to have something to say by this. Here's a quote from the December 15th story, which I strongly encourage everyone to read. Though China's authoritarian government bans use of Meta social media by its citizens, Beijing lets Chinese companies advertise to foreign consumers on the globe spanning platforms. As a result, Meta's advertising business was thriving in China, ultimately reaching over $18 billion in annual sales in 2024, more than a tenth of the company's global revenue. But Meta calculated that about 19% of that money, more than $3 billion, was coming from ads for scams, illegal gambling, pornography and other banned content, according to Meta internal documents reviewed by Reuters. Then it goes on. The documents show that Meta believed China was the country of origin of roughly a quarter of all ads for scams and banned products on Meta's platforms worldwide. Victims ranged from shoppers in Taiwan who purchased bogus health supplements to investors in the United States and Canada who were swindled out of their savings. Quote, we need to make significant investment to reduce growing harm, end quote. Meta staffers warned in an internal April 2024 presentation to the leaders of its safety operations. To that end, Meta created an anti fraud team that went beyond previous efforts to monitor scams and other banned activity from China. Using a variety of stepped up enforcement tools, it slashed the problematic ads by about half during the second half of 2024, from 19% to 9% of the total advertising revenue coming from China. Then Meta Chief executive Mark Zuckerberg weighed in. Quote, as a result of integrity, strategy pivot and follow up from Zuck, end quote. A late 2024 document notes the China ads enforcement team was, quote, asked to pause and quote its work, Reuters was unable to learn the specifics of the CEO's involvement or what the so called integrity strategy pivot entailed. But after Zuckerberg's input, the documents show Meta disbanded its China focused anti scam team. It also lifted a freeze it had introduced on granting new Chinese ad agencies access to its plat forms. And the final thing here is by mid-2025, banned ads climbed up to about 16% of Meta's China revenue. So what does this mean? It means they know the harm is being done. They know how to stop it. They proved they can stop it, but when it worked, they didn't like the results and so they shut down the operation to stop it. I am sure that there's a lot of folks on Capitol Hill whose blood is boiling as they read these documents. It is an extraordinary pattern of behavior. It's difficult for me to imagine that this story is not headed to congressional hearings in addition to the requested fec, ftc, sorry, SEC and FTC investigations. It's just truly, truly remarkable stuff, Sadie.
- [45:05]
B
It is pretty mind blowing and I'm so glad that you did all the research so we didn't have to.
- [45:10]
A
No, I didn't do all that much research. I mean, I'm just reading what Reuters is reporting here. So all credit to them and whatever sources they have inside Meta that are so forthcoming with these pretty juicy internal documents.
- [45:22]
B
Sure. So let's bring this back to policy real quick, since we are on the AI Policy podcast and we were just talking about the executive order. So what do examples like this say about what the right level of regulation is for technology and AI more broadly?
- [45:37]
A
Yeah, I think at the risk of driving my audience insane, I think the best answer comes in the form of a quote from the Reuters article. So here's a quote from the November 6 piece from Reuters. The documents note that Meta plans to try and cut the share of Facebook and Instagram revenue derived from scam ads. In the meantime, Meta has internally acknowledged that regulatory fines for scam ads are certain and anticipates penalties of up to $1 billion according to one internal document. But those fines would be much smaller than Meta's revenue from scam ads. A separate document from November 2024 states, every six months, Meta earns $3.5 billion from just the portion of scam ads that, quote, present higher legal risk, end quote, the document says. And then it goes on. Here it is. That figure almost certainly exceeds, quote, the cost of any regulatory settlement involving scam ads, end quote. So this is them saying, we know we're going to lose in coin. We know if the government investigates us, we're going to be found guilty, but we don't care because our revenue from this pattern of behavior exceeds the expected fines. So the fines are just the cost of doing business. What that says is even when it is illegal, even when they know it's illegal, it's so profitable that the existing regulation is not an adequate deterrent against the illegal behavior.
- [47:23]
B
And there's no intention of behavior change or fixing this.
- [47:26]
A
Yes. So I think I would bring this back to the conversation we're having around the world on AI regulation at this moment. It is true that in the United States there is already a lot of AI regulation. Right. You know, if you take out a gun and you put AI on it and you shoot somebody, you're still guilty of murder. Right. AI is not a get out of jail free card. And that applies to financial crimes, that applies to harassment crimes that, you know, just using AI doesn't mean it's not illegal. So all the stuff that is normally illegal is still illegal in the era of AI. But as we see, you know, the progress of technology changes the balance of power between the actors in the overall ecosystem and also changes their abilities to intervene and prevent harm. This is the difference between the phone company in the 1930s not, you know, shutting off phone access because they don't know who's in the mafia, and Facebook. Knowing every single thing that every single person says on their platform and every single thing they do on their platform and knowing when bad behavior is going on, knowing when users are complaining about bad behavior to the tune of hundreds of reports of illegal or fraudulent or policy violating behavior and not doing anything about it. And so I think as I look at the AI policy conversation, I'm sympathetic to both sides on this debate. I know it is true that you can design dumb regulations that will stifle economic growth, that will slow innovation. That is a real thing to be concerned about. That can happen. That has happened. But it is also the case that there's an extraordinary amount of bad behavior going on out there. Malicious activity, you know, if it's not illegal, it should be kind of activity. And that is also a drain on the American economy and innovation. I mean, my goodness, the fact that there is, according to the uk, half of all financial scams to the tune of tens of billions of dollars annually are being facilitated by meta platforms, or the fact that 1/3 of all fraud in the United States is being facilitated by meta platforms, that is a big drain on the American economy. That is a big drain on people's well being and livelihoods. And I'm sure no member of Congress is going to think that it's okay that Chinese cybercriminals are robbing Americans to the tunes of tens of billions of dollars a year. Nobody's going to think it's okay just because Facebook gets a cut. Right. That is not a great argument. And so we have to find, you know, a pattern of regulations that can adequately disincentivize bad behavior. Because according to Meta's own internal analysis, the penalties that they expect are not severe enough to make it worth their while to stop what's going on on this platform. And now think about, you know, what's going on in the debate between the United States and Europe. You know, the Europeans have regulations that are designed to increase transparency into these platforms, forcing them to fork over data about what takes place on these platforms. And I think the wisdom of that approach should be obvious at this stage. And what do I mean when I say that? It's because if these documents had not been leaked, we would not have found out about all of this. And I think it has been reported in other outlets that in previous scenarios where Meta has had internal documents leaked, their policy is not to stop the bad behavior. Their policy is to ban the creation of documents about their bad behavior so that they can't be leaked next time. And so what I'm getting at here is that if the government does not have a mechanism for visibility into these platforms, the window of opportunity for detecting this and shutting it down and adequately taking any steps to bring sanity to this regime is going to be pretty tough. And again, this is social media. There are relationships to AI. And I think one of the strongest connections to AI comes from Demis Hassabis, who is the CEO of Google DeepMind. And he said, quote, we should learn the lessons from social media where this attitude of maybe move fast and break things went ahead of the understanding of what consequent second and third order effects were going to be. And Senator Blumenthal, who was the co author on that letter, he said during a congressional hearing several years ago, quote, we had the same choice when we face social media, we failed to seize that moment. The result is predators on the Internet, toxic content, exploiting children, creating dangers for them. And Senator Blackburn and I and others like Senator Durbin on the Judiciary Committee are trying to deal with it. But Congress failed to meet the moment on social media. Now we have the obligation to do it on AI before the threats and the risks become real. So I think that is sort of where I come down here. There's an AI part of this story specifically on what's going on with Meta, the fact that they're using algorithms to recommend scams to their users. But there's also just the precedent of like, what have we done in tech regulation? Where was it wise to not regulate and where was it not wise to regulate? I think about gdpr, for example, I recall being in a room in Europe with proponents of privacy legislation and one of the speakers said, said, would anybody please raise their hand if they feel like GDPR has actually protected Europeans privacy? And no hands went up. And that was in a room of people who were pro privacy legislation. And this isn't even really a privacy argument here. This is about fraud scams, really harming people, robbing them of their, of their life savings in some instances. And you know, so I, I, I'm, I'm sympathetic to the arguments that, that regulations like GDPR didn't protect privacy, did harm innovation, did slow economic growth, but when you're looking at hundreds of billions of dollars of fraud, that can really harm economic growth too. And we need a regulatory regime that can prevent us from making these categories of mistakes in the era of AIs. And I have to say, if there's one thing Meta's internal documents say incredibly persuasively, it's don't take Meta's public statements at their word, to be frank. And I think the same has to be true of all technology companies. Because while we want to assume the best of intent, while we want to assist American enterprises in succeeding in the marketplace, there's gotta be some degree of transparency that allows for a trust but verify approach when the stakes are this high.
- [55:01]
B
Well, that's not a very optimistic way to end the episode, but I think we've covered a lot, lot today. You've shocked me and I've been entertained. So thanks so much for breaking all of this down for us. I'm sure our audience found it as helpful as I did, and I had a great time chatting with you. So we'll as always be following these topics and I'm sure we'll talk about them again. But thanks so much for having me back, Greg.
- [55:26]
A
Well, thank you, Sadie. And to, to all of our listeners, let me just say you gotta read these Reuter stories. We've been talking for an hour here and we didn't even cover half of it. It's an extraordinary amount of journalism. I am told that there's more stuff that's going to be coming out in the coming weeks. And as I said at the top of this podcast, I really think a good understanding of this issue is a prerequisite for intelligent participation in the AI regulatory debate going forward.
- [56:03]
B
Great. Thanks so much Greg.
- [56:05]
A
Thanks Sadie. Thanks for listening to this episode of the AI Policy Podcast. If you like what you heard, there's an easy way for you to help us. Please give us a five star review on your favorite podcast platform and subscribe and tell your friends. It really helps when you spread the word. This podcast was produced by Sarah Baker, Sadie McCullough, and Matt Mann. See you next time.