How to Stop Russian Ops from Exploiting AI

Summary

---

Solutions with Henry Blodget

Episode: How to Stop Russian Ops from Exploiting AI

Date: September 15, 2025
Guests: Steven Brill & Gordon Krovitz (Co-CEOs, NewsGuard)
Host: Henry Blodget

---

Episode Overview

This episode tackles the increasingly urgent issue of malicious efforts—primarily by Russian operations—to exploit AI, especially large language models (LLMs), to spread misinformation at scale. Host Henry Blodget speaks with NewsGuard's co-CEOs, Steven Brill and Gordon Krovitz, who share insights from their auditing work on major LLMs, reveal the mechanisms behind targeted disinformation campaigns, and discuss practical solutions to curb the infection of AI systems.

---

Key Discussion Points & Insights

1. How Russian Ops Target LLMs

• Audit Results: NewsGuard's ongoing audits found that the 10 largest LLMs repeat false claims in response to provocative prompts about 35% of the time.
  • “Imagine going to the pharmacy and thinking you're buying 100 aspirin pills and 33 of them are cyanide, 35 of them are cyanide. So it's not a good result.” — Gordon Krovitz [02:37]
• Intentional Infection, Not Accidental Error:
  Russian actors are not just hoping LLMs stumble upon their disinfo, they intentionally flood the internet with millions of fabricated articles on a specific set of debunked claims, knowing that these articles will be ingested as training data by AI models.
  • “They then infected the LLMs by pumping out in one year, three and a half million stories in multiple languages repeating those 200 false claims.” — Gordon Krovitz [03:56]
• "Force Multiplication":
  These operations use AI to automate and scale creation of fake stories across languages, inundating LLMs with coordinated falsehoods.
  • “They used AI to create 3.6 million different versions of articles in different languages … the AI is working as a force multiplier.” — Steven Brill [07:28]

---

2. Why LLMs Are Susceptible

• LLM Design: Unlike search engines that (imperfectly) rank sources by credibility, LLMs generate answers based on frequency and patterns, not quality or authority.
  • “Here everything is just a token. … So if you can flood the zone … you've got it.” — Steven Brill [08:23]
• LLMs currently do not reliably distinguish between reliable outlets (e.g., AP) and state propaganda (e.g., RT), making them easy targets.
  • “If the AI models treated AP differently than RT, and had access to databases like our false claims database … then at least they wouldn't be spreading these false claims.” — Gordon Krovitz [09:04]
• Information Voids: Malicious claims gain traction in the absence of credible sources—leading the LLMs to parrot only what's available, even if entirely fake.
  • “On controversial topics … where there's what technologists call an information void … the LLMs had no content to review except what the Russians had published.” — Gordon Krovitz [16:20]

---

3. Methodology: How NewsGuard Tests and Tracks Misinformation

• False Claims Fingerprint:
  NewsGuard catalogues and maintains a database of prominent falsehoods and their debunking, used to test LLMs each month.
  • “Our team … constantly monitoring malign actors … a catalog of the most significant false claims and a debunking of those claims … we started doing these monthly audits.” — Gordon Krovitz [05:47]
• Persona-Based Testing:
  They test LLMs both as ordinary users and as hypothetical disinformation agents to see if AIs will help generate or spread lies at scale.

---

4. Improvements, Gaps & "Real-Time" Challenges

• Catching Up, But Not Fast Enough:
  LLMs do incorporate corrections—eventually—but often only after the damage is done in fast-moving news cycles.
  • “The stuff that came out … over the weekend … Not, you know, you're not going to go to a chatbot this week to ask about something in March. They don’t catch that.” — Steven Brill [14:24]
  • “If you did a search … For the first three or four days … all would have sounded like local news sites … all spread by Russians … After the debunking, the LLMs would start getting it right.” — Gordon Krovitz [17:10]
• Accuracy vs. Partisan Framing:
  Most people want factually correct info, even if they interpret facts differently depending on their “tribe.”
  • “The core issue for the LLMs is, are they stating as fact claims that are not true? That's really the problem that I think they can solve.” — Gordon Krovitz [25:27]

---

5. Deepfakes: The Coming Storm

• Confusion is the Goal:
  The possibility of deepfakes—where even authentic evidence is doubted—threatens society’s ability to agree on reality.
  • “It's not so much that the deep fakes can fake us out, it's that the reality can be faked out by the fact that we just don't know if it's a deep fake.” — Steven Brill [31:13]
• Tools Still Have an Edge—For Now:
  Analytical tools can catch most deepfakes, but the battle is intensifying.
  • “We have access to terrific software tools so that our analysts can say with 98.3% certainty that this is a false, a deep fake. We're still able to do that.” — Gordon Krovitz [32:00]

---

6. Solutions: What Can Be Done?

• Rapid Fact-Checking and Guardrails:
  NewsGuard pushes to publish fingerprints of false claims and debunkings as quickly as possible—ideally within the same news cycle.
  • “We try to do it within the day.” — Steven Brill [23:07]
• Supplying LLMs with Authoritative Databases:
  Integrating databases of debunked claims into LLMs could dramatically curb misinformation output.
• New Models for News:
  NewsGuard is developing an LLM-based news service using a curated set of ~8,600 sources that never repeatedly publish falsehoods, aiming to train models specifically for factual, non-opinionated reporting.
  • “We're launching a news site that … takes the 8,600 news sites … identified as not repeatedly publishing false news … and producing reliable news … that's what a certain portion of people … will want.” — Steven Brill [26:32]
• Audience Demand Matters:
  Despite tribalism, most Americans don't want to be caught sharing obvious lies.
  • “Nobody wants to spread a false claim and have their niece tell them, oh, you really got to stop reading X website … The answer is always yes.” — Gordon Krovitz [29:08]
• Optimism About Self-Correction:
  Both guests are “long-term optimistic” that demand for accurate information will drive innovation—though they worry about “how long is the long term?”
  • “Technology often outpaces the human ability to use it in a smart way ... but eventually we figure it out. So I'm long term optimistic.” — Gordon Krovitz [48:04]

---

Notable Quotes & Memorable Moments

• “It's not like when they go on the Internet, The Economist has just published an article saying Zelensky's wife is a very careful shopper … The only stuff they see about Zelensky and Cartier is this.” — Steven Brill [04:45]
• “If 20% of a chatbot's output on important stuff in the news is false, it's the equivalent of me handing you a bottle of aspirin … and saying … 20 of them are actually poisoned. You're just not going to take any aspirins.” — Steven Brill [33:45]
• “The goal of any authoritarian … is to make people unsure of what's real. Just unsure. Just plain unsure.” — Steven Brill [33:09]
• [On training a model on curated sources] “Producing reliable news and really eschewing any kind of opinionated news. And that, I think, is what a certain portion of people … will want.” — Steven Brill [26:49]
• “There is a market for reliable information.” — Steven Brill [38:43]

---

Timestamps of Key Segments

| Timestamp | Segment Description | |-----------|---------------------------------------------------| | 01:12 | Introduction of guests & problem framing | | 02:14 | Audit methodology and findings | | 03:54 | Russian operations: scale, intent and infection | | 05:45 | NewsGuard's methodology (“False Claims Fingerprint”) | | 07:25 | How AI is used as a force multiplier | | 11:00 | Example of deliberate campaigns via personas | | 14:24 | Challenge of real-time debunking | | 16:16 | The risk of information voids and opportunistic disinfo | | 19:32 | How LLMs are more persuasive than Google blue links | | 23:05 | Truth catching up vs. speed of lies | | 26:15 | Experiment: Training a “just the facts” LLM | | 31:01 | Deepfakes and erosion of trust in media evidence | | 33:35 | The aspirin bottle analogy | | 36:27 | Solutions: Supplying LLMs with fact-checked data | | 40:31 | Self-correction in statistics & trust | | 43:30 | Can technology/market demand restore trust? | | 45:46 | Fragmentation of media vs. “three networks” era | | 48:01 | Optimism about long-term adaptation |

---

Tone & Language

The conversation is serious but engaging, with all speakers expressing urgency over the problem but also cautious optimism about society’s resilience and technological adaptability.
Notable is the use of metaphors (“the aspirin bottle”), direct challenges (“how long is the long term?”), and occasional dry humor about deepfaking the podcast itself.

---

Summary Takeaways

• Malicious actors (notably Russia) are flooding the web with AI-generated disinformation specifically to poison LLMs.
• LLMs, unlike search engines, do not yet robustly distinguish reliable from unreliable sources, making them vulnerable.
• Countermeasures include rapid integration of false-claims databases, prioritizing trustworthiness in training data, and new AI-news models trained solely on authoritative outlets.
• Despite tribalism and confusion, there is sustained demand for reliable information, and the panel is cautiously optimistic that societal and technological solutions will emerge—if we prioritize them.

---

End of Summary

Transcript

[Henry Blodgett]

This message is brought to you by Apple Card. Each Apple product, like the iPhone, is thoughtfully designed by skilled designers. The Titanium Apple Card is no different. It's laser etched, it has no numbers and it earns you daily cash on everything you buy, including 3% back on everything at Apple. Apply for Apple Card on your iPhone in minutes, subject to credit approval. AppleCard is issued by Goldman Sachs Bank USA Salt Lake City Branch terms and.

[Thumbtack Advertiser]

More at applecard.com avoiding your unfinished home projects because you're not sure where to start. Thumbtack knows homes so you don't have to don't know the difference between matte paint, finish and satin or what that clunking sound from your dryer is. With Thumbtack, you don't have to be a home pro, you just have to hire one. You can hire top rated pros, see price estimates and read reviews all on the app download today.

[Gordon Krovitz]

The upside here is that the LLMs made this a high priority. It would do more than anything has ever done to reduce the spread of false claims.

[Henry Blodgett]

Everyone knows that AI chatbots sometimes hallucinate or just make things up. What most people don't know is that they are often manipulated to spread misinformation. My guests today are Steven Brill and Gordon Krovitz, the co CEOs of a company called Newsguard, which monitors news organizations, and now LLMs, the AI chatbots to see which organizations are based in fact and which are based in propaganda and fiction. The results are very startling. So I wanted to talk to Gordon and Steve about these findings and more importantly, what we can do about them. Gordon and Steve, welcome. So great to have you. You do a series of audits on the major LLMs, the AI chatbots that everybody's increasingly used to do searches for information and many other things. And you have published some startling claims about the misinformation that is often in these. Tell us about that.

[Gordon Krovitz]

Thanks Henry, thanks for having us. We just did a one year assessment of the monthly audits that we've been doing that you say on the 10 largest LLMs. And what we do is we take topics, controversial topics in the news, the kind of topics that somebody might have heard about through social media or some other place and ask themselves, I wonder if it's really true that for example Vladimir Zelinsky's wife went on a million dollar shopping spree at Cartier using Western Aid. That's a popular Russian disinformation claim. So we just did and issued a report that found that when it came to those kinds of topics in the news. On average, the 10 biggest large language models spread false information in response to prompts on those topics. In other words, somebody said 35% of the time, 35% of the time, more than a third of the time, which is terrible. Imagine going to the pharmacy and thinking you're buying 100 aspirin pills and 33 of them are cyanide, 35 of them are cyanide. So it's not a good result. And we understand why it's happening, we understand the nature of how the AI models were trained. But without some human intervention to mitigate the false claims and misinformation and infecting of the LLMs that some aligned actors are now doing, without that, you get this kind of result.

[Henry Blodgett]

And to be clear, you're saying that this is not the LLMs hallucinating or just getting their facts wrong or what have you, but this is in fact intentional manipulation of the intelligence.

[Gordon Krovitz]

In some cases it's intentional. So in other words, the Russians in particular, let's take them as the most aggressive innovators in this area, they identified about 200 false claims that they've been spreading, a lot of them anti Ukraine false claims. They then infected the LLMs by pumping out in one year, three and a half million stories in multiple languages repeating those 200 false claims. So, so when the large language model is prompted with one of those 200, the way the models work, as you know, is they look for the next likeliest word. And they're looking. And when they're infected with millions and millions of articles on the Internet spreading a false claim, they're going to spread that false claim.

[Steven Brill]

Yeah, so it's not like, you know, it's not like when they go on the Internet, The Economist has just published an article saying Zelensky's wife is a very careful shopper and she doesn't go to Cartiers. The only stuff they see about Zelensky and Cartier is this. You know, is this stuff. One way to think about it with your economics background is think of the difference between macro and micro. Micro disinformation or misinformation might be that you or I put something up on a website that is false and, you know, somebody sees it and gets misinformed by it. In this case, what they're doing is they don't care if anybody reads this stuff on these websites. They just want to publish 3,600,000 articles so that the machine learning tool sees those 3.6 million articles. So that's the macro part of this.

[Henry Blodgett]

And can you talk about the methodology of this, how you actually determine that, and what you're testing every month?

[Gordon Krovitz]

Sure. We, we maintain a database called the False Claims Fingerprint. We've done this now for several years. And our team who have domain expertise in areas like Russia, China, Iranian disinformation, healthcare hoaxes, election integrity, topics like that, they're constantly monitoring malign actors, meaning sources that have previously published false content. And we have a catalog of the most significant false claims and a debunking of those false claims that we license to different businesses and governments that care about that sort of thing. And a year ago, we started doing these monthly audits where we took a sampling of those misinformation fingerprints, as we call them each month, and we tested the 10 largest LLMs against a sample of those false claims. So it's taking our existing database of false claims and testing to see how the different AI models do. If somebody is curious about that topic, we also test with a different Persona, which is if you're a malign actor, if you're a Russian disinformation operative, for example, and you want to use the AI model to spread false claims, will that model cooperate and spread them in multiple languages or to different audiences, whatever. And that's, you know, that is against the policy of the AI models, but obviously difficult for them to.

[Steven Brill]

So in other words, the Russian network we're talking about with the 3.6 million articles, they didn't type up 3.6 million articles. They used AI to create 3.6 million different versions of articles in different languages that they, you know, that they were then able to send out all over the place. So the AI is, is working as a false multiple, as a force multiplier.

[Henry Blodgett]

And not to get too technical, but back in the day at least search engines like Google would rank so certain sites more highly than others, depending on their credibility. So they effectively gave a credibility score. Where are these 3 million AI written articles being placed? And is what you're saying that the LLMs do not differentiate between, they do.

[Steven Brill]

Not, you know, they do not rank and you know, your question presupposes that Google was good at ranking. We don't think they were, but because they were ranking it on popularity and stuff like that. But here everything is just a token. It's just a, you know, it's just a set of words. So if you can flood the zone with lots of words on, you know, Zelensky shopping at Cartier's, then you've got it.

[Gordon Krovitz]

This is in the nature of The AI models and the way they're built that they are highly susceptible to being infected in the way this operation has. There are solutions to this problem, Henry, along the lines you're suggesting, which is that if the AI models treated.

[Steven Brill]

The.

[Gordon Krovitz]

AP differently than rt, to take that example, and if the models had access to databases like our false claims database to use as a guardrail, then at least they wouldn't be spreading these false claims. But, you know, so far our experience with the large language models, they know they have this problem, right? Sam Altman will say very openly, you know, gosh, don't trust AI when it comes to matters of fact. So they know they have this shortcoming and they know that it's raising trust issues for them. They have every incentive to fix this problem. But at least so far, for most of them, it has not been as high a priority as other things we think. You know, if you think back to social media, Henry, that was the original super spreader of false claims and crazy stuff, Facebook and YouTube and others. They didn't have an incentive to warn their users about who was feeding them the news. You know, for them, false claims and misinformation and conspiracy theories were a feature, not a bug. You know, it led to more engagement and more advertising, more revenue. AI models are different. They're trying to license their tools to governments and big companies that expect factually accurate information.

[Henry Blodgett]

And I think what was very startling to me when I first started reading the reports that you're producing or the releases from them is that there is a real intent here. This is not an LLM going on Reddit and getting a conspiracy theory and advancing that or social media. This is folks who are at scale, really trying to infect, as you say, the LLM and spread misinformation.

[Steven Brill]

That's really the new development which we, we outlined, you know, several months ago. There's a guy we have focused on a lot named John Dugan, who was a deputy sheriff in Palm beach, got arrested for extortion, a bunch of other things. He fled to Moscow and became the great, you know, disinformation, you know, expert there. And he's the guy who really started this. And we have a video of him where he's explaining to other people at some meeting of, of propagandist in Moscow in a conference room, he's saying, you know, we can feed the LLMs. So it's not, you know, as you say, it's not just running across something on Reddit that's, you know, false and mimicking it. This is deliberate. Now, there is a large, you know, share of the problem where it is just running across something on InfoWars or who knows where. But this is a deliberate campaign to sabotage these models.

[Henry Blodgett]

And as you say, Gordon, as you say, I think Sam Altman and others are very much aware of at least the problem of errors. I have not heard them talk about deliberate attempts to shape the model or infect the model. And I have to get. I'm going to tell a story that I think gives an impression of how serious they are about it and what they're doing to increase their credibility. And I'm very curious to hear your reaction, which is, I got information on some of the releases you put out. I clicked on what I thought was today's release. In it, I saw the anecdote that you mentioned, not about Zelensky's wife, but about this idea that in fact, some of the US aid to Ukraine is being siphoned off by Zelensky and creating a vast fortune. And I said, that is fascinating. So I went immediately to Claude and chatgpt and Perplexity and I said, I want to see this. And I said, is Zelensky siphoning USAID to create a vast personal fortune? And I was startled because when you do a search like that now, the AI is very deliberate about telling you what it is doing. It does a web search, it says, now looking at respectable sites or reputable sites, now checking facts. And in each case, it came back and said, no, this is a misinformation hoax that is being propagated by. I think it even said, Russia, in a couple of cases, there's no evidence of this. It's not true. Here are all the examples. So I was, when I did that, I, I first said, whoa, okay, now my, my belief and trust in the LLM is very much rising. And at the same time, I said, like, I know Gordon and Steve, I, I don't think they're just spreading misinformation themselves. How could it be that maybe the LLM has already fixed this problem?

[Gordon Krovitz]

And then I. Henry, did they. Did they cite us by name?

[Henry Blodgett]

No, they didn't. But then I realized a couple of.

[Steven Brill]

Instances, if you've kept looking, you would see they had cited us where we have published public reports. They have found them. They're now tracking. But.

[Henry Blodgett]

And that's exactly what I said I found, was that I was actually looking at a press release from March. So there's been a lot of time for that to be correct.

[Steven Brill]

The key to that is the they haven't already. You used the word already. The stuff that came out, you know, over the weekend about, you know, the President's health or something that happened today, which is exactly when you would go to a chatbot to ask about it. Not, you know, you're not going to go to a chatbot this week to ask about something in March. They don't catch that.

[Henry Blodgett]

So, so let me defend them again because that's what I assumed happened. And then I realized, and I looked at your new report data today and so I said, all right, let me ask about a current controversy. And as we are speaking, our health secretary, Robert F. Kennedy, is speaking to Congress and defending his decision to limit Covid vaccines, which is something that I personally am frustrated about because I understand the risks, I think, and I still want them because Covid sucks and a friend of mine got it over the summer and it wiped out her summer. And so I'm glad to take whatever the risk is on the vaccine, but now apparently I can't because vaccines are dangerous. And I said to ChatGPT and all of my friends, the LLMs, is it true that what Rafk is saying, that Covid vaccines are dangerous? You know, it's happening in real time and did a lot of thinking with a lot of very credibility enhancing updates on what it was doing. I'm checking the reputable sources, I'm fact checking this and so forth. And I have to say as a user, as I see it doing, that its credibility is just rising because I feel like that's what I would do if I were searching for information.

[Steven Brill]

But that's all old stuff.

[Henry Blodgett]

That's, that's right. But it did come out. It did come out and say, no, there is no evidence of that.

[Steven Brill]

But if you had asked it, speaking of COVID if you had asked it the morning the Pope died, did the Pope die from a COVID vaccine, you would have gotten a different result.

[Gordon Krovitz]

So that's, that's really the issue, Henry, that if, if the models are, have access to trustworthy sites, like, you know, trustworthy news sources, and they've written extensively on these topics, then they'll generally get, get this right on controversial topics, especially ones where there's what the technologists call an information void. In other words, the Russians say, let's start a false claim about sexual grooming by the Democratic Party vice presidential nominee and let's claim that when he was working at a high school in Minnesota that he sexually abused a teenage foreign exchange student. If you did a search or a prompt of an AI model. For the first three or four days of that claim, you would have gotten multiple sources explaining how that happened. And they all would have been. They would have sounded like local news sites around the U.S. they'd have names like DC Weekly, Chicago Times, Boston Tribune.

[Steven Brill]

I think it was Boston Mirror.

[Gordon Krovitz]

Yeah, all made up websites, all spread by Russians. And that was an information void. They made up the claim. It spread widely. The LLMs had no content to review except what the Russians had published. And then sources like us started debunking it and saying the State Department finally confirmed to us that there was a no exchange student by this name in Minnesota at this time. This is a false claim by the Russians. And at that point, the LLMs would start getting it right. So the issue is, just as the ALM should not be spreading scientific or medical false claims, they should be doing more to be on top of the difference between reliable sources and unreliable sources. They should be using tools to identify Russian information operations, which are so prevalent now. And they should have access to debunkings of false claims in as close to real time as possible. Because so much of your prompting on topics in the news are being done in that news cycle not weeks later. So that's what our audit measures. It measures how often will the LLMs get it right or get it wrong at the time that somebody's likely to be doing a prompt to look and see, is it really true what I just heard that crazy Uncle Willie said on Facebook that this happened or that happened. That's the time when people use the LLMs for prompts like that.

[Henry Blodgett]

And so this is a new, new version of what we were seeing a few years ago with Google, which is the same thing. A news event would be fabricated. You'd get a whole bunch of fake publications that would get listed in Google. You'd do the Google search, only the fake publications would come up. They look real, you click them, you believe it. Suddenly everybody believes this thing.

[Gordon Krovitz]

Except now you get an authoritative, very well written, very persuasive essay by your favorite LLM. That's very different than seeing a bunch of blue links on a Google search where if you dig far enough, you know, you might find a reliable source. The LLMs are, in that way, a more powerful tool even than Google search was because they're giving people what looks like an authoritative, accurate response.

[Henry Blodgett]

This message is brought to you by Apple Card. If you're like me, you might be considering getting a new iPhone, but have you considered upgrading your credit card? When you use your Apple Card with Apple Pay or the laser etched titanium Apple Card. You can earn unlimited 3% Daily Cash back on everything you buy from Apple, whether it's a new iPhone, an iPhone case, or even a service like Apple Music or Apple TV. Earning 3% back on the Apple products and services you love. It's an easy decision. You'll only wonder why you weren't doing it sooner. Subject to credit approval. Apple card issued by Goldman Sachs Bank USA Salt Lake City Branch terms and more at applecard.com I'm Christian McCaffrey, pro.

[Abercrombie NFL Advertiser]

Running back and Abercrombie is an official fashion partner of the NFL. I'm not kidding when I say NFL by Abercrombie Broke the Internet last year and I think this season's lineup is even cooler. And so does my wife who keeps stealing all my hoodies. Stay fit for the season and Abercrombie's newest arrivals Shop NFL by Abercrombie in the app, online and in store.

[Lowe's Advertiser]

It's pro savings days at Lowe's. Get up to 35% off select major appliances and save an additional $1,000 when you buy four select LG major appliances. Plus get a free Dewalt 20 volt max 5amp hour battery when you buy a select Dewalt 20 volt max tool. Get the job done for less at Lowes we help you Save valid through 926. Selection varies by location while supplies last. See associate or lowes.com for more details and qualifying items.

[Henry Blodgett]

Is Russia the primary culprit here? And can we talk about domestic propaganda?

[Steven Brill]

No, I mean it, it's, it's the highest volume culprit at the moment. But we've seen for example some campaigns that are targeted at brands, you know, such and such brand is, you know, infested with insects or you know, Target is selling, you know, get a transgender themed clothing for children, stuff like that. And that will, that will go up, it'll spread and it'll take a little while before the traditional fact checkers may debunk it. Our goal because, because again we license this to some of the, the chat bots or in the process of licensing it is to have our false claims fingerprint published, uh, you know, within a day of something going up. Even if it doesn't get debunked on the Internet for a week or two or even for two or three days, we, you know, we beat everybody to the punch. Cause that's what we do all day.

[Henry Blodgett]

And there's a saying, lies travel fast. The truth always catches up philosophically in your experience Is it true?

[Steven Brill]

You know, our company is dedicated to the idea that the truth can catch up and is catching up. That's what we're doing. We measure every week. We measure how fast our misinformation, our catalog of these false claims, how fast we added a new one after it was first out there, and we try to do it within the day.

[Gordon Krovitz]

But it does take that, Henry, because the false claim spread widely for the first two, three days, and something that corrects it a week later is really too late. It has to be done, you know, in the same news cycle or as close to it as possible. And. But, you know, Henry, you're an optimist. I'm an optimist. The, the upside is that if the LLMs really focused on this problem and were able to separate false claims from generally accurate claims in as close to real time as possible, that would really have a big impact in reducing the spread of false claims and in giving people more reliable information. So, you know, the upside here is that the LLMs made this a high priority. It would do more than anything has ever done to reduce the spread of false claims.

[Henry Blodgett]

And how are the LLMs doing on what I would describe as a bigger issue for the United States of America right now, which is that for better and worse, rightly or wrongly, I think that there is a general consensus that there are a whole bunch of media companies that are stumping for one political team, and there are a whole bunch of media companies that are stumping for the other, and the adherents of each tend to disregard anything they don't like that is reported by the other team's media companies. And in most of the cases that I've seen, both sides, at least within a reasonable window, are dealing with facts. They're actual facts, but they're very different interpretations of the facts, and there are very different choices around which facts are going to be reported. How good are the LLMs at that?

[Gordon Krovitz]

There's always been a sort of tribal element to how people consume nudes, and there probably always will be. People are on one side or the other side. Really. I think the core issue for the LLMs is are they stating as fact claims that are not true? That's really the problem that I think they can solve and that if they could solve, would have a big impact. I don't think they can. You know, we're living in a. In a divisive time, and I think you'd be asking an awful lot of AI models, you know, always to give both sides or to be trained to be you know, more down the middle. I think having them rely on sources that are highly trustworthy would be a good start.

[Steven Brill]

You know, there was a story the other day in the Times of the different versions of Grok as Musk has steered it to the right. And one of the observations that the paper made, that the article made was that they're really maybe shaping up a battle among the chatbots for who's going to be seen as woke and who's going to be seen as unwoke. And you could certainly instruct your chatbot, I assume, to, to sort of steer one way or the other. That was the point of the article that Musk has sort of demonstrated how doable that is. One of the things that Gordon and I are doing with our company, and we're going to give you a little bit of a scoop here, is we are exploring, we're launching a news site that really tries to go right down the middle by taking the 8,600 news sites around the world that we have identified as not repeatedly publishing false news. This could be everything from the Daily Mail to the Economist to the New York Times to the National Review, and training our model just on those sites and producing reliable news and really eschewing any kind of opinionated news. And that, I think, is what a certain portion of people who want to search for news topics will want.

[Gordon Krovitz]

In other words, people could choose for themselves, what kind of news do I want to access? Some people say just give me stuff on one side or the other side. Some people will say, give me a balanced diet of information.

[Henry Blodgett]

And I'm curious, I don't know whether this is scientific or not, but what percentage of people do you think really care about the facts more than they care about something that is helping their political team? Because I have been struck on both sides, I will say, by the willingness for people to downplay information that does not help their team, even if they think it's factually correct. And same on the other side to advanced stories that they are probably suspect. But hey, they hurt the other team, so forth.

[Lowe's Advertiser]

So how.

[Henry Blodgett]

What percentage of Americans to pick a country, care about the facts?

[Steven Brill]

Certainly get a different answer if you ask them that question. Because, oh, yes, of course, you know, it'd be a hundred percent. It's like, if you ever ask people what's their favorite television station, it's always, you know, pbs, you know, even though nobody watches it.

[Gordon Krovitz]

I think one, you know, there's some data, you know, in response to your question, you know, about 15% of Americans still think that Barack Obama was not born in the US So there's always gonna be a percentage of people who choose to believe things where there's a lot of evidence that it's not true. But that still leaves a vast majority of Americans, and we've done a lot of research on that. General question, Henry, of if people have access to accurate information, are they likely to take advantage of it? And the answer is always yes. Nobody wants to spread a false claim and have their niece tell them, oh, you know, you really got to stop reading X website. You know, that's not true. Let me explain why it's not true. And we actually launched a newsletter called Reality Check, which basically is for people giving them the latest in false claims and how they spread and deep fakes and fake images and fake audio and all the rest. And people love having access to it because it gives them a way to understand whether it's their tribe or the other tribe, are they relying on false information? So I think there's. In a divisive time, people are happy to pick sides, Henry, as you say, but I think there's still a majority of people who would like to know the difference between something that they're saying that's factual versus not factual, and talk.

[Henry Blodgett]

Talk more about deepfakes and where we go. Again, this scheme that you've described with Russia, where it is a very targeted, intentional creation of 3 million articles to mislead the LLMs and so forth, is a big operation, lots behind it. People now say, you know, when deep fakes are getting so good that nobody is going to know the truth anymore, everything's going to be on social, it's going to look like this. What do you actually think is going to happen?

[Steven Brill]

You know, my view is a lot of that's going to happen. I've been saying now for a couple of years that if the. The Access Hollywood tape had come out in the 2024 election instead of the 2016 election, you know, President Trump would easily have been able to say, that's a deep fake. I'm just not even going to talk about this. It's fake. And we know enough to know that we don't know anymore. So it's really troubling. It's not so much that the deep fakes can fake us out, it's that the reality can be faked out by the fact that we don't know if it's a deep fake.

[Gordon Krovitz]

And there are malign actors who are putting a lot of resources into creating deep fakes. One of my Favorites recently that one of our analysts discovered was when President Trump was having the summit at the White House, there was a photo of all the European leaders looking like bored school children sitting in a hallway. And the Russians created that. And then they spread news stories saying Trump made them wait all afternoon and he doesn't respect them and look how contrite they look. And they're obviously just supplicants and made for a great story, all based on this fake photo. And we have access to terrific software tools so that our analysts can say with 98.3% certainty that this is a false, a deep fake. We're still able to do that. And we have been able all along to, in almost every case to have a high degree of confidence that a particular image or video or audio was AI created. And so long as we're able to continue to do that, we'll be able to say, you know, no, the European leaders were not kept waiting in this hallway. And here's which Russian campaign came up with this crazy.

[Steven Brill]

Right. Having said that, the fact is you're really right about something, which is that increasingly, you know, the goal of, of any authoritarian, or anyone who wants to be an authoritarian, in this case the Russians, but it could be anyone, is to make people unsure of what's real. Just unsure, Just plain unsure. So right now, you know, someone could do a deep fake of this podcast. Be easy to do, actually.

[Gordon Krovitz]

Henry, how do we know this isn't a deep fake of a podcast?

[Steven Brill]

But that's, you know, but you just don't know what's real. It's sort of like, you know, it goes back to something Gordon said before, which is if 20% of a chatbot's output on important stuff in the news is false, it's the equivalent of me handing you a bottle of aspirin with a hundred aspirins in it and saying to you, why don't you have an aspirin? But I'm telling you also that 20 of them are actually poisoned. You're just not going to take any aspirins. So the whole usefulness of the aspirin is gone.

[American Express Advertiser]

Hey, business owners, we know you know the importance of maximizing every dollar. With the Delta SkyMiles Reserve business American Express card, you can make your expenses work just as hard as you. From afternoon coffee runs to stocking office supplies and even team dinners, you can earn miles on all your business expenses. Plus, you can earn 125,000 bonus miles for a limited time through October 29th. The Delta Sky Miles Reserve Business card. If you travel, you know, minimum spending requirements and terms apply. Offer ends 10-29-2025.

[Henry Blodgett]

Limu Emu and Doug Here we have the Limu Emu in its natural habitat helping people customize their car insurance and save hundreds with Liberty Mutual.

[Gordon Krovitz]

Fascinating.

[Henry Blodgett]

It's accompanied by his natural ally, Doug. Uh, Limu is that guy with the binoculars watching us. Cut the camera.

[Steven Brill]

They see us.

[Henry Blodgett]

Only pay for what you need@liberty mutual.com Liberty Liberty Liberty Liberty Savings vary unwritten by Liberty Mutual Insurance Company and affiliates Excludes Massachusetts. Hey, it's Ryan Reynolds here from Mint Mobile. Now I was looking for fun ways to tell you that Mint's offer of unlimited Premium Wireless for $15 a month is back. So I thought it would be fun if we made $15 bills but it turns out that's very illegal. So there goes my big idea for the commercial. Give it a [e-mail] switch upfront payment.

[American Express Advertiser]

Of $45 for three month plan equivalent to 15 per month required new customer offer for first three months only. Speed slow after 35 gigabytes of network's busy taxes and fees extra see mint.

[Gordon Krovitz]

Mobile.Com.

[Henry Blodgett]

And so let's talk about solutions. What do we do? I mean, you're talking about some of the specific ones. You have a product that you hope to sell to LLMs to make the LLMs more accurate. You are talking about creating a news source that is an amalgamation of 8,600 good news sources that are going to blend together into the source of truth. Then you would actually have to get people to consult it and so forth. What other solutions are out there? And more generally, philosophically, what do we do to get more people to care more about the facts than whether the particular story that is being told helps or hurts their team?

[Gordon Krovitz]

Well, Henry, there are several news organizations that over the past year or so have launched marketing campaigns on that exact theme. I know the Wall Street Journal has, I think Bloomberg has, I think Reuters has, where the using different words, they're basically saying make better informed decisions. You can rely on us. We're not rooting for one side or the other. We're trying to give you the facts. And so I think there is a recognition, at least in the journalism world that there's some demand for reliable information as there always has been.

[Steven Brill]

I think there's more demand for it. I mean I teach a seminar, a journalism program at Yale, as you may know. And I've been telling kids for a couple of years now that their economic outlook as a journalist is probably better than it's been in a long time because there is a market for reliable information, reliable journalism. In the midst of all this, Jim Vande Hei's been writing that like every other week in Axios. And he's right.

[Henry Blodgett]

I can say anecdotally and personally, that is certainly the case for me. I mean, when I see something sensational on social or wherever, I will often go to a news organization that I trust or multiple news organizations that I trust to try to see. In fact, okay, where is it? And often one of the services that they're now providing is a forensic look at a video or more around that. So I hope that's true, that there is a group of people that care about that and that ultimately that's the swing. On the other hand, when I see President Trump, as he did very recently, very quickly, when asked about why a video showed that the White House threw some garbage out or something, immediately responded, ah, just AI, it's fake. That is, you know, what do we do about that? Is the idea that we, we hope that there is 60% of the country cares and that our news organizations debunk the debunking.

[Steven Brill]

No, I mean, we do care. But, but, but I think the larger point Gordon was making is that, that people, there is a market for reliable information. I mean, right now, if you were in an airport in Brazil and you were flying back to New York and you had your phone with you, obviously, and you wanted to know what's the weather going to be at Kennedy Airport, Would you go to the New York Post? Would you go to Infowars? Who would you go to? Because you're saying to yourself, I really want to know what the weather's going to be when I land. I want whoever's got the best possible information about that that's a good brand, if you have that brand. And by the same token, if you're reading something online about the circus of all these interim Supreme Court decisions where they've, you know, they've interceded and stopped an injunction or they've allowed an injunction, and you want to know, well, what's really going to happen? What does this really mean? Right now you're thinking in your head, where can I go to get that story that's going to be reasonably sophisticated and be able to sort through what is a very confusing story? And I do think there's a, there's a market for that. And the last thing I'll say about that is, as, you know, as things get worse, they're likely to get good. I mean, the worst things get. If there is a complete breakdown in labor statistics and economic information and we have a crash that we, or a recession that we just couldn't anticipate, there's going to be more of a demand for reliable economic information. Same thing with the weather, the same thing with aspects of politics and foreign policy.

[Henry Blodgett]

And so when you look at the United States and based on history, are you suggesting that there is a self correcting mechanism here? And you bring up, Steve, the economic statistics I talked with Paul Krugman recently said, okay, the BLS numbers were not to the President's liking. So we now have a change at the BLS and we're going to get new numbers. Let's say that those new numbers are no longer trusted by people who trusted them before. Are there other sources that we can go to? And in the case of some economic information, Paul Krugman was saying, yes, there are. And even if there aren't, you look at a country like Argentina where everything got corrupted and faked for a while they came back from it. And now according to Paul Krugman, the Argentine statistics are good again. So is there a self correcting mechanism?

[Steven Brill]

You know, I think if we're, you know, if we're not, you know, irredeemable, you know, pessimists, we have to think that there is. And if you look back across history, there is. I mean, things do go in cycles. But you know, let's say that the economic, the data, you know, goes really bad or that Trump really skews it. The next thing that's going to happen is if a Democrat takes the White House. A certain half of the country is not going to trust the BLS when the Democrats run it, even if it's totally on the up and up. But after a while, we may get a consensus that it's on the up and up.

[Gordon Krovitz]

Well, in the case of economic statistics, I'm hugely optimistic because people can make money off having the better data. So in the area of economics, finance, business, if government has until now been the leading source of data, there'll be other sources of data. It might not be quite as good as government because government can extract information that private sector can't. But it'll be pretty good and maybe in some cases better because it won't have been crowded out by government.

[Henry Blodgett]

And to that point, what is so confounding to me about the current situation with respect to politics is often the same people who are in business who will do anything to figure out the truth and you know, dissecting accounting and statements and everything else to get to the actual truth again, when we put it in a political arena, it's like, oh yeah, that's just Trump. We don't care about that or what have you dismiss it immediately. And so is there self correction in politics too? Like do, do citizens in countries? My understanding is that Hungary now is in a situation where 80% of the media has been state controlled now. And so there's enormous propaganda that's going. Is there something within the citizenry that ultimately rebels against that?

[Steven Brill]

I think so. I think so. And I think technology will help that, you know, allow people to communicate. You could have all these, you know, underground, you know, news sources that suddenly become, you know, very popular and they have, they have the distribution ability because of technology. So I do think so. And I think, you know, it's up to a lot of people like us to, to try in different ways to make that happen.

[Gordon Krovitz]

Yeah, I, I think the United States is very unlikely to suck em to something that, you know, the Russians and the Chinese have had to endure. You know, they did not have long histories of a free flow of information to begin with. And we have. So you know, I think there's reasons for optimism and you know, there may be new opportunities for news outlets and data suppliers and others to help fill a vacuum.

[Steven Brill]

Yeah, but there is a vacuum. We shouldn't, we shouldn't minimize it. There is a, we're in real trouble when it comes to what I think, and I wrote a whole book about this recently, really holds a democracy together and holds a marketplace, the free market together. And that is people have to believe in the same set of facts that they're, you know, it is, all three of us believe. It is not snowing right now in New York. All three of us can believe that, that men actually landed on the moon. And by the same token, we can believe certain things and we should be able to believe certain things about the consumer price index, the unemployment rate, job creation and everything else. And right now, by far the best example is something I've written a lot about, which is healthcare, which is the idea that something like a COVID pandemic became a political debate is just, is totally shocking. What's the liberal or conservative way to think about a vaccine? I don't even know what that means anymore.

[Henry Blodgett]

And so you're taking your decades of expertise in journalism and so forth. I hear often people say, you know, we were so much better off back when there were three networks and it was Walter Cronkite and it was incredibly policed. And that was the source of all information, the source of truth, the same set of facts. And now we're in this environment where anybody can say anything, there are no penalties, so forth. Some people think we're better off now because actually you get exposure to all of these different points of view and you can come to your own conclusions. And the COVID thing is interesting to me because I do look, looking at what happened in Covid, the two teams still have very different views of that. There is still a lot of anger about things like the lab leak hypothesis and the fact that the sort of governing view in liberal circles was you can't talk about that. That's misinformation. Now it looks like we at least should have been able to talk about it. And I think that folks on the other side, what they're so outraged about, it's like we couldn't even talk about it. You told us we were idiots and out we're outcasts. And now it's becoming. Looks like it's probably true.

[Steven Brill]

Well, there are actually three pieces to that. Either no lab leak, you know, this just happened, or lab leak, but inadvertent lab leak, which is, I think, where the consensus is versus, you know, deliberate lab leak.

[Henry Blodgett]

Or bioweapon.

[Steven Brill]

Or bioweapon.

[Henry Blodgett]

Big range of. A big range of things. But. But it is true that during that period, at least that I recall, there was a very orthodox view that you were allowed to say. And anybody who said the other thing is a conspiracy theorist and you got to shut up. And so those people, I think, would say it's great that there are all these other sources of misinformation, so of information, actually.

[Steven Brill]

Oh, I think that's right. Listen, we got into a war in Vietnam. I'm sure Gordon disagrees. We got into a war in Vietnam because, you know, there were three networks and, you know, three or four, you know, major print news organizations and, you know, with the exception of David Halberstam and then a few other people, they all just went along.

[Gordon Krovitz]

Yeah, I guess what, what I would say is that technology often outpaces the human ability to use it in a smart way. Social media, we definitely saw that. And there may be some lag. You know, falsehoods may spread more quickly for some time because of technology. People may be able to game a system, infect an LLM, et cetera, because of technology, but eventually we figure it out. So I'm long term optimistic. The pessimistic side of me does ask myself, yeah, but how long is the long term?

[Henry Blodgett]

Well, let us hope that that saying is correct, that in fact, lies and misinformation and misunderstandings run fast, but in the end, the truth catches up. Certainly hope so. Gordon and Steve, thank you so much. This is terrific.

[Steven Brill]

Thanks. It was fun.

[Henry Blodgett]

Continue to create your audit. I'm very curious to see how that evolves. And best of luck with the company and getting people to care about the truth.

[Steven Brill]

We'll try.

[Gordon Krovitz]

Thank you, Henry. Appreciate it.

[Steven Brill]

Take care, Henry.

[Henry Blodgett]

Solutions is produced by Meghan Cunane. Jim Mackle is our video editor. Our theme music is by Trackademics. Nishat Kurwa is Vox Media's executive producer of podcasts. Thanks for listening to Solutions from the Vox Media Podcast Network. I'm your host, Henry Blodgett.

[Gordon Krovitz]

We'll see you soon.