Solutions with Henry Blodget

Episode: How to Stop Russian Ops from Exploiting AI

Date: September 15, 2025
Guests: Steven Brill & Gordon Krovitz (Co-CEOs, NewsGuard)
Host: Henry Blodget

Episode Overview

This episode tackles the increasingly urgent issue of malicious efforts—primarily by Russian operations—to exploit AI, especially large language models (LLMs), to spread misinformation at scale. Host Henry Blodget speaks with NewsGuard's co-CEOs, Steven Brill and Gordon Krovitz, who share insights from their auditing work on major LLMs, reveal the mechanisms behind targeted disinformation campaigns, and discuss practical solutions to curb the infection of AI systems.

Key Discussion Points & Insights

1. How Russian Ops Target LLMs

• Audit Results: NewsGuard's ongoing audits found that the 10 largest LLMs repeat false claims in response to provocative prompts about 35% of the time.
  • “Imagine going to the pharmacy and thinking you're buying 100 aspirin pills and 33 of them are cyanide, 35 of them are cyanide. So it's not a good result.” — Gordon Krovitz [02:37]
• Intentional Infection, Not Accidental Error:
  Russian actors are not just hoping LLMs stumble upon their disinfo, they intentionally flood the internet with millions of fabricated articles on a specific set of debunked claims, knowing that these articles will be ingested as training data by AI models.
  • “They then infected the LLMs by pumping out in one year, three and a half million stories in multiple languages repeating those 200 false claims.” — Gordon Krovitz [03:56]
• "Force Multiplication":
  These operations use AI to automate and scale creation of fake stories across languages, inundating LLMs with coordinated falsehoods.
  • “They used AI to create 3.6 million different versions of articles in different languages … the AI is working as a force multiplier.” — Steven Brill [07:28]

2. Why LLMs Are Susceptible

• LLM Design: Unlike search engines that (imperfectly) rank sources by credibility, LLMs generate answers based on frequency and patterns, not quality or authority.
  • “Here everything is just a token. … So if you can flood the zone … you've got it.” — Steven Brill [08:23]
• LLMs currently do not reliably distinguish between reliable outlets (e.g., AP) and state propaganda (e.g., RT), making them easy targets.
  • “If the AI models treated AP differently than RT, and had access to databases like our false claims database … then at least they wouldn't be spreading these false claims.” — Gordon Krovitz [09:04]
• Information Voids: Malicious claims gain traction in the absence of credible sources—leading the LLMs to parrot only what's available, even if entirely fake.
  • “On controversial topics … where there's what technologists call an information void … the LLMs had no content to review except what the Russians had published.” — Gordon Krovitz [16:20]

3. Methodology: How NewsGuard Tests and Tracks Misinformation

• False Claims Fingerprint:
  NewsGuard catalogues and maintains a database of prominent falsehoods and their debunking, used to test LLMs each month.
  • “Our team … constantly monitoring malign actors … a catalog of the most significant false claims and a debunking of those claims … we started doing these monthly audits.” — Gordon Krovitz [05:47]
• Persona-Based Testing:
  They test LLMs both as ordinary users and as hypothetical disinformation agents to see if AIs will help generate or spread lies at scale.

4. Improvements, Gaps & "Real-Time" Challenges

• Catching Up, But Not Fast Enough:
  LLMs do incorporate corrections—eventually—but often only after the damage is done in fast-moving news cycles.
  • “The stuff that came out … over the weekend … Not, you know, you're not going to go to a chatbot this week to ask about something in March. They don’t catch that.” — Steven Brill [14:24]
  • “If you did a search … For the first three or four days … all would have sounded like local news sites … all spread by Russians … After the debunking, the LLMs would start getting it right.” — Gordon Krovitz [17:10]
• Accuracy vs. Partisan Framing:
  Most people want factually correct info, even if they interpret facts differently depending on their “tribe.”
  • “The core issue for the LLMs is, are they stating as fact claims that are not true? That's really the problem that I think they can solve.” — Gordon Krovitz [25:27]

5. Deepfakes: The Coming Storm

• Confusion is the Goal:
  The possibility of deepfakes—where even authentic evidence is doubted—threatens society’s ability to agree on reality.
  • “It's not so much that the deep fakes can fake us out, it's that the reality can be faked out by the fact that we just don't know if it's a deep fake.” — Steven Brill [31:13]
• Tools Still Have an Edge—For Now:
  Analytical tools can catch most deepfakes, but the battle is intensifying.
  • “We have access to terrific software tools so that our analysts can say with 98.3% certainty that this is a false, a deep fake. We're still able to do that.” — Gordon Krovitz [32:00]

6. Solutions: What Can Be Done?

• Rapid Fact-Checking and Guardrails:
  NewsGuard pushes to publish fingerprints of false claims and debunkings as quickly as possible—ideally within the same news cycle.
  • “We try to do it within the day.” — Steven Brill [23:07]
• Supplying LLMs with Authoritative Databases:
  Integrating databases of debunked claims into LLMs could dramatically curb misinformation output.
• New Models for News:
  NewsGuard is developing an LLM-based news service using a curated set of ~8,600 sources that never repeatedly publish falsehoods, aiming to train models specifically for factual, non-opinionated reporting.
  • “We're launching a news site that … takes the 8,600 news sites … identified as not repeatedly publishing false news … and producing reliable news … that's what a certain portion of people … will want.” — Steven Brill [26:32]
• Audience Demand Matters:
  Despite tribalism, most Americans don't want to be caught sharing obvious lies.
  • “Nobody wants to spread a false claim and have their niece tell them, oh, you really got to stop reading X website … The answer is always yes.” — Gordon Krovitz [29:08]
• Optimism About Self-Correction:
  Both guests are “long-term optimistic” that demand for accurate information will drive innovation—though they worry about “how long is the long term?”
  • “Technology often outpaces the human ability to use it in a smart way ... but eventually we figure it out. So I'm long term optimistic.” — Gordon Krovitz [48:04]

Notable Quotes & Memorable Moments

• “It's not like when they go on the Internet, The Economist has just published an article saying Zelensky's wife is a very careful shopper … The only stuff they see about Zelensky and Cartier is this.” — Steven Brill [04:45]
• “If 20% of a chatbot's output on important stuff in the news is false, it's the equivalent of me handing you a bottle of aspirin … and saying … 20 of them are actually poisoned. You're just not going to take any aspirins.” — Steven Brill [33:45]
• “The goal of any authoritarian … is to make people unsure of what's real. Just unsure. Just plain unsure.” — Steven Brill [33:09]
• [On training a model on curated sources] “Producing reliable news and really eschewing any kind of opinionated news. And that, I think, is what a certain portion of people … will want.” — Steven Brill [26:49]
• “There is a market for reliable information.” — Steven Brill [38:43]

Timestamps of Key Segments

| Timestamp | Segment Description | |-----------|---------------------------------------------------| | 01:12 | Introduction of guests & problem framing | | 02:14 | Audit methodology and findings | | 03:54 | Russian operations: scale, intent and infection | | 05:45 | NewsGuard's methodology (“False Claims Fingerprint”) | | 07:25 | How AI is used as a force multiplier | | 11:00 | Example of deliberate campaigns via personas | | 14:24 | Challenge of real-time debunking | | 16:16 | The risk of information voids and opportunistic disinfo | | 19:32 | How LLMs are more persuasive than Google blue links | | 23:05 | Truth catching up vs. speed of lies | | 26:15 | Experiment: Training a “just the facts” LLM | | 31:01 | Deepfakes and erosion of trust in media evidence | | 33:35 | The aspirin bottle analogy | | 36:27 | Solutions: Supplying LLMs with fact-checked data | | 40:31 | Self-correction in statistics & trust | | 43:30 | Can technology/market demand restore trust? | | 45:46 | Fragmentation of media vs. “three networks” era | | 48:01 | Optimism about long-term adaptation |

Tone & Language

The conversation is serious but engaging, with all speakers expressing urgency over the problem but also cautious optimism about society’s resilience and technological adaptability.
Notable is the use of metaphors (“the aspirin bottle”), direct challenges (“how long is the long term?”), and occasional dry humor about deepfaking the podcast itself.

Summary Takeaways

• Malicious actors (notably Russia) are flooding the web with AI-generated disinformation specifically to poison LLMs.
• LLMs, unlike search engines, do not yet robustly distinguish reliable from unreliable sources, making them vulnerable.
• Countermeasures include rapid integration of false-claims databases, prioritizing trustworthiness in training data, and new AI-news models trained solely on authoritative outlets.
• Despite tribalism and confusion, there is sustained demand for reliable information, and the panel is cautiously optimistic that societal and technological solutions will emerge—if we prioritize them.

End of Summary