Loading summary
Transcript95 lines
- [00:03]
Dina Temple Raston
From Recorded Future News and prx, this is Click Here. Hey, it's Dina. The Click Here team is taking a break from producing brand new episodes this month so we can get ahead on some stories we want to bring you in 2025. So we started going through the archives and we thought there was one story we should return to. It's about what worries some people about TikTok. Congress passed a law earlier this year that said TikTok was a national security threat, and the law demands that ByteDance, TikTok's mainland Chinese parent company, either has to sell the app or face a ban in the U.S. tikTok, for its part, maintains that the ban violates the First Amendment rights of its 170 million American users. The story we did back in September took a look at what might have lawmakers so worried. It's about how the Chinese Communist Party could use the app's algorithm to influence American audiences. And we decided to bring it to you again now because ByteDance has just asked the US Court of Appeals for the District of Columbia to temporarily freeze the ban in hopes that the Supreme Court might agree to hear the case. The ban takes effect on January 19th. Take a listen. Looking back on it, Esma Memtimin had a feeling that her TikTok video might be taken down, though at first blush, it didn't seem particularly sinister. There was no violence, no swear words, there wasn't even dialogue. Just this music that you're hearing right now and a series of jump cuts of young women putting these small square stickers in public places in Munich. One on a lamppost, another on a pay phone or a snack machine. These were stickers Esma and her friends had made themselves to send a message.
- [02:17]
Esma Memtimin
We decided to go through Munich, and everywhere we go, we stick like a sticker of that and then film it.
- [02:24]
Dina Temple Raston
This is Esma. And the sticker that they were plastering around the city said this very basic thing. Google Uyghur, or you may have heard it pronounced Uyghur. Esma is Uyghur, a Muslim minority that has been fighting for autonomy from China for years. And the sticker was supposed to spark some curiosity, to get people to learn for themselves what the Chinese government was doing to Uighurs. But TikTok took the video down. Basically, they said, you went against our community rules.
- [02:56]
Esma Memtimin
Yeah, but it was something super hilarious. It was super. It didn't make sense because they said something like, it's aggressive or insulting or whatever.
- [03:10]
Dina Temple Raston
She sent us a screenshot of the TikTok notice it actually said that the video was taken down because it showed violent behavior and criminal activities, even though they were literally just putting stickers around town. So they decided to make another video. This one pointed people to a trove of leaked documents that revealed details about the archipelago of prisons in northwestern China where the Chinese have been detaining Uighurs. China has claimed for years that they're just vocational camps.
- [03:43]
Tim Harford
These are the faces China never intended us to see.
- [03:47]
Dina Temple Raston
This is from a BBC story about.
- [03:49]
Tim Harford
The hacked documents from inside its system of mass incarceration in Xinjiang.
- [03:55]
Dina Temple Raston
Esma's second video told viewers where to go online to see the files for themselves.
- [04:01]
Esma Memtimin
The files consist of over 10 gigabytes of speeches, protocols, spreadsheets, and images.
- [04:07]
Dina Temple Raston
TikTok took that video down, too. Esma thought maybe if she edited the video, it would stay up. So she removed words TikTok sometimes doesn't like, words like murder and sexual abuse, both of which the leaked papers suggest China was doing to its Uighur prisoners.
- [04:25]
Esma Memtimin
And then it got taken down again.
- [04:27]
Dina Temple Raston
This time, TikTok said it contained too much personal information. This was the third time that Esma's videos had been taken down for what felt like nonsensical reasons. Now, as a general matter, social media is all about regular people like Esma making their voices heard. But this social media company didn't seem interested in the things Esma had to say. And she started to wonder if that had to do not with her videos, but with TikTok's parent company. TikTok is owned by a Chinese company called ByteDance, headquartered in Beijing. I'm Dina Temple Reston, and this is Click Here, a podcast about all things cyber and intelligence. We tell true stories about the people making and breaking our digital world. And today, the trouble with TikTok and its Chinese parent company. For years now, questions have swirled about whether the Chinese Communist Party has a say in what appears on the popular video app. We talk to Esma, who appears to be a victim of exactly that, and look at a new report that seems to confirm that there's something odd about the algorithm at TikTok. Stay with us.
- [05:57]
Tim Harford
Do nice guys really finish last? I'm Tim Harford, host of the Cautionary Tales podcast, and I'm exploring that very question. Join me for my new miniseries on the art of fairness. From New York to Tahiti, we'll examine villains undone by their villainy, monstrous self devouring egos, and accounts of the extraordinary power of decency. Listen on the iHeartRadio app, Apple Podcasts, or wherever you listen to podcasts.
- [06:27]
Dina Temple Raston
From Recorded Future News. This is Click Here. I'm Dina Temple Rest. Esma was born in southern Germany, and for the longest time she didn't know any kids who look like she did or talk like she did.
- [06:49]
Esma Memtimin
I grew up in a small town and there are like no Urs at all here.
- [06:54]
Dina Temple Raston
Her parents had left northwestern China in the 1990s and settled in a suburb of Munich, where Esma learned Germany but spoke her parents language at home. Her parents went out of their way to make the house reflect their culture too.
- [07:09]
Esma Memtimin
So we have a lot of very cultural books with like a lot of. Wait, I don't know the word in English right now. I think tales.
- [07:19]
Dina Temple Raston
Yeah, fables or tales.
- [07:20]
Esma Memtimin
Yes, exactly. So my mom brought a lot of books and then in general, like more oriental vibes. A lot of like carpets. They're very. And stuff. Like the whole vibe is just in.
- [07:33]
Dina Temple Raston
Her small town though, Esma said people didn't know what to make of her.
- [07:37]
Esma Memtimin
They were just saying, oh, okay, so you're from Uruguay, or they were saying, okay, yeah, well then you're Chinese.
- [07:43]
Dina Temple Raston
It wasn't until Esma happened to find a local girl on Instagram who was also Uyghur that her life began to shift.
- [07:51]
Esma Memtimin
It was the first kind of moment where I realized, oh, wow, there are other people like me. It felt like I finally had some hope that there are other girls as well and other Uyghurs as well in my age.
- [08:11]
Dina Temple Raston
She reached out and they became friends and she realized that there were hundreds of Uyghurs in Munich and lots of people her age, which she loved.
- [08:21]
Esma Memtimin
That already felt very wholesome to just know, I don't have to explain my culture and my background to them. I can just be myself and they know where I'm from and who I am.
- [08:33]
Dina Temple Raston
And as she spent more time with fellow Uyghurs, she had a sort of political awakening. She went to protests, started learning more about what the Chinese authorities were doing to Uyghurs in China.
- [08:43]
Esma Memtimin
I felt very inspired and very motivated to do something. And after that, it really kind of got started.
- [08:55]
Dina Temple Raston
And she and her friends wanted to get that narrative out there to educate people. And the best way to do that, they thought, was TikTok.
- [09:03]
Esma Memtimin
We knew that even back then in 2022, TikTok was the platform if you want to go viral and if you want to have a bigger audience.
- [09:12]
Dina Temple Raston
And they had a great idea for a series of Tiktoks they were sure would resonate with Gen z. Esma was 18 at the time.
- [09:21]
Esma Memtimin
And our generation, like Gen Z, is all about self presentation on social media. So I thought about a campaign using mirrors because our generation is addicted to how we look and we look at mirrors and in our camera the whole time, Right.
- [09:41]
Dina Temple Raston
So ESMA applied for a grant from a German Uyghur organization to try to get some funding for a kind of educational art installation that would travel around Germany.
- [09:51]
Esma Memtimin
So the whole idea was to get five mirrors and then on those mirrors to place different kind of quotes and a QR code under those quotes. And for example, one quote was, I begged them to kill me, which is a quote of an ex detainee, an ex ural detainee. And if you scan the QR code, you'll be directed to a website that is explaining what is going on.
- [10:18]
Dina Temple Raston
Each mirror would focus on a different topic related to the Uyghur's plight. They sent in the proposal and it was accepted. And they got $2,000 and they went straight to work. They posted the project on TikTok and this one didn't get taken down, but it also didn't get much traction. So they just kept trying to dream up videos that would get traction. And that's what led to that sticker video and hacked documents, video we talked about before, the ones that were taken down. We sent TikTok the videos and asked if they could tell us why they were removed because they clearly weren't violent or personal, as the removal notices had said. A TikTok spokesman did comment on other parts of the story, but they said they couldn't get us a response on Esma's videos. By press time, Esma is pretty sure the issue isn't community rules, is that she was focused on Uygur issues. A lot of people have wondered if they're just imagining that things seem a little different on TikTok. And when we come back, a new report makes the case that we weren't so crazy to think so.
- [11:26]
Joel Finkelstein
They're suppressing anything that could be damning to the ccp, whether users like it or not.
- [11:34]
Dina Temple Raston
Stay with us. Joel Finkelstein is the director and chief research scientist at the Network Contagion Research Institute at Rutgers University. Essentially, he studies the spread of contagious.
- [12:06]
Joel Finkelstein
Online threats, whether it's crypto scams, antisemitism, Hinduphobia, public health threats.
- [12:13]
Dina Temple Raston
And the way Joel sees it, these kinds of threats are manageable if they're identified early, before they pick up momentum. And you also sort of mix psychology and neuroscience into all of that.
- [12:24]
Joel Finkelstein
Well, that's guilty is charged. So I'm a neuroscientist by training, and I think I just noticed a ton of parallels between that, between how the brain works and what's happening on social media platforms.
- [12:36]
Dina Temple Raston
Neuroscientists and psychologists who specialize in the teenage brain have been studying social media for years out of concern that the rapidly developing adolescent brain may be uniquely vulnerable to what these platforms serve up. And as part of that, if algorithms are censoring what young people see on platforms like TikTok, that's an important thing to know.
- [12:58]
Joel Finkelstein
We know that the platform could be used for mass indoctrination, for sparking riots, et cetera, but it's really hard to show that that's happening algorithmically.
- [13:09]
Dina Temple Raston
TikTok says no one is tweaking the algorithm, that the things that go viral do so organically. But the only way to prove that is by looking at the algorithm itself, which TikTok says is secret. So in late 2023, Joel and his team came up with a way to assess it indirectly by comparing content on particular topics and then seeing how much of it appears on TikTok versus other platforms, like Instagram, for instance.
- [13:38]
Joel Finkelstein
Like, I can look up Trump on both, and I could look up Biden on both, and I could look up. I don't know who's a famous. Who's the most famous musician you know, and Taylor Swift. Taylor Swift. I can look up Taylor Swift. Right.
- [13:50]
Dina Temple Raston
So they did that. They looked up political hashtags like Joe Biden and Donald Trump and pop culture references like Barbie. And very quickly they spotted a pattern.
- [14:01]
Joel Finkelstein
You could look those folks up, and the basic ratio is two occurrences of that hashtag on Instagram for every one occurrence on TikTok.
- [14:09]
Dina Temple Raston
Essentially, every time Joe Biden got mentioned on TikTok, he'd get twice as many shout outs on Instagram. Same was true of Donald Trump. They worked with the Barbie movie, too. And this two to one ratio makes a lot of sense. Instagram is twice the size of TikTok, so it stands to reason that it would have about twice as much content.
- [14:31]
Joel Finkelstein
And that's true both for politics and it's true for popular culture. And that's true almost across the board.
- [14:38]
Dina Temple Raston
So that was Joel's baseline. Roughly speaking, there should be about twice as much content about a topic on Instagram than there is on TikTok. And then they started to look at things the Chinese Communist Party saw as hot button issues.
- [14:53]
Joel Finkelstein
Taiwan, the Dalai Lama, Tibet. We looked at the Uyghurs, of course, and we also looked at foreign Issues which we knew had an interest to the ccp. We looked up Israel and Palestine, we looked up hashtags related to India and Pakistan, and we looked up hashtags related to Ukraine.
- [15:09]
Dina Temple Raston
And that two to one ratio that they tracked when you looked at Biden or Barbie was completely different when you looked at issues the CCP cared about. In every case, they found that there was a lot less content about these issues on TikTok than on a similar platform like Instagram.
- [15:26]
Joel Finkelstein
And these were not minor differences. It wasn't 2 to 1 on the Dalai Lama. It was. It was something like 50 to 1 on these kinds of hashtags, where you'd have 50 times more of this content on Instagram than you would on TikTok.
- [15:40]
Dina Temple Raston
That is, every time the Dalai Lama got mentioned on TikTok, he got 50 more shouts on Instagram. And the results for Tiananmen Square, the site of the 1989 crackdown on student demonstrators, showed a ratio of 80:1 to 1 for every one time Tiananmen was mentioned on TikTok. There were 81 more instances of it on Instagram. And the term a Hong Kong protest, even worse, 181 to 1.
- [16:10]
Joel Finkelstein
There's so much more interest on Instagram for anything that's damaging to the CCP, and it's like a desert of that information on TikTok.
- [16:24]
Dina Temple Raston
On the flip side, content on TikTok about things the Chinese leadership cares about and wants to promote. Well, there was lots of that. Take the issue of cashmere. For decades now, China has been fighting India over the border region of Kashmir. On TikTok, the pro China hashtag standwithcashmir was 600 times more likely to appear than it was on Instagram. The ratios were so dramatically different that Joel and the team concluded the only way that could happen is if the Chinese leadership had some say in what was appearing on TikTok, that its content moderation had a little help from the ccp, or maybe a lot.
- [17:06]
Joel Finkelstein
I could take this report to Xi Jinping and I could say, listen, if this is a deliberate effort, I give you an A plus. You have done a terrific job. These are very sizable impacts.
- [17:17]
Dina Temple Raston
We asked TikTok about this, and their spokesman, Michael Hughes, said that the report's methodology was flawed to reach a predetermined false conclusion. And he added that the report hadn't been peer reviewed. So we reached out to three different researchers in the field to ask about the report, and they said while they might quibble with some parts of the methodology, the report's conclusions tracked what they had found in their own research. The Department of Justice and the National Security Council at the White House found the data compelling enough that they used what Joel discovered to help gin up support for a bill that would require ByteDance, TikTok's parent company, to sell off its stake or face being banned in the App Store. And then this other thing happened. After the Rutgers report came out, the analytical tool that Joel's team had used to do their analysis, TikTok straight up removed it.
- [18:15]
Joel Finkelstein
They did that without any announcement. This was a major feature, and they did it surreptitiously, and they certainly didn't tell us it was down. So after saying that the methodology was bogus, they then stopped the capacity for the rest of the research community to be able to examine them in this fashion, which is, I think, as much an admission of a problem as one could make, right?
- [18:36]
Dina Temple Raston
Earlier this year, Joel and his team decided to try to find a different way to prove that the CCP was manipulating TikTok's content. They created 40 sock puppet accounts, essentially fake accounts made to look and act like a real person.
- [18:53]
Joel Finkelstein
We wanted to see what could be returned when we looked up topics that were sensitive to the Chinese across these platforms. And what we found was that when we looked up Tiananmen Square, especially when we looked up Uyghurs, it was absolute remarkable disparities between the platforms.
- [19:08]
Dina Temple Raston
If you looked up Uighur or Uyghur on YouTube or Instagram, you'd see mainstream news about forced detention and abuse. You might even see those videos that Esma Memtimim and her friends dreamed up. But if you looked it up on TikTok, you'd be served up videos about Uyghur culture and music.
- [19:28]
Joel Finkelstein
There's a lot of cultural influencers that are frontier influencers, talking about how beautiful China is. There's a ton of influencers that are showing out the dance and the dress of Uighurs, their culture, their food, how wonderful things are. And this is pretty profane. I mean, it's like. It's like having an advertisement about how good the cheeseburgers are at Auschwitz.
- [19:48]
Dina Temple Raston
If one of these sock puppets looked up Tiananmen, same thing. The cultural history of the area, or a focus on Mao's mausoleum that sits in its center. Not a word about the tanks that rolled in 35 years ago.
- [20:01]
Joel Finkelstein
It was absolute remarkable, you know, disparities between the platforms.
- [20:06]
Dina Temple Raston
None of this surprises Fergus Ryan. He's a senior China analyst at the Australian Strategic Policy Institute, which does research for the Aussie Department of Defense, and he's been studying TikTok for years in effort. He says that has gotten more challenging over time.
- [20:24]
Fergus Ryan
When the app was first expanding around the world, its censorship efforts were quite blunt. But since there has been a lot of scrutiny on the app, it's become more subtle and very difficult to detect.
- [20:40]
Dina Temple Raston
Fergus said it isn't a stretch to think that ByteDance would have to respond if leaders in Beijing objected to something on the platform.
- [20:48]
Fergus Ryan
All Chinese tech firms are required to assist the government in intelligence work if asked, and that means that it can't realistically refuse requests from the Chinese government.
- [21:02]
Dina Temple Raston
The ccp, by law, can tell Chinese companies what to do, and people who disobey them or seem to go their own way have a tendency to disappear. The founder of the B2B business, Alibaba, a guy named Jack Ma, is a perfect example. Alibaba is massive, and Jack Ma was very prominent, not just in China, but all around the world. But after he criticized the Chinese government, he simply disappeared from view. He returned to the country in March 2023 as a teacher, not a businessman. The point here is that the US Government just doesn't hold that kind of power over social media platforms like Instagram or Facebook, and Fergus Ryan says that's an important distinction.
- [21:48]
Fergus Ryan
The CCP's ability to directly influence the platform means that TikTok, it's not just another social media app. It's a strategic asset for the Chinese state.
- [22:00]
Dina Temple Raston
Which is why he said TikTok is different from other platforms. It's pretty clear it isn't just audience preference or likes that are driving content. Some other mysterious force appears to be.
- [22:12]
Fergus Ryan
TikTok plays a significant role in subtly shaping how people, particularly younger users, perceive China and its government policies.
- [22:23]
Dina Temple Raston
This isn't just conjecture. Recently, Joel Finkelstein's team led a survey to see what TikTok users think about China.
- [22:32]
Joel Finkelstein
We directly asked subjects, you know, what is Tiananmen best known for? Is it best known as a great tourist attraction? China's human record? What do you think of that? You think it's pretty good, or you think it's an awful record?
- [22:45]
Dina Temple Raston
The Results Joel said TikTok users were 33% more likely to believe that Tiananmen Square was just a tourist attraction than people on other social media platforms. Remember Esma Memtimim, the Uyghur in Germany at the beginning of the episode? She's now the co founder of something called the Uyghur Youth Initiative. It's a nonprofit aimed at educating people about human rights abuses against Uyghurs. And you'd think after all the videos TikTok took down that Esma and her friends would have given up on it. But instead they figured out how to jiu jitsu the algorithm, how to get around the censorship and still take advantage of TikTok's reach.
- [23:31]
Esma Memtimin
Because we were thinking we could use. We wanted to use China's weapons against them. We want to use their toxic algorithm for our benefit.
- [23:40]
Dina Temple Raston
And they've been doing exactly that. You could say they're using the beating heart of social media against itself. They have burrowed their content inside TikTok trends so their videos go viral.
- [23:54]
Esma Memtimin
Trends are literally the way you can trick the algorithm. If you hop on a trend, then TikTok will want to push your videos, because TikTok knows that this trend is getting a lot of attention. And then we just try to use those trends for our cause.
- [24:10]
Dina Temple Raston
So they make videos that echo a current trend, and then once they're posted, they get swept up into a larger tide, making them harder to spot and harder to censor. Like this new meme that swept the platform late last year.
- [24:25]
Esma Memtimin
We're librarians, of course.
- [24:26]
Dina Temple Raston
I know what book you're looking for when you just give me a color.
- [24:29]
Fergus Ryan
I'm a sound engineer. Of course I'm going to have you.
- [24:31]
Dina Temple Raston
Do weird stuff when you're recording vocals.
- [24:34]
Fergus Ryan
Can you give me that in a whisper, please?
- [24:37]
Joel Finkelstein
We're actors, of course. We don't know the difference between make believe and reality.
- [24:43]
Dina Temple Raston
Millions of people were sharing this new meme format. So Esma and her colleagues grabbed their phones and adapted the meme to their very Uyghur messages.
- [24:52]
Esma Memtimin
We are Uyghurs, of course. Nobody can guess my nationality. We are Uyghurs, of course we need four hours to prepare one dish. We are Uyghurs, of course we don't have basic human rights. It was super, super improvised. We were done filming after five or ten minutes, and we were just. We just knew that we wanted to make it very sarcastic and we wanted to smile and we wanted to make people wonder. We are Uyghurs, of course China is selling our organs. We're Uyghurs, of course we don't know if families are alive or not. One time, we wanted to kind of focus more on human rights. And then for the other scene, we decided to go more into the cultural aspect.
- [25:37]
Dina Temple Raston
They posted it on TikTok and the crowds went wild. Almost right away, they had nearly a million views.
- [25:43]
Esma Memtimin
We were like, okay, well, I think this is it now. I think it's not going to be more than that. It cannot be more than a million. Like that's already insane.
- [25:52]
Dina Temple Raston
She went to sleep, woke up the.
- [25:54]
Esma Memtimin
Next morning, and it just kept growing. And that was super insane. Especially getting the reactions of my parents who are not very used to social media.
- [26:07]
Dina Temple Raston
To date, their TikTok video has had nearly 6 million million views. So if, as researchers suggest, TikTok's parent company and the CCP are using an algorithm to tamp down content they don't agree with, people like Esma are going to have to keep coming up with creative ways to fool it.
- [26:27]
Esma Memtimin
Our social media team, they have it kind of in their intuition. They just do. They just kind of understand. They speak the TikTok language, kind of.
- [26:39]
Dina Temple Raston
This is Click Here. Today's episode was produced by Megan Dietrich, Sean Powers, Erica Gaida, and me, Dina Temple Raston. It was edited by Karen Duffin, Fact Checked by Darren Ankrum, and contains original music by Ben Levingston with some other music by Blue Dot Sessions. Our staff writer is Lucas Riley, and our illustrator is Megan Gough. Martin Peralta is our sound designer and engineer. Click Here is a production of Recorded Future News. We'll be back on Friday with Mic Drop.
- [27:34]
Martin Peralta
Looking for more of the cybersecurity and intelligence coverage you get on Click Here, Then check out our sister publication the Record from Recorded Future News. You'll get breaking cyber news from reporters in New York, Washington, London and Kyiv, among others, and you'll see for yourself why it attracts hundreds of thousands of page views every month. Just go to therecord Media.