← More or Less
More or Less
The maths of spies and terrorists
00:09:30
In the wake of the Boston Marathon bombing and the killing of a British soldier on the...
Loading summary
Transcript27 lines
- [00:01]
A
Thank you for downloading from the BBC. For details of our complete range of
- [00:06]
B
podcasts and our terms of use, go
- [00:08]
A
to bbcworldserveys.com podcasts hello and welcome to More or Less on the BBC World Service. I'm Tim Harford. This week we look at the statistics behind catching terrorists. In the wake of April's Boston Marathon bombing and the killing of a British soldier, Lee Rigby, on the streets of Woolwich in London, the intelligence services have come in for heavy criticism. In both cases, we're told the security services knew about the suspects and had carried out some investigations. In the British case, they were on some kind of terrorism watch list. Why then weren't they being followed around? Shouldn't the security services be paying more attention to people who are known to be dangerous? The UK Parliament's Intelligence and Security Committee is to investigate possible intelligence failings in the Woolwich case. I've been speaking to the former head of the British domestic intelligence service, MI5, Dame Stella Rimmington. Is this sort of criticism, I asked, really reasonable?
- [01:08]
C
Well, if the implication of that criticism is that the security service should know everything that everybody who is known to them is doing every hour of every day, then it clearly isn't reasonable. I mean, I don't know how many suspects there are, but let's say, and one has read this in the newspaper, that there are about 2,000. And if you want to follow an individual around 24 hours a day, you would probably need a team of, let's say, six people, and that's probably under egging it with cars. But that team of people would work for, let's say, six hours a day. So you need to do it 24 hours a day. You need six, three shifts of that for one person. And then you might need other people, static surveillance people sitting in a house, for example, to alert these people when the person was coming out. Because you can't just sit outside the house, you've obviously got to sit somewhere else, then you've got to have a control center, then all that information has to go to a desk officer. So you're building up a large group of people and we're only talking about one of these manifold suspects we hear about, and doing that 24 hours a day, seven days a week. While you're the mathematician, you do the sums. It's an awful lot of people.
- [02:32]
A
I'm an economist and that sounds expensive. So we're talking about tens of thousands of people.
- [02:38]
C
Clearly, clearly.
- [02:39]
A
One question that occurred to me is we sometimes talk about this list of Suspects as though the bad guys are definitely on the list and there's nobody on the list by mistake.
- [02:48]
C
The people who will be on the list, if there is a list, will be people who have, for one reason or another, come to the attention of the security services. But there are degrees of potential involvement and the whole art of the intelligence operation is to try and identify the people who are most likely to do something and focus their attention on those people. And of course that means that you might sometimes get it wrong as a
- [03:16]
A
matter of simple economics, then it's not possible to follow every suspicious character around the clock. But even if we could, that would present mathematical problems of a different kind. Imagine that we had unlimited resources to spy on the bad guys. Or more plausibly imagine that the intelligence services could use computers to monitor everyone's email and phone lines and spot trouble. Well the Internet comic and musician Professor Elemental has just such an idea.
- [03:44]
B
So professor, we turn to you. Design for us a machine that can detect the character of a rogue even when he's concealed by a huge crowd of decent law abiding countrymen. Yours, etc etc sincerely and so forth. The Home Office of course it can be done. I'll simply recalibrate my needle in a haystack machine. I'll change it so it searches for ne' er do wells in the data
- [04:08]
A
ray rather than Professor Elemental there from the intertubes. Since we weren't sure where to reach him by telegram, we called another Professor, Howard Wehner, a professor of statistics at the Wharton School at the University of Pennsylvania in the United States. He's imagined something much like Professor Elemental's great machine for catching villains. Unlimited wiretapping tied to advanced voice analysis software on everyone's telephone line that could detect would be terrorists after SC the first three words they say on the phone it's 99% accurate. And there's just one tiny problem.
- [04:45]
D
We said, suppose there are 3,000 terrorists in the United States. If the software is 99% accurate, you would be able to pick up almost all of them, 99% of them. However, if you're listening to everybody, if they're listening to all 300 million of us, 1% of those are going to be picked up by mistake. 1% of 300 million is 3 million. And so mixed in with the 3,000 true terrorists that you've identified are going to be 3 million completely innocent people who are now being sent off to Guantanamo Bay. And so for every terrorist you have on Guantanamo, you've got 999 innocent but really pissed off people.
- [05:24]
A
The problem arises because terrorists are rare. The bad news is that in reality, your terrorist detector would be nowhere near 99% effective now. The good news is that security services are much more selective in who they monitor. If you narrowed your target population to the point that the prevalence is up to one actual terrorist per hundred people wiretapped, and assume that your test is 90% effective, even then, the chance of a false positive is still high. In those circumstances, when someone triggers an arrest, Howard Wehner says, The odds are 11 to 1 that they're not a terrorist. A problem that Professor Elemental may be about to encounter.
- [06:04]
B
Dear Professor, I enclose a photographic plate showing three members of the Bournemouth Amateur Dramatic Society following their arrest and public flogging. It transpired they were only engaged in a performance of the Pirates of Penzance. It seems the machine mistook their stage attire for that of undesirables or scallywags.
- [06:27]
A
Professor Elemental, who has since abandoned such exploits and lent his support to the Open Rights group, which is campaigning in the UK against a potential law to expand government powers to examine electronic communication. Wherever you stand on this question of civil liberties, there's no getting away from the use of algorithms to find patterns. They're used everywhere, from major retailers such as Amazon, Walmart and Tesco to the credit scores used to decide whether. Whether we can get a loan from the bank.
- [06:55]
E
Of course, the problem of a false positive in Tesco club card data is very different from the problem of a false positive in a national security question.
- [07:03]
A
That's Professor Louisa Moore from the University of Durham in the uk. She's been studying how algorithms can be used to spot suspicious patterns in the data. For instance, in information on air passengers, these patterns might be used to identify terrorists.
- [07:18]
E
So one example of that might be to travel to Pakistan and to spend three months there before returning. Naturally, you may already begin to think about how the algorithms used to detect possible risky connections might be adapting. For example, post Boston, there may now be more attention in the US to travel to particular parts of the world, perhaps including Chechnya and Dagestan. We could imagine post Woolwich, that there might be greater attention in the refining of algorithms to think about the patterns of travel and links to deportation. But of course, the question remains, though this is an automated system, it's using data from past events. Our research is suggesting that the tuning of the algorithm reflects almost always past events.
- [08:09]
A
So the algorithms are always fighting the last war?
- [08:12]
E
In a sense, yes. Yes, they are.
- [08:15]
A
Professor Louisa Moore it's certainly not an easy task, as Dame Stella Rimmington explains.
- [08:21]
C
If you have too much information, of course you suffer from what the Stasi in eastern Germany suffered from, which is an overdose of information. And intelligence services can strangle themselves if they have too much information because they can't sort out what they need to know and what they don't need to know, etc. So in all this search for information, you've got to be pretty focused and targeted.
- [08:46]
A
Dame Stella Rimmington, formerly the head of the British intelligence service MI5, and she was at pains to add a loyal more or less listener. And that's all we have time for this week, but you can download more editions of the program via our website bbcworldservice.com moreorless the website is also the place to read more or to send us your comments and your questions until we return next week. Goodbye.
- [09:13]
B
There are dozens of different podcasts now
- [09:16]
A
available from the BBC, including news, documentaries,
- [09:19]
B
science, business, arts and sports.
- [09:22]
A
The details of them all go to bbcworldservice.com podcasts.