B (3:31)

Okay, so we can express probabilities with numbers and we can also express them with words like probable, unlikely and so on. Ideally the words and numbers match up somehow, but that's not always the case.

C (3:47)

A real standout example for this kind of problem came in 1951 when a CIA analyst called Sherman Kent wrote a report that said that there was a serious possibility that what was then USSR would try and invade Yugoslavia that year.

B (4:03)

A serious possibility of war seems like quite an important thing to be clear about.

C (4:08)

He then bumped into someone who was a chairman of an important committee in a corridor and the chairman said, what did you mean by serious possibility? And Kent said he thought about 65%. And the chairman said, I thought you meant something much lower than that. And Kent kind of had this panic and went back and talked to a lot of his team who'd been writing these dozens of reports and realised that even in the team who are co writing these things, they had very different numbers in mind for the kind of language they were using.

B (4:34)

The invasion never happened. But this ambiguity between language and numbers rumbled on over the decades. However, Adam says that in the last 25 years there's been a real effort to pin them down, particularly from the intelligence agencies which are trying to communicate serious risks to politicians.

C (4:54)

So there was a real effort to introduce in UK what was known as a probability yardstick, where it's saying if you're using these kind of words, this is the probability that you mean. So if you're using words like highly likely, you're saying there that there's an 80 to 90% chance that that's going to occur. If you're using the phrase we realistic possibility, you mean a 40 to 50% chance.

B (5:18)

James Bond and his analysts in the back office should be communicating clearly now, but what about the rest of us? This probability language isn't just found in government reports. It also percolates out into the public in news stories about wars and pandemics and climate change. Does the public have a clear idea what these words actually mean? Adam says some small studies have looked at this, but he wanted much more data, so he built a quiz.

C (5:52)

There were two main parts to the quiz. The first was comparing phrases. So you had a series of pairs, and it might say for each pair, which has the higher probability of something occurring. The second part of the quiz got people to put specific numbers to phrases. So if you're given a phrase like may happen, pick a number between 1 and 100% that you think matches the probability of that phrase occurring.

B (6:18)

More than 5,000 people took part in Adam's probability quiz. You can have a go yourself if you click on the link on our website. And this gave Adam a pretty good idea of which words were highly ambiguous and which everyone agreed on.

C (6:33)

One of the things that was possible was to see where the highest amount of ambiguity was. So, in terms of just the variation in what people say, the most consistent was the phrase about even, which you'd hope that perhaps people would land on about 50%.

B (6:51)

There was also more clarity in some high and low probability phrases. For instance, most people thought the phrase will happen meant there was a 100% chance and almost certain something around 95%. At the other end, most people thought the phrase almost no chance meant between a 1 and 5% chance. The phrases remote chance and highly unlikely both came in between 1 and 10%. Those are the phrases where you can be pretty sure people know what you're saying. However, some were far less clear.

C (7:31)

The most variable phrase was realistic possibility. So one that actually happens to be in intelligence guidelines and often used internationally in headlines to report risks of events.

B (7:42)

Respondents to Adam's quiz thought the phrase realistic possibility could mean anything from a 10% chance of it happening to it being a near certainty, with the average answer just over 50%.

C (7:56)

So it really does seem like a word that means a lot of different things to a lot of different people.

B (8:00)

The terms probable and likely both had ranges from around 50 to 100%, with the averages just under 75%. Adam also asked people to compare probability words side by side to see if people were using probability in a consistent way. So here goes. Let's try a little quiz of our own. Which is more likely to happen, Something that is very unlikely or something that is highly probable? Ok, that one was easy. Obviously, highly probable is the answer. How about this, though? Which is more likely, something that might happen or something that could happen? Turns out people's judgments on these close comparisons are not consistent at all.

C (8:54)

Some phases particularly that might happen, it could happen, which arguably are just very hard to pin down. But even if you give people that and then give them the same judgment a few questions later, they'll flip between them somewhat arbitrarily.

B (9:07)

While the study was self selecting, he did end up getting a reasonable spread of responses from people in different groups and he could see a few patterns start to emerge.

C (9:17)

There was a bit of a trend, for example, with values in education, so typically people who had more degrees tended to be a bit more pessimistic. We also found that people responding from the US were slightly more optimistic in their answers, all things being equal compared to the uk.

B (9:32)

These, however, were only small effects in the scheme of things, moving the numbers by a few percent. Really the big picture here is that some probability language is just very ambiguous. That's a problem if you're trying to warn people about serious risks. But what should we do about that? For Adam, one thing is just making sure that the people communicating risk, whether they're scientists, politicians or journalists, know the risk.

C (10:00)

Just ensuring that what the person coming up with an estimate is saying is what the person is hearing at the other end.

B (10:08)

The second is to push the people calculating risks not to hide behind these words to avoid accountability.

C (10:15)

All of us, I think, can have some benefit in actually challenging ourselves to make a judgment that we can really go back and ask if we genuinely got that right, rather than having these kind of ambiguous words where we can always wriggle out of it later if we want.

B (10:30)

Thanks to Adam Kocharski. If you've seen a number in the news you think we should take a look at email more or lessbc.co.uk. we will be back next week. Until then, goodbye.

D (10:49)

At Energy Trust of Oregon, we know it isn't easy the tremendous weight of today's operating costs working in cold, drafty spaces with inefficient heating systems or under lights that have seen brighter days. But we also know how to help you upgrade those systems, lower those costs and meet the demands of your business with smart energy efficient solutions. Find cash incentives@energytrust.org Energy Trust of Oregon

E (11:17)

More power to you if you work in university maintenance. Grainger considers you an MVP because your playbook ensures your arena is always ready for tip off. And Grainger is your trusted partner offering the products you need all in one place, from H Vac and plumbing supplies to lighting and more. And all delivered with plenty of time left on the clock. So your team always gets the win, call 1-800-GRAINGER visit grainger.com or just stop by Grainger for the ones who get it done.