Missing planes

Summary

Podcast Summary: More or Less — "Missing Planes"

Host: Charlotte MacDonald
First Broadcast: March 22, 2014
Episode Theme: Exploring how Bayesian statistics and search theory are applied to find missing aircraft, using the Air France Flight AF447 and Malaysia Airlines Flight 370 as case studies.

---

Episode Overview

This episode of More or Less examines the mathematical approaches used in the search for missing aircraft, particularly focusing on the role of Bayesian statistics—a method developed during the Second World War to hunt German U-boats. The host, Charlotte MacDonald, explores how these probabilistic tools are being used by search teams today and discusses their successes and limitations, illustrated by detailed accounts of the Air France Flight 447 search and the then-ongoing search for Malaysia Airlines Flight 370.

---

Key Discussion Points & Insights

1. Historical Roots of Bayesian Statistics in Search Operations

• Charlotte outlines the origins of Bayesian statistics, explaining how it allows analysts to combine multiple, sometimes contradictory, scenarios and revise probabilities as new data emerge.
• These methods were crucial in WWII for tracking German submarines and continue to underpin modern search operations.

2. The Air France Flight 447 Case: Applying Bayesian Methods

• MacDonald tells the story of the 2009 disappearance of Air France Flight AF447 over the Atlantic, illustrating how difficult it is to locate lost aircraft even with debris found.
• The French BEA called upon American statistician Colleen Keller (of Metron Incorporated) and her team to apply Bayesian theory to the search effort.

Step-by-Step Search Methodology:

• Analyzing Theories and Probabilities:
  • Each plausible cause of the crash was weighted, and historical crash data were considered.
  • Radius of possible locations: "they estimated it could not have flown more than 40 nautical miles in any direction from that point." (03:31)
• Mapping Probability Grids:
  • Quote: “You take that area and you grid it. … Each of those scenarios has a terminus of where the airplane might have ended up, and those areas get probability based on how much you weighted that scenario.” — Colleen Keller (03:54)
• Updating Models with Search Outcomes:
  • If areas had already been searched, probabilities for those cells were lowered.

Notable Moments & Quotes:

• After initial failures, “everyone was throwing up their hands, like, what do we do next?” — Colleen Keller (04:41)
• Keller’s team re-evaluated assumptions about whether the black box was emitting a signal, leading to a revised search zone.
• Memorable description: “It basically just took a big cutout of the hot areas and made them completely cold. When we went back and said no, those searches maybe weren’t effective at all...” — Colleen Keller (05:59)
• The renewed search based on the updated model led to actually finding the wreckage.

Reflections on Success and Luck:

• “It still was a minor miracle that we found it. … It was so lucky that the wreckage … was on the bottom in a very sandy open area. … There were some areas … that looked like the Himalayas in terms of mountains and crags and valleys.” — Colleen Keller (07:05)
• Emphasis on the role of luck and suitable geography in the eventual success.

3. Comparisons with Malaysia Airlines Flight 370 Search

• Charlotte explains that the MH370 search area is vastly bigger and more uncertain than for AF447.
• Even finding debris might not help in narrowing search areas because of time elapsed and currents.
• Notable quote: “It's a big world out there, and I know a lot of people are saying, how could you possibly hide or not find a 777? And I think the answer is it's easy.” — Colleen Keller (08:24)
• “I think that it's very likely if we don't get any major breakthroughs, it could be in the bottom of the Indian Ocean and we will never find it, sadly.” — Colleen Keller (08:55)

---

Notable Quotes & Memorable Moments

• On Bayesian Search Grids:
  “You take that area and you grid it… Each of those scenarios has a terminus of where the airplane might have ended up, and those areas get probability based on how much you weighted that scenario.” — Colleen Keller (03:54)

• On Revisiting Assumptions:
  “…those searches maybe weren't effective at all because there was no target to hear. … All of a sudden it was like, wow, we gotta go back and hit that area again.” — Colleen Keller (05:59)

• On the Role of Luck:
  “It still was a minor miracle that we found it. … There were some areas … that looked like the Himalayas … And it could have been undetected forever, but it just happened to be right where they looked. It was fairly miraculous.” — Colleen Keller (07:05)

• On the Difficulty of the MH370 Search:
  “It's a big world out there, and I know a lot of people are saying, how could you possibly hide or not find a 777? And I think the answer is it's easy.” — Colleen Keller (08:24)

---

Important Timestamps

• [00:19] — Introduction and WWII background on Bayesian methods
• [02:04] — Air France 447 disappearance and the challenge of finding debris
• [02:54] — Introduction to search theory expert Colleen Keller
• [03:54] — Keller describes the probabilistic mapping process
• [04:41] — Initial search fails, motivating a model update
• [05:59] — Rethinking assumptions about black box signals leads to the critical breakthrough
• [07:05] — Reflections on the narrow luck in finding AF447
• [07:40] — Start of discussion on Malaysia Airlines Flight 370
• [08:24] — Keller discusses the vastness of the MH370 search area and likelihood of never finding it

---

Conclusion

This episode demonstrates both the power and limitations of Bayesian statistical methods in real-world search operations. While such tools can narrow down vast search areas and bring rigor to complex, uncertain situations, the outcome often remains deeply reliant on luck and the unforgiving vastness of our world’s oceans. The story of AF447 is both a testament to statistical perseverance and a cautionary tale for the ongoing—and much more daunting—search for Malaysia Airlines Flight 370.

Transcript

[A]

This is the short edition of More or Less, first broadcast on the BBC World Service.

[B]

Thank you for downloading from the BBC. For details of our complete range of podcasts and our terms of use, go to bbcworldservice.com podcasts.

[A]

Hello and welcome to More or less on the BBC World Service. I'm Charlotte MacDonald. This week, can statistics help find the missing Malaysia Airlines plane? In the spring of 1943, the Second World War was raging and German U boats were sinking ships in the Atlantic. Desperate to find the Nazi submarines before they could strike, the American and British developed a model to find them. Using what is called Bayesian statistics. Named after an 18th century British Presbyterian minister called Thomas Bayes, this type of thinking allows you to assess various scenarios at once, even contradictory ones. The probability of each being true is brought together to give you the most likely solution. And if you find new information, you can revise your model easily. Mathematicians working for the American and British navies thought about the different scenarios of where the U boats could be. They then devised a method of assigning a probability to each one to work out the most probable location of the submarines, adapting their model as they went along. Today, those methods are still being used to find everything from missing people to missing planes. So could techniques from the Second World War be used to help find the missing Malaysia Airlines plane? To give you an idea of how, I'm going to tell you a story about another missing plane.

[C]

An extensive search has so far failed to find any trace of an Air France plane thought to have crashed. Flight AF447 should have landed here several hours ago, but no one knows where the aircraft is, nor the fate of its 216 passengers and 12 crew.

[A]

This was June 2009, five days after the plane went missing. Debris from the aircraft was found floating on the surface of the Atlantic. But the anguished relatives of those on the plane still didn't know what happened to their loved ones. And the mystery of why the plane crashed could only be answered by finding the black box and the cockpit voice recorder. You may think that having found the debris, it would be easy to find the rest of the plane, but it's not that simple. The area near the equator is known for unpredictable currents. There was no way of working out where that debris had been. Five days earlier, ships, planes and submarines searched for the airliner, but they couldn't find it. At this point, the French authority responsible for investigating aviation accidents, the bea, made a call to a group of statisticians

[D]

in the U.S. my name is Colleen Keller. I'm a senior analyst at Metron Incorporated, which is a consulting firm that has expertise in search theory, Keller went out

[A]

to France to help the search. With her colleagues, they went through all the available information and assessed the uncertainties of each piece of data, applying Bayesian principles of probability to work out the most likely location of the plane. To calculate these figures, they first looked at the theories about what caused the plane to crash. For instance, they assessed the likeliness of various mechanical failures and came up with a probability for each scenario. They then assessed historical data from previous crashes, noting, for example, that planes were usually found very close to where they were last known to have been. They took the last known location of the plane and estimated it could not have flown more than 40 nautical miles in any direction from that point. Using Bayesian maths, Keller and her team created what they call a probability map of the search area.

[D]

You take that area and you grid it. You divide it into cells, depending on how fine grained you want to get, and then you run your scenarios. And your scenarios might say, it flew this way, it flew this way. Each of those scenarios has a terminus of where the airplane might have ended up, and those areas get probability based on how much you weighted that scenario.

[A]

Finally, they looked at all the data from searches that had already been carried out, and they lowered the probability of the plane being found in those locations because they'd already been searched. The Metron analysts gave a map to a search team indicating the area they thought the plane was most likely to be in. A team went to look, but nothing was found. It seemed the Bayesian mass could not help after all.

[D]

So at that point, everybody was throwing up their hands, like, what do we do next?

[A]

The months passed, and the French and Brazilian authorities tried a few more things, but still no luck. Metron got another call.

[D]

So they came back to us one last time, and they said, well, we have enough money to do this once again, but this is probably the last chance. Metron did the analysis and factored in all the searches they'd done to date.

[A]

The Bayesian methods underpinning their model made it easy for Keller to input all the new bits of data from the most recent searches. While they were looking back on their work, they also decided they weren't happy with one of their initial assumptions. The historical data showed that following a plane crash, the black box will be emitting a signal in 90% of cases. In the immediate aftermath of the Air France crash, search teams had spent a lot of time to sweep their areas close to the last known location, listening for the ping of the black box or voice recorder, they had heard nothing. So Keller and her team had decided there was a very low probability the plane would be found there. But what if the black box and voice recorder were not sending a signal? The Metron statisticians now adapted their model to this possible scenario and came up with a new map.

[D]

Well, I wish I could show you the picture because it's really striking. The original probability map showed heavy probability along the intended flight path of the airplane from the last known point to the very edge of the circle. But they searched along that area. It basically just took a big cutout of the hot areas and made them completely cold. When we went back and said no, those searches maybe weren't a effective at all because there was no target to hear. And they had actually not tried to search that area again because they figured they'd searched it very well in the first 30 days. So all of a sudden it was like, wow, we gotta go back and hit that area again.

[A]

So a team returned to the scene to look in this location, and this time they found the plane. The mystery of the crash was solved. The black box recorder data appeared to show that the pilots were given faulty speed readings, responded inappropriately, and lost control of the plane. The problem is, although the Air France mystery was a success story in that they found the plane, perhaps it should be seen as an example of just how hard it is to find a missing aircraft.

[D]

It still was a minor miracle that we found it. I mean, we don't want to take full credit for it. We just kept them from looking everywhere haphazardly. But it was so lucky that the wreckage for Air France was on the bottom in a very sandy open area. There were some areas in there that looked like the Himalayas in terms of mountains and crags and valleys. And a side scan sonar being towed by a submarine might not see over the next ridge. And it could have been undetected forever, but it just happened to be right where they looked. It was fairly miraculous.

[A]

So what do we know about Malaysia Airlines Flight 370? The plane took off from Kuala Lumpur in Malaysia just after midnight on 8 March, but never arrived in Beijing. The last signal picked up from the plane seems to have been after seven hours. But we don't have an exact location, just a range of where it might be. There's been lots of speculation in the press about what happened. The Air France aircraft was thought to be in a 40 nautical mile radius of its last known location. Now, that sounds small, but actually it's thousands of square miles. What is clear is that the Malaysia Airlines Flight 370 is missing in a much bigger, bigger area. Keller says even finding debris might not mean finding the bulk of the plane.

[D]

If we found wreckage at this point, it would tell us it's in one body of water versus the other. But it's so long since the aircraft crashed that I don't think the wreckage is going to be very helpful in narrowing the search area down. It's a big world out there, and I know a lot of people are saying, how could you possibly hide or not find a 777? And I think the answer is it's easy. I think that it's very likely if we don't get any major breakthroughs, it could be in the bottom of the Indian Ocean and we will never find it, sadly.

[A]

If you have any statistical queries, then get in touch by emailing moreorlessbc.co.uk or tweet us using the hashtag bbcmoreorless. To find out more information, go to our [e-mail] More or less there are

[B]

dozens of different podcasts now available from the BBC, including news, documentaries, science, business, arts and sport. For details of them all, go to bbcworldservice.com podcasts.