The Artificial Future of Forecasting

Summary

Podcast Summary: Off the Radar – The Artificial Future of Forecasting

Episode Details:

• Title: The Artificial Future of Forecasting
• Host: Emily Gracie, Meteorologist at The National Weather Desk
• Release Date: February 25, 2025

---

Introduction: The Dawn of AI in Weather Forecasting

In this enlightening episode of Off the Radar, host Emily Gracie delves deep into the transformative role of artificial intelligence (AI) in weather forecasting. Setting the stage at [00:00], Emily poses a captivating question: "What if we could predict exactly when a storm will hit your neighborhood?" She highlights the rapid advancements in AI, where supercomputers process trillions of data points to detect patterns beyond human capability. However, she also underscores the inherent complexity of the atmosphere, describing it as "a chaotic dance of variables that still defies complete prediction." This episode aims to unravel the cutting-edge AI technologies enhancing weather forecasts, differentiate genuine breakthroughs from mere hype, and explore the potential and limitations of AI in achieving unprecedented forecasting precision.

---

Segment 1: Expert Insights with Dr. Amy McGovern

Guest: Dr. Amy McGovern, Lead of the National Science Foundation's AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography at the University of Oklahoma.

Background and Historical Context At [03:13], Dr. McGovern shares her extensive background, emphasizing her long-standing involvement in AI prior to its recent surge in popularity. She cautions about the critical responsibility tied to weather predictions, stating, "That somebody's going to predict the weather and people are going to die because it wasn't verified well. And that is a serious worry for me."

Evolution of AI in Meteorology Dr. McGovern traces the roots of AI in weather forecasting back to the 1990s, noting early applications involved "post processing" of physics-based models to correct biases like cold or spatial discrepancies. She explains how AI has progressed from these rudimentary corrections to more sophisticated real-time data integration, enhancing forecast accuracy.

Current Applications and Successes Explaining AI's role in everyday forecasts at [06:30], Dr. McGovern illustrates how AI enables personalized, hyperlocal weather predictions by downscaling large-scale models. She acknowledges recent successes, such as AI accurately predicting an unexpected cold spell, yet she remains cautiously optimistic about AI's consistency across diverse weather events.

Strengths and Challenges in Predicting Extreme Weather Addressing the ability of AI to predict specific weather phenomena like tornadoes and hurricanes, Dr. McGovern discusses the development of the Extreme Weather Bench by Brightband, where she consults. This benchmark assesses AI models' performance across various high-impact weather categories, highlighting that while AI shows promise in certain areas, challenges remain, especially in representing extreme events accurately.

Biases in AI Models At [11:27], the conversation shifts to the potential biases in AI beyond meteorological data. Dr. McGovern explains that societal biases can inadvertently infiltrate weather AI models through uneven data collection, such as more crowdsourced data in socioeconomically advantaged regions. She provides concrete examples, like riptide detection algorithms performing variably across different geographic areas due to inconsistent radar coverage.

Data Limitations and the Road Ahead Dr. McGovern identifies data scarcity, particularly in underrepresented regions like Africa and South America, as a significant hurdle for AI's global forecasting capabilities. She emphasizes the need for high-resolution, comprehensive data to train effective AI models and discusses ongoing efforts to overcome these limitations.

Unexpected AI Applications in NOAA She also highlights innovative, albeit unexpected, AI applications within NOAA, such as recognizing whale songs for tracking migratory patterns and detecting rip currents to enhance beach safety. These applications showcase AI's versatility beyond traditional weather forecasting.

Ensuring Trustworthiness and Ethical Considerations Dr. McGovern voices concerns about the commercialization of AI weather models without adequate verification, stressing the importance of rigorous testing to prevent erroneous forecasts that could have life-threatening consequences. She advocates for strong collaborations between government, private sector, and academia to advance trustworthy AI forecasting.

---

Segment 2: Breakthroughs in AI Forecasting with Elon Price

Guest: Elon Price, Lead Author for Google DeepMind's groundbreaking AI model, Gencast.

Introducing Gencast At [21:21], Elon Price introduces Gencast, Google DeepMind's latest AI-driven probabilistic medium-range weather forecasting model. He explains that Gencast outperforms traditional physics-based models, delivering highly accurate forecasts up to 15 days in advance. Unlike its predecessor, GraphCast, which was deterministic, Gencast is an ensemble model providing multiple scenarios to better understand the range and likelihood of various weather outcomes.

Traditional Models vs. AI-Based Forecasting Elon elucidates the fundamental differences between traditional and AI-based forecasting at [23:31]. Traditional models simulate atmospheric physics by solving complex equations numerically, a process that is computationally intensive. In contrast, AI models like Gencast learn directly from historical weather data, identifying patterns over decades to predict future conditions more efficiently.

AI Training and Data Utilization He details the training process, where Gencast is fed decades of historical data (ERA5 reanalysis dataset) to learn weather patterns. During forecasting, the model inputs current weather conditions and generates an ensemble of possible future states, offering probabilistic insights rather than single-point predictions.

Performance and Case Studies Addressing the model's efficacy, Elon cites Hurricane Milton as a case study at [33:32]. Gencast predicted a 60-80% probability of Milton making landfall in Florida eight and a half days in advance, increasing to over 90% just days prior. This early and probabilistic forecasting can significantly enhance preparedness and risk management.

Comparing GraphCast and Gencast At [38:04], Elon contrasts GraphCast with Gencast, highlighting that while GraphCast was a pivotal step in AI-based deterministic forecasting, Gencast's ensemble approach aligns more closely with operational needs by providing a range of possible scenarios. This shift enables better risk assessment and decision-making for extreme weather events.

Limitations and Future Improvements Elon acknowledges current limitations, notably Gencast's challenges in accurately predicting the intensity of tropical cyclones due to insufficient extreme event data in the training set. However, he remains optimistic about addressing these issues through enhanced data collection and model refinement. Additionally, he mentions potential expansions, such as predicting cloud cover and increasing the model's spatial resolution for more detailed forecasts.

Open Source Philosophy and Collaborative Goals Emphasizing DeepMind's commitment to advancing science, Elon discusses the open-source nature of Gencast. By making the code and model weights publicly available for non-commercial use, DeepMind fosters scientific collaboration and encourages researchers to build upon their work. This approach aligns with DeepMind's mission to leverage AI for the betterment of humanity, particularly through tools that have widespread societal benefits like weather forecasting.

---

Conclusion: The Synergistic Future of AI and Meteorology

In wrapping up, Emily Gracie reflects on the crucial interplay between AI advancements and the traditional meteorological sciences. The episode underscores the rapid progress AI is making in forecasting accuracy and the importance of addressing data limitations and biases to ensure reliable and equitable weather predictions. Both Dr. Amy McGovern and Elon Price highlight the necessity of collaborative efforts across government, academia, and the private sector to harness AI's full potential while maintaining ethical standards and public trust.

As AI continues to evolve, its integration into weather forecasting promises not only more precise and timely predictions but also innovative applications that can save lives and protect communities. However, the journey forward requires careful navigation of challenges related to data quality, model bias, and the ever-present need for rigorous validation to uphold the integrity and reliability of weather forecasts.

Notable Quotes:

• Emily Gracie [00:00]: "AI is transforming weather forecasting in ways that seemed impossible just years ago."

• Dr. Amy McGovern [03:13]: "That somebody's going to predict the weather and people are going to die because it wasn't verified well. And that, that is a serious worry for me."

• Elon Price [21:47]: "Gencast is our new AI based probabilistic medium range weather forecasting model. And it's exciting because it is the first of these models that really outperforms the traditional, the state of the art up until this point."

• Elon Price [23:42]: "The AI approach, the model is going to look at historical weather data, basically decades worth of historical weather data... it has the ability to hopefully learn more realistic and more complex dynamics than what those equations that we are solving describe."

• Dr. Amy McGovern [15:35]: "The AI models going forward, the big bump in the road is how good can we make them everywhere when we don't have data everywhere?"

---

Final Thoughts: The Artificial Future of Forecasting provides a comprehensive exploration of how AI is revolutionizing weather forecasting. Through expert discussions, the episode highlights both the groundbreaking advancements and the significant challenges that lie ahead. Listeners gain valuable insights into the mechanics of AI models like Gencast, the importance of data integrity, and the ethical considerations essential for the responsible deployment of AI in meteorology. This episode not only educates but also inspires confidence in the potential of AI to enhance our ability to anticipate and respond to the ever-changing weather patterns that shape our world.

Transcript

[Emily Gracie]

What if we could predict exactly when a storm will hit your neighborhood? Not just a 60% chance of rain tomorrow, but a heavy downpour at 2:17pm on your street corner? AI is transforming weather forecasting in ways that seemed impossible just years ago. Supercomputers running sophisticated networks are now digesting trillions of data points to see patterns humans never could. But this technology revolution comes with an important reality check. The atmosphere remains one of the most complex systems we study, a chaotic dance of variables that still defies complete prediction. Today, we're going off the radar and exploring the cutting edge of AI weather forecasting. From the systems already making your daily forecast more accurate to research that's pushing the boundaries of what's possible, we'll separate genuine breakthroughs from hype and glimpse the meteorological crystal ball that may one day predict weather with unprecedented precision. How close are we to the perfect forecast? And what fundamental limits might we never overcome? I'm meteorologist Emily Gracie, and you're listening to off the Radar, a production of the National Weather Desk. On the show, we dig deep into topics about weather, climate, the ocean, space, and much more. Our goal is to help you better understand the weather and to love it as much as we do. Machine learning and artificial intelligence are being studied and implemented across all parts of the weather enterprise. NOAA or the national oceanic and Atmospheric Administration is using AI in a number of different ways, like for urban heat mapping, marine life identification, and even rip current detection, which we'll take a deeper dive into in an upcoming episode. Today we have two AI experts joining us. Elon Price is the lead author on Google DeepMind's latest public. It's about Gencast, a breakthrough AI model that's pushing weather prediction beyond what we thought was possible, delivering remarkably accurate forecasts up to 15 days in advance. He'll share how Gencast is redefining the boundaries between what's predictable and what's not. Elon will take us inside this game changing technology and discuss why their forecasting is so important to the safety of people.

[Elon Price]

Sometimes it's really important to know that there's a small chance of a really bad thing happening. And if you're only ever going to give the most likely prediction, you might miss that. That risk is something that you need to account for and you need to prepare for.

[Emily Gracie]

But first up, we're going to hear from Dr. Amy McGovern, who leads the National Science Foundation's AI Institute for Research on Trustworthy AI in Weather, Climate and Coastal Oceanography at the University of Oklahoma. Amy's been in this field since before most of us ever even heard the term artificial intelligence. She's an expert on what is possible, what the bumps in the road may be, and what she's concerned about when it comes to the future of AI and weather forecasting.

[Dr. Amy McGovern]

That somebody's going to predict the weather and people are going to die because it wasn't verified well. And that, that is a serious worry for me.

[Emily Gracie]

From enhancing daily forecasts to potentially saving lives during extreme weather events, AI is transforming how we understand the atmosphere above us. Stay with us to discover how these computational breakthroughs might help us navigate an increasingly unpredictable future. Amy McGovern, thank you so much for coming on to talk about this important issue today. A lot of buzz out there. So I want to get the stitch from you on everything going on, but can you give me a little background on yourself first of what your educational background is and then what you research right now?

[Dr. Amy McGovern]

So I'm a professor, I'm generally appointed in meteorology and computer science. And so you ask for educational background. My PhD is actually in computer science and I like to tell people I've been doing AI since before AI was cool because I think a lot of people seem to think that AI just appeared in the last few years and that's not true. So my PhD was in AI and then, you know, I wanted to get into meteorology because it allowed me to do something that's interesting and it lets me develop AI for a way that could save lives.

[Emily Gracie]

Oh good. Because I want to talk about the history of AI in our country too, because like you said, it's gotten a lot of buzz in recent years. But can you tell me how long it's been in use and kind of the beginnings here?

[Dr. Amy McGovern]

It's been around a long time since like the 50s and 60s, 1950s and 60s, so that that's clear. You know, it started with some really early systems they called like expert systems where they were trying to do just help decision making and complicated decisions like healthcare systems. And they're really simple. Like expert systems were rule based systems. They were, can we codify sort of what's going on in human decision making so that we can help them go through according to data and perhaps remove some of the biases that are happening in these human decisions. So there were some of the early applications were in medicine and then some of the other early applications of course have been in robotics because robotics has been around a very long time too. And trying to make the robots do something more intelligent than be an RC system is a Great part of AI.

[Emily Gracie]

Interesting. Okay, what about with weather? What about weather forecasting? When did that start?

[Dr. Amy McGovern]

So AI for weather. AI for weather has probably been around. So I've been part of the AMS AI conference for a while and I think it started before I got out of grad school. So in the 90s, but they weren't meeting every year. There were just initial applications and a lot of the initial stuff was basically taking the existing physics based weapon models and doing post processing. It's no longer called that anymore, they have cool new words for everything. But post processing is basically you take the physics based model, you take the output, and then you try to correct the model biases because those do happen. Everybody knows one model has a cold bias, one bottle has a quorum bias, one in the middle has spatial bias. So a lot of that was the early applications. And then people started to be able to div into what can we do that's even better than that? Can we take the real time data that's available and put that on top of the weather, you know, the physics based model and, and try to give you a, a better forecast in real time, for example?

[Emily Gracie]

Okay. A lot of our listeners are probably not degreed meteorologists or scientists. So can you tell me kind of in layman's terms how AI is being used right now? Like if you're someone who's just looking for a forecast, maybe you pull up your phone every now and then. Where are we seeing practical uses of AI in our forecast now?

[Dr. Amy McGovern]

Okay, so that' a great question. I think that, you know, the AI is happening behind the scenes and so it's happening. You know, most people are grabbing their smartphone and looking at the weather forecast, right? And what they're getting is a personalized forecast, right? The forecast is giving, is trying to personalize it for where they are. It's trying to give them a hyperlocal forecast. And that's being done with AI primarily because what it's doing is, is back to that post processing, trying to take the, the large scale model and try to downscale it. For example. It's happening in other places. There's some new. The, the reason that AI is getting sort of hot right now is that you're hearing about these new global scale AI models, like Google's models and Nvidia models and all these other models and you're hearing about them on the news and you're saying, wow, they're doing these great things. What they're really doing is AI based weather forecasting across the Whole globe. They're not going to right now. They're not giving you something that's going to tell you where the next tornado is going to be. What they're doing is giving you a good idea of what the temperature is going to be. When's the next cold spell coming, when's the next heat wave coming? And those things they're doing really well at. And again, I think from the public, it's still behind the scenes. What's happening with those is those models are being used by the forecasters to put the human over the loop and give you a new forecast. But that's coming in as input to them. This latest cold freeze that just happened. I don't, I know this is airing in a couple of weeks, but there's a cold spell right now. And you know, some of the models didn't predict it to be as cold as it was. And if you go look at some of the AI models, some of them nailed it. And you know, that's a good question. Are they going to nail it all the time?

[Emily Gracie]

That brings up the point of specific types of weather. Do you see AI being successful with certain types of weather over other certain types of weather? As far as research going on right now, is there that possibility that it will predict where the next tornado is going to be or exactly where hurricanes can make landfall?

[Dr. Amy McGovern]

Those are excellent questions and it actually relates to one of the things I'm doing outside of academia. So I'm consulting for a company called Brightband. We're a new startup company. And one of the things I'm doing there is developing a new benchmark suite called Extreme Weather Bench. And the goal is that we can enable people to evaluate them on the different types of high impact weather. So we have a whole tropical cyclone or hurricane section, right. And you can look and see how well it's doing at landfall prediction, how well it's doing at intensity of the the hurricane, how well it's doing at the rainfall. But then you can also look at the severe weather, you can look at the flooding, you can look at the cold spell so that you can actually break the performance up by the different categories of high impact weather. And I think that's going to be really important because right now primarily the way people are evaluating them is they're looking at like the global root mean squared error. So they're saying, oh well, on average we're off by 2 degrees. Well, you know what, when it's 2 degrees off at 75, no one really cares. But when it's 2 degrees off. I'm sitting here just on the line of a freezing rain event. And the 2 degrees difference between 33 and 31 is a big difference in impact. And so an RMSE kind of obscures that. And having something that lets you dive into what the real impacts where I think is good. And so the answer to your question is I think that the AI models are going to do well. We're already seeing in the news people looking at specific tropical cyclones, but they're not, when they do that, they don't tend to look at the other events. Right. So what we need to do is start diving into each of these high impact events and say, how well is it doing at that? You ask about tornadoes. Yes, I think that there are models in development that are working on those tornadoes. Are they ready tomorrow? Now?

[Emily Gracie]

But it does seem that AI progresses much more quickly than like the science of the past. Is that. Am I crazy or is that true? Is it progressing really fast?

[Dr. Amy McGovern]

It is progressing really quickly. I don't know that it's that science of the past is slow so much as computers are getting faster all the time. So we're able to do things that we weren't able to do before. And the exponential growth in computer speed is enabling things to grow exponentially faster. Rate you might have thought up. So the deep learning algorithm, for example, was thought up much earlier than it was actually been able to be used because the computers didn't exist for it. But as soon as they existed, people were able to just go, gotcha.

[Emily Gracie]

Okay, that makes sense. All right, you brought up a company you're working with, private sector. I would like to bring up Noah Hot Topic in the news right now. But what is your knowledge of what's going on with Noah when it comes to the use of AI? Is there stuff in the works?

[Dr. Amy McGovern]

They have good stuff in the works because they have a NOAA center for AI. It's a little bit different because of the way that NOAA is funded through Congress. They can get mandates to do something, but they don't necessarily get handed funding for that. And so the NOAA center for AI was created a couple of years ago and they've really been working hard, but they were not actually granted an appropriation to the budget to actually fund them. And so they've been working with what resources they can do, but they are doing a lot of AI throughout noaa. And if you go to the NOAA center for AI webpage, you can actually see a bunch of the applications of AI throughout noaa. I Mean, because I do have such close colleagues in noaa, I see a lot of the things they're doing. They're testing these global AI models, they're testing the regional models. The tornado thing that you mentioned, there's a tornado model that's being developed that's actually based on Google's GraphCast model. It's called WAFscast. WAF stands for Warnarm Forecast Model. So they're trying to make a graphcast based version of the warnarm forecast model. It's not operational, but that's another thing that NOAA does really well. NOAA does trustworthiness. They focus on making sure the models are trustworthy. And so they bring them up in test beds. And so everything, all of these models that are getting test tested in the test beds before they go out and get thrown at the humans in the public.

[Emily Gracie]

So something I want to talk to you about too. You mentioned bias as far as warm air, cold air, cold bias. But I want to talk about other biases that occur when it comes to AI. Can AI be, and you know, people can be this way too. Can AI be biased against certain types of people?

[Dr. Amy McGovern]

Well, AI can certainly. I think that's been in the news.

[Emily Gracie]

Right.

[Dr. Amy McGovern]

The most obvious example is if you train the database on all white faces and you give it black faces to recognize, it struggles to recognize black faces. We have a paper that actually talks about some, how some of these biases can transition over to weather too, because you're not doing face recognition in weather. And so that's not an obvious translation. But there are ways in which those sort of societal biases that are happening do transition over. And that has to do with, for example, with the data that you've collected, there tends to be more crowdsourced data in the more socioeconomic, socioeconomic regions. But there's also just biases that exist in weather data that don't exist in other types of data. So physics bias. So the laws of physics apply to everything. But let me give you a simple one that's nice and easy. When the radar goes out in a straight line, if you're trying to detect the tornadoes, you know the radar is only seeing the tornadoes, which are down at the lowest level nearest the radar. And so if you were training an AI model and it learned somehow that it had to see a particular signature in the radar, it might end up not giving you tornado detection farther away when that might actually be a problem. Because the radar coverage in the United States isn't exactly uniform. Everywhere there's topography that gets in the way. And so you might deploy an algorithm that was only doing radar detection unintentionally, really well in certain areas and not in others. And then people might have over reliance. Oh, this algorithm was great. It gave me 100% every tornado. So trust it. And then it misses tornado.

[Emily Gracie]

Okay. Any other instances you can think of? Any specific examples? I read one that you. I think it was an interview you did that was about air quality and air quality sensors.

[Dr. Amy McGovern]

Yeah, air quality sensors are in that same thing because they're crowdsourced. Right. And any crowdsourced data has biases to it. Because humans have biases to it. The other we. A paper that was out that I'm referring. We've had two papers about biases. One of them is about categorization of the biases and one that we haven't talked about yet that's going to apply to AI no matter what you're doing in AI is the biases that exist in our human nature. And that isn't to mean that we're bad. It just means that we humans make quick decisions. That's what psychology shows. Right. And some of that comes from being overwhelmed. And we might be making quick decisions about data or how we process data or something that might make the data not as good for the AI model as it could have otherwise if we'd made a more careful. There's a book over there called a Fast Thinking, Slow Thinking, and that's kind of what I'm referring to that it isn't like it's inherently bad in us. It's just a way that we humans work. We have to make fast decisions when the cheetah is running out of progress. Right. But you want to make a more slow decision when you need to ponder things. And sometimes when we're making these fast decisions, we're not aware that we're making the fast decisions. And so we might be making choices of metrics that aren't the best metric to actually check things. We might be making choices of how we process the data. We might be making choices when we're looking at model output and just saying quickly, I like that one. Without diving into it deeply to understand where it hits.

[Emily Gracie]

I guess I'm curious what the solution is then. Is it, you know, more ethical training? Is it more variety of people working on the project? What's the answer?

[Dr. Amy McGovern]

All of the above. We've got a project that we just got funded and that we're working on developing some measurements so that you could actually ask your AI for weather model. How biased it is in certain situations. Like, is it suffering from the physics bias? Is it suffering from a geographic bias? That's another example we could give. Like, you know, you trained it all in the Southeast, but now it doesn't work all that well in the Northwest because they are very different weather regimes. So ask it these things so that you could then have a way to mitigate it. So if you had a list, a checklist, then you could go through that and say, okay, now I can be more confident of that. So, but I think if you have. The more, the more people you have working on it, the more people you have are going to ask those kinds of questions. If you really literally only know the weather in one area, you're not even going to know that you're wrong in other areas.

[Emily Gracie]

Okay, so AI as a whole, when it comes to weather forecasting, what are some of the limitations other than biases? What are some of the other things you see as popping up as kind of bumps in the road going into the future?

[Dr. Amy McGovern]

I think big bump in the road is data. And that is because the a, the AI models don't actually know physics. They're learning physics from observations. And the observations they're getting are 40 or 50 or 60 years of data. Almost every model is training from Era 5, so that's ECMWF. The European model, they give out this reanalysis, forecast Beta, that it's a great data set, but it's a coarse data set and it's known to have issues. The precipitation data isn't all that great. It's better in North America and Europe because we have more observations here. And so the AI models going forward, the big bump in the road is going to be how good can we make them everywhere when we don't have data everywhere? Right. Can we make a great AI model for Africa when we don't have any data in Africa? Same for South America. We have some data, but not very much. It's just, it's a challenge. I think data is going to be the big bump in the road. We need higher resolution data and we need data globally, everywhere. And even if we solved it, by the way, in all the countries I just mentioned, we need ocean data. There's a large part of the world that's ocean and it's not very well sensed.

[Emily Gracie]

Yeah, not just ocean. I interviewed a guy that's studying Everest and putting weather stations on Everest, and that's kind of another inhospitable environment that's hard to study too, Right?

[Dr. Amy McGovern]

Very true. And it affects the rest of the world. That's the weird people are studying something like, okay, I need my tornado and I need to know whether or not it's going to be here in the next hour in Oklahoma. I really don't care what it's doing in Everest right now from Oklahoma. Right. But if I want to talk to you about drought in the next two months, I really need to know what's going on around the whole globe. And I can't just have these blind spots. And that's going to be something that we really need to address. I saw a really cool talk at ams. This is not my work work, so I'm promoting somebody else's work. But it was a really cool talk. Greg Hakam from the University of Washington gave a talk asking about what the limits of predictability were and whether or not AI could break the limits of predictability that are known right now with the physics based models. And I think that the talk he gave was really cool because he was asking the question of if you knew that there was going to be this really big event, how far back in advance could you get the AI model to predict it? And he showed that you could do it farther back than any of the physics based models could do now. No, it still requires you to know the right answer. But that's another thing that I see coming up. Like, how can AI break that current predictability barrier? How could we start predicting things four weeks in advance? And by the way, for the listeners, I'm not saying this is happening tomorrow.

[Emily Gracie]

So when is it happening?

[Dr. Amy McGovern]

This is still in development.

[Emily Gracie]

Any timeline at all. Are we talking months, years, decades?

[Dr. Amy McGovern]

Years. But not in the decades. Not in the decades. I think that a lot of the stuff is going to be in the. Especially because of what you ask at the beginning. AI feels like it's echoing exponentially. I think we're talking in the next couple of years we're going to see some real breaks or something. Lot of this.

[Emily Gracie]

Are there any unexpected uses for AI that you've seen come up that weren't really planned that have been useful when it comes to weather or climate?

[Dr. Amy McGovern]

No. NOAA has a couple of examples of those that I didn't really think about. And that's because I didn't. I don't think as much about the ocean. So they have two that I've seen that I think are pretty cool. One is recognizing whale songs and whales and different whales. Like a whale Id not just songs, but like faces. And so you can actually figure out the migratory patterns of individual whales. Which is just pretty cool, because to me, it's a whale. And another one I saw that they've got deployed is riptide detection.

[Emily Gracie]

Yes.

[Dr. Amy McGovern]

And that's a place where you could save lives. And it probably isn't unexpected to somebody who studies the riptides on a more regular basis, but I live in a landlocked state, and I just thought it was cool because imagine that you could deploy that and all the people that don't understand riptides, they could at least automatically have something on the beach that says, right now there's a high riptide, and you could visually show it. Here's where it is. Please don't go in, because I think people sometimes don't always read the sign. Oh, that sign's been up since yesterday.

[Emily Gracie]

There's an upcoming podcast episode, actually, with Greg Dusek that I'm doing.

[Dr. Amy McGovern]

Oh, awesome. Awesome.

[Emily Gracie]

Amy, is there anything else you want the general public to know when it comes to AI, what they should be looking for, what they should be wary of in the future when it comes to weather?

[Dr. Amy McGovern]

That's an interesting question. I think. I think a lot of the places that are out there right now are trying to make sure that they're testing what they're putting out. But I have a concern that there might be people who are putting out models that are. They're selling them, overselling them before they're ready because of the AI hype. And that is my concern and my thing to be wary of. I think that if you're. If you're running purchasing for a company, please make sure that what you're looking at is an actual model that's done a good set of evaluations, I think. I'm not worried about that for the general public so much, because I think the general public, the models that are being deployed on your phone are getting pretty well verified because most people are getting in from Noah, and Noah doesn't really good job of doing verification and validation. But it's a concern for me that somebody's going to predict the weather and people are going to die because it wasn't verified well. And that is a serious worry for me.

[Emily Gracie]

Yeah, it's a legitimate concern. Yeah, it really kind of drills in the whole point of the weather enterprise and the necessity of this piece of government, private sector, and academia all working together at this.

[Dr. Amy McGovern]

Yes. Having these collaborations and these partnerships is going to be really key to going forward, because that's another thing. You asked about a bump in the road earlier. Here's a bump in the road. NOAA doesn't have the computing resources that private industry does. Academia doesn't have the computing resources that private industry does. We need to find a better way to do those partnerships. Right, because academia can take a little bit more of the risks because that's what we, you know, we write the grant, but like we're not selling the product, whereas private industry needs to sell the product. But maybe we can do some of that exploration together so that it can actually be a great MIS management strategy where we can do something really cool with the resources. But, but, but developing something that maybe somebody in private industry couldn't do it on a quarter basis. It's fun stuff. I'm very excited about the future of AI for weather.

[Emily Gracie]

Elon Price, I want to talk to you about something that's getting a lot of buzz online right now, especially in my community, which is the weather community, the meteorologist community. And this was just kind of released recently. But Google DeepMind has done some amazing work when it comes to AI use of weather forecasting. So tell me about this latest announcement that you guys wrote about. What is gencast?

[Elon Price]

Gencast is our new AI based probabilistic medium range weather forecasting model. And it's exciting because it is the first of these models that really outperforms the traditional, the state of the art up until this point, which have been weather forecasting models which use physics or simulate physics to, to, to forecast the weather. So it's an ensemble model, it's a probabilistic model and it's an ensemble model. What that means is that instead of giving us one prediction of how the weather will develop, it gives us multiple scenarios. It helps us understand the range of possible outcomes, range of possible scenarios and how likely each of those is to occur. And that really makes it much more useful for decision making. And in fact, the operational forecasts that we rely on day to day are ensemble forecasts like this. So we did a rigorous evaluation of this model and we found that it outperforms this prior state of the art, which is called, which is an ens, which is the ensemble system from the European center for Medium Range Weather Forecasting. And we found that it did better not just on, on day to day weather, but also on extreme weather, things like extreme heat, extreme cold and predicting the tracks of tropical cyclones. And yeah, and that's really, really important. It means, you know, knowing the risks, these risks and these, the chance of these things happening earlier and being able to prepare better, it's also much faster. So we're talking about doing on, in about eight minutes on a chip that's a bit bigger than a laptop, what could take up to hours for these traditional methods on supercomputers that have tens or hundreds of thousands of processes.

[Emily Gracie]

Can you kind of break down, in really simple terms the difference between traditional computer modeling to do a forecast and the use of machine learning or AI to do a forecast?

[Elon Price]

Yeah, absolutely. It's a very important question because they are taking quite fundamentally different approaches to predicting the weather. So the traditional approach to forecasting the weather is essentially to simulate the physics, or at least the dominant physics of the atmosphere. So what that means is that we, we kind of write, we, we know, we write down some of the main physics equations that, that kind of describe the way that the atmosphere develops over time. And then we apply, we use computers to solve these equations numerically. And that's a process which is a very computationally intensive process. But what essentially does is simulate the physics and we kind of roll the physics forward in time and that tells us, okay, how is the atmosphere going to change? In other words, how is the weather going to change over time? And that's really fundamentally quite different to how an AI or machine learning based model would do that. The AI approach, the model is going to look at historical weather data, basically decades worth of historical weather data. And we task the model, the machine learning model, with learning the weather patterns directly by looking at that data. So in simple terms, what we're going to do is we're going to say, we're going to give it that, what was the weather, you know, at some day in 1980, you know, on, on some Monday? And we're gonna, we're gonna ask it pretty at what was the weather going to, how did the weather change in the next 12 hours? What was that weather 12 hours later? And then we'll show the model what really happened 12 hours later. And it'll be shown essentially what error did it make. And it'll make a little correction based on the mistake it made. And we're going to do that over and over and again, over and over again for data points for over four, over the last four decades. And in that process is what we call training the model. It learns to pick up, okay, if this is what the conditions were like, how is it, how, how is it likely that the weather will develop from that point? And so really it's learned directly from looking at kind of historical weather data or estimates of what, what the historical weather was, what these weather patterns are. And that's important because the equations that we solve, where the traditional models solve, they are really only an approximation to the full set of atmospheric dynamics. They don't capture everything thing. And machine learning models aren't restricted to learning those dynamics that we can write down in an equation because they're going to look at estimates of what really happened. They have the ability to hopefully learn more realistic and more complex dynamics than what those equations that we are solving describe. And that's fundamentally how it's possible that machine learning models can improve upon traditional models.

[Emily Gracie]

Okay, so what about current weather conditions? Because I always hear about, you know, like, they're flying into hurricanes. And here's the latest data. So we're going to take this data, we're going to put it into the computer model and make the forecast better. So how does that work with AI? Do you use any current conditions?

[Elon Price]

The historical data is really only for the learning process to get the model good at predicting the weather. And then when we want to use the model to predict the weather, we use it in the same way essentially that we would use a current model. So we give, we say, we now tell the model, okay, you've seen all of this historical data. You've understood the weather patterns or learned how to predict the weather patterns. Here is the weather. Now tell us how the weather is going to change. And so you input what the weather conditions are now and what the weather conditions were, say, 12 hours ago. And then you get out of the model a prediction for what the weather will be over the course of the next 15 days. And as I mentioned, you don't get actually get one prediction out. You get get, say, 50 predictions out. And that is what we would call an ensemble of forecasts. And so it tells you, okay, this, these are the ways that the weather might change and that, and allows you to calculate, okay, some things are more likely than others, but maybe something is. There's a small chance of the cycling going this way. And so it's worth preparing for.

[Emily Gracie]

Anyway, let me get this straight. Let's say a traditional computer model is a college student. They're just taking like math and science classes. Right? But like an AI computer model went to college and they took math, science, social studies, sociology, communications, and then they're both going out into the real world and kind of doing the same things, but AI is better prepared. Is that kind of a good analogy of what they are?

[Elon Price]

Maybe. I would say, you know, the AI model didn't do maths and science.

[Emily Gracie]

A lot of history, though, right? They're big on the history Interested history.

[Elon Price]

Yeah, and they picked up, up, they, they learned, they basically learned the patterns of, of how the world changes. And, and the good thing about the weather is that for the most part it's, you know, the underlying physics of the weather are fundamentally the same as they, as they were over the course of this period of this tr. The training data period, the data that the model saw when it was learning how to predict the weather. And so, so we verify through this process of validating how well the model performs, we are checking that the kind of underlying processes of the weather are the same. And so it works to learn from the past and then predict in the future because the fundamental patterns that it's learned are still true today, even though it was learned them by looking at data from the past.

[Emily Gracie]

Is an AI model going to do a better job of predicting, predicting weather and a changing climate then? Because it's not having to go back and look at new physics all the time. Instead it's just looking at like the past 20 or 40 years worth of data.

[Elon Price]

It's a really good question and it's actually not something I'd say that we have a, a really definitive answer for. What I can say is that one of the main concerns, one of the, one of the chief immediate concerns of climate change is extreme weather. You know. And so an important part of the paper that we published in Nature was an evaluation not just of, you know, the kind of average case weather over the course of the whole year, which we did do, but was also to look at how, when, how does the model do at predicting the risk of extreme weather, so extreme heat, extreme cold and the tracks of tropical cyclones or extreme wind speed. And what we found was that, that even though, you know, the climate, you know, has been changing and the weather in 2019 was not exactly the same as the weather, you know, in 1980. This machine learning model, Gencast, which was trained on this historical data, did still outperformed traditional models when predicting extreme weather. And so that tells us that at least for now, these models are capturing the extremes well, they are predicting them well, they're predicting them better than traditional models. We can. It's another important and good feature of machine learning models is that we can retrain them as we get more data and more recent data, we can keep updating them. So, you know, the model that we have in the paper was trained up looking at data up until 2018 and then we tested the model by using it in the future in 2019 and seeing how it did. If we want to make predictions. Now though, there's no reason to stop the training data at 2018. We can use training data all the way up until, you know, yesterday in theory. And so that allows the model to learn the fact that you know what the, what the kind of recent weather patterns are, what is kind of more or less prominent, more or less likely, given the most recent data. And we have verified that as you do that, as you include more and more data and more and more up to date data, that makes the model do better and better and better. So that's another way in which the model can adapt. So that being said, the reason I say it isn't a, it isn't an open and shut case. It's not a, it's not, I don't say we have a definitive answer on that, on that question. Question is that if we reach, say for some sort of tipping point, for example, where the weather dynamics really do stop representing what they used to look like, if the weather fundamentally changes in a way that historical patterns aren't really informative, well, then machine learning models will start to struggle more because of course they're only really as good as the data that they're trained on. And so it's important that the training data represents the circumstances that it's going to be used in day to day.

[Emily Gracie]

And then where does that data come from?

[Elon Price]

So the data that we train on is called reanalysis data. And what that means is it's basically an estimate of what the atmosphere was like in the past. And it comes from a data set that's called the ERA5 reanalysis dataset, which was produced, it's publicly available and it was produced by the ecmwf, the European center for Media Range Weather Forecasting. So that provides us with decades worth of estimates of what the weather conditions were like at different times. And that gives us our training data set. Now when we evaluate the model, we can either use that data set to evaluate it, but we can also use the model real time. And to do that we, we would feed in what would be, would call an operational analysis. So again, it's an estimate of the, of what the atmosphere is like at the moment rather than, you know, in the past.

[Emily Gracie]

So you're talking about extreme weather events. You know, we've certainly had our fair share here in the U.S. hurricanes. Were, did you, were you able to apply this to any hurricanes in that the US may have experienced this year? Any specific relatable that we can be like? Oh yeah, we remember that, that it.

[Elon Price]

Did really well with, yeah, Absolutely we did. So I can, I could talk about Milton, for example, which was. We did, we did essentially we did a retrospective analysis. So looking back, making predictions, you know, in advance of, of Hurricane Milton, you know, we can ask how, how would, how would gencast how would this AI model have done if we were making predictions live in advance of the, of the hurricane? So we could ask questions like according to the model, you know, according to the model's predictions, what was the probability that Hurricane Milton would make landfall in Florida? And we can ask that question about forecasts that were made a day before or five days before or 10 days before, before that landfall actually did happen. We did that, we asked those questions about our model and what we found is that eight and a half days in advance of land of that landfall, GenCost was predicting 60 to 80% probability of that landfall happening. And from about 5.75 days before landfall it was predicted, it predicted that landfall in Florida was more than 90% probability. And we could also, if we want to, we can look at, open a map and we can look at, you know, what, what locations, landfill locations are more or less likely. That kind of prediction, you know, 60 to 80% of our probability of landfill about eight and a half days before, before, before it happened. That's actually even a, the, before the hurricane formed. And that's, you know, that sort of warning that can be really important allows, you know, whether it's, if, whether it's preparing evacuation plans or, or you know, stocking supplies, you know, that can, that could be really, really important. But having mentioned that example, I think important to mention probably two other things or caveats, so to speak. The first is that there's merit in looking at individual case studies, but probably also worth taking each individual case study with a pinch of salt because all, you know, all models are going to do better on, going to do really well on some examples and slightly worse on other examples. And really to make a scientific claim about the reliability of one's model, you really have to do a kind of extended evaluation of all cyclones over an extended period of time. So that's what we did in, in the Nature paper. So we took, in our case, we took the full year of 2019 and we looked at all the cyclones and we compared our performance of the predictions of these tracks with the predictions made by ENS in that year, this state of the art physics based system. And so it's on that basis, you know, that we can, that we are able to claim, you know, okay, we're doing better at predicting the tracks of tropical cyclones rather than on the basis of cherry picking a particular case study. Because you know, we did really well in predicting Milton. So that's anyway, so we have done that analysis. That analysis is in the paper. And then the second thing would be to bring up a limitation that gen costs that this model currently faces and that is that while it's currently really good at predicting the tracks of these storms, it currently doesn't do very well at predicting intensity of tropical cyclones. Now this, the, the reason for that is essentially a function of the data that it was trained on. This ERA 5 reanalysis data set that I mentioned, the training data that our model learns where the pattern's from. Those very extreme winds, very extreme cyclones are not really well represented in that data set. And so the model doesn't really learn to represent them well. We don't think that's an unsolvable problem and we're confident that that's something we're going to be able to address. But it does mean that in the immediate term this, the tropical, the track predictions of models like this should be used in combination with other models that are better tailored to predicting cyclone intensity.

[Emily Gracie]

Gotcha. Okay. And you're talking about like the next step here and what you're going to do in the future, but the past was not that long ago because I feel like it was just about a year ago. I was talking to one of your colleagues about graphcast and how amazing that was. And then here we are, this is now, now, just a short time later talking about the next big thing. So this is really progressing pretty quickly. Right? Can you tell me also the difference between graphcast and gencast?

[Elon Price]

Firstly, you're absolutely right, the pace is really rapid and especially if you think about the fact that, you know, we got to where we are now with traditional models really over the course of decades and they have been fundamental in creating the data that has allowed this accelerated advance of AI based models. Based on that, you know, based on those advances, based on that data that it, that those models, those traditional models created really in the course of a few years AI models have caught up and surpassed this decade's worth of research in traditional models. And now we have state of the art weather predictions coming from, coming from machine learning. So really the pace is very fast. And yes, the kind of gap between Graphcast and Gencast, the previous medium range weather model from Google DeepMind and this most recent one is also quite big. The fundamental difference between the previous model and this model is that the previous model was a deterministic model that gave us 10 day predictions. So it gave us a single kind of best guess at how the weather is going to develop. And that was a really important milestone because, because it really showed that machine learning models could learn some fundamental weather dynamics and they could compete with single deterministic simulations by physics based models. But even after graphcast, the best operational systems were still using traditional based methods. And that's because operational models have been ensemble models. They have to be probabilistic models. Because the weather is inherently uncertain, we can't predict it exactly. And so the best, what we need to do is be able to predict the risks of different events happening. And that aspect of what's necessary for a really useful weather model. That's what our new AI model gives us. The fact that it's an ensemble model that gives us a range of possible scenarios and shows us what's more likely, what's less likely, and allows us to, to make sure we capture risks which are important even if they're small. Sometimes it's really important to know that there's a small chance of a really bad thing happening. And if you're only ever going to give the most likely prediction, you might miss that. That risk is something that you need to account for and you need to prepare for. You know, machine learning AI models are kind of ready for prime time in a sense. They're now ready to start being incorporated into operational weather forecasting systems and used alongside these traditional models.

[Emily Gracie]

It's really interesting what you said about conveying a small risk, even if it's really small, conveying that it exists. I'd love to talk to a social scientist about that and understand what the world would think of that. You mentioned hurricane intensity and wind speed. Are there any other weaknesses or limitations of the model?

[Elon Price]

Well, there's definitely opportunities to improve it. So for example, there are certain variables, atmospheric variables, which the model doesn't at the moment predict. So for example, it doesn't predict cloud cover at the moment. There isn't really a kind of fundamental reason why it can't. We just didn't train it with that data. So the model could be retrained to do that. It could also further improve its resolution. So the model at the moment is a quarter degree resolution model. That means that essentially we kind of split, divide up the world into these little squares. Each one is a quarter degree latitude by a quarter degree longitude. And then when we make a prediction, we make a prediction for all of the different weather variables for each of the squares. And the smaller the squares, the higher the resolution of the model. And one option would be to train a model like this just at a higher resolution than what we've currently trained.

[Emily Gracie]

Okay, you talked about open source and you know, Google DeepMind's always been really good about like just sharing everything. Like this is what we've done, this is what we have, we're putting it out there. But I'm curious, kind of the why of that. You know most people are in something to make money and so I love a little background on like the ultimate goals of DeepMind and what the plan is here because everybody thinks of Google as a search engine or their email. So what's with the science part?

[Elon Price]

Well, you're absolutely right that the team has a great track record of open source. In particular the weather team here that developed our previous model, that model, the code was made open source, the weights were made available for non commercial use so researchers could use them, could actually run the model themselves. We really saw the kind of scientific impact of sharing, of sharing our work like that. And it's something that's really important to the team and we've done it again with this model. So the code code is open source. The weights of the model, both the research model that generated the nature paper results and an operational model that can be kind of run live. Both the weights of those are available for non commercial use. We've also just released an archive of his both historical forecasts made by the model and live forecasts that are being made by the model. Which means that researchers can access, interrogate, you know, do research on the outputs of these models without actually running the models themselves. You know, we care about kind of advancing the front, the scientific frontier in this regard. And that really, I mean it squares very much with Google DeepMind's mission which is to advance AI, you know, for the benefit of humanity and the benefit of society. And this is one of those instances where machine learning can really have, you know, dramatic and very far reaching benefits to everyone because, you know, because the weather touches everyone's lives and in both small ways and sometimes you know, life changing or life devastating ways. So we're definitely kind of excited and motivated by the fact that this sort of research can, can kind of help advance a field which will really benefit so many people.

[Emily Gracie]

I think this is really important information for people to understand and get. Thank you so much for your time.

[Elon Price]

Thank you for having me. It was a great chat, thanks a lot.

[Emily Gracie]

Off the Radar is a production of the National Weather Desk. Make sure you're following the show on Apple Podcasts, Spotify, or wherever you listen to podcasts. New episodes publish every Tuesday morning. You can also rate and review us there, let us know what you think of the show and give me ideas for future episodes. Off the Radar is also on Instagram now. Make sure you give us a follow so you can see snippets from the show and lots of other great weather content there as well. Thank you to Elon Price and Dr. Amy McGovern for joining me today. Thank you to the National Weather Desk and Sinclair Broadcast Group for their ongoing support of the podcast as well. Also thank you to my associate producer, Brian Petras. I'm meteorologist Emily Gracie. Make it a great day.