CISL Seminar - Towards AI-based Data Assimilation for Weather Forecasting

ABSTRACT

In recent years, significant advances in employing machine learning to build emulators of Numerical Weather Prediction (NWP) models has given rise to a cadre of “machine learning weather prediction” or MWLP models. In contrast with earlier deployments of AI technologies to weather – which primarily focused on either weather/climate downscaling, numerical model bias correction, or nowcasting using computer vision techniques applied to radar and satellite data, MLWP models integrate a full 3D state of the atmosphere forward in time. By initializing them with a current analysis state (and in some cases, a recently passed one), MLWP models can be autoregressively rolled out forward in time to produce weather forecasts. State-of-the art models such as GraphCast (Google DeepMind), Aurora (Microsoft), or CREDIT (NCAR/NSF) regularly achieve forecast skill comparable to leading operational NWP systems.

A significant limitation of MLWP modeling as explored up to today has been the reliance on large reanalysis datasets for model training and traditional data assimilation analyses for inference. As a consequence, operational MLWP still critically depends on some degree of NWP/DA. However, a new category of “data-driven” or “observation-driven” models are beginning to emerge which, at least for inference, sever this reliance on analysis states for initialization. 

In our talk, we will present two workstreams which aim to develop new, observation-driven forecast models. In the first half, we present “NNJA-AI”, an AI-ready observational dataset design to facilitate training these models. In the second half, we present “AIDA”, a novel “AI data assimilation” model which produces inference solely from observations, and can be used to initialize a large class of extant MLWP models. Brightband operates an operational data processing and assimilation system demonstrating this technology.