SIParCS 2024 - Suman Shekhar

A first prototype for in-memory data transfer between Earth System models and data assimilation

Recorded Talk

Earth system models such as NSF NCAR’s Community Earth System Model (CESM) are essential tools for simulating and predicting environmental phenomena, with applications including weather forecasting, flood anticipation, drought assessment, and climate change projections. These models integrate many interlinked physical processes, often represented by independently developed components coupled through advanced software infrastructure. Despite their sophistication, these models often exhibit biases due to missing physics and coarse model grid representations of the domain. Data assimilation (DA) algorithms, which integrate observational data with model predictions, are critical for mitigating these biases and steering models towards more accurate states.

Currently, DA systems like NSF NCAR’s Data Assimilation Research Testbed (DART) operate by writing restart files to disk, generating up to terabytes of data at each 6-hour timestep. We hypothesize that this I/O-intensive approach leads to significant performance bottlenecks. To address this limitation, we developed a first prototype for in-memory field transfer between DART system and the CESM. By leveraging the NUOPC (National Unified Operational Prediction Capability) software layer, DART can be integrated as a CESM model component, enabling direct, in-memory exchange of fields. This approach eliminates the I/O bottleneck, enhancing performance and scalability, and represents a significant advancement in Earth system modeling and data assimilation performance. 

Mentors: Helen Kershaw, Dan Amrhein, Ufuk Turuncoglu

Slides and poster