A new parallel python tool for the standardization of earth system model data
Paul, K., Mickelson, S., Dennis, J. M.. (2017). A new parallel python tool for the standardization of earth system model data. IEEE International Conference on Big Data 2016, doi:10.1109/BigData.2016.7840946
| Title | A new parallel python tool for the standardization of earth system model data |
|---|---|
| Author(s) | Kevin Paul, Sheri Mickelson, John M. Dennis |
| Abstract | We have developed a new parallel Python tool for the standardization of Earth System Model (ESM) data for publication as part of Model Intercomparison Projects (MIPs). It was specifically designed to aid Community Earth System Model (CESM) scientists at the National Center for Atmospheric Research (NCAR) in preparation for the Coupled Model Intercomparison Project, Phase 6 (CMIP6), expected to start in early 2017. However, the tool is general to any and all MIPs and ESMs. The tool is implemented with MPI parallelism using mpi4py, and it performs the data standardization computation with a directed acyclic graph (DAG) data structure capable of streaming data from ESM input data to standardized output files. In this paper, we describe the tool, its design and testing. |
| Publication Title | IEEE International Conference on Big Data 2016 |
| Publication Date | Feb 6, 2017 |
| Publisher's Version of Record | https://dx.doi.org/10.1109/BigData.2016.7840946 |
| OpenSky Citable URL | https://n2t.net/ark:/85065/d7bg2rj3 |
| OpenSky Listing | View on OpenSky |
| CISL Affiliations | TDD, ASAP, IOWAG |