Toward a multi-method approach: Lossy data compression for climate simulation data
Baker, A., Xu, H., Hammerling, D., Li, S., Clyne, J. P., et al. (2017). Toward a multi-method approach: Lossy data compression for climate simulation data. High Performance Computing. ISC High Performance 2017, doi:https://doi.org/10.1007/978-3-319-67630-2_3
| Title | Toward a multi-method approach: Lossy data compression for climate simulation data |
|---|---|
| Genre | Chapter |
| Author(s) | Allison Baker, Haiying Xu, Dorit Hammerling, Samuel Li, John P. Clyne, J. Kunkel, R. Yokota, M. Taufer, J. Shalf |
| Abstract | Earth System Model (ESM) simulations are increasingly constrained by the amount of data that they generate rather than by computational resources. The use of lossy data compression on model output can reduce storage costs and data transmission overheads, but care must be taken to ensure that science results are not impacted. Choosing appropriate compression algorithms and parameters is not trivial given the diversity of data produced by ESMs and requires an understanding of both the attributes of the data and the properties of the chosen compression methods. Here we discuss the properties of two distinct approaches for lossy compression in the context of a well-known ESM, demonstrating the different strengths of each, to motivate the development of an automated multi-method approach for compression of climate model output. |
| Publication Title | High Performance Computing. ISC High Performance 2017 |
| Publication Date | Jan 1, 2017 |
| Publisher's Version of Record | https://doi.org/10.1007/978-3-319-67630-2_3 |
| OpenSky Citable URL | https://n2t.org/ark:/85065/d7vx0k24 |
| OpenSky Listing | View on OpenSky |
| CISL Affiliations | TDD, ASAP, IOWAG, AIML, VAST |