Development of NCL equivalent serial and parallel python routines for meteorological data analysis
Gharat, J., Kumar, B., Ragha, L., Barve, A., Jeelani, S. M., et al. (2022). Development of NCL equivalent serial and parallel python routines for meteorological data analysis. The International Journal of High Performance Computing Applications, doi:10.1177/10943420221077110
| Title | Development of NCL equivalent serial and parallel python routines for meteorological data analysis |
|---|---|
| Author(s) | Jatin Gharat, Bipin Kumar, Leena Ragha, Amit Barve, Shaik Mohammad Jeelani, John Clyne |
| Abstract | The NCAR Command Language (NCL) is a popular scripting language used in the geoscience community for weather data analysis and visualization. Hundreds of years of data are analyzed daily using NCL to make accurate weather predictions. However, due to its sequential nature of execution, it cannot properly utilize the parallel processing power provided by High-Performance Computing systems (HPCs). Until now very few techniques have been developed to make use of the multi-core functionality of modern HPC systems on these functions. In the recent trend, open-source languages are becoming highly popular because they support major functionalities required for data analysis and parallel computing. Hence, developers of NCL have decided to adopt Python as the future scripting language for analysis and visualization and to enable the geosciences community to play an active role in its development and support. This study focuses on developing some of the widely used NCL routines in Python. To deal with the analysis of large datasets, parallel versions of these routines are developed to work within a single node and make use of multi-core CPUs to achieve parallelism. Results show high accuracy between NCL and Python outputs and the parallel versions provided good scaling compared to their sequential counterparts. |
| Publication Title | The International Journal of High Performance Computing Applications |
| Publication Date | May 1, 2022 |
| Publisher's Version of Record | https://dx.doi.org/10.1177/10943420221077110 |
| OpenSky Citable URL | https://n2t.net/ark:/85065/d7988bq8 |
| OpenSky Listing | View on OpenSky |
| CISL Affiliations | TDD, VAST |