Improving the scalability of the ocean barotropic solver in the Community Earth System Model

Hu, Y., Huang, X., Baker, A. H., Tseng, Y., Bryan, F. O., et al. (2016). Improving the scalability of the ocean barotropic solver in the Community Earth System Model. SC15: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, doi:10.1145/2807591.2807596

Title Improving the scalability of the ocean barotropic solver in the Community Earth System Model
Author(s) Yong Hu, Xiaomeng Huang, Allison H. Baker, Yuheng Tseng, Frank O. Bryan, John M. Dennis, Gwangwen Yang
Abstract High-resolution climate simulations are increasingly in demand and require tremendous computing resources. In the Community Earth SystemModel (CESM), the Parallel Ocean Model (POP) is computationally expensive for high-resolution grids (e.g., 0.1°) and is frequently the least scalable component of CESM for certain production simulations. In particular, the modified Preconditioned Conjugate Gradient (PCG), used to solve the elliptic system of equations in the barotropic mode, scales poorly at the high core counts, which is problematic for high-resolution simulations. In this work, we demonstrate that the communication costs in the barotropic solver occupy an increasing portion of the total POP execution time as core counts are increased. To mitigate this problem, we implement a preconditioned Chebyshev-type iterative method in POP (called P-CSI), which requires far fewer global reductions than PCG. We also develop an effective block preconditioner based on the Error Vector Propagation Method to attain a competitive convergence rate for P-CSI. We demonstrate that the improved scalability of P-CSI results in a 5.2x speedup of the barotropic mode in high-resolution POP on 16,875 cores, which yields a 1.7x speedup of the overall POP simulation. Further, we ensure that the new solver produces an ocean climate consistent with the original one via an ensemble-based statistical method.
Publication Title SC15: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
Publication Date Jan 1, 2016
Publisher's Version of Record https://dx.doi.org/10.1145/2807591.2807596
OpenSky Citable URL https://n2t.net/ark:/85065/d7fb54np
OpenSky Listing View on OpenSky
CISL Affiliations TDD, ASAP

< Back to our listing of publications.