.png){kind=icon}
.png){kind=icon}
Return
Performance optimization and evaluation for parallel processing of big data in earth system models
.png){kind=icon}
PRE
AI
DOI：10.1007/s10586-017-1477-0.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Big data and high performance computing in Earth System Models (ESMs) are receiving increased attention in earth science research. When scaling to large-scale multi-core computing, efficient parallelization of an ESM, which demands fast parallel computing for long-term integration or climate simulation, becomes extremely challenging because of time-consuming internal big data communication. In this paper, an optimization algorithm for the massive data communication between the Weather Research and Forecasting model and Coupler version 7 in the Chinese Academy of Sciences-Earth System Model (CAS-ESM) is proposed. The optimization strategy is to transmit data from a small packet into a larger packet. Through experiments on a multi-core cluster, the efficiency of the algorithm is confirmed. Then, the parallel performance of the CAS-ESM is evaluated fully. Results show that the parallel efficiency of the CAS-ESM on 1024 CPU cores reaches nearly 70%, indicating that the CAS-ESM has desirable parallel performance and strong scalability. In addition, a generic performance evaluation method for ESMs from perspectives of optimal load balance and efficiency is proposed. Results show that the computing speed is the fastest and computational efficiency is the highest when the CAS-ESM runs on a certain number of cores.
Keywords:
Big data
High performance computing
Performance optimization
Earth system model
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
C
IF:
4.1
Papers:
5.0K
Citations:
7.5K
Organization
View more organizations