.png){kind=icon}
.png){kind=icon}
Return
Efficient multi-GPU implementations of preconditioned conjugate gradient method
.png){kind=icon}
PRE
AI
DOI：10.1016/j.advengsoft.2025.103936.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Many numerical methods rely on solving sparse linear systems, such as finite element analysis and computational fluid dynamics. In many simulations, solving sparse linear systems has become a bottleneck, often consuming a significant portion of the total simulation time. The Preconditioned Conjugate Gradient (PCG) method is an efficient iterative method for solving large sparse linear systems, and it is well-suited for GPU parallelization. As the scale of the problems to be solved increases, higher efficiency in PCG solvers is critical. This paper proposes two efficient implementations of the PCG method, aimed at further accelerating its computational performance on GPUs. First, we optimize one of the most time-consuming operation SpMV (Sparse Matrix-Vector Multiplication) in PCG using a block storage format for efficient global memory access. To this end, we introduce two novel SpMV algorithms: warp-fixed-mma and warpbalanced-mma. Next, we present two PCG implementations: PCG-cuda-graphs and PCG-persistent-kernel. Both implementations use the proposed SpMV algorithms. The experimental results on the 2904 matrices from the SuiteSparse Matrix Collection show that our best-performing SpMV algorithm in FP64 precision outperforms four latest SpMV methods cuSPARSE, DASP (Specific Dense Matrix Multiply-Accumulate Units Accelerated General Sparse Matrix-Vector Multiplication), CSR5 and TileSpMV by a factor of on average 2.2x, 3.1x, 2.48x and 3.20x on A100, respectively. Compared to the PCG implementation in PETSc (the Portable, Extensible Toolkit for Scientific Computation), the two proposed PCG implementations achieve average speedups of 1.41x and 1.63x on the NVIDIA A100 GPU, and 1.43x and 1.66x on the NVIDIA H100 GPU on 21 representative matrices. Additionally, for large matrices, our proposed multi-GPU parallel PCG-persistent-kernel implementation delivers impressive speedup compared to the single-GPU implementation.
Keywords:
PCG
GPU
Tensor core
SpMV
Multi-GPU
Journal
IF:
5.7
Papers:
3.3K
Citations:
1.2W