.png){kind=icon}
返回
A load-balanced acceleration method for small and irregular batch matrix multiplication on GPU
.png){kind=icon}
PRE
AI
DOI：10.1016/j.sysarc.2025.103341.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
As an essential mathematical operation, GEneral Matrix Multiplication (GEMM) plays a vital role in many applications, such as high-performance computing, machine learning, etc. In practice, the performance of GEMM is limited by the dimension of matrix and the diversity of GPU hardware architectures. When dealing with batched, irregular and small matrices, the efficiency of GEMM usually performs poorly. To this end, common approach is to segment the matrix into multiple tiles and utilize parallelism between workgroups in GPU to compute the results. However, previous works only consider tile size and inter-workgroup parallelism and ignore the issues of low computational efficiency and hardware resource utilization caused by the difference in workloads between wavefronts. To address these issues, we propose a load-balanced batch GEMM acceleration method, consisting of a multi-thread kernel design and an efficient tiling algorithm. The multithread kernel design can address the workload unbalance between wavefronts indifferent workgroups, and the efficient tiling algorithm can choose the optimal tiling scheme with the new thread-level parallelism calculation method to achieve load-balanced task allocation. Finally, various comparative experiments were conducted on two GPU platforms: AMD and NVIDIA. Experimental results indicate the proposed method outperforms previous methods.
Keyword:
Batch GEMM
Thread workload
Multi-thread kernel
Tiling algorithm
期刊
IF:
4.1
论文数:
3.0K
被引数:
4.2K
机构
引用论文
分享收藏Efficient and portable GEMM-based convolution operators for deep neural network training on multicore processors基于GEMM的高效可移植卷积算子，用于多核处理器上的深度神经网络训练
OAAI
OAAI分享收藏Detection of novelty, but not memory of spatial habituation, is associated with an increase in phosphorylated cAMP response element‐binding protein levels in the hippocampus.png)
HippocampusOAAI
HippocampusIF0
OAAI分享收藏分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏
查看更多内容