.png){kind=icon}
返回
STuning-DL: Model-Driven Autotuning of Sparse GPU Kernels for Deep Learning
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3402326.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The relentless growth of modern Machine Learning models has spurred the adoption of sparsification techniques to simplify their architectures and reduce the computational demands. Network pruning has demonstrated success in maintaining original network accuracy while shedding significant portions of the original weights. However, leveraging this sparsity efficiently remains challenging due to computational irregularities, particularly in GPU kernels. A new trend of template-based GPU kernels for semi-structured sparsity shows promise in efficiency but lacks autotuning capabilities to adapt to input dynamics, often underperforming in scenarios where they have not been meticulously hand-tuned. We present STuning-DL, the first pruning-aware autotuner for third-party template-based implementations enabling efficient optimization of sparse kernels for Deep Learning, spanning from high-level aspects (CUDA C++ level) down to GPU-native instructions specifics (assembly-level). STuning-DL tunes and optimizes at run-time sparse kernels' performance for each input problem, yielding speedups of up to 5.42 x on NVIDIA T4-16GB and up to 3.6 x on NVIDIA A100-40GB GPU in sparse matrices from real world models compared to existing heuristics from sparse libraries like cuSparse and cuSparseLt.
Keyword:
CUDA
GPU
learning-based predictive model
network pruning
sparse computation
SpMM
Tensor Core
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Transcription Impacts the Efficiency of mRNA Translation via Co-transcriptional N6-adenosine Methylation.png)
Cell
OAAI
CellIF0
OAAI分享收藏Research on the Efficiency of Wireless Power Transfer System Based on Multi-Auxiliary Transmitting Coils基于多辅助发射线圈的无线电能传输系统效率研究
PREAI
分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏分享收藏
查看更多内容