.png){kind=icon}
返回
Accelerating Sparse DNNs Based on Tiled GEMM
.png){kind=icon}
OA
AI
DOI：10.1109/TC.2024.3365942.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Network pruning can reduce the computation cost of deep neural network (DNN) models. However, sparse models often produce randomly-distributed weights to maintain accuracy, leading to irregular computations. Consequently, unstructured sparse models cannot achieve meaningful speedup on commodity hardware built for dense matrix computations. Accelerators are usually modified or designed with structured sparsity-optimized architectures for exploiting sparsity. For example, the Ampere architecture introduces a sparse tensor core, which adopts the 2:4 sparsity pattern. We propose a pruning method that builds upon the insight that matrix multiplication generally breaks the large matrix into multiple smaller tiles for parallel execution. We present the tile-wise sparsity pattern, which maintains a structured sparsity pattern at the tile level for efficient execution but allows for irregular pruning at the global scale to maintain high accuracy. In addition, the tile-wise sparsity is implemented at the global memory level, and the 2:4 sparsity executes at the register level inside the sparse tensor core. We can combine these two patterns into a tile-vector-wise ( TVW ) sparsity pattern to explore more fine-grained sparsity and further accelerate the sparse DNN models. We evaluate the TVW on the GPU, achieving averages of 1.85 x , 2.75 x , and 22.18 x speedups over the dense model, block sparsity, and unstructured sparsity.
Keyword:
Computational modeling
Tensors
Graphics processing units
Sparse matrices
Computer architecture
Hardware
Costs
Pruning
sparse DNN
sparse tensor core
期刊
IF:
3.8
论文数:
5.3K
被引数:
9.8K
机构
引用论文
.png){kind=icon}
Quantitative Evaluation of the Impact of the Great East Japan Earthquake and Tsunami on the Rocky Intertidal Community对日本东北大地震和海啸对岩石潮间带群落影响的定量评估
PREAI
分享收藏Design and Development of the Graphology-based Career Analysis and Prediction System (G-CAPS) for Engineering Students基于笔迹学的职业分析与预测系统（G-CAPS）的设计与开发——面向工程学生
PREAI
分享收藏Research on the Efficiency of Wireless Power Transfer System Based on Multi-Auxiliary Transmitting Coils基于多辅助发射线圈的无线电能传输系统效率研究
PREAI
分享收藏
查看更多内容