.png){kind=icon}
返回
Quantization Robust Pruning With Knowledge Distillation
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3257864.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
To resolve the problem that deep neural networks (DNN) require a large number of network parameters, many researchers have sought to compress the network. Network pruning, quantization and knowledge distillation have been studied for this purpose. Considering realistic scenarios such as deploying DNN on the resource constraint device where the network uploaded in the device performs wells in various bit-widths without re-training and the network with reasonable performance, we propose quantization robust pruning with knowledge distillation (QRPK) method. In QRPK, model weights are divided into essential weigths and inessential weights based on their magnitude value. Then, QRPK trains the quantization robustness model with a high pruning ratio by making the distribution of essential weights as a quantization friendly distribution. We conducted experiments on CIFAR-10 and CIFAR-100 to verify the effectiveness of QRPK and a QRPK trained model performs well in various bit-width, as it designed by pruning, quantization robustness and knowledge distillation.
Keyword:
Quantization (signal)
Computational modeling
Convolutional neural networks
Knowledge engineering
Robustness
Performance evaluation
Neural networks
network quantization
knowledge distillation
network pruning
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
.png){kind=icon}
分享收藏 没有更多内容
没有更多内容