arrow
返回

Attention Round for post-training quantization

delete2024-01-01
delete6
delete
OA
AI
S
Shaoyun Xu
C
Chao Kong
W
Wei Wang
DOI:10.1016/j.neucom.2023.127012delete
delete原文链接
delete分享
delete收藏
查看原文
摘要

摘要

En 中文
Quantization methods for convolutional neural network models can be broadly categorized into post-training quantization (PTQ) and quantization aware training (QAT). While PTQ offers the advantage of requiring only a small portion of the data for quantization, the resulting quantized model may not be as effective as QAT. To address this limitation, this paper proposes a novel quantization function named Attention Round. Unlike traditional quantization function that map 32 bit floating-point value w to nearby quantization levels, Attention Round allows w to be mapped to all possible quantization levels in the entire quantization space, expanding the quantization optimization space. The possibilities of mapping w to different quantization levels are inversely correlated with the distance between w and the quantization levels, regulated by a Gaussian decay function. Furthermore, to tackle the challenge of mixed precision quantization, this paper introduces a lossy coding length measure to assign quantization precision to different layers of the model, eliminating the need for solving a combinatorial optimization problem. Experimental evaluations on various models demonstrate the effectiveness of the proposed method. Notably, for ResNet18 and MobileNetV2, the PTQ approach achieves comparable quantization performance to QAT while utilizing only 1024 training data and 10 min for the quantization process.
Keyword:
Convolutional neural networks
Post-training quantization
Attention Round
Mixed precision
AI总结

AI总结

对已上传原文的论文进行重点信息的提取,主要内容包括:简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。

期刊

Neurocomputing 封面图
Neurocomputing
IF:
6.5
论文数:
2.5W
被引数:
8.4W

机构

安徽工程大学 封面图
安徽工程大学
学者数:
569
论文数: 174
被引数: 2.0K
中
中国科学院
学者数:
3.0W
论文数: 8.3K
被引数: 39.2W
引用论文

引用论文

Thermal Plasmas
err
IF0
err1994-01-01
err0
PREAI
errMaher I. Boulos; Pierre Fauchais; Emil Pfender
err分享
err收藏
Loss aware post-training quantization损失感知训练后量化
err2021-10-01
err54
errOAAI
errNahshan, Yury; Chmiel, Brian; Baskin, Chaim; Zheltonozhskii, Evgenii; Banner, Ron; Bronstein, Alex M.; Mendelson, Avi
err分享
err收藏
Thermally induced formation of metastable nanocomposites in amorphous Cr-Zr-O thin films deposited using reactive ion beam sputtering
err2016-08-01
err0
PREAI
errDavid Rafaja; Christina Wüstefeld; Gintautas Abrasonis; Stefan Braeunig; Carsten Baehtz; Florian Hanzig; Milan Dopita; Matthias Krause; Sibylle Gemming
err分享
err收藏
学者 查看更多内容