.png){kind=icon}
.png){kind=icon}
Return
ISQ: Intermediate-Value Slip Quantization for Accumulator-Aware Training
.png){kind=icon}
PRE
AI
DOI：10.1109/LSP.2025.3539579.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The development of lightweight technologies has made deploying convolutional neural networks on edge devices popular. However, the overflow caused by low-bit accumulators significantly degrades the accuracy of the model. Therefore, there is a challenge in balancing accuracy and low-bit accumulators. In this letter, we propose a novel method applying for training low-bit quantized neural network named Intermediate-Value Slip Quantization (ISQ). ISQ is used to constrain weights to decrease the risk of accumulator's overflow. Besides, we also set a criterion for ISQ to be aware of the bit width of the accumulator. In addition, we propose a method to integrate bias and Batch Normalization (BN) into the ISQ. This allows the computations to be shifted from the floating-point domain to the fixed-point. The experiment results demonstrate that ISQ effectively suppresses the overflow. The model accuracy under the 16-bit accumulator can be restored to 73.16% from 13.7% on CIFAR-100 with the same quantization configuration. Through our method, the design space of the hardware can be explored and the target accuracy can be achieved with the lowest hardware resources.
Keywords:
Quantization (signal)
Training
Accuracy
Hardware
Neural networks
Optimization
Backpropagation
Field programmable gate arrays
Data mining
Convolution
Convolutional neural network
quantization
quantization-aware training
overflow
accumulator
Journal
IF:
9.6
Papers:
1.1W
Citations:
1.7W