VLCQ: Post-training quantization for deep neural networks using variable length coding

doi:10.1016/j.future.2024.107654

2025-05-01

PRE

Reem Abdel‐Salam

Ahmed H. Abdel-Gawad *

Amr G. Wassal

DOI：10.1016/j.future.2024.107654

原文链接

原文求助

摘要

En 中文

Quantization plays a crucial role inefficiently deploying deep learning models on resources constraint devices. Post-training quantization does not require either access to the original dataset or retraining the full model. Current methods that achieve high performance (near baseline results) require INT8 fixed-point integers. However, to achieve high model compression by achieving lower bit-width, significant degradation to the performance becomes the challenge. In this paper, we propose VLCQ, which relaxes the constraint of fixedpoint encoding which limits the quantization techniques from better quantizing the weights. Therefore, this work utilizes variable-length encoding which allows for exploring the whole space of quantization techniques. Thus, achieving much better results (close to or even better than the baseline results) while achieving lower bit-widths without the need to access any training data or to fine-tune the model. Extensive experiments were carried out on various deep-learning models for the image classification and segmentation, and object detection tasks. When compared to state-of-the-art post-training quantization approaches, experimental results reveal that our suggested method offers improved performance with better model compression (lower bit-rate). For per- channel quantization, our method surpassed the FP32 accuracy and Piece-Wise Linear Quantization (PWLQ) method inmost models while achieving up-to 6X model compression ratio compared to the FP32 and up-to 1.7X compared to PWLQ. If the model compression is the concern with little effect on performance, our method achieves up-to 12.25X compression ratio compared to FP32 within 4% performance loss. For per-tensor, our method is competitive with Data-Free Quantization scheme (DFQ) in achieving the best performance. However, our method is more flexible in getting lower bit rates than DFQ across the different tasks and models.

Keyword:

Post-training quantization

Deep neural networks

Non-uniform quantization

Uniform quantization

Variable length encoding

Data-free quantization

Model compression