Towards accurate post-training quantization for reparameterized models

Luoming Zhang

doi:10.1007/s10489-025-06418-0

Towards accurate post-training quantization for reparameterized models

2025-05-01

0

PRE

AI

DOI：10.1007/s10489-025-06418-0

原文链接

原文求助

摘要

En 中文

Model reparameterization is a widely accepted technique for improving inference speed without compromising performance. However, current Post-training Quantization (PTQ) methods often lead to significant accuracy degradation when applied to reparameterized models. This is primarily caused by channel-specific and sample-specific outliers, which appear only at specific samples and channels and impact on the selection of quantization parameters. To address this issue, we propose RepAPQ, a novel framework that preserves the accuracy of quantized reparameterization models. Different from previous frameworks using Mean Squared Error (MSE) as a measurement, we utilize Mean Absolute Error (MAE) to mitigate the influence of outliers on quantization parameters. Our framework consists of two core components: Quantization Protecting Reparameterization and Across-block Calibration. For effective calibration, Quantization Protecting Reparameterization combines multiple branches into a single convolution with an affine layer. During training, the affine layer accelerates convergence and amplifies the output of the convolution to better accommodate samples with outliers. Additionally, Across-block Calibration leverages the measurement of stage output as supervision to address the gradient problem introduced by MAE and enhance the interlayer correlation with quantization parameters. Comprehensive experiments demonstrate the effectiveness of RepAPQ across various models and tasks. Our framework outperforms previous methods by approximately 1% for 8-bit PTQ and 2% for 6-bit PTQ, showcasing its superior performance. The code is available at https://github.com/ilur98/DLMC-QUANT.

Keyword:

Model compression

Model quantization

Model reparameterization

Efficient deep learning

Towards accurate post-training quantization for reparameterized models

摘要

期刊

机构

引用论文