Bsqat: block-wise shared quantization-aware training for large language models

Senbao Hou

doi:10.1007/s40747-026-02394-1

Bsqat: block-wise shared quantization-aware training for large language models

2026-07-17

0

OA

AI

DOI：10.1007/s40747-026-02394-1

原文链接

原文求助

摘要

En 中文

块级共享量化感知训练（BSQAT）作为一种关键技术，通过结合局部和全局优化实现高效且准确的模型量化。在局部QAT（LQAT）阶段，BSQAT采用重叠滑动窗口策略合并相邻Transformer块，并分配共享量化参数以捕捉块间关联，同时设计轻量级低秩缩放模块以保持权重精细度。全局QAT（GQAT）阶段则通过最小开销微调全局缩放参数以增强模型鲁棒性。实验表明，BSQAT在LLaMa、OPT、Vicuna和Mistral等模型架构上表现良好，参数规模从6.7B扩展至123B均有效。在LLaMa-3-70B模型的INT2量化中，BSQAT将平均零样本准确率从65.18%提升至67.87%，同时内存使用减少约7%。

Keyword:

Large language models (LLMs)

Quantization-aware training

Low-bit quantization

Model compression

Bsqat: block-wise shared quantization-aware training for large language models

摘要

期刊

机构

引用论文