SEMQ: Efficient non-uniform quantization with sensitivity-based error minimization for large language models

doi:10.1016/j.future.2025.108120

SEMQ: Efficient non-uniform quantization with sensitivity-based error minimization for large language models

2025-09-05

0

PRE

AI

Athanasios V. Vasilakos

M

Man-Fai Leung

DOI：10.1016/j.future.2025.108120

Original

Original request for help

Save

Abstract

En 中文

• A novel non-uniform quantization algorithm designed for Large Language Models (LLMs), featuring an iterative optimization strategy for quantization point selection that effectively minimizes quantization errors. • We approximate LLM weight distributions to bell-shaped curves, coupled with a robust outlier detection and separation mechanism, thereby enhancing quantization resilience. • Conducting a comprehensive empirical evaluation on a series of benchmark datasets, demonstrating the algorithm’s efficacy in achieving 2-bit quantization while preserving competitive model performance.

SEMQ: Efficient non-uniform quantization with sensitivity-based error minimization for large language models

Abstract

Journal

Organization