Token-based dynamic bit-width assignment for ViT quantization

Dohyung Kim

doi:10.1016/j.patcog.2025.112269

Return

Token-based dynamic bit-width assignment for ViT quantization

2025-08-16

PRE

DOI：10.1016/j.patcog.2025.112269

Original

Original request for help

Save

Abstract

En 中文

• We introduce a novel token-based dynamic bit-allocation method for ViT quantization that assigns bit-widths of individual tokens dynamically, including the class token, for each input instance. • We present a shifting quantizer, specially designed for post-GELU activations, that considers the asymmetric distribution across positive and negative values. • We demonstrate that our approach boosts the performance of the token-wise quantization scheme significantly, and show that it achieves a good compromise between efficiency and efficacy.

Keywords:

token-based dynamic bit-allocation

ViT quantization

shifting quantizer

post-GELU activations

efficiency and efficacy