.png){kind=icon}
.png){kind=icon}
Return
MIX-PC: Enabling Efficient DNN Inference with Mixed Numeric Precision Compilation Optimization
.png){kind=icon}
PRE
AI
DOI：10.1145/3785473.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Mixed-precision quantization has shown great potential in reducing memory requirements and improving inference efficiency and has received extensive attention. However, narrowing bit lengths does not always translate into significant efficiency gains for various reasons. Firstly, computation kernels often lack support for mixed-precision operations, limiting their utility in systems that rely on such kernels. Secondly, as DNN computation involves multiple stages, the precision pattern can vary significantly across these stages. This complicates the development of precision-aware optimizations for achieving end-to-end efficiency gains. Ultimately, effective precision-aware optimization necessitates additional support throughout the entire vertical stack. This article proposes Mix-PC, a framework that enables efficient mixed-precision compilation for highperformance DNN inference. On one hand, we delve into the cause of inefficiencies in the current compilation for supporting mixed-precision operators. On the other hand, we propose Mix-PC, a general-purpose compilation framework that leverages adaptive patterns for different numeric precision. It leverages adaptive patterns tailored for different numeric precision and progressive fusion strategies, enabling enhanced compatibility with diverse models. The results show that compared to the SOTA quantization efforts, Mix-PC achieves 1.70 & times;, 1.34 & times;, and 2.13 & times; speedup, respectively, in BERT, ViT, and HuBERT.
Keywords:
AI compiler
model compression
mixed-precision quantization
Journal
A
IF:
1.8
Papers:
96
Citations:
1.1K