.png){kind=icon}
返回
Long-range zero-shot generative deep network quantization
.png){kind=icon}
OA
AI
DOI：10.1016/j.neunet.2023.07.042.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Quantization approximates a deep network model with floating-point numbers by the model with low bit width numbers, thereby accelerating inference and reducing computation. Zero-shot quantization, which aims to quantize a model without access to the original data, can be achieved by fitting the real data distribution through data synthesis. However, it has been observed that zero-shot quantization leads to inferior performance compared to post-training quantization with real data for two primary reasons: 1) a normal generator has difficulty obtaining a high diversity of synthetic data since it lacks long-range information to allocate attention to global features, and 2) synthetic images aim to simulate the statistics of real data, which leads to weak intra-class heterogeneity and limited feature richness. To overcome these problems, we propose a novel deep network quantizer called long-range zero-shot generative deep network quantization (LRQ). Technically, we propose a long-range generator (LRG) to learn long-range information instead of simple local features. To incorporate more global features into the synthetic data, we use long-range attention with large-kernel convolution in the generator. In addition, we also present an adversarial margin add (AMA) module to force intra-class angular enlargement between the feature vector and class center. The AMA module forms an adversarial process that increases the convergence difficulty of the loss function, which is opposite to the training objective of the original loss function. Furthermore, to transfer knowledge from the full-precision network, we also utilize decoupled knowledge distillation. Extensive experiments demonstrate that LRQ obtains better performance than other competitors.(c) 2023 Elsevier Ltd. All rights reserved.
Keyword:
Deep network quantization
Long-range generator
Adversarial margin add
Synthetic data generation
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.3
论文数:
8.3K
被引数:
3.0W
机构
引用论文
Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
PREAI
分享收藏Low Dose Vaccination with Attenuated Francisella tularensis Strain SchuS4 Mutants Protects against Tularemia Independent of the Route of Vaccination用减毒的土拉弗朗西斯菌菌株SchuS4突变体进行低剂量疫苗接种可预防土拉热病，而与疫苗接种途径无关.png)
PLoS ONEOAAI
PLoS ONEIF0
OAAI分享收藏
分享收藏分享收藏The power of obfuscation techniques in malicious JavaScript code: A measurement study恶意JavaScript代码中混淆技术的功能: 一项测量研究OAAI
OAAI分享收藏
查看更多内容