.png){kind=icon}
.png){kind=icon}
Return
Conditional generative data-free knowledge distillation
.png){kind=icon}
OA
AI
DOI：10.1016/j.imavis.2023.104627.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Knowledge distillation has made remarkable achievements in model compression. However, most existing methods require the original training data, which is usually unavailable due to privacy and security issues. This paper proposes a conditional generative data-free knowledge distillation (CGDD) framework for training light-weight networks without real data. This framework realizes efficient knowledge distillation based on conditional image generation. Specifically, we treat the preset labels as ground truth to train a semi-supervised conditional generator. The trained generator can produce specified classes of training images. During training, we force the student model to extract the hidden knowledge in teacher feature maps, which provide crucial cues to the learn-ing process. Meanwhile, we construct an adversarial training framework to promote distillation performance. The framework will help the student model to explore larger data space. To demonstrate the effectiveness of the proposed method, we conduct extensive experiments on different datasets. Compared with other data-free works, our method obtains state-of-the-art results on CIFAR100, Caltech101, and different versions of ImageNet datasets. The codes will be released.(c) 2023 Elsevier B.V. All rights reserved.
Keywords:
Data -free knowledge distillation
Generative adversarial networks
Model compression
Convolutional neural networks
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
4.2
Papers:
4.0K
Citations:
6.7K