.png){kind=icon}
返回
D3K: Dynastic Data-Free Knowledge Distillation
.png){kind=icon}
PRE
AI
DOI：10.1109/TMM.2023.3236212.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Data-free knowledge distillation further broadens the applications of the distillation model. Nevertheless, the problem of providing diverse data with rich expression patterns needs to be further explored. In this paper, a novel dynastic data-free knowledge distillation ((DK)-K-3) model is proposed to alleviate this problem. In this model, a dynastic supernet generator (D-SG) with a flexible network structure is proposed to generate diverse data. The D-SG can adaptively alter architectural configurations and activate different subnet generators in different sequential iteration spaces. The variable network structure increases the complexity and capacity of the generator, and strengthens its ability to generate diversified data. In addition, a novel additive constraint based on the differentiable dhash (D-Dhash) is designed to guide the structure parameter selection of the D-SG. This constraint forces the D-SG to constantly jump out of the fixed generation mode and generate diverse data in semantics and instance. The effectiveness of the proposed model is verified on the experimental benchmark datasets (MNIST, CIFAR-10, CIFAR-100, and SVHN).
Keyword:
Generators
Data models
Knowledge engineering
Training data
Neural networks
Training
Task analysis
Dhash
dynastic network
image classification
generated data diversity
knowledge distillation
期刊
IF:
9.7
论文数:
4.5K
被引数:
2.4W
机构
引用论文
.png){kind=icon}
A Comprehensive Review on Face Recognition Methods and Factors Affecting Facial Recognition Accuracy
PREAI
分享收藏
查看更多内容