.png){kind=icon}
Return
Interactive Knowledge Distillation for image classification
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2021.04.026.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Knowledge distillation (KD) is a standard teacher-student learning framework to train a light-weight student network under the guidance of a well-trained, large teacher network. As an effective teaching strategy, interactive teaching has been widely employed at school to motivate students, in which teachers not only provide knowledge, but also give constructive feedback to students upon their responses, to improving their learning performance. In this work, we propose Interactive Knowledge Distillation (IAKD) to leverage the interactive teaching strategy for efficient knowledge distillation. In the distillation process, the interaction between the teacher network and the student one is implemented by swapping-in operations: randomly replacing the blocks in the student network with the corresponding blocks in the teacher network. In this way, we directly involve the teacher's powerful feature transformation ability for largely boosting the performance of the student network. Experiments with typical settings of teacher-student networks demonstrate that the student networks trained by our IAKD achieve better performance than those trained by conventional knowledge distillation methods on diverse image classification datasets. (c) 2021 Elsevier B.V. All rights reserved.
Keywords:
Interaction mechanism
Knowledge distillation
Model compression
Image classification
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
Cited Papers
.png){kind=icon}
ShareSave
ShareSaveShareSaveShareSaveColorimetric detection of pesticides based on calixarene modified silver nanoparticles in water
PREAI
ShareSaveShareSave
View more