.png){kind=icon}
返回
Interactive Knowledge Distillation for image classification
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2021.04.026.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Knowledge distillation (KD) is a standard teacher-student learning framework to train a light-weight student network under the guidance of a well-trained, large teacher network. As an effective teaching strategy, interactive teaching has been widely employed at school to motivate students, in which teachers not only provide knowledge, but also give constructive feedback to students upon their responses, to improving their learning performance. In this work, we propose Interactive Knowledge Distillation (IAKD) to leverage the interactive teaching strategy for efficient knowledge distillation. In the distillation process, the interaction between the teacher network and the student one is implemented by swapping-in operations: randomly replacing the blocks in the student network with the corresponding blocks in the teacher network. In this way, we directly involve the teacher's powerful feature transformation ability for largely boosting the performance of the student network. Experiments with typical settings of teacher-student networks demonstrate that the student networks trained by our IAKD achieve better performance than those trained by conventional knowledge distillation methods on diverse image classification datasets. (c) 2021 Elsevier B.V. All rights reserved.
Keyword:
Interaction mechanism
Knowledge distillation
Model compression
Image classification
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
.png){kind=icon}
Colorimetric detection of pesticides based on calixarene modified silver nanoparticles in water
PREAI
分享收藏Oxidative stress in liver of grass carp Ctenopharyngodon idella naturally infected with Saprolegnia parasitica and its influence on disease pathogenesis天然感染水蚤的草鱼肝脏氧化应激及其对疾病发病机制的影响
PREAI
分享收藏Early Memories from Outpatients With and Without a History of Childhood Sexual Abuse有和没有儿童期性虐待史的门诊患者的早期记忆
PREAI
分享收藏
查看更多内容