.png){kind=icon}
返回
Knowledge distillation via instance-level sequence learning
.png){kind=icon}
OA
AI
DOI：10.1016/j.knosys.2021.107519.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, distillation approaches for extracting general knowledge from a teacher network to guide a student network have been suggested. Most existing methods transfer knowledge from the teacher to the student network by feeding a sequence of random minibatches sampled uniformly from the data. We argue that, instead, a compact student network should be guided gradually using samples ordered in a meaningful sequence. Thus, the gap in feature representation between the teacher and student network can be bridged step by step. In this paper, we provide a curriculum learning knowledge distillation framework via instance-level sequence learning. It employs the student network of the early epoch as a snapshot to create a curriculum for the student network's next training phase. We performed extensive experiments using the CIFAR-10, CIFAR-100, SVHN, and CINIC-10 datasets. When compared with several state-of-the-art methods, our framework achieved the best performance with fewer iterations. (C) 2021 Elsevier B.V. All rights reserved.
Keyword:
Neural networks compression
Knowledge distillation
Computer vision
Deep learning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
K
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
.png){kind=icon}
GEV-NN: A deep neural network architecture for class imbalance problem in binary classification
PREAI
分享收藏分享收藏Computations of Standard Binding Free Energies with Molecular Dynamics Simulations用分子动力学模拟计算标准结合自由能OAAI
OAAI分享收藏
查看更多内容