.png){kind=icon}
返回
Knowledge Distillation: A Survey
.png){kind=icon}
PRE
AI
DOI：10.1007/s11263-021-01453-z.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters. However, it is a challenge to deploy these cumbersome deep models on devices with limited resources, e.g., mobile phones and embedded devices, not only because of the high computational complexity but also the large storage requirements. To this end, a variety of model compression and acceleration techniques have been developed. As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. It has received rapid increasing attention from the community. This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, teacher-student architecture, distillation algorithms, performance comparison and applications. Furthermore, challenges in knowledge distillation are briefly reviewed and comments on future research are discussed and forwarded.
Keyword:
Deep neural networks
Model compression
Knowledge distillation
Knowledge transfer
Teacher– student architecture
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
9.3
论文数:
3.9K
被引数:
2.8W
机构
引用论文
Tumour‐associated changes in intestinal epithelial cells cause local accumulation of KLRG1+GATA3+ regulatory T cells in mice.png)
Immunology
OAAI
ImmunologyIF0
OAAI分享收藏
查看更多内容