.png){kind=icon}
返回
Collaborative knowledge distillation via filter knowledge transfer
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2023.121884.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Knowledge distillation is a promising model compression technique that generally distills the knowledge from a complex teacher model to a lightweight student model. However, the performance gain of a student model is usually limited by the capacity gap between the large teacher model and the small student model. In this paper, we propose a new collaborative knowledge distillation method that makes use of a new strategy, named Filter Knowledge Transfer (FKT) to detect and learn the valuable filter information following from the teacher to the student. To be specific, the useful knowledge of filters measured by using information entropy is transferred between different peer networks and unimportant filters are reactivated according to the ratio based on the characterized filter information entropy during the online distillation process. Experimental results derived on four popular datasets, CIFAR-10/100, Market-1501, and Tiny-ImageNet, demonstrate the superiority of our proposed method over the others we considered.
Keyword:
Model compression
Knowledge distillation
Filter pruning
Filter knowledge transfer
期刊
IF:
7.5
论文数:
3.0W
被引数:
10.2W
机构
查看更多机构
引用论文
.png){kind=icon}
没有更多内容