.png){kind=icon}
Return
Explainability-based knowledge distillation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.111095.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Knowledge distillation (KD) is a popular approach for deep model acceleration. Based on the knowledge distilled, we categorize KD methods as label-related and structure-related. The former distills the very abstract (high-level) knowledge, e.g., logits; and the latter uses the spatial (low- or medium-level feature) knowledge. However, existing KD methods are usually not explainable, i.e., we do not know what knowledge is transferred during distillation. In this work, we propose a new KD method, Explainability-based Knowledge Distillation (Exp-KD). Specifically, we propose to use class activation map (CAM) as the explainable knowledge which can effectively capture both label- and structure-related information during the distillation. We conduct extensive experiments, including image classification tasks on CIFAR-10, CIFAR-100 and ImageNet datasets, and explainability tests on ImageNet and ImageNet-Segmentation. The results show the great effectiveness and explainability of Exp-KD compared with the state-of-the-art. Code is available at https://github.com/ Blenderama/Exp-KD.
Keywords:
Explainability
Knowledge distillation
CAM
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
Cited Papers
ShareSaveShareSaveShareSaveColorimetric detection of pesticides based on calixarene modified silver nanoparticles in water
PREAI
ShareSaveDeep metric learning for few-shot image classification: A Review of recent developments.png)
PATTERN RECOGNITION
OAAI
PATTERN RECOGNITIONIF7.6
OAAIShareSave
ShareSave
View more