.png){kind=icon}
.png){kind=icon}
Return
Multi-view Feature Adjustment and Alignment for knowledge distillation
.png){kind=icon}
PRE
AI
R
Y
H
T
DOI：10.1016/j.displa.2026.103455.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Knowledge Distillation (KD) is a widely used model compression technique that typically relies on soft labels generated by a teacher model as supervision signals to guide the training of a student model. However, significant structural gaps between teachers and students can cause semantic misalignment and uncertainty, thereby weakening the effectiveness of supervision. To address this, we propose Multi-view Feature Adjustment and Alignment (MFAA), a hierarchical feature-based distillation framework. MFAA constructs a multi-view distillation pathway to generate fine-grained supervision, encompassing input interrelation modeling, feature interaction, and output decision, thereby significantly improving the student model's generalization and performance. We evaluated MFAA across classification, transfer, few-shot, and robustness tasks on six benchmarks: CIFAR-100, CIFAR-100-C, STL-10, SVHN, Tiny ImageNet, and ImageNet. Objective metrics included accuracy, mean Corruption Error (mCE), KL divergence, and Centered Kernel Alignment (CKA); subjective metrics were assessed using Class Activation Maps (CAM) and t-SNE visualizations to evaluate the consistency and interpretability of the learned representations. Experimental results demonstrate that MFAA consistently outperforms existing methods, achieving a 5.86% average gain in robustness evaluations. It also enhances feature consistency and semantic alignment, confirming its strengths in representation preservation and structural alignment. Our source code is available at https://github.com/lambett/MFAA.
Keywords:
Knowledge distillation
Feature representation
Image classification
Journal
IF:
3.4
Papers:
2.1K
Citations:
3.2K
Organization
Cited Papers