.png){kind=icon}
Return
Making Knowledge Distillation Open Again
.png){kind=icon}
PRE
AI
DOI：10.1109/tip.2026.3721359.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Knowledge distillation (KD) has become a pivotal technique for transferring knowledge from large-scale teacher models to lightweight student models. However, traditional feature-based distillation methods necessitate the direct exposure of the teacher’s intermediate representations, raising concerns regarding data privacy and the leakage of proprietary model details. These concerns often hinder the deployment of distillation in collaborative or cloud-based scenarios. To address these challenges, we propose a privacy-aware teacher-oriented projected feature distillation framework that reduces direct feature exposure while preserving distillation effectiveness. Specifically, our method employs a low-rank projection strategy to obfuscate the teacher’s features into a compact subspace. Authorized student models leverage this projection matrix to align their own features, enabling effective knowledge transfer while reducing direct exposure of the teacher’s original spatial patterns. Extensive experiments on benchmark datasets demonstrate empirical feature obfuscation and reconstruction resistance while maintaining competitive performance in object detection and semantic segmentation, with gains in several settings even when learning from projected representations.
Keywords:
Knowledge distillation
object detection
feature obfuscation
feature projection
Journal
IF:
13.7
Papers:
1.0W
Citations:
8.4W
Organization
Cited Papers
ShareSaveShareSaveSelective knowledge sharing for privacy-preserving federated distillation without a good teacher.png)
NATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAIShareSaveShareSaveShareSave
View more