.png){kind=icon}
.png){kind=icon}
Return
Enhancing knowledge distillation for semantic segmentation through text-assisted modular plugins
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.111329.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Compared with other model compression methods, such as pruning and quantization, knowledge distillation offers superior compatibility and flexibility. Current knowledge distillation (KD) methods for semantic segmentation predominantly guide the student model to replicate the structured information of the teacher model solely through image data. However, these approaches often overlook the potential benefits of incorporating auxiliary modalities, such as textual information, into the distillation process, thereby failing to effectively bridge the gap between the student and teacher models. This paper introduces a novel text-assisted distillation methodology. Leveraging the framework of Contrastive Language-Image Pretraining (CLIP), we propose two modular plugins: the Text-Channel Distillation module and the Text-Region Distillation module, designed to integrate textual priors into the distillation process. These modules serve as abridge between the student and teacher models, enhancing the emulation of teacher networks by student models. Characterized by their simplicity, versatility, and seamless integration with existing knowledge distillation frameworks, these modules facilitate improved performance. Experimental evaluations conducted on the Cityscapes, Pascal VOC, and CamVid datasets demonstrate that augmenting state-of-the-art distillation techniques with these plug-and-play modules yields significant improvements in distillation effectiveness.
Keywords:
Semantic segmentation
Knowledge distillation
Modality fusion
Plug-and-play module
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
No organization information available