.png){kind=icon}
.png){kind=icon}
Return
Cross-Architecture Knowledge Distillation for Deep Joint Source-Channel Coding
.png){kind=icon}
PRE
AI
DOI：10.1109/TMC.2025.3633266.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep learning-based joint source-channel coding (DeepJSCC) has shown significant benefits in emerging semantic and task-oriented communications, providing a promising solution for reducing latency and bandwidth requirements in next-generation mobile networks. However, its deployment on resource-constrained devices is limited by model complexity. Devices with varying computational capacities require models of distinct architectures and complexity levels, motivating the design of a cross-architecture model compression scheme for DeepJSCC. In this paper, we propose a cross-architecture knowledge distillation framework called CAKDJSCC for heterogeneous DeepJSCC models. Specifically, we design a teaching assistant network with feature fusion modules (FFMs) that dynamically perceive architecture gaps between teacher and student models, thereby generating student-adaptive feature representations to alleviate feature space misalignment caused by architectural inconsistencies. In addition, we introduce a conditional information bottleneck (CIB) loss to optimize the distillation process, which prevents students from overfitting to teacher-specific inductive biases while enhancing knowledge transfer efficiency in cross-architecture scenarios. Extensive experiments demonstrate that our approach significantly improves the student model’s reconstruction accuracy and perceptual quality without increasing the inference latency while minimizing the performance degradation during model compression.
Keywords:
DeepJSCC
knowledge distillation
teacher assistant network
conditional information bottleneck
semantic communications
Journal
IF:
9.2
Papers:
5.6K
Citations:
1.8W