.png){kind=icon}
Return
Cross-Modal Coding for Task-Oriented Communications: A Rate-Distortion Perspective
.png){kind=icon}
PRE
AI
DOI：10.1109/TMC.2025.3620465.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Task-oriented communications for multi-modal applications emerge with the intelligence-oriented evolution of Internet of Things, where massive computation offloading with heterogeneous streaming requirements greatly challenges the existing mobile networks. Compared with semantic coding which reduces intra-modality redundancy by feature extraction, cross-modal coding further exploits inter-modality association and thus acts as a promising solution. However, unresolved information-theoretic issues hinder its full promise: 1) how to characterize the achievable region of cross-modal coding for task-oriented communications, and 2) to what extent can task-oriented communications benefit from exploiting inter-modality association. Therefore, this work first establishes a cross-modal rate-distortion function for task-oriented communications, and proves the feasibility of optimizing its information-bottleneck inspired transformation for guiding the design of learnable codec. In particular, the optimal feature representation is specified by a converging iterative solver under perfect statistical knowledge. Second, we prove a new bound on compression gains of cross-modal coding in task-oriented communications, based on a sufficient condition for cross-modal representation to be effective. Furthermore, a typical learnable codec is designed, whose loss function can be theoretically interpreted by our derived results. Finally, experimental evaluations verify the positive correlation between cross-modal coding gains and inter-modality association levels.
Keywords:
Task-oriented communications
semantic coding
multi-modal learning
edge computing
Journal
IF:
9.2
Papers:
5.9K
Citations:
1.8W
Organization
Cited Papers
ShareSaveShareSaveShareSave
ShareSaveShareSaveA Wireless AI-Generated Content (AIGC) Provisioning Framework Empowered by Semantic Communication.png)
IF0
PREAI
ShareSaveShareSaveShareSave
View more