.png){kind=icon}
返回
Cross-Modal Coding for Task-Oriented Communications: A Rate-Distortion Perspective
.png){kind=icon}
PRE
AI
DOI：10.1109/TMC.2025.3620465.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Task-oriented communications for multi-modal applications emerge with the intelligence-oriented evolution of Internet of Things, where massive computation offloading with heterogeneous streaming requirements greatly challenges the existing mobile networks. Compared with semantic coding which reduces intra-modality redundancy by feature extraction, cross-modal coding further exploits inter-modality association and thus acts as a promising solution. However, unresolved information-theoretic issues hinder its full promise: 1) how to characterize the achievable region of cross-modal coding for task-oriented communications, and 2) to what extent can task-oriented communications benefit from exploiting inter-modality association. Therefore, this work first establishes a cross-modal rate-distortion function for task-oriented communications, and proves the feasibility of optimizing its information-bottleneck inspired transformation for guiding the design of learnable codec. In particular, the optimal feature representation is specified by a converging iterative solver under perfect statistical knowledge. Second, we prove a new bound on compression gains of cross-modal coding in task-oriented communications, based on a sufficient condition for cross-modal representation to be effective. Furthermore, a typical learnable codec is designed, whose loss function can be theoretically interpreted by our derived results. Finally, experimental evaluations verify the positive correlation between cross-modal coding gains and inter-modality association levels.
Keyword:
Task-oriented communications
semantic coding
multi-modal learning
edge computing
期刊
IF:
9.2
论文数:
5.6K
被引数:
1.8W
机构
引用论文
An Indirect Rate-Distortion Characterization for Semantic Sources: General Model and the Case of Gaussian Observation语义源的间接率失真表征: 一般模型和高斯观察的情况
OAAI
OAAI分享收藏分享收藏The Role of Fidelity in Goal-Oriented Semantic Communication: A Rate Distortion Approach保真度在面向目标的语义通信中的作用: 速率失真方法OAAI
OAAI分享收藏DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor With Application to In-Hand Manipulation数字: 一种低成本紧凑型高分辨率触觉传感器的新颖设计，可应用于手动操作
OAAI
OAAI分享收藏分享收藏A Wireless AI-Generated Content (AIGC) Provisioning Framework Empowered by Semantic Communication.png)
IF0
PREAI
分享收藏Cross-Modal Generative Semantic Communications for Mobile AIGC: Joint Semantic Encoding and Prompt Engineering移动AIGC的跨模态生成语义通信: 联合语义编码和提示工程OAAI
OAAI分享收藏
查看更多内容