.png){kind=icon}
.png){kind=icon}
Return
Learning Latent Multimodal Dynamics for Optimized Resource Planning
.png){kind=icon}
PRE
AI
DOI：10.1109/TWC.2025.3644600.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In this work, we study the joint scheduling and power allocation problem of vision-based remote control systems, where multiple devices upload their image states to a central controller and receive control actions. Due to the high dimensionality of the image states and to manage the lack of radio resources, we propose a novel self-supervised learning approach to predict the devices’ joint control and wireless dynamics in latent space, enabling wireless resource optimization without compromising the control objectives of the remote control system. Our method leverages two coupled joint-embedding predictive architectures (JEPAs): a control JEPA models the control transition dynamics and guides the predictions of a wireless JEPA, which captures the dynamics of the device’s channel state information (CSI) through cross-modal conditioning. We then train a deep reinforcement learning (RL) algorithm to derive a control policy from latent control dynamics and a power predictor to estimate scheduling slots with favorable channel conditions based on latent CSI representations. To enhance control reliability, we employ an efficient ensemble technique to estimate the uncertainty of JEPA predictions. The two JEPAs are used by the remote controller to forecast future latent trajectories of the devices’ control and wireless states, allowing the controller to proactively plan its scheduling policy using model predictive control (MPC). Simulation results, conducted in a customized image-based control environment with ray tracing, demonstrate that our proposed approach converges three times faster and reduces transmit power by over 50% while maintaining control performance comparable to baseline methods that do not account for wireless resource optimization.
Keywords:
Self-supervised learning
resource management
joint-embedding predictive architecture
cross-modal prediction
Journal
IF:
10.7
Papers:
1.3W
Citations:
5.3W