.png){kind=icon}
返回
Adaptive model splitting with sample-efficient reinforcement learning for federated learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2025.130875.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The Reinforcement Learning (RL)-based adaptive model splitting strategy effectively addresses the challenges posed by heterogeneous devices in Federated Learning (FL) for Internet of Things (IoT) scenarios. However, in IoT scenarios, the scarcity of high-quality state samples leads to unstable splitting actions by RL agents. This is primarily because the RL agent can explore only a limited state-action space, hindering the learning of the optimal splitting strategy. To address this problem, we propose SplitRLC, a novel adaptive splitting with sample-efficient RL that utilizes virtual state-action pairs under cycle-consistency constraints, further enhanced by a reliable sample filtering mechanism. Specifically, SplitRLC predicts future states from current states and actions using transition models. The backward transition model then predicts previous states, forming a trajectory cycle. Based on this mechanism, we generated numerous virtual state-action pairs and applied the cycle consistency constraint to ensure their correctness. By utilizing these rich self-supervised sample (virtual state-action pairs), SplitRLC significantly reduces the dependence of RL on real state samples. Furthermore, the framework incorporates the consensus of the transition model to minimize the bias of low-reliability samples during training. This design significantly improves the efficiency of sample utilization in the learning process. We provide a convergence analysis and validate the effectiveness of the scheme through experiments on a real testbed. Our method outperforms state-of-the-art baselines in both training efficiency and robustness compared to traditional RL approaches. SplitRLC accelerates the training process of typical IoT devices by more than 50% by adaptively splitting the model from device to server. The source code is available at https://github.com/NiantaoZ/SplitRLC.
Keyword:
Federated learning
Reinforcement learning
Cycle consistency
Internet of Things (IoT)
期刊
IF:
7.5
论文数:
3.0W
被引数:
10.2W
机构
暂无机构信息
引用论文
.png){kind=icon}
Edge-assisted U-shaped split federated learning with privacy-preserving for Internet of Things面向物联网的边缘辅助U形分割隐私保护联合学习
OAAI
OAAI分享收藏Accelerating Federated Learning With Data and Model Parallelism in Edge Computing在边缘计算中通过数据和模型并行性加速联合学习
PREAI
分享收藏A Joint Communication and Learning Framework for Hierarchical Split Federated Learning用于分层分裂联邦学习的联合交流和学习框架
OAAI
OAAI分享收藏Continuous improvement of self-driving cars using dynamic confidence-aware reinforcement learning使用动态置信度感知强化学习持续改进自动驾驶汽车
PREAI
分享收藏ESFL: Efficient Split Federated Learning Over Resource-Constrained Heterogeneous Wireless DevicesESFL: 资源受限异构无线设备上的高效分体式联邦学习OAAI
OAAI分享收藏
查看更多内容