.png){kind=icon}
返回
Federated Ensemble Model-Based Reinforcement Learning in Edge Computing
.png){kind=icon}
OA
AI
DOI：10.1109/TPDS.2023.3264480.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Federated learning (FL) is a privacy-preserving distributed machine learning paradigm that enables collaborative training among geographically distributed and heterogeneous devices without gathering their data. Extending FL beyond the supervised learning models, federated reinforcement learning (FRL) was proposed to handle sequential decision-making problems in edge computing systems. However, the existing FRL algorithms directly combine model-free RL with FL, thus often leading to high sample complexity and lacking theoretical guarantees. To address the challenges, we propose a novel FRL algorithm that effectively incorporates model-based RL and ensemble knowledge distillation into FL for the first time. Specifically, we utilise FL and knowledge distillation to create an ensemble of dynamics models for clients, and then train the policy by solely using the ensemble model without interacting with the environment. Furthermore, we theoretically prove that the monotonic improvement of the proposed algorithm is guaranteed. The extensive experimental results demonstrate that our algorithm obtains much higher sample efficiency compared to classic model-free FRL algorithms in the challenging continuous control benchmark environments under edge computing settings. The results also highlight the significant impact of heterogeneous client data and local model update steps on the performance of FRL, validating the insights obtained from our theoretical analysis.
Keyword:
Computational modeling
Data models
Heuristic algorithms
Training
Edge computing
Reinforcement learning
Analytical models
Deep reinforcement learning
distributed machine learning
edge computing
federated learning
期刊
IF:
6
论文数:
5.2K
被引数:
1.1W
机构
引用论文
User Access Control in Open Radio Access Networks: A Federated Deep Reinforcement Learning Approach
PREAI
分享收藏分享收藏分享收藏Carbohydrate metabolism of the cat 1. Activity of amylase in the gastrointestinal tract of the cat1
PREAI
分享收藏Challenges of real-world reinforcement learning: definitions, benchmarks and analysis现实世界强化学习的挑战: 定义、基准和分析.png)
MACHINE LEARNING
OAAI
MACHINE LEARNINGIF2.9
OAAI分享收藏
查看更多内容