.png){kind=icon}
返回
Offline Reinforcement Learning for Wireless Network Optimization With Mixture Datasets
.png){kind=icon}
OA
AI
DOI：10.1109/TWC.2024.3395624.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The recent development of reinforcement learning (RL) has boosted the adoption of online RL for wireless radio resource management (RRM). However, online RL algorithms require direct interactions with the environment, which may be undesirable given the potential performance loss due to the unavoidable exploration in RL. In this work, we first explore the use of offline RL algorithms in solving the RRM problem. We evaluate several state-of-the-art offline RL algorithms for a practical RRM problem that aims at maximizing a linear combination of total rates and 5-percentile rates via user scheduling. Our findings indicate that the performance of offline RL for the RRM problem is heavily contingent upon the behavior policy deployed for data collection. We propose an innovative offline RL approach utilizing heterogeneous datasets from various behavior policies. This method demonstrates that a strategic mixture of datasets enables near-optimal RL policy generation, even with suboptimal behavior policies. Additionally, we introduce two enhancements: an ensemble-based policy to augment dataset mixture training efficiency, and a novel offline-to-online strategy for seamless adaptation to new environments. Our data mixture approach achieves over 95% efficiency of an online RL agent in the absence of expert data. The ensemble algorithm notably reduces training duration by half compared to the data mixture method. Furthermore, our model, when applied with offline-to-online fine-tuning, surpasses existing benchmarks by approximately 5% in our user scheduling problem.
Keyword:
Wireless networks
Resource management
Optimization
Reinforcement learning
Scheduling
Interference
Data models
Radio resource management
offline reinforcement learning
deep reinforcement learning
期刊
IF:
10.7
论文数:
1.3W
被引数:
5.3W
机构
引用论文
.png){kind=icon}
Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning基于多Agent深度强化学习的无线网络资源管理
OAAI
OAAI分享收藏Generalized Global Bandit and Its Application in Cellular Coverage Optimization广义全局强盗及其在元胞覆盖优化中的应用
PREAI
分享收藏
查看更多内容