.png){kind=icon}
返回
Offline Reinforcement Learning With Reverse Diffusion Guide Policy
.png){kind=icon}
PRE
AI
DOI：10.1109/TII.2024.3413308.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Offline reinforcement learning (ORL) learns policy from a static dataset without further interaction with the environment, which holds significant promise in industrial control systems characterized by inefficient online interaction and inherent safety concerns. To mitigate the extrapolation error induced by distribution shift, it is essential for ORL to constrain the learned policy to perform actions within the support set of behavior policy. Existing methods fail to represent the behavior policy properly and typically tend to prefer actions with higher densities within the support set, resulting suboptimal learned policy. This article proposes a novel ORL method which represents the behavior policy with a diffusion model and trains a reverse diffusion guide policy to instruct the pretrained diffusion model in generating actions. The diffusion model exhibits stable training and strong distribution expression ability, and the reverse diffusion guide policy can effectively explore the entire support set to help generate the optimal action. When facing low-quality datasets, a trainable perturbation can be further added to the generated action to help the learned policy escape the performance limitation of behavior policy. Experimental results on D4RL Gym-MuJoCo benchmark demonstrate the effectiveness of the proposed method, surpassing several state-of-the-art ORL methods.
Keyword:
Reinforcement learning
Training
Standards
Diffusion processes
Perturbation methods
Informatics
Extrapolation
Diffusion model
offline reinforcement learning (ORL)
perturbation model
policy constraint
reverse diffusion guide policy (RDGP)
期刊
IF:
9.9
论文数:
8.6K
被引数:
6.0W
机构
暂无机构信息
引用论文
A Data-Driven Solution for Energy Management Strategy of Hybrid Electric Vehicles Based on Uncertainty-Aware Model-Based Offline Reinforcement Learning基于不确定性感知模型离线强化学习的混合动力汽车能量管理策略数据驱动方案
OAAI
.png){kind=icon}
OAAI分享收藏Transformer-Based Imitative Reinforcement Learning for Multirobot Path Planning基于Transformer的多机器人路径规划模拟强化学习
PREAI
分享收藏Study of bi-directional buck-boost converter topologies for application in electrical vehicle motor drives应用于电动汽车电机驱动的双向buck-boost变换器拓扑研究
PREAI
分享收藏An Intelligent Train Operation Method Based on Event-Driven Deep Reinforcement Learning基于事件驱动的深度强化学习的列车智能运行方法
PREAI
分享收藏分享收藏 没有更多内容
没有更多内容