.png){kind=icon}
返回
Dual Behavior Regularized Offline Deterministic Actor-Critic
.png){kind=icon}
PRE
AI
DOI：10.1109/TSMC.2024.3388007.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
To mitigate the extrapolation error arising from offline reinforcement learning (RL) paradigm, existing methods typically make learned Q-functions over-conservative or enforce global policy constraints. In this article, we propose a dual behavior regularized offline deterministic Actor-Critic (DBRAC) by simultaneously performing behavior regularization on the coupling-iterative policy evaluation (PE) and policy improvement (PI) in the policy iteration process. In the PE phase, the difference between the Q-function and behavior value is first taken as the anti-exploration behavior value regularization term to drive the Q-function toward its true Q-value, which significantly reduces the conservatism of learned Q-function. In the PI phase, the estimated action variances of behavior policy in different states are then utilized for designing the weight and threshold of mild-local behavior cloning regularization term, which standardizes the local improvement potential of learned policy. Experiments on the well-known datasets for deep data-driven RL (D4RL) demonstrate that the DBRAC can quickly learn more competitive task-solving policies in various offline situations with different data qualities, significantly outperforming state-of-the-art offline RL baselines.
Keyword:
Anti-exploration behavior value
dual behavior regularization (DBR)
mild-local behavior cloning (BC)
offline deterministic Actor-Critic
reinforcement learning (RL)
期刊
IF:
10.5
论文数:
1.1W
被引数:
5.0W
机构
暂无机构信息
引用论文
A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat面向六自由度UCAV空空作战的层次化深度强化学习框架
OAAI
.png){kind=icon}
OAAI分享收藏Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded MicrogridsOAAI
OAAI分享收藏Multi-Kernel Online Reinforcement Learning for Path Tracking Control of Intelligent Vehicles面向智能车辆路径跟踪控制的多核在线强化学习
PREAI
分享收藏
查看更多内容