.png){kind=icon}
返回
Offline Generalized Actor-Critic with Distance Regularization
.png){kind=icon}
PRE
AI
DOI：10.1109/JAS.2025.125633.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In order to address the issue of overly conservative offline reinforcement learning (RL) methods that limit the generalization of policy in the out-of-distribution (OOD) region, this article designs a surrogate target for OOD value function based on dataset distance and proposes a novel generalized Q-learning mechanism with distance regularization (GQDR). In theory, we not only prove the convergence of GQDR, but also ensure that the difference between the Q-value learned by GQDR and its true value is bounded. Furthermore, an offline generalized actor-critic method with distance regularization (OGACDR) is proposed by combining GQDR with actor-critic learning framework. Two implementations of OGACDR, OGACDR-EXP and OGACDR-SQR, are introduced according to exponential (EXP) and open-square (SQR) distance weight functions, and it has been theoretically proved that OGACDR provides a safe policy improvement. Experimental results on Gym-MuJoCo continuous control tasks show that OGACDR can not only alleviate the overestimation and overconservatism of Q-value function, but also outperform conservative offline RL baselines.
Keyword:
Actor-critic
distance regularization
generalized Q-learning
offline reinforcement learning
out-of-distribution (ODD)
期刊
I
IF:
0
论文数:
116
被引数:
0
机构
引用论文
.png){kind=icon}
Autonomous Vehicle Platoons in Urban Road Networks: A Joint Distributed Reinforcement Learning and Model Predictive Control Approach城市道路网络中的自动车辆队列: 联合分布式强化学习和模型预测控制方法
PREAI
分享收藏
Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learning基于磁场的奖励塑造，用于目标条件强化学习
OAAI
OAAI分享收藏
查看更多内容