.png){kind=icon}
返回
Alternated Greedy-Step Deterministic Policy Gradient
.png){kind=icon}
PRE
AI
DOI：10.1109/TCDS.2023.3242274.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The greedy-step $Q$ -learning (GQL) can effectively accelerate the $Q$ -value updating process. However, since it is an improved version of $Q$ -learning, the problem of $Q$ -value overestimation also exists. Since there are in total two max operators used to iteratively calculate $Q$ -value in GQL, many existing solutions to reduce the $Q$ -value estimation bias are invalid for GQL. To address the issue, an alternated greedy-step update (AGU) framework that consists of two independent $Q$ -value estimators is proposed in this study. In the proposed AGU framework, one estimator is to determine the time step that can maximize the estimated $n$ -step return and the other estimator is to update the prior estimator using the target value calculated on the basis of the determined time step. The convergence of the AGU framework is proved in theoretical. In addition, an alternated greedy-step deterministic policy gradient (AGDPG) that can be applied to continuous-action tasks is proposed by combining the AGU framework with deep deterministic policy gradient (DDPG). Experiments of AGDPG on continuous-action tasks of the MuJoCo platform highlight its superior performance.
Keyword:
Alternated greedy-step update (AGU)
deep deterministic policy gradient (DDPG)
greedy-step Q-learning (GQL)
Q-value overestimation
期刊
IF:
4.9
论文数:
1.1K
被引数:
3.5K
机构
暂无机构信息
引用论文
.png){kind=icon}
Study of bi-directional buck-boost converter topologies for application in electrical vehicle motor drives应用于电动汽车电机驱动的双向buck-boost变换器拓扑研究
PREAI
分享收藏Reducing Estimation Bias via Triplet-Average Deep Deterministic Policy Gradient通过三元组平均深度确定性策略梯度减少估计偏差
OAAI
OAAI分享收藏
查看更多内容