.png){kind=icon}
Return
Alternated Greedy-Step Deterministic Policy Gradient
.png){kind=icon}
PRE
AI
DOI：10.1109/TCDS.2023.3242274.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The greedy-step $Q$ -learning (GQL) can effectively accelerate the $Q$ -value updating process. However, since it is an improved version of $Q$ -learning, the problem of $Q$ -value overestimation also exists. Since there are in total two max operators used to iteratively calculate $Q$ -value in GQL, many existing solutions to reduce the $Q$ -value estimation bias are invalid for GQL. To address the issue, an alternated greedy-step update (AGU) framework that consists of two independent $Q$ -value estimators is proposed in this study. In the proposed AGU framework, one estimator is to determine the time step that can maximize the estimated $n$ -step return and the other estimator is to update the prior estimator using the target value calculated on the basis of the determined time step. The convergence of the AGU framework is proved in theoretical. In addition, an alternated greedy-step deterministic policy gradient (AGDPG) that can be applied to continuous-action tasks is proposed by combining the AGU framework with deep deterministic policy gradient (DDPG). Experiments of AGDPG on continuous-action tasks of the MuJoCo platform highlight its superior performance.
Keywords:
Alternated greedy-step update (AGU)
deep deterministic policy gradient (DDPG)
greedy-step Q-learning (GQL)
Q-value overestimation
Journal
IF:
4.9
Papers:
1.1K
Citations:
3.5K
Organization
No organization information available
Cited Papers
.png){kind=icon}
ShareSave
ShareSaveShareSave
View more