.png){kind=icon}
返回
Stochastic cubic-regularized policy gradient method
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2022.109687.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Policy-based reinforcement learning methods have achieved great achievements in real-world decisionmaking problems. However, the theoretical understanding of policy-based methods is still limited. Specifically, existing works mainly focus on first-order stationary point policies (FOSPs); in some very special reinforcement learning settings (e.g., tabular case and function approximation with restricted parametric policy classes) some works consider globally optimal policy. It is well-known that FOSPs could be undesirable local optima or saddle points, and obtaining a global optimum is generally NP-hard. In this paper, we propose a policy gradient method that provably converges to secondorder stationary point policies (SOSPs) for any differentiable policy classes. The proposed method is computationally efficient, and it judiciously uses cubic-regularized subroutines to escape saddle points while at the same time minimizing the Hessian-based computations. We prove that the method enjoys the sample complexity of O tilde (epsilon-3.5), which improves upon the current optimal complexity O tilde (epsilon-4.5). Finally, experimental results are provided to demonstrate the effectiveness of the method.(c) 2022 Elsevier B.V. All rights reserved.
Keyword:
Reinforcement learning
Policy gradient
Stochastic optimization
Non -convex optimization
Second-order stationary point
期刊
K
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
.png){kind=icon}
分享收藏
Fabrication of porous hollow γ-Al2O3 nanofibers by facile electrospinning and its application for water remediation静电纺丝法制备多孔中空 γ-Al2O3纳米纤维及其在水体修复中的应用
PREAI
分享收藏
查看更多内容