.png){kind=icon}
返回
Off-policy algorithm based Hierarchical optimal control for completely unknown dynamic systems
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2021.11.077.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This paper proposes an online reinforcement learning (RL) for solving Stackelberg games with completely unknown dynamic systems. To deal with the hierarchical optimal control problem, the key is to find the solution of a two-level optimal control with the leader's optimal problem being constrained. Firstly, the leader-follower coupled Hamiltonian-Jacobi (HJ) equations subject to the follower's costate equation is derived. Secondly, an off-policy scheme is designed following the policy iteration (PI) algorithm based on model. The improved off-policy algorithm is given for obtaining the solution to the leader-follower coupled HJ equations. The algorithm is built without requiring any knowledge of the dynamic system, and the leader-follower optimal control policies are directly constructed by the extracted data. Meanwhile, the existence of the Stackelberg equilibrium is demonstrated. Lastly, NNs for each player are built, and NN learning is accomplished online with Kronecker product technique and vectorization, which simples the NN form and could decrease computation burden. Simulation examples are presented to demonstrate the proposed learning algorithm. (C) 2021 Published by Elsevier B.V.
Keyword:
Stackelberg games
Reinforcement learning
Hierarchical control
Off-policy
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
.png){kind=icon}
分享收藏Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances具有扰动的未知系统最优控制的非策略参与者-批评者结构
PREAI
分享收藏Data-Based Reinforcement Learning for Nonzero-Sum Games With Unknown Drift Dynamics基于数据的具有未知漂移动态的非零和游戏的强化学习
PREAI
分享收藏
查看更多内容