.png){kind=icon}
返回
A Q-Learning Algorithm to Solve the Two-Player Zero-Sum Game Problem for Nonlinear Systems
.png){kind=icon}
OA
AI
DOI：10.1002/acs.3958.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This paper deals with the two-player zero-sum game problem, which is a bounded L2$$ {L}_2 $$-gain robust control problem. Finding an analytical solution to the complex Hamilton-Jacobi-Issacs (HJI) equation is a challenging task. Hence, a novel Q-learning algorithm for unknown continuous-time (CT) affine-in-inputs nonlinear systems is proposed for generating an approximate solution to the HJI equation, which is valid in a local domain due to the use of a local approximator, that is, a Neural Network (NN) structure. The approach is model-free and does not require the knowledge of system drift dynamics, and input and disturbance gains. The algorithm learns online from measurements of state variables in real time. To generate the local approximate solution of the HJI equation for the two-player zero-sum game problem for nonlinear systems, the proposed non-iterative algorithm requires only a single critic NN instead of the commonly used triple NN approximator structure. A persistence of excitation condition is required to guarantee Uniformly Ultimately Boundedness (UUB) and convergence to the optimal solution. The effectiveness of the proposed Q-learning approach for the two-player zero-sum game problem is demonstrated via simulations of a linear F-16 aircraft plant and a highly complex nonlinear system. Proof of closed-loop system stability is provided using Lyapunov Analysis, and convergence of the approximate solution to the true saddle-point solution is guaranteed in a UUB-sense.
Keyword:
adaptive control
approximate dynamic programming
Q-learning
zero-sum game
期刊
IF:
3.8
论文数:
2.6K
被引数:
3.6K
机构
引用论文
Adaptive optimal control for continuous-time linear systems based on policy iteration基于策略迭代的连续时间线性系统自适应最优控制.png)
AUTOMATICA
AUTOMATICAIF5.9
PREAI
分享收藏分享收藏A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems一种用于不确定非线性系统的近似最优控制的新型参与者-批评者-标识符体系结构AUTOMATICA
AUTOMATICAIF5.9
PREAI
分享收藏
查看更多内容