.png){kind=icon}
Return
Hamiltonian-Driven Adaptive Dynamic Programming With Approximation Errors
.png){kind=icon}
OA
AI
DOI：10.1109/TCYB.2021.3108034.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In this article, we consider an iterative adaptive dynamic programming (ADP) algorithm within the Hamiltonian-driven framework to solve the Hamilton-Jacobi-Bellman (HJB) equation for the infinite-horizon optimal control problem in continuous time for nonlinear systems. First, a novel function, ``min-Hamiltonian,'' is defined to capture the fundamental properties of the classical Hamiltonian. It is shown that both the HJB equation and the policy iteration (PI) algorithm can be formulated in terms of the min-Hamiltonian within the Hamiltonian-driven framework. Moreover, we develop an iterative ADP algorithm that takes into consideration the approximation errors during the policy evaluation step. We then derive a sufficient condition on the iterative value gradient to guarantee closed-loop stability of the equilibrium point as well as convergence to the optimal value. A model-free extension based on an off-policy reinforcement learning (RL) technique is also provided. Finally, numerical results illustrate the efficacy of the proposed framework.
Keywords:
Costs
Mathematical model
Stability analysis
Approximation error
Approximation algorithms
Dynamic programming
Iterative algorithms
Hamilton-Jacobi-Bellman (HJB) equation
Hamiltonian-driven framework
inexact adaptive dynamic programming (ADP)
optimal control
Journal
IF:
10.5
Papers:
1.1W
Citations:
5.0W
Organization
View more organizations
Cited Papers
ShareSaveShareSaveAdaptive optimal control for continuous-time linear systems based on policy iteration.png)
AUTOMATICA
AUTOMATICAIF5.9
PREAI
ShareSaveError Bound Analysis of Q-Function for Discounted Optimal Control Problems With Policy Iteration
PREAI
ShareSave View more