.png){kind=icon}
Return
Q-Learning for Continuous-Time Linear Systems: A Data-Driven Implementation of the Kleinman Algorithm
.png){kind=icon}
PRE
AI
DOI：10.1109/TSMC.2022.3145693.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
A data-driven strategy to estimate the optimal feedback and the value function in an infinite-horizon, continuous-time, linear-quadratic optimal control problem for an unknown system is proposed. The method permits the construction of the optimal policy without any knowledge of the model, without requiring that the time derivatives of the state are available for the design, and without even assuming that an initial stabilizing feedback policy is available. Two alternative architectures are discussed: the first scheme revolves around the periodic computation of some matrix inversions involving the Q-function, whereas the second approach relies on a purely continuous-time implementation of some dynamic systems whose trajectories are uniformly attracted by the solutions to the above algebraic equations. Interestingly, the proposed strategy essentially constitutes a (direct) data-driven implementation of the celebrated Kleinman algorithm, hence subsuming the particularly appealing features of the latter, such as quadratic monotone convergence to the optimal solution. The theory is then validated by the means of practically motivated applications.
Keywords:
Optimal control
Trajectory
Costs
Convergence
Symmetric matrices
Riccati equations
Q-learning
Linear systems
optimal control
reinforcement learning
uncertain
unknown systems
Journal
IF:
10.5
Papers:
1.1W
Citations:
5.0W
Organization
Cited Papers
ShareSaveShareSaveValue iteration and adaptive dynamic programming for data-driven adaptive optimal control design.png)
AUTOMATICA
OAAI
AUTOMATICAIF5.9
OAAIShareSaveData-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics
PREAI
ShareSaveShareSaveAdaptive optimal control for continuous-time linear systems based on policy iterationAUTOMATICA
AUTOMATICAIF5.9
PREAI
ShareSave
ShareSaveShareSave
View more