.png){kind=icon}
.png){kind=icon}
Return
Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problems
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2023.3235967.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This article introduces and analyzes an improved Q-learning algorithm for discrete-time linear time-invariant systems. The proposed method does not require any knowledge of the system dynamics, and it enjoys significant efficiency advantages over other data-based optimal control methods in the literature. This algorithm can be fully executed offline, as it does not require to apply the current estimate of the optimal input to the system as in on-policy algorithms. It is shown that a PE input, defined from an easily tested matrix rank condition, guarantees the convergence of the algorithm. A data-based method is proposed to design the initial stabilizing feedback gain that the algorithm requires. Robustness of the algorithm in the presence of noisy measurements is analyzed. We compare the proposed algorithm in simulation to different direct and indirect data-based control design methods.
Keywords:
Q-learning
Heuristic algorithms
Data models
Convergence
Trajectory
Prediction algorithms
Linear systems
Data-based control
optimal control
reinforcement learning (RL)
Journal
IF:
7
Papers:
1.3W
Citations:
6.7W