Constrained deterministic policy gradient adaptive dynamic programming using projected gradient update

Timotei Lala

doi:10.1080/21642583.2025.2546823

Constrained deterministic policy gradient adaptive dynamic programming using projected gradient update

2025-10-25

0

OA

AI

T

Timotei Lala *

DOI：10.1080/21642583.2025.2546823

原文链接

查看原文

摘要

En 中文

Learning controllers for nonlinear complex systems using data-driven model-free adaptive dynamic programming (ADP) algorithms is a challenging problem. Traditional ADP methods often suffer from non-smooth convergence or divergence during learning, potentially leading to unstable control. In this paper, we propose a novel model-free constrained deterministic policy gradient adaptive dynamic programming (CDPGADP) for discrete-time general nonlinear systems. Our method constrains the difference between consecutive ADP iteration controllers using the method of projected gradient descent. Neural networks (NNs) are used to approximate both the state-action function and the controller. The constraint operates directly in the controller command space, effectively limiting the controller output during successive learning iterations. This algorithm is validated on a wind turbine pitch control task, learning to maintain a constant angular velocity at a prescribed nominal value under varying wind disturbances.

Keyword:

ADP

constrained ADP

deterministic policy gradient

projected gradient descent

Q-learning

Constrained deterministic policy gradient adaptive dynamic programming using projected gradient update

摘要

AI总结

期刊

机构

引用论文