.png){kind=icon}
.png){kind=icon}
Return
Optimization methods rooted in optimal control
.png){kind=icon}
PRE
AI
DOI：10.1007/s11432-024-4207-5.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In the paper, we investigate the optimization problem (OP) by applying the optimal control method. The optimization problem is reformulated as an optimal control problem (OCP) where the controller (iteration updating) is designed to minimize the sum of costs in the future time instant, which thus theoretically generates the optimal algorithm (fastest and most stable). By adopting the maximum principle and linearization with Taylor expansion, new algorithms are proposed. It is shown that the proposed algorithms have a superlinear convergence rate and thus converge more rapidly than the gradient descent; meanwhile, they are superior to Newton's method because they are not divergent in general and can be applied in the case of a singular or indefinite Hessian matrix. More importantly, the OCP method contains the gradient descent and the Newton's method as special cases, which discovers the theoretical basis of gradient descent and Newton's method and reveals how far these algorithms are from the optimal algorithm. The merits of the proposed optimization algorithm are illustrated by numerical experiments.
Keywords:
optimal control
optimization methods
optimization algorithm
maximum principle
superlinear convergence
Journal
IF:
7.6
Papers:
4.9K
Citations:
8.9K
Organization
No organization information available