.png){kind=icon}
.png){kind=icon}
Return
Efficient gradient descent algorithm with anderson acceleration for separable nonlinear models
.png){kind=icon}
PRE
AI
DOI：10.1007/s11071-024-10651-6.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Separable nonlinear models are pervasively employed in diverse disciplines, such as system identification, signal analysis, electrical engineering, and machine learning. Identifying these models inherently poses a non-convex optimization challenge. While gradient descent (GD) is a commonly adopted method, it is often plagued by suboptimal convergence rates and is highly dependent on the appropriate choice of step size. To mitigate these issues, we introduce an augmented GD algorithm enhanced with Anderson acceleration (AA), and propose a Hierarchical GD with Anderson acceleration (H-AAGD) method for efficient identification of separable nonlinear models. This novel approach transcends the conventional step size constraints of GD algorithms and considers the coupling relationships between different parameters during the optimization process, thereby enhancing the efficiency of the solution-finding process. Unlike the Newton method, our algorithm obviates the need for computing the inverse of the Hessian matrix, simplifying the computational process. Additionally, we theoretically analyze the convergence and complexity of the algorithm and validate its effectiveness through a series of numerical experiments.
Keywords:
Anderson acceleration
Separable nonlinear problem
Robust parameter estimation
Hierarchical identification algorithm