.png){kind=icon}
.png){kind=icon}
Return
Adaptive stepsize estimation based accelerated gradient descent algorithm for fully complex-valued neural networks
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2023.121166.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Nesterov accelerated gradient (NAG) method is an efficient first-order algorithm for optimization problems. To ensure the convergence, it usually takes a relatively conservative constant as the stepsize. However, the choice of stepsize has a great impact on the optimization process. In this paper, two adaptive stepsize estimation methods are proposed for complex-valued NAG algorithm for efficient training of fully complex-valued neural networks. The basic idea of the first one is to adaptively determine suitable stepsize by estimating the local smoothness constant of the loss function with the norm of approximate complex Hessian matrix. Its validity is theoretically analyzed by means of the decomposition of complex matrix. Furthermore, by introducing a new parameter design method for multi-step quasi-Newton condition, an improved stepsize estimation is presented. Experimental results on pattern recognition, channel equalization, wind forecasting and synthetic aperture radar (SAR) target classification demonstrate the effectiveness of the proposed methods.
Keywords:
Accelerated gradient descent
Adaptive stepsize
Local smoothness constant
Curvature information
Fully complex-valued neural networks
Journal
IF:
7.5
Papers:
2.9W
Citations:
10.2W