.png){kind=icon}
.png){kind=icon}
Return
The Pontryagin Maximum Principle for Training Convolutional Neural Networks\ast
.png){kind=icon}
PRE
AI
S
A
DOI：10.1137/24M1675369.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
A novel batch sequential quadratic Hamiltonian (bSQH) algorithm for training convolutional neural networks (CNNs) with L0-based regularization is presented. This methodology is based on a discrete-time Pontryagin maximum principle (PMP). It uses forward and backward sweeps together with the layerwise approximate maximization of an augmented Hamiltonian function, where the augmentation parameter is chosen adaptively. A technique for determining this augmentation parameter is proposed, and the loss-reduction and convergence properties of the bSQH algorithm are analyzed theoretically and validated numerically. Results of numerical experiments in the context of image classification with a sparsity-enforcing, L0-based regularizer demonstrate the effectiveness of the proposed method in full-batch and mini-batch modes.
Keywords:
convolutional neural network
discrete Pontryagin maximum principle
sequential quadratic Hamil-tonian method
method of successive approximations
numerical optimization
Journal
S
IF:
2.6
Papers:
17
Citations:
0
Organization
Cited Papers