.png){kind=icon}
Return
Controlled descent training
.png){kind=icon}
OA
AI
DOI：10.1002/rnc.7194.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In this work, a novel and model-based artificial neural network (ANN) training method is developed supported by optimal control theory. The method augments training labels in order to robustly guarantee training loss convergence and improve training convergence rate. Dynamic label augmentation is proposed within the framework of gradient descent training where the convergence of training loss is controlled. First, we capture the training behavior with the help of empirical Neural Tangent Kernels (NTK) and borrow tools from systems and control theory to analyze both the local and global training dynamics (e.g., stability, reachability). Second, we propose to dynamically alter the gradient descent training mechanism via fictitious labels as control inputs and an optimal state feedback policy. In this way, we enforce locally Script capital H2$$ {\mathscr{H}}_2 $$ optimal and convergent training behavior. The novel algorithm, Controlled Descent Training (CDT), guarantees local convergence. CDT unleashes new potentials in the analysis, interpretation, and design of ANN architectures. The applicability of the method is demonstrated on standard regression and classification problems.
Keywords:
convergent learning
gradient decent training
label augmentation
label selection
neural Tangent Kernel
optimal labels
Journal
IF:
3.2
Papers:
7.0K
Citations:
1.4W
Organization
Cited Papers
An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
ShareSave
Online learning-based model predictive control with Gaussian process models and stability guaranteesOAAI
OAAIShareSaveShareSaveModel-free self-triggered control based on deep reinforcement learning for unknown nonlinear systems
PREAI
ShareSaveShareSaveShareSave
View more