.png){kind=icon}
返回
Controlled descent training
.png){kind=icon}
OA
AI
DOI：10.1002/rnc.7194.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this work, a novel and model-based artificial neural network (ANN) training method is developed supported by optimal control theory. The method augments training labels in order to robustly guarantee training loss convergence and improve training convergence rate. Dynamic label augmentation is proposed within the framework of gradient descent training where the convergence of training loss is controlled. First, we capture the training behavior with the help of empirical Neural Tangent Kernels (NTK) and borrow tools from systems and control theory to analyze both the local and global training dynamics (e.g., stability, reachability). Second, we propose to dynamically alter the gradient descent training mechanism via fictitious labels as control inputs and an optimal state feedback policy. In this way, we enforce locally Script capital H2$$ {\mathscr{H}}_2 $$ optimal and convergent training behavior. The novel algorithm, Controlled Descent Training (CDT), guarantees local convergence. CDT unleashes new potentials in the analysis, interpretation, and design of ANN architectures. The applicability of the method is demonstrated on standard regression and classification problems.
Keyword:
convergent learning
gradient decent training
label augmentation
label selection
neural Tangent Kernel
optimal labels
期刊
IF:
3.2
论文数:
7.0K
被引数:
1.4W
机构
引用论文
An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools自然语言处理中的深度学习: 模型、技术和工具.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏
Online learning-based model predictive control with Gaussian process models and stability guarantees具有高斯过程模型和稳定性保证的在线学习模型预测控制OAAI
OAAI分享收藏分享收藏Model-free self-triggered control based on deep reinforcement learning for unknown nonlinear systems基于深度强化学习的未知非线性系统无模型自触发控制
PREAI
分享收藏
查看更多内容