.png){kind=icon}
返回
Modularizing Deep Learning via Pairwise Learning With Kernels
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2020.3042346.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
By redefining the conventional notions of layers, we present an alternative view on finitely wide, fully trainable deep neural networks as stacked linear models in feature spaces, leading to a kernel machine interpretation. Based on this construction, we then propose a provably optimal modular learning framework for classification that does not require between-module backpropagation. This modular approach brings new insights into the label requirement of deep learning (DL). It leverages only implicit pairwise labels (weak supervision) when learning the hidden modules. When training the output module, on the other hand, it requires full supervision but achieves high label efficiency, needing as few as ten randomly selected labeled examples (one from each class) to achieve 94.88% accuracy on CIFAR-10 using a ResNet-18 backbone. Moreover, modular training enables fully modularized DL workflows, which then simplify the design and implementation of pipelines and improve the maintainability and reusability of models. To showcase the advantages of such a modularized workflow, we describe a simple yet reliable method for estimating reusability of pretrained modules as well as task transferability in a transfer learning setting. At practically no computation overhead, it precisely described the task space structure of 15 binary classification tasks from CIFAR-10.
Keyword:
Kernel
Artificial neural networks
Training
Task analysis
Backpropagation
Deep learning
Symmetric matrices
Deep learning (DL)
kernel methods
modular training
neural networks (NNs)
task transferability
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
.png){kind=icon}
Visuomotor perimetry in fish: A new approach to the functional analysis of altered visual pathways鱼的视觉运动视野检查: 一种对改变的视觉通路进行功能分析的新方法
PREAI
分享收藏分享收藏
查看更多内容