.png){kind=icon}
返回
Robust Student Network Learning
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2019.2929114.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep neural networks bring in impressive accuracy in various applications, but the success often relies on heavy network architectures. Taking well-trained heavy networks as teachers, classical teacher-student learning paradigm aims to learn a student network that is lightweight yet accurate. In this way, a portable student network with significantly fewer parameters can achieve considerable accuracy, which is comparable to that of a teacher network. However, beyond accuracy, the robustness of the learned student network against perturbation is also essential for practical uses. Existing teacher-student learning frameworks mainly focus on accuracy and compression ratios, but ignore the robustness. In this paper, we make the student network produce more confident predictions with the help of the teacher network, and analyze the lower bound of the perturbation that will destroy the confidence of the student network. Two important objectives regarding prediction scores and gradients of examples are developed to maximize this lower bound, to enhance the robustness of the student network without sacrificing the performance. Experiments on benchmark data sets demonstrate the efficiency of the proposed approach to learning robust student networks that have satisfying accuracy and compact sizes.
Keyword:
Perturbation methods
Convolution
Neural networks
Training
Redundancy
Feature extraction
Learning systems
Deep learning
knowledge distillation (KD)
teacher-student learning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
8.9
论文数:
7.5K
被引数:
7.2W
机构
引用论文
A Hierarchical Cardiac Rhythm Classification Methodology Based on Electrocardiogram Fiducial Points一种基于心电图基准点的分层心律分类方法OAAI
OAAI分享收藏Synthesis of n-type semiconducting diamond film using diphosphorus pentaoxide as the doping source以五氧化二磷为掺杂源合成n型半导体金刚石膜
PREAI
分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏Inactivation of the 20S proteasome in Streptomyces lividans and its influence on the production of heterologous proteins.png)
MicrobiologyOAAI
MicrobiologyIF0
OAAI分享收藏Review of Design Experiments Using the Taguchi Approach: 16 Steps to Product and Process Improvement
PREAI
分享收藏
查看更多内容