.png){kind=icon}
返回
Generalization and Expressivity for Deep Nets
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2018.2868980.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Along with the rapid development of deep learning in practice, theoretical explanations for its success become urgent. Generalization and expressivity are two widely used measurements to quantify theoretical behaviors of deep nets. The expressivity focuses on finding functions expressible by deep nets but cannot be approximated by shallow nets with similar number of neurons. It usually implies the large capacity. The generalization aims at deriving fast learning rate for deep nets. It usually requires small capacity to reduce the variance. Different from previous studies on deep nets, pursuing either expressivity or generalization, we consider both the factors to explore theoretical advantages of deep nets. For this purpose, we construct a deep net with two hidden layers possessing excellent expressivity in terms of localized and sparse approximation. Then, utilizing the well known covering number to measure the capacity, we find that deep nets possess excellent expressive power (measured by localized and sparse approximation) without essentially enlarging the capacity of shallow nets. As a consequence, we derive near-optimal learning rates for implementing empirical risk minimization on deep nets. These results theoretically exhibit advantages of deep nets from the learning theory viewpoint.
Keyword:
Deep learning
expressivity
generalization
learning theory
localized approximation
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
8.9
论文数:
7.5K
被引数:
7.2W
机构
引用论文
分享收藏Improved Particle Size Control for the Dispersion Polymerization of Methyl methacrylate in Supercritical Carbon Dioxide超临界二氧化碳中甲基丙烯酸甲酯分散聚合的改进粒度控制OAAI
OAAI分享收藏
Probabilistic lower bounds for approximation by shallow perceptron networks.png)
NEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
分享收藏
查看更多内容