.png){kind=icon}
返回
Understanding Double Descent Using VC-Theoretical Framework
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2024.3388873.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In spite of many successful applications of deep learning (DL) networks, theoretical understanding of their generalization capabilities and limitations remains limited. We present analysis of generalization performance of DL networks for classification under VC-theoretical framework. In particular, we analyze the so-called double descent phenomenon, when large overparameterized networks can generalize well, even when they perfectly memorize all available training data. This appears to contradict conventional statistical view that optimal model complexity should reflect an optimal balance between underfitting and overfitting, i.e., the bias-variance trade-off. We present VC-theoretical explanation of double descent phenomenon, under classification setting. Our theoretical explanation is supported by empirical modeling of double descent curves, using analytic VC-bounds, for several learning methods, such as support vector machine (SVM), least squares (LS), and multilayer perceptron classifiers. The proposed VC-theoretical approach enables better understanding of overparameterized estimators during second descent.
Keyword:
Complexity control
deep learning (DL)
double descent
generalization bounds
networks with random weights
structural risk minimization (SRM)
VC-dimension
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
暂无机构信息
引用论文
.png){kind=icon}
分享收藏Improved Particle Size Control for the Dispersion Polymerization of Methyl methacrylate in Supercritical Carbon Dioxide超临界二氧化碳中甲基丙烯酸甲酯分散聚合的改进粒度控制
OAAI
OAAI分享收藏分享收藏
查看更多内容