.png){kind=icon}
返回
Neural Network Gaussian Processes by Increasing Depth
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2022.3185375.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recent years have witnessed an increasing interest in the correspondence between infinitely wide networks and Gaussian processes. Despite the effectiveness and elegance of the current neural network Gaussian process theory, to the best of our knowledge, all the neural network Gaussian processes (NNGPs) are essentially induced by increasing width. However, in the era of deep learning, what concerns us more regarding a neural network is its depth as well as how depth impacts the behaviors of a network. Inspired by a width-depth symmetry consideration, we use a shortcut network to show that increasing the depth of a neural network can also give rise to a Gaussian process, which is a valuable addition to the existing theory and contributes to revealing the true picture of deep learning. Beyond the proposed Gaussian process by depth, we theoretically characterize its uniform tightness property and the smallest eigenvalue of the Gaussian process kernel. These characterizations can not only enhance our understanding of the proposed depth-induced Gaussian process but also pave the way for future applications. Lastly, we examine the performance of the proposed Gaussian process by regression experiments on two benchmark datasets.
Keyword:
Artificial neural networks
Gaussian processes
Deep learning
Neurons
Kernel
Topology
Network topology
Deep neural networks
generalized central limit theorem
neural network Gaussian processes (NNGPs)
smallest eigenvalue
uniform tightness
weak dependence
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
查看更多机构
引用论文
.png){kind=icon}
分享收藏Synthesis of dimethacrylates monomers with low polymerization shrinkage and its application in dental composites materials低聚合收缩二甲基丙烯酸酯单体的合成及其在牙科复合材料中的应用
PREAI
分享收藏 没有更多内容