.png){kind=icon}
返回
Correspondence between neuroevolution and gradient descent
.png){kind=icon}
OA
AI
DOI：10.1038/s41467-021-26568-2.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Gradient-based and non-gradient-based methods for training neural networks are usually considered to be fundamentally different. The authors derive, and illustrate numerically, an analytic equivalence between the dynamics of neural network training under conditioned stochastic mutations, and under gradient descent. We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations, for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.
Keyword:
MONTE-CARLO METHOD
NEURAL-NETWORKS
DYNAMICS
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
15.7
论文数:
9.4W
被引数:
91.2W
机构
引用论文
.png){kind=icon}
Real-time and offline techniques for identifying obstructive sleep apnea patients用于识别阻塞性睡眠呼吸暂停患者的实时和离线技术
PREAI
分享收藏分享收藏
查看更多内容