.png){kind=icon}
返回
Newton Methods for Convolutional Neural Networks
.png){kind=icon}
OA
AI
DOI：10.1145/3368271.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep learning involves a difficult non-convex optimization problem, which is often solved by stochastic gradient (SG) methods. While SG is usually effective, it may not be robust in some situations. Recently, Newton methods have been investigated as an alternative optimization technique, but most existing studies consider only fully connected feedforward neural networks. These studies do not investigate some more commonly used networks such as Convolutional Neural Networks (CNN). One reason is that Newton methods for CNN involve complicated operations, and so far no works have conducted a thorough investigation. In this work, we give details of all building blocks, including the evaluation of function, gradient, Jacobian, and Gauss-Newton matrix-vector products. These basic components are very important not only for practical implementation but also for developing variants of Newton methods for CNN. We show that an efficient MATLAB implementation can be done in just several hundred lines of code. Preliminary experiments indicate that Newton methods are less sensitive to parameters than the stochastic gradient approach.
Keyword:
Convolution neural networks
newton methods
large-scale classification
subsampled Hessian
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.6
论文数:
1.5K
被引数:
6.2K
机构
引用论文
分享收藏Investigation of Ti mesh-supported anodes for direct borohydride fuel cells直接硼氢化物燃料电池用钛网支撑阳极的研究
PREAI
分享收藏
An experimental study on the environmental impact of hydrogen and natural gas blend burning.png)
Chemosphere
ChemosphereIF0
PREAI
分享收藏
查看更多内容