.png){kind=icon}
返回
Projection based weight normalization: Efficient method for optimization on oblique manifold in DNNs
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2020.107317.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Optimizing deep neural networks (DNNs) often suffers from the ill-conditioned problem. We observe that the scaling based weight space symmetry (SBWSS) in rectified nonlinear network will cause this negative effect. Therefore, we propose to constrain the incoming weights of each neuron to be unit-norm, which is formulated as an optimization problem over the Oblique manifold. A simple yet efficient method referred to as projection based weight normalization (PBWN) is also developed to solve this problem. This proposed method has the property of regularization and collaborates well with the commonly used batch normalization technique. We conduct comprehensive experiments on several widely-used image datasets including CIFAR-10, CIFAR-100, SVHN and ImageNet for supervised learning over the state-of-the-art neural networks. The experimental results show that our method is able to improve the performance of different architectures consistently. We also apply our method to Ladder network for semi-supervised learning on permutation invariant MNIST dataset, and our method achievers the state-of-the-art methods: we obtain test errors as 2.52%, 1.06%, and 0.91% with only 20, 50, and 100 labeled samples, respectively. (C) 2020 Elsevier Ltd. All rights reserved.
Keyword:
Deep learning
Weight normalization
Oblique manifold
Image classification
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Active multi-kernel domain adaptation for hyperspectral image classification.png)
PATTERN RECOGNITION
OAAI
PATTERN RECOGNITIONIF7.6
OAAI分享收藏Spatio-temporal deformable 3D ConvNets with attention for action recognition用于动作识别的具有注意的时空可变形3D ConvNetsPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏 没有更多内容
没有更多内容