Shakedrop Regularization for Deep Residual Learning

Yoshihiro Yamada

doi:10.1109/ACCESS.2019.2960566

2019-01-01

DOI：10.1109/ACCESS.2019.2960566

原文链接

查看原文

摘要

En 中文

Overfitting is a crucial problem in deep neural networks, even in the latest network architectures. In this paper, to relieve the overfitting effect of ResNet and its improvements (i.e., Wide ResNet, PyramidNet, and ResNeXt), we propose a new regularization method called ShakeDrop regularization. ShakeDrop is inspired by Shake-Shake, which is an effective regularization method, but can be applied to ResNeXt only. ShakeDrop is more effective than Shake-Shake and can be applied not only to ResNeXt but also ResNet, Wide ResNet, and PyramidNet. An important key is to achieve stability of training. Because effective regularization often causes unstable training, we introduce a training stabilizer, which is an unusual use of an existing regularizer. Through experiments under various conditions, we demonstrate the conditions under which ShakeDrop works well.

Keyword:

Computer vision

image classification

neural networks

AI总结

对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。

期刊

IEEE Access

IF:

3.6

论文数:

9.8W

被引数:

29.4W

机构

Osaka Metropolitan University

学者数:

1.2W

论文数: 9.7K

被引数: 1.6K

引用论文

Discovery of anticancer agents of diverse natural origin

Pure and Applied Chemistry

IF0

2013-10-31

OAAI

A. Douglas Kinghorn; Esperanza J. Carcache de Blanco; Hee-Byung Chai; Jimmy Orjala; Norman R. Farnsworth; D. Doel Soejarto; Nicholas H. Oberlies; Mansukh C. Wani; David J. Kroll; Cedric J. Pearce; Steven M. Swanson; Robert A. Kramer; William C. Rose; Craig R. Fairchild; Gregory D. Vite; Stuart Emanuel; David Jarjoura; Frederick O. Cope

Thermal comfort, perceived air quality, and cognitive performance when personally controlled air movement is used by tropically acclimatized persons 当热带适应的人使用个人控制的空气运动时，热舒适性，感知的空气质量和认知表现

Indoor Air

IF0

2016-11-12

OAAI

S. Schiavon; B. Yang; Y. Donner; V. W.-C. Chang; W. W. Nazaroff

Synthesis of n-type semiconducting diamond film using diphosphorus pentaoxide as the doping source 以五氧化二磷为掺杂源合成n型半导体金刚石膜

Applied Physics A Solids and Surfaces

IF0

1990-10-01

PREAI

Ken Okano; Hideo Kiyota; Tatsuya Iwasaki; Yoshitaka Nakamura; Yukio Akiba; Tateki Kurosu; Masamori Iida; Terutaro Nakamura