.png){kind=icon}
.png){kind=icon}
Return
A Dynamic Rectified Linear Activation Units
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2019.2959036.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep neural network regression models produce substantial gains in big data prediction systems. Multilayer perceptron neural (MPL) networks have more various properties than single-layer feedforward neural networks. A deeper neural network is more intelligent and sophisticated, which is one of the main research directions. However, the disappearing gradient is the primary problem that restricts the research. The appropriate activation function is one of the effective methods for solving this problem. A bold idea about activation functions emerged: if the activation function is different in two adjacent training epochs, the probability of the same gradient value will be small. We proposed a novel activation function whose shape can be changed dynamically in training. Our experimental results show that this activation function with dynamic characteristics can effectively avoid the disappearing gradient and can make the multilayer perceptron neural networks deeper.
Keywords:
Deep neural networks
convergent dynamic activation function
vanishing gradient
non-additional calculation
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.