.png){kind=icon}
Return
A note on factor normalization for deep neural network models
.png){kind=icon}
OA
AI
DOI：10.1038/s41598-022-09910-6.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep neural network (DNN) models often involve high-dimensional features. In most cases, these high-dimensional features can be decomposed into two parts: a low-dimensional factor and residual features with much-reduced variability and inter-feature correlation. This decomposition has several interesting theoretical implications for DNN training. Based on these implications, we develop a novel factor normalization method for better performance. The proposed method leads to a new deep learning model with two important characteristics. First, it allows factor-related feature extraction, and second, it allows for adaptive learning rates for factors and residuals. These model features improve the convergence speed on both training and testing datasets. Multiple empirical experiments are presented to demonstrate the model's superior performance.
Journal
IF:
3.9
Papers:
28.3W
Citations:
83.5W
Organization
Cited Papers
Facile Synthesis, Characterization, Nanocrystal Growth and Photoluminescence Properties of GeS Nanowires.png)
Nano
NanoIF0
PREAI
ShareSaveMechanical, but not infective, pacemaker erosion may be successfully managed by re-implantation of pacemakers.HeartOAAI
HeartIF0
OAAIShareSaveShareSaveShareSave
View more