.png){kind=icon}
Return
Grafting constructive algorithm in feedforward neural network learning
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-022-04082-2.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Constructive algorithm provides a gradually building mechanism by increasing nodes from zero. By this means, the neural network can independently and efficiently determine its structure. However, this mechanism has an essential issue: the algorithm that adds nodes one by one is too greedy to keep an efficient construction way and the global optimal solution may be missed. Therefore, this paper proposes a novel grafting mechanism to add block nodes of any number by training a sub-network during the construction. Then, a fast-training approach of the added block neurons is presented by selecting a small sub-network from the large initialized network and the corresponding grafting constructive algorithm (GCA) is established. To obtain a compact network structure, a fine-tuning scheme is developed according to GCA to adjust all parameters as a hybrid fashion and the hidden weights are extended to deal with matrix input in image classification. The experimental results on regression and classification tasks demonstrate that the proposed GCA can achieve a more compact network than other constructive algorithms and a faster error convergence rate than traditional gradient-based optimization algorithms.
Keywords:
Constructive algorithm
Grafting mechanism
Ultra-wide network
Feedforward neural network learning
Journal
IF:
3.5
Papers:
7.7K
Citations:
1.7W
Organization
Cited Papers
ShareSave
ShareSaveShareSaveApplication of cascade correlation networks for structures to chemistry.png)
APPLIED INTELLIGENCE
APPLIED INTELLIGENCEIF3.5
PREAI
ShareSave
View more