.png){kind=icon}
.png){kind=icon}
Return
A hierarchical evolution of neural architecture search method based on state transition algorithm
.png){kind=icon}
PRE
AI
DOI：10.1007/s13042-023-01794-w.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Among different search strategies to realize neural architecture search (NAS), evolutionary algorithms (EAs) have gained much attention due to their global optimization capability. However, the large number of performance evaluation procedure makes the EA-based methods extremely time-consuming. To address this issue, an efficient framework called HENA (Hierarchical Evolution of Neural Architecture) is proposed in this paper. In HENA, NAS is hierarchically divided into two continuous phases, candidate operation search and connection relationship search. For this purpose, a supernet is defined to subsume the whole search space, where all weights can be inherited by all architectures. A novel evolutionary algorithm called state transition algorithm (STA) is used to traverse the search space continuously. In the first phase, several sampled architectures are trained on mini-group data to search better operation for different positions within the network. In the second phase, better connection relationship among operations can be directly determined without training via inheriting trained weights. Finally, the proposed method is evaluated on the widely used datasets, and the experimental results show that (1) the architecture learned by HENA obtains state-of-the-art performance (2.93% and 20.44% error rate on CIFAR-10 and CIFAR-100 respectively). (2) the learned architecture achieves 24.8% top-1 error on ImageNet dataset.
Keywords:
Deep neural networks
Neural architecture search
State transition algorithm
Evolutionary algorithm
Journal
IF:
2.7
Papers:
3.1K
Citations:
5.6K