.png){kind=icon}
返回
Self-Supervised Representation Learning for Evolutionary Neural Architecture Search
.png){kind=icon}
OA
AI
DOI：10.1109/MCI.2021.3084415.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently proposed neural architecture search (NAS) algorithms adopt neural predictors to accelerate architecture search. The capability of neural predictors to accurately predict the performance metrics of the neural architecture is critical to NAS, but obtaining training datasets for neural predictors is often time-consuming. How to obtain a neural predictor with high prediction accuracy using a small amount of training data is a central problem to neural predictor-based NAS. Here, a new architecture encoding scheme is first devised to calculate the graph edit distance of neural architectures, which overcomes the drawbacks of existing vector-based architecture encoding schemes.To enhance the predictive performance of neural predictors, two self-supervised learning methods are proposed to pre-train the architecture embedding part of neural predictors to generate a meaningful representation of neural architectures. The first method designs a graph neural network-based model with two independent branches and utilizes the graph edit distance of two different neural architectures as a supervision to force the model to generate meaningful architecture representations. Inspired by contrastive learning, the second method presents a new contrastive learning algorithm that utilizes a central feature vector as a proxy to contrast positive pairs against negative pairs. Experimental results illustrate that the pre-trained neural predictors can achieve comparable or superior performance compared with their supervised counterparts using only half of the training samples. The effectiveness of the proposed methods is further validated by integrating the pre-trained neural predictors into a neural predictor guided evolutionary neural architecture search (NPENAS) algorithm, which achieves state-of-the-art -performance on NASBench-101, NASBench-201, and DARTS benchmarks.
期刊
IF:
11.2
论文数:
613
被引数:
3.1K
机构
引用论文
Concomitant mobilization of plasma cells and hematopoietic progenitors into peripheral blood of multiple myeloma patients: positive selection and transplantation of enriched CD34+ cells to remove circulating tumor cells.png)
BloodOAAI
BloodIF0
OAAI分享收藏Admixture Effects on the Rheological/Mechanical Behavior and Micro-Structure Evolution of Alkali-Activated Slag BackfillsMineralsOAAI
MineralsIF0
OAAI分享收藏Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification使用遗传算法自动设计CNN架构进行图像分类OAAI
OAAI分享收藏分享收藏Transplantation of CD34+ peripheral blood progenitor cells after high- dose chemotherapy for patients with advanced multiple myelomaBloodOAAI
BloodIF0
OAAI分享收藏
查看更多内容