.png){kind=icon}
.png){kind=icon}
Return
Auto CNN classifier based on knowledge transferred from self-supervised model
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-023-04598-1.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Training with unlabeled datasets using self-supervised models has the edge over training with labeled datasets, reducing human effort, and no need for annotated data during training. The knowledge obtained from the pre-trained self-supervised model can be transferred to an image classifier as a target task. Transferring knowledge from source to target task requires tuning a large number of hyperparameters, such as number of CNN layers, number of filters in each CNN layer, kernel size, stride, number of fully connected (FC) layers, units in each FC layer, dropouts, learning rate, and many more. An efficient process is required to automatically tune these hyperparameters to reduce the manual efforts and enhance the performance of the self-supervised model. This paper uses a pre-trained self-supervised model to transfer knowledge for image classification. We propose an efficient Bayesian optimization-based method for automatically tuning (autotuning) the hyperparameters of the self-supervised model during the knowledge transfer. The proposed autotuned image classifier consists of a few CNN layers followed by an FC layer. Finally, we use a softmax layer to obtain the probability of classes for the images. To evaluate the performance of the proposed method, state-of-the-art self-supervised models such as SimClr, SWAV, and SWAV 2x are used to extract the learned features. According to our experiments on three benchmarked datasets (CIFAR-10, CIFAR-100, and Tiny Imagenet datasets), the proposed method not only enhances the performances of the self-supervised models, but also provides state-of-the-art results.
Keywords:
Neural architecture search
Autotuning
Self-supervised learning
Hyperparameter optimization
BO-TPE
Journal
IF:
3.5
Papers:
7.5K
Citations:
1.7W
Organization
No organization information available