.png){kind=icon}
返回
MinEnt: Minimum entropy for self-supervised representation learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2023.109364.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Self-supervised representation learning is becoming more , more popular due to its superior perfor-mance. According to the information entropy theory, the smaller the information entropy of a feature, the more certain it is and the less redundant it is. Based on this, we propose a simple yet effective self-supervised representation learning method via Minimum Entropy (MinEnt). From the perspective of reducing information entropy, our MinEnt takes the output of the projector towards its nearest minimum entropy as the optimization target. The core of our MinEnt consists of three important steps: 1) normal-ize along the batch dimension to avoid model collapse, 2) compute the nearest minimum entropy to get the target, 3) compute the loss and backpropagate to optimize the network. Our MinEnt can learn ef-ficient representations, even without the need for techniques such as negative sample pairs, predictors, momentum encoders, cross-correlation matrices, etc. Experimental results on four widely used datasets show that our method achieves competitive results in a simple manner.(c) 2023 Elsevier Ltd. All rights reserved.
Keyword:
Self-supervised learning
Minimum entropy
Unsupervised representation learning
Image classification
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Unsupervised deep clustering via contractive feature representation and focal loss.png)
PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏Dual space latent representation learning for unsupervised feature selection用于无监督特征选择的对偶空间潜在表示学习PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏Deep anomaly detection with self-supervised learning and adversarial training基于自监督学习和对抗训练的深度异常检测PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏Unsupervised representation learning by discovering reliable image relationsPATTERN RECOGNITION
OAAI
PATTERN RECOGNITIONIF7.6
OAAI分享收藏
查看更多内容