logo
科言猫学术研究的AI总结
language
手机端手机端
  • Phone Feature微信小程序
简体中文简体中文
notice消息通知
博客博客
delete
arrow
返回

Investigating Self-Supervised Methods for Label-Efficient Learning

delete2025-03-10
delete0
delete
OA
AI
S
Srinivasa Rao Nandam *
Z
Zhenhua Feng
J
Josef Kittler
M
Muhammed Awais
DOI：10.1007/s11263-025-02397-4delete
delete原文链接
delete原文求助
delete分享
delete收藏
摘要

摘要

En 中文
视觉变压器与自监督学习相结合，已使模型能够在大型数据集上扩展至多个下游任务，包括分类、分割和检测。然而，这些模型在多个下游任务中的低样本学习潜力仍 largely 未被探索。在本工作中，我们系统地考察了不同的自监督预文本任务，即对比学习、聚类和掩码图像建模，通过比较不同的预训练模型来评估其低样本能力。此外，我们探讨了各种崩溃避免技术，如中心化、ME-MAX 和 Sinkhorn，对下游任务的影响。基于我们的详细分析，我们提出了一种框架，将掩码图像建模和聚类作为预文本任务。该框架在所有考察的低样本下游任务中表现出更优性能，包括多类分类、多标签分类和语义分割。此外，在大型数据集上测试模型时，我们展示了各种任务上的性能提升。
Keyword:
Vision transformers
Self-supervised learning
Low-shot learning
Masked image modelling

期刊

International Journal of Computer Vision 封面图
International Journal of Computer Vision
IF:
9.3
论文数:
3.9K
被引数:
2.8W

机构

U
Univ Surrey
学者数:
405
论文数: 295
被引数: 111
引用论文

引用论文

ImageNet: A large-scale hierarchical image databaseImageNet: 一个大规模的分层图像数据库
err2009 IEEE Conference on Computer Vision and Pattern Recognition
IF0
err2009-06-01
err0
PREAI
errJia Deng; Wei Dong; Richard Socher; Li-Jia Li; Kai Li; Li Fei-Fei
err分享
err收藏
PANet: Few-Shot Image Semantic Segmentation With Prototype AlignmentPANet: 具有原型对齐的少镜头图像语义分割
err2019 IEEE/CVF International Conference on Computer Vision (ICCV)
IF0
err2019-10-01
err0
errOAAI
errKaixin Wang; Jun Hao Liew; Yingtian Zou; Daquan Zhou; Jiashi Feng
err分享
err收藏
Emerging Properties in Self-Supervised Vision Transformers自监督视觉变换器中的新兴特性
err2021 IEEE/CVF International Conference on Computer Vision (ICCV)
IF0
err2021-10-01
err0
errOAAI
errMathilde Caron; Hugo Touvron; Ishan Misra; Herve Jegou; Julien Mairal; Piotr Bojanowski; Armand Joulin
err分享
err收藏
Automated Flower Classification over a Large Number of Classes在大量课程上进行自动花卉分类
err2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing
IF0
err2008-12-01
err0
PREAI
errMaria-Elena Nilsback; Andrew Zisserman
err分享
err收藏
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
errINTERNATIONAL JOURNAL OF COMPUTER VISION
IF9.3
err2017-02-06
err3.1K
errOAAI
errKrishna, Ranjay; Zhu, Yuke; Groth, Oliver; Johnson, Justin; Hata, Kenji; Kravitz, Joshua; Chen, Stephanie; Kalantidis, Yannis; Li, Li-Jia; Shamma, David A.; Bernstein, Michael S.; Li Fei-Fei
err分享
err收藏
Barely-Supervised Learning: Semi-supervised Learning with Very Few Labeled Images
errProceedings of the AAAI Conference on Artificial Intelligence
IF0
err2022-06-28
err0
errOAAI
errThomas Lucas; Philippe Weinzaepfel; Gregory Rogez
err分享
err收藏
Exploring Simple Siamese Representation Learning探索简单的暹罗表征学习
err2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
IF0
err2021-06-01
err0
errOAAI
errXinlei Chen; Kaiming He
err分享
err收藏
An Empirical Study of Training Self-Supervised Vision Transformers训练自监督视觉转换器的实证研究
err2021 IEEE/CVF International Conference on Computer Vision (ICCV)
IF0
err2021-10-01
err0
errOAAI
errXinlei Chen; Saining Xie; Kaiming He
err分享
err收藏
Cats and dogs猫和狗
err2012 IEEE Conference on Computer Vision and Pattern Recognition
IF0
err2012-06-01
err0
PREAI
errO. M. Parkhi; A. Vedaldi; A. Zisserman; C. V. Jawahar
err分享
err收藏
Learning What Not to Segment: A New Perspective on Few-Shot Segmentation学习不分割的内容: 少镜头分割的新视角
err2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
IF0
err2022-06-01
err0
errOAAI
errChunbo Lang; Gong Cheng; Binfei Tu; Junwei Han
err分享
err收藏
学者 查看更多内容