.png){kind=icon}
返回
Visualizing and Understanding Contrastive Learning
.png){kind=icon}
OA
AI
DOI：10.1109/TIP.2023.3346295.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Contrastive learning has revolutionized the field of computer vision, learning rich representations from unlabeled data, which generalize well to diverse vision tasks. Consequently, it has become increasingly important to explain these approaches and understand their inner workings mechanisms. Given that contrastive models are trained with interdependent and interacting inputs and aim to learn invariance through data augmentation, the existing methods for explaining single-image systems (e.g., image classification models) are inadequate as they fail to account for these factors and typically assume independent inputs. Additionally, there is a lack of evaluation metrics designed to assess pairs of explanations, and no analytical studies have been conducted to investigate the effectiveness of different techniques used to explaining contrastive learning. In this work, we design visual explanation methods that contribute towards understanding similarity learning tasks from pairs of images. We further adapt existing metrics, used to evaluate visual explanations of image classification systems, to suit pairs of explanations and evaluate our proposed methods with these metrics. Finally, we present a thorough analysis of visual explainability methods for contrastive learning, establish their correlation with downstream tasks and demonstrate the potential of our approaches to investigate their merits and drawbacks.
Keyword:
Data models
Task analysis
Measurement
Computational modeling
Image classification
Data augmentation
Adaptation models
Explaining contrastive learning
explaining similarity networks
期刊
IF:
13.7
论文数:
1.0W
被引数:
8.4W
机构
引用论文
Awareness of activity limitations, goal setting and rehabilitation outcome in patients with brain injuries.png)
Brain Injury
Brain InjuryIF0
PREAI
分享收藏分享收藏Generation of hydroxyl radicals by urban suspended particulate air matter. The role of iron ions城市悬浮颗粒物产生羟基自由基。铁离子的作用:
PREAI
分享收藏Unmasking Clever Hans predictors and assessing what machines really learn揭开聪明的汉斯预测器并评估机器真正学到的东西NATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAI分享收藏
查看更多内容