.png){kind=icon}
返回
Improving ViT interpretability with patch-level mask prediction
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patrec.2024.11.018.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Vision Transformers (ViTs) have demonstrated remarkable performances on various computer vision tasks. Attention scores are often used to explain the decision-making process of ViTs, showing which tokens are more important than others. However, the attention scores have several limitations as an explanation for ViT, such as conflicting with other explainable methods or highlighting unrelated tokens. In order to address this limitation, we propose a novel method for generating a visual explanation map from ViTs. Unlike previous approaches that rely on attention scores, our method leverages ViT features and conducts a single forward pass through our Patch-level Mask prediction (PM) module. Our visual explanation map provides class-dependent and probabilistic interpretation that can identify crucial regions of model decisions. Experimental results demonstrate that our approach outperforms previous techniques in both classification and interpretability aspects. Additionally, it can be applied to the weakly-supervised object localization (WSOL) tasks using pseudo mask labels. Our method requires no extra parameters and necessitates minimal locality supervision, utilizing less than 1% of the ImageNet-1k training dataset.
Keyword:
Vision Transformer
Interpretability
Weak supervision
Object localization
期刊
IF:
3.3
论文数:
8.0K
被引数:
1.6W
机构
引用论文
.png){kind=icon}
Oxidative stress in liver of grass carp Ctenopharyngodon idella naturally infected with Saprolegnia parasitica and its influence on disease pathogenesis天然感染水蚤的草鱼肝脏氧化应激及其对疾病发病机制的影响
PREAI
分享收藏
查看更多内容