.png){kind=icon}
返回
Semantic-Aware Message Broadcasting for Efficient Unsupervised Domain Adaptation
.png){kind=icon}
OA
AI
DOI：10.1109/TIP.2024.3437212.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Vision transformer has demonstrated great potential in abundant vision tasks. However, it also inevitably suffers from poor generalization capability when the distribution shift occurs in testing (i.e., out-of-distribution data). To mitigate this issue, we propose a novel method, Semantic-aware Message Broadcasting (SAMB), which enables more informative and flexible feature alignment for unsupervised domain adaptation (UDA). Particularly, we study the attention module in the vision transformer and notice that the alignment space using one global class token lacks enough flexibility, where it interacts information with all image tokens in the same manner but ignores the rich semantics of different regions. In this paper, we aim to improve the richness of the alignment features by enabling semantic-aware adaptive message broadcasting. Particularly, we introduce a group of learned group tokens as nodes to aggregate the global information from all image tokens, but encourage different group tokens to adaptively focus on the message broadcasting to different semantic regions. In this way, our message broadcasting encourages the group tokens to learn more informative and diverse information for effective domain alignment. Moreover, we systematically study the effects of adversarial-based feature alignment (ADA) and pseudo-label based self-training (PST) on UDA. We find that one simple two-stage training strategy with the cooperation of ADA and PST can further improve the adaptation capability of the vision transformer. Extensive experiments on DomainNet, OfficeHome, and VisDA-2017 demonstrate the effectiveness of our methods for UDA.
Keyword:
Transformers
Broadcasting
Training
Computer vision
Task analysis
Semantics
Adaptation models
Unsupervised domain adaptation
vision transformer
semantic-aware message broadcasting
adversarial-based feature alignment
self-training
期刊
IF:
13.7
论文数:
1.0W
被引数:
8.4W
机构
引用论文
分享收藏Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
PREAI
分享收藏
Using SEPIC Topology for Improving Power Factor in Distributed Power Supply Systems.png)
EPE Journal
EPE JournalIF0
PREAI
分享收藏Separation of high-density polyethylene/poly(ethylene terephthalate)/poly(vinyl chloride) mixtures based on differences in their hardness基于硬度差异的高密度聚乙烯/聚对苯二甲酸乙二醇酯/聚氯乙烯混合物的分离Polimery
PolimeryIF0
PREAI
分享收藏
查看更多内容