.png){kind=icon}
返回
Aligning Image Semantics and Label Concepts for Image Multi-Label Classification
.png){kind=icon}
PRE
AI
DOI：10.1145/3550278.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Image multi-label classification task is mainly to correctly predict multiple object categories in the images. To capture the correlation between labels, graph convolution network based methods have to manually count the label co-occurrence probability from training data to construct a pre-defined graph as the input of graph network, which is inflexible and may degrade model generalizability. Moreover, most of the current methods cannot effectively align the learned salient object features with the label concepts, so that the predicted results of model may not be consistent with the image content. Therefore, how to learn the salient semantic features of images and capture the correlation between labels, and then effectively align them is one of the key to improve the performance of image multi-label classification task. To this end, we propose a novel image multi-label classification framework which aims to align Image Semantics with Label Concepts (ISLC). Specifically, we propose a residual encoder to learn salient object features in the images, and exploit the selfattention layer in aligned decoder to automatically capture the correlation between labels. Then, we leverage the cross-attention layers in aligned decoder to align image semantic features with label concepts, so as to make the labels predicted by model more consistent with image content. Finally, the output features of the last layer of residual encoder and aligned decoder are fused to obtain the final output feature for classification. The proposed ISLC model achieves good performance on various prevalent multi-label image datasets such as MS-COCO 2014, PASCAL VOC 2007, VG-500, and NUS-WIDE with 87.2%, 96.9%, 39.4%, and 64.2%, respectively.
Keyword:
Multi-label classification
transformer
self-attention
salient features
label correlation
visual analysis
期刊
IF:
6
论文数:
2.0K
被引数:
5.4K
机构
引用论文
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
OAAI
OAAI分享收藏分享收藏Synthesis of n-type semiconducting diamond film using diphosphorus pentaoxide as the doping source以五氧化二磷为掺杂源合成n型半导体金刚石膜
PREAI
分享收藏The role of phase interface energy in martensitic transformations: A lattice Monte-Carlo simulationOAAI
OAAI分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏A disintegrin and metalloprotease 10 (ADAM10) is a central regulator of murine liver tissue homeostasis.png)
Oncotarget
OAAI
OncotargetIF0
OAAI分享收藏A Semi-supervised Learning Approach Based on Adaptive Weighted Fusion for Automatic Image Annotation基于自适应加权融合的半监督学习图像自动标注方法
PREAI
分享收藏
查看更多内容