.png){kind=icon}
.png){kind=icon}
Return
Exploring inter-and intra-modal relations in compositional zero-shot learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2025.130213.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Compositional Zero-Shot Learning (CZSL) aims to recognize unknown compositions by leveraging learned concepts of states and objects. Prior methods have typically emphasized either inter-modal relation for multi-modal fusion, ignoring the entanglement within state-object pairs, or solely intra-modal relation for enhancing representations, neglecting the association between vision and language domains. To tackle these limitations, we propose a CZSL framework that simultaneously learns inter-and intra-modal relations to improve image-label alignment. Firstly, we explore inter-modal relation to enable image features to grasp the cross-modal information from states and objects. The image-text fusion method facilitates the modeling of text-aware image features and image-aware text features, improving the model's compositional recognition capability. Secondly, due to the contextuality within state-object pairs, we further explore intra-modal relation to exploit semantic information from various representation subspaces, facilitating the comprehensive semantic expression of text features. Moreover, we propose a composition fusion module to establish semantic entanglement within state-object compositions. Extensive experiments demonstrate that our method significantly surpasses the state-of-the-art methods in both closed-world and open-world settings.
Keywords:
Compositional zero-shot learning
Multimodal learning
Inter-modal relation
Intra-modal relation
Compositional recognition