.png){kind=icon}
Return
Multi-Modal Primitive Retrieval for Compositional Zero-Shot Learning
.png){kind=icon}
PRE
AI
DOI：10.1007/s11263-026-02763-w.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Compositional generalization, understanding unseen combinations composed of seen primitives, is one of the fundamental properties of human intelligence. Aiming to evaluate such ability of vision models, compositional zero-shot learning (CZSL) requires recognizing unseen attribute-object compositions by learning from seen compositions. It’s essential for CZSL to compose the learned knowledge of seen primitives, i.e., attributes or objects, into novel compositions. In this work, we propose a retrieval-augmented method to explicitly retrieve knowledge of seen primitives from both vision domain and language domain, for compositional zero-shot learning. Our method augments standard multi-path classification methods with retrieval modules. Specifically, we first construct several databases storing abundant and diverse primitive knowledge, including the attribute and object representations of training images, and textual representations for descriptions of attributes and objects, respectively. For an input training/testing image, we use visual and textual retrieval modules to retrieve representations of relevant training images and text descriptions with the same attribute and object, respectively. The primitive representations and image representation of the input image are augmented by using the retrieved representations, for composition recognition. By referencing semantically similar images and texts, the proposed method is capable of recalling knowledge of seen primitives for compositional generalization. Experiments on three widely used datasets show the effectiveness of the proposed method.
Keywords:
Multi-Modality
Retrieval Augmentation
Compositional Zero-Shot Learning
Compositional Generalization
Journal
IF:
9.3
Papers:
3.9K
Citations:
2.8W
Organization
Cited Papers
.png){kind=icon}
ShareSave
ShareSave
ShareSaveShareSaveShareSave
View more