Multi-Level Contextual Prototype Modulation for Compositional Zero-Shot Learning

doi:10.1109/TIP.2025.3592560

Multi-Level Contextual Prototype Modulation for Compositional Zero-Shot Learning

2025-01-01

0

PRE

AI

DOI：10.1109/TIP.2025.3592560

Original

Original request for help

Save

Abstract

En 中文

Compositional Zero-Shot Learning (CZSL) aims to recognize unseen attribute-object compositions by leveraging prior knowledge of known primitives. However, real-world visual features of attributes and objects are often entangled, causing distribution shifts between seen and unseen combinations. Existing methods often ignore intrinsic variations and interactions among primitives, leading to poor feature discrimination and biased predictions. To address these challenges, we propose Multi-level Contextual Prototype Modulation (MCPM), a transformer-based framework with a hierarchical structure that effectively integrates attributes and objects to generate richer visual embeddings. At the feature level, we apply contrastive learning to improve discriminability across compositional tasks. At the prototype level, a subclass-driven modulator captures fine-grained attribute-object interactions, enabling better adaptation to long-tail distributions. Additionally, we introduce a Minority Attribute Enhancement (MAE) strategy that synthesizes virtual samples by mixing attribute classes, further mitigating data imbalance. Experiments on four benchmark datasets (MIT-States, C-GQA, UT-Zappos, and VAW-CZSL) show that MCPM brings significant performance improvements, verifying its effectiveness in complex composition scenes.

Keywords:

Compositional zero-shot learning

vision transformer

subclass discrimination

contrastive learning

data augmentation

Multi-Level Contextual Prototype Modulation for Compositional Zero-Shot Learning

Abstract

Journal

Organization