Modality-Specific Prototypes for Disentangled Reconstruction in modality-missing object re-identification

doi:10.1016/j.patcog.2026.114641

Modality-Specific Prototypes for Disentangled Reconstruction in modality-missing object re-identification

2026-08-14

0

PRE

AI

DOI：10.1016/j.patcog.2026.114641

Original

Original request for help

Save

Abstract

En 中文

• Prototype-based disentangled reconstruction for modality-missing object Re-ID. • Adaptive prototype disentanglement with orthogonality and entropy-guided updates. • Hierarchical prototype injection for controllable two-stage modality reconstruction. • Strong results on four multi-modal Re-ID benchmarks in complete and missing settings. Abstract The common strategy for handling modality-missing in object re-identification (Re-ID) is to transform the available modalities into the missing one. However, direct feature-level or image-level transformation may suffer from the entanglement of modality-specific information and identity information, which leads to discrepancies in the reconstructed features in terms of both modality and identity. To address this issue, we propose the Modality-Specific Prototypes for Disentangled Reconstruction Network (PDRNet), which learns modality-specific priors and identity-discriminative residual representations while mitigating the adverse impact of information entanglement during the reconstruction process. This enables robust object Re-ID under missing-modality conditions. Specifically, we introduce modality prototypes to characterize modality-specific information and design an Adaptive Prototype-driven Disentanglement (APD) module. This module encourages the disentanglement between modality-specific components and identity-discriminative residual features through an orthogonality-based decorrelation constraint, thereby reducing their mutual interference. In addition, an entropy-distribution-based adaptive update strategy is employed to enable the prototypes to dynamically reflect modality statistics. Furthermore, we propose a Hierarchical Prototype Injection (HPI) module, which utilizes the missing-modality prototype in two stages during reconstruction. The prototype is first injected as a condition to guide the content reconstruction of identity representations, and then re-injected and refined through a feed forward network, achieving controlled and progressive modality-style injection while preserving identity discrimination and restoring modality consistency. Extensive experiments on multiple multi-modal object re-identification datasets demonstrate that the proposed PDRNet significantly outperforms existing methods under missing-modality scenarios. The code is publicly available at: https://github.com/skye-1201/PDRNet.

Keywords:

Object re-identification

Multi-modal learning

Prototype learning

Modality-missing challenge

Modality-Specific Prototypes for Disentangled Reconstruction in modality-missing object re-identification

Abstract

Journal

Organization