.png){kind=icon}
Return
Multi-semantic depth collaborative multi-modality image fusion network
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2026.115756.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multi-modality image fusion aims to integrate complementary data from diverse sensors to achieve a more comprehensive representation of the imaging scene. Existing methods primarily focus on surface-level interactions between pixel-level features and visual or textual semantics, and do not fully integrate the global consistency between text modalities, image fusion, and downstream tasks. The lack of this deep collaboration between multiple semantics cannot effectively capture multi-modal information and the fine-grained association between fusion and downstream tasks. Therefore, we propose a Multi-Semantic depth collaborative Image Fusion (MSIF) network. Firstly, different from the traditional cascaded network structure, MSIF adopts a dual-task parallel structure that efficiently establishes deep correlations between the fusion and segmentation tasks. Second, MSIF enhances the semantic consistency between fusion and downstream tasks by leveraging deeper and detailed semantic information in text modalities to guide image fusion and segmentation. Additionally, MSIF uses the designed Multi-Semantic Deep Synergistic Module (MSDSM) to deeply integrate text features, pixel-level features and segmentation semantic features, so as to realize the effective fusion of multi-modal information and meet the semantic requirements of downstream tasks. Finally, the combined semantic features are fed into a fusion decoder to yield the final fusion result. Extensive quantitative and qualitative experiments demonstrate that our approach achieves state-of-the-art (SOTA) performance on multimodal datasets.
Journal
K
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
View more organizations
Cited Papers
ShareSaveAT-GAN: A generative adversarial network with attention and transition for infrared and visible image fusion.png)
INFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
ShareSavePIAFusion: A progressive infrared and visible image fusion network based on illumination awareINFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
ShareSaveRXDNFuse: A aggregated residual dense network for infrared and visible image fusionINFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
ShareSaveShareSaveShareSaveShareSaveRethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelityINFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
ShareSave
View more