.png){kind=icon}
返回
Motion perception-driven multimodal self-supervised video object segmentation
.png){kind=icon}
PRE
AI
DOI：10.1007/s00371-024-03597-8.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Unsupervised video object segmentation segments foreground objects from videos without annotations. However, existing methods rely mainly on a single modality to process motion information and perform poorly with occlusions and static objects. To address this problem, we propose a multimodal motion perception network (M2PNet) based on self-supervised training for completely unlabeled video object segmentation tasks. M2PNet adopts a unique dual-path encoder-decoder structure to model spatial and temporal features to capture richer spatiotemporal information. Specifically, the spatial path computes spatial contrast matrices based on coattention mechanisms to enhance motion-related region correlations and accurately represent motion areas. The temporal path first utilizes residual connections and attention mechanisms to strengthen fused representations of different modalities and then iteratively learns motion change patterns based on slot attention to capture motion characteristics. In addition, we innovatively use residual maps to capture subtle interframe changes and build connections between different modal features. Extensive experiments on public datasets demonstrate that our multimodal method achieves significant improvements over other methods, demonstrating the advantage of multisource feature deep fusion. Our implementation is available at https://github.com/cao3082423114/M2PNet.
Keyword:
Video object segmentation
Motion perception
Self-supervised learning
Multimodal fusion
期刊
IF:
2.9
论文数:
4.6K
被引数:
6.5K
机构
引用论文
Effects of multimedia on document browsing and navigation: an exploratory empirical investigation
PREAI
分享收藏
FUN-SIS: A Fully UNsupervised approach for Surgical Instrument Segmentation.png)
MEDICAL IMAGE ANALYSIS
OAAI
MEDICAL IMAGE ANALYSISIF11.8
OAAI分享收藏Forced acoustical response of a cavity coupled with a semi-infinite space using coupled mode theoryWave Motion
Wave MotionIF0
PREAI
分享收藏Chiral cage molecules generated by regioselective O-alkylation of a doubly-bridged calix[8]arene
PREAI
分享收藏
查看更多内容