.png){kind=icon}
返回
Multimodal Progressive Modulation Network for Micro-Video Multi-Label Classification
.png){kind=icon}
PRE
AI
DOI：10.1109/TMM.2024.3405724.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Micro-videos, as an increasingly popular form of user-generated content (UGC), naturally include diverse multimodal cues. However, in pursuit of consistent representations, existing methods neglect the simultaneous consideration of exploring modality discrepancy and preserving modality diversity. In this paper, we propose a multimodal progressive modulation network (MPMNet) for micro-video multi-label classification, which enhances the indicative ability of each modality through gradually regulating various modality biases. In MPMNet, we first leverage a unimodal-centered parallel aggregation strategy to obtain preliminary comprehensive representations. We then integrate feature-domain disentangled modulation process and category-domain adaptive modulation process into a unified framework to jointly refine modality-oriented representations. In the former modulation process, we constrain inter-modal dependencies in a latent space to obtain modality-oriented sample representations, and introduce a disentangled paradigm to further maintain modality diversity. In the latter modulation process, we construct global-context-aware graph convolutional networks to acquire modality-oriented category representations, and develop two instance-level parameter generators to further regulate unimodal semantic biases. Extensive experiments on two micro-video multi-label datasets show that our proposed approach outperforms the state-of-the-art methods.
Keyword:
Modulation
Correlation
Semantics
Representation learning
Visualization
Task analysis
Fans
Micro-video
multi-label classification
multimodal representation
期刊
IF:
9.7
论文数:
4.6K
被引数:
2.4W
机构
引用论文
Clinicopathologic profile of 470 giant cell tumors of bone from a cancer hospital in western India
.png){kind=icon}
PREAI
分享收藏
分享收藏
Dual-Domain Aligned Deep Hierarchical Matrix Factorization Method for Micro-Video Multi-Label Classification用于微视频多标签分类的双域对齐深层分层矩阵分解方法
PREAI
分享收藏分享收藏MG-GCN: Multi-Granularity Graph Convolutional Neural Network for Multi-Label Classification in Multi-Label Information SystemMg-gcn: 多标签信息系统中多标签分类的多粒度图卷积神经网络
PREAI
分享收藏
查看更多内容