.png){kind=icon}
返回
Multimodal deep hierarchical semantic-aligned matrix factorization method for micro-video multi-label classification
.png){kind=icon}
PRE
AI
DOI：10.1016/j.ipm.2024.103798.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
As one of the typical formats of prevalent user -generated content in social media platforms, micro -videos inherently incorporate multimodal characteristics associated with a group of label concepts. However, existing methods generally explore the consensus features aggregated from all modalities to train a final multi -label predictor, while overlooking fine-grained semantic dependencies between modality and label domains. To address this problem, we present a novel multimodal deep hierarchical semantic -aligned matrix factorization (DHSAMF) method, which is devoted to bridging the dual -domain semantic discrepancies and the inter -modal heterogeneity gap for solving the multi -label classification task of micro -videos. Specifically, we utilize deep matrix factorization to individually explore the hierarchical modality -specific representations. A series of semantic embeddings is introduced to facilitate latent semantic interactions between modality -specific representations and label features in a layerwise manner. To further improve the representation ability of each modality, we leverage underlying correlation structures among instances to adequately mine intra-modal complementary attributes, and maximize the inter -modal alignment by aggregating consensus attributes in an optimal permutation. The experimental results conducted on the MTSVRC and VidOR datasets have demonstrated that our DHSAMF outperforms other state-of-the-art methods by nearly 3% and 4% improvements in terms of the AP metric.
Keyword:
Micro-video understanding
Multi-label learning
Multimodal learning
Low-rank learning
Deep matrix factorization
期刊
I
IF:
6.9
论文数:
5.2K
被引数:
1.4W
机构
引用论文
Two-step multi-view and multi-label learning with missing label via subspace learning基于子空间学习的带缺失标签的两步多视图多标签学习
PREAI
分享收藏Dual-Domain Aligned Deep Hierarchical Matrix Factorization Method for Micro-Video Multi-Label Classification用于微视频多标签分类的双域对齐深层分层矩阵分解方法
PREAI
分享收藏分享收藏An epiilluminator/detector unit permitting arc lamp illumination for fluorescence activated cell sorters.png)
Cytometry
OAAI
CytometryIF0
OAAI分享收藏Co-consensus semi-supervised multi-view learning with orthogonal non-negative matrix factorization
PREAI
分享收藏
查看更多内容