.png){kind=icon}
返回
Semantic Collaborative Learning for Cross-Modal Moment Localization
.png){kind=icon}
PRE
AI
DOI：10.1145/3620669.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Localizing a desired moment within an untrimmed video via a given natural language query, i.e., cross-modal moment localization, has attracted widespread research attention recently. However, it is a challenging task because it requires not only accurately understanding intra-modal semantic information, but also explicitly capturing inter-modal semantic correlations (consistency and complementarity). Existing efforts mainly focus on intra-modal semantic understanding and inter-modal semantic alignment, while ignoring necessary semantic supplement. Consequently, we present a cross-modal semantic perception network for more effective intra-modal semantic understanding and inter-modal semantic collaboration. Concretely, we design a dual-path representation network for intra-modal semantic modeling. Meanwhile, we develop a semantic collaborative network to achieve multi-granularity semantic alignment and hierarchical semantic supplement. Thereby, effective moment localization can be achieved based on sufficient semantic collaborative learning. Extensive comparison experiments demonstrate the promising performance of our model compared with existing state-of-the-art competitors.
Keyword:
Cross-modal moment localization
intra-modal semantic understanding
inter-modal semantic collaboration
期刊
IF:
9.1
论文数:
1.2K
被引数:
4.7K
机构
引用论文
A retrospective 5-year review of rubella in South Africa prior to the introduction of a rubella-containing vaccine.png)
PLOS ONE
OAAI
PLOS ONEIF0
OAAI分享收藏
查看更多内容