.png){kind=icon}
.png){kind=icon}
Return
Semantics-Aware Spatial-Temporal Binaries for Cross-Modal Video Retrieval
.png){kind=icon}
PRE
AI
M
Q
Y
Y
J
DOI：10.1109/TIP.2020.3048680.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
With the current exponential growth of video-based social networks, video retrieval using natural language is receiving ever-increasing attention. Most existing approaches tackle this task by extracting individual frame-level spatial features to represent the whole video, while ignoring visual pattern consistencies and intrinsic temporal relationships across different frames. Furthermore, the semantic correspondence between natural language queries and person-centric actions in videos has not been fully explored. To address these problems, we propose a novel binary representation learning framework, named Semanticsaware Spatial-temporal Binaries (S(2)Bin), which simultaneously considers spatial-temporal context and semantic relationships for cross-modal video retrieval. By exploiting the semantic relationships between two modalities, S(2)Bin can efficiently and effectively generate binary codes for both videos and texts. In addition, we adopt an iterative optimization scheme to learn deep encoding functions with attribute-guided stochastic training. We evaluate our model on three video datasets and the experimental results demonstrate that S(2)Bin outperforms the state-of-the-art methods in terms of various cross-modal video retrieval tasks.
Keywords:
Semantics
Binary codes
Feature extraction
Visualization
Task analysis
Natural languages
Stochastic processes
Cross-modal hashing
video retrieval
binary representation
spatial-temporal features
natural language
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
13.7
Papers:
1.0W
Citations:
8.4W
Organization
Cited Papers