Dynamic scale position embedding for cross-modal representation learning

Jungkyoo Shin

doi:10.1016/j.neunet.2025.108087

Return

Dynamic scale position embedding for cross-modal representation learning

2025-09-08

PRE

DOI：10.1016/j.neunet.2025.108087

Original

Original request for help

Save

Abstract

En 中文

• We propose a position-based method for dynamic temporal scaling. • We design a single-encoder multi-scale temporal model. • We achieve consistent gains on 4 datasets in video retrieval and video captioning.

Keywords:

Multi-modal learning

Position embedding

Representation learning

Journal

Neural Networks

IF:

6.3

Papers:

8.3K

Citations:

3.0W

Organization

No organization information available

Cited Papers

What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

IF0

2020-01-01

OAAI

Yu-An Wang; Yun-Nung Chen

Save

End-to-End Dense Video Captioning with Masked Transformer

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IF0

2018-06-01

OAAI

Luowei Zhou; Yingbo Zhou; Jason J. Corso; Richard Socher; Caiming Xiong

Save

Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2023-06-01

OAAI

Peng Jin; Jinfa Huang; Pengfei Xiong; Shangxuan Tian; Chang Liu; Xiangyang Ji; Li Yuan; Jie Chen

Save

Shin, J., Kim, B., & Kim, E. (2025). Generative modeling of class probability for multi-modal representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 20737–20746).

IF0

PREAI

Save

X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval

Proceedings of the 30th ACM International Conference on Multimedia

IF0

2022-10-10

OAAI

Yiwei Ma; Guohai Xu; Xiaoshuai Sun; Ming Yan; Ji Zhang; Rongrong Ji

Save

Zhang, Z., & Gong, X. (2023). Positional label for self-supervised vision transformer. In Proceedings of the AAAI conference on artificial intelligence (pp. 3516–3524). (vol. 37).

IF0

2023-06-26

PREAI

Zhemin Zhang; Xun Gong

Save

Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2023-06-01

OAAI

Wenhao Wu; Haipeng Luo; Bo Fang; Jingdong Wang; Wanli Ouyang

Save

Text-Video Retrieval via Multi-Modal Hypergraph Networks

Proceedings of the 17th ACM International Conference on Web Search and Data Mining

IF0

2024-03-04

PREAI

Qian Li; Lixin Su; Jiashu Zhao; Long Xia; Hengyi Cai; Suqi Cheng; Hengzhu Tang; Junfeng Wang; Dawei Yin

Save

TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval

Lecture Notes in Computer Science

IF0

2022-10-23

PREAI

Yuqi Liu; Pengfei Xiong; Luhui Xu; Shengming Cao; Qin Jin

Save

Li, Y., Min, K., Tripathi, S., & Vasconcelos, N. (2023). SViTT: Temporal learning of sparse video-text transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 18919–18929).

IF0

2023-06-01

PREAI

Yi Li; Kyle Min; Subarna Tripathi; Nuno Vasconcelos

Save