.png){kind=icon}
返回
Image-text matching using multi-subspace joint representation
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-022-01038-x.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Joint representation learning has been an attractive way to solve image-text retrieval problem due to its efficiency on both time and storage. On the one hand, the most classical methods model the joint semantic subspace with respect to only semantic relationship at the level of holistic images and sentences, which fails to explore the fine-grained semantic relationship. On the other hand, visual-linguistic pretrain-finetune scheme has achieved an impressive performance in many downstream image-text tasks, but more storage cost and computation burden still notably limit their use in some real applications which have strict requirement for low storage cost or rapid response to image acquired in real-time, such as retrieving textual description of some just taken photos on a storage-limited mobile platform. To mitigate the problem above, we proposed a lightweight cross-modal retrieval for learning the joint representation. Instead of only modeling whole joint semantic space, the proposed model captures semantic relationship in multiple subspaces. Specially, we treat the retrieval problem as not only a ranking process but also a decision process, and propose an entropy-based constraint to preserve as much hierarchy-aware information as possible across various semantic subspaces. Experiments show that the proposed method can achieve competitive performance when compared with the state-of-the-art joint representation learning methods on two publicly available datasets.
Keyword:
Joint representation
Image-text retrieval
Multi-subspace learning
Cross-modal matching
期刊
IF:
3.1
论文数:
2.8K
被引数:
2.7K
机构
暂无机构信息
引用论文
.png){kind=icon}
Sodium channel kinetic changes that produce Brugada syndrome or progressive cardiac conduction system disease产生Brugada综合征或进行性心脏传导系统疾病的钠通道动力学变化
PREAI
分享收藏分享收藏Using collective system design to define and communicate organization goals and related solutions
PREAI
分享收藏分享收藏
查看更多内容