.png){kind=icon}
返回
Multi-views contrastive learning for dense text retrieval
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2023.110624.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Dense text retrieval has become a widely used paradigm for recalling existing language models, and efficient dense text retrieval is essential for obtaining sufficiently accurate candidate samples. However, the existing methods for dense text retrieval, which typically use dual-encoder architectures to match similar queries and documents, suffer from a lack of information interaction at low data volumes, resulting in suboptimal performance. Additionally, existing research relies on negative sampling techniques that do not take into account the negative effects of single negative sampling bias on the robustness of the model. These limitations hinder the development of more robust dense text retrieval models. In this paper, we propose a multi-view contrast learning architecture, named MvCR, to address these issues. MvCR improves the performance of dense text retrieval by performing contrast learning with multiple views while significantly increasing the model's ability to discriminate between positive and negative samples. Additionally, we propose a data augmentation method that focuses on increasing the number of hard negative samples with accurate and semantic matching features. The experimental results have shown that MvCR can perform as well as strong baseline models even when the data volume is small. Furthermore, MvCR achieved better results on two popular retrieval benchmarks with comparable amounts of data. Specifically, MRR@10 was 39.1(+0.9%) and Recall@50 was 87.8(+1.4%) on the MS-MARCO dataset. And Recall@5 increased to 77.2(+1.8%) and Recall@50 increased to 85.3(+1.0%) on the Natural Questions dataset.(c) 2023 Elsevier B.V. All rights reserved.
Keyword:
Dense text retrieval
Data augmentation
Contrastive learning
Pre-training language model
期刊
K
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
.png){kind=icon}
分享收藏 没有更多内容
没有更多内容