.png){kind=icon}
.png){kind=icon}
Return
On Representation Learning-based Methods for Effective, Efficient, and Scalable Code Retrieval
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2024.128172.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Code retrieval consists of finding relevant code snippets regarding a programmer's query - an increasingly important task due to software ubiquity. Although significant progress has been made, there is still a need for refinement and improvement in current code retrieval solutions. Besides effective, , we argue that solutions for the code retrieval task should be efficient and scalable to ever-increasing large code repositories. This paper introduces xCoFormer, , the first representation learning-based model for the code retrieval task in the literature, which meets all these requirements. Our contributions include: (i) an interactive tag-training method that efficiently places a query closer to its relevant codes in the embedding space and (ii) use of a specialized loss function (N N-pair) better suited to the code retrieval task. To evaluate our proposal, we conducted experiments with different versions of xCoFormer against several state-of-the-art deep learning models in datasets with different properties. In our experiments, xCoFormer produced the best cost-effectiveness tradeoff - it is as effective as our closest competitor while being thousands of times faster at search time. xCoFormer's attention mechanism also (partially) meets the desirable requirement of explainability by exposing the reasoning of the model's predictions. Finally, when used as a proxy method, fine-tuned with domain-specific code language models, such as Unixcoder and CodeBERT, our proposal achieves gains of more than 33% in effectiveness when compared to the use of general purpose language models such as BERT and RoBERTa.
Keywords:
Code retrieval
Representation learning
Language models
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W