A code completion approach combining pointer network and Transformer-XL network

Xiangping Zhang

doi:10.1007/s10489-025-06315-6

Return

A code completion approach combining pointer network and Transformer-XL network

2025-02-14

PRE

DOI：10.1007/s10489-025-06315-6

Original

Original request for help

Save

Abstract

En 中文

Code completion is a crucial aspect of contemporary integrated development environments (IDEs), as it not only streamlines the software development process but also bolsters the quality of software products. By leveraging large-scale codes to learn the probability distribution among code token units, deep learning methods have demonstrated significant improvements in the accuracy of token unit recommendations. However, the efficacy of code completion employing deep learning is often compromised by information loss. To mitigate this issue, we introduce a novel code language model that incorporates both the pointer network and the Transformer-XL architecture to surpass the constraints of current approaches in code completion. Our proposed model accepts as input the original code snippet and its corresponding abstract syntax tree (AST), utilizing the Transformer-XL model as the foundational architecture for capturing long-term dependencies. Additionally, we incorporate a pointer network as an adjunct component to forecast Out-of-Vocabulary (OoV) words. Our approach has been rigorously evaluated on the authentic PY150 and JS150 datasets. The comparative experimental results demonstrate the effectiveness of our model in improving the accuracy of the code completion task at the token unit level.

Keywords:

Code completion

Transformer-XL

Pointer network

Out-of-vocabulary