Dynamically Relative Position Encoding-Based Transformer for Automatic Code Edit

doi:10.1109/TR.2022.3194370

Dynamically Relative Position Encoding-Based Transformer for Automatic Code Edit

2023-09-01

2

OA

AI

DOI：10.1109/TR.2022.3194370

Original

Original request for help

Save

Abstract

En 中文

Adapting Deep Learning (DL) techniques to automate non-trivial coding activities, such as code documentation and defect detection, has been intensively studied recently. Learning to predict code changes is one of the popular and essential investigations. Prior studies have shown that DL techniques such as Neural Machine Translation (NMT) can benefit meaningful code changes, including bug fixing and code refactoring. However, NMT models may encounter bottleneck when modeling long sequences, thus are limited in accurately predicting code changes. In this work, we design a Transformer-based approach, considering that Transformer has proven effective in capturing long-term dependencies. Specifically, we propose a novel model named DTrans. For better incorporating the local structure of code, i.e., statement-level information in this paper, DTrans is designed with dynamically relative position encoding in the multi-head attention of Transformer. Experiments on benchmark datasets demonstrate that DTrans can more accurately generate patches than the state-of-the-art methods, increasing the performance by at least 5.45\%-46.57\% in terms of the exact match metric on different datasets. Moreover, DTrans can locate the lines to change with 1.75\%-24.21\% higher accuracy than the existing methods.

Keywords:

Codes

Transformers

Encoding

Task analysis

Decoding

Predictive models

Computer bugs

Code edit

position encoding

Transformer

Dynamically Relative Position Encoding-Based Transformer for Automatic Code Edit

Abstract

Journal

Organization