4.7 Article

Dynamically Relative Position Encoding-Based Transformer for Automatic Code Edit

Journal

IEEE TRANSACTIONS ON RELIABILITY
Volume 72, Issue 3, Pages 1147-1160

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TR.2022.3194370

Keywords

Codes; Transformers; Encoding; Task analysis; Decoding; Predictive models; Computer bugs; Code edit; position encoding; Transformer

Ask authors/readers for more resources

This study proposes a Transformer-based model, named DTrans, for learning to predict code changes. By incorporating dynamically relative position encoding in the multi-head attention of Transformer, DTrans can accurately generate patches and locate lines to change with higher accuracy compared to existing methods.
Adapting Deep Learning (DL) techniques to automate non-trivial coding activities, such as code documentation and defect detection, has been intensively studied recently. Learning to predict code changes is one of the popular and essential investigations. Prior studies have shown that DL techniques such as Neural Machine Translation (NMT) can benefit meaningful code changes, including bug fixing and code refactoring. However, NMT models may encounter bottleneck when modeling long sequences, thus are limited in accurately predicting code changes. In this work, we design a Transformer-based approach, considering that Transformer has proven effective in capturing long-term dependencies. Specifically, we propose a novel model named DTrans. For better incorporating the local structure of code, i.e., statement-level information in this paper, DTrans is designed with dynamically relative position encoding in the multi-head attention of Transformer. Experiments on benchmark datasets demonstrate that DTrans can more accurately generate patches than the state-of-the-art methods, increasing the performance by at least 5.45\%-46.57\% in terms of the exact match metric on different datasets. Moreover, DTrans can locate the lines to change with 1.75\%-24.21\% higher accuracy than the existing methods.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available