logo
CanyamAI summaries for academic research
language
MobileMobile
  • Phone FeatureWeChat Mini Program
EnglishEnglish
noticeNotice
BlogBlog
delete
arrow
Return

Improving code completion efficiency through grouped attention

delete2025-11-01
delete0
PRE
AI
Y
Yiming Yin
J
Jianxun Liu *
Y
Yi Liu
D
Deng Jia
DOI：10.1093/comjnl/bxaf117delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Although the code completion model based on the Transformer architecture has achieved remarkable results, the computational complexity of its multi-head self-attention grows quadratically with the increase of sequence length, resulting in low efficiency. To this end, we propose a code completion model based on grouped attention, referred to as GACC. This method groups the attention heads of a representation in the query, key, or value so that the attention heads in the same group share the single attention head of other representations, thereby reducing the number of parameters and computational complexity. We conducted experiments on public datasets of Python and JavaScript, and the experimental results show that compared to models based on multi-head self-attention, GACC effectively reduces the time taken for the model to suggest the next code token while achieving comparable performance on Top-k and MRR metrics.

Journal

C
COMPUTER JOURNAL
IF:
1.5
Papers:
102
Citations:
0

Organization

No organization information available