.png){kind=icon}
返回
Improving code completion efficiency through grouped attention
.png){kind=icon}
PRE
AI
DOI：10.1093/comjnl/bxaf117.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Although the code completion model based on the Transformer architecture has achieved remarkable results, the computational complexity of its multi-head self-attention grows quadratically with the increase of sequence length, resulting in low efficiency. To this end, we propose a code completion model based on grouped attention, referred to as GACC. This method groups the attention heads of a representation in the query, key, or value so that the attention heads in the same group share the single attention head of other representations, thereby reducing the number of parameters and computational complexity. We conducted experiments on public datasets of Python and JavaScript, and the experimental results show that compared to models based on multi-head self-attention, GACC effectively reduces the time taken for the model to suggest the next code token while achieving comparable performance on Top-k and MRR metrics.
期刊
C
IF:
1.5
论文数:
103
被引数:
0
机构
暂无机构信息
引用论文
.png){kind=icon}
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head CheckpointsGQA: 从多头检查点训练广义多查询转换器模型
OAAI
OAAI分享收藏分享收藏
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and GenerationCodeT5: 用于代码理解和生成的标识符感知的统一预训练编码器-解码器模型OAAI
OAAI分享收藏
查看更多内容