.png){kind=icon}
.png){kind=icon}
Return
Analyzing the performance of large language models on statement-level code summarization
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-026-07214-0.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Code summarization provides concise descriptions of code, covering functionality, logic, and usage, which is essential for enhancing code quality and maintainability. Recent advancements in large language models (LLMs) have demonstrated strong performance in this task, gradually replacing traditional deep learning methods. However, most studies focus on function-level code summarization, leaving statement-level code summarization underexplored. Statement-level code summarization describes the functionality of key statements in code and plays a vital role in understanding higher-level code units like functions and classes. However, this task faces challenges due to its high context dependence and other factors. This paper evaluates LLMs for statement-level code summarization, focusing on dataset construction, comment generation, and automated evaluation. We created a high-quality dataset by incorporating functions as context and using rule-based filtering and relevant tools. During dataset construction, we verified the consistency of code-comment pairs using LLMs, resulting in the exclusion of 60% of low-quality data. In the comparative experiments, using functions as context significantly enhanced performance across all metrics, with the BLEU score increasing by 25.04%. Next, we tested mainstream prompt methods, and the few-shot method outperformed others, excelling in both automated metric evaluation and human assessment. Finally, we tested LLMs in automated evaluation, finding a strong positive correlation and consistency with human evaluation, highlighting their potential as a new method for automated assessment.
Keywords:
Large language models
Code summarization
Statement-level code summarization
Journal
IF:
3.5
Papers:
7.5K
Citations:
1.7W