Context-aware code summary generation

doi:10.1016/j.jss.2025.112580

Context-aware code summary generation

2025-08-06

0

PRE

AI

DOI：10.1016/j.jss.2025.112580

Original

Original request for help

Save

Abstract

En 中文

Recent advances in Large Language Models (LLMs) have helped make automatic code summarization a reality. However, the summaries these approaches write tend to focus on a narrow area of code. The results are summaries that explain what that function does internally, but lack a description of its purpose in the broader context of the program. In this paper, we present an approach for including this context in recent LLM-based code summarization. The input to our approach is a Java method and the summary of the caller methods. The output is a succinct English description of purpose of the method. The advantage of this approach is that we reduce 81% of the tokens in smaller models. We train the model in two steps. First we distill knowledge about code summarization from a large model, then we fine-tune the model using data from a study of human programmers who were asked to write code summaries. We evaluate our method and analyze the rationale behind the decision in the tournament style human study. We find in our human study that human programmers prefer the summary from our approach rather than GPT-4 on this task and accuracy is the most common reason a summary is better than others. In terms of the automatic metrics, we also observed 10% improvement in USE score.

Keywords:

code summarization

large language models

context-aware summaries

model distillation

human evaluation

Context-aware code summary generation

Abstract

Journal

Organization