.png){kind=icon}
Return
LLMs for Commit Messages: A Survey and an Agent-Based Evaluation Protocol on CommitBench
.png){kind=icon}
OA
AI
DOI：10.3390/computers14100427.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Commit messages are vital for traceability, maintenance, and onboarding in modern software projects, yet their quality is frequently inconsistent. Recent large language models (LLMs) can transform code diffs into natural language summaries, offering a path to more consistent and informative commit messages. This paper makes two contributions: (i) it provides a systematic survey of automated commit message generation with LLMs, critically comparing prompt-only, fine-tuned, and retrieval-augmented approaches; and (ii) it specifies a transparent, agent-based evaluation blueprint centered on CommitBench. Unlike prior reviews, we include a detailed dataset audit, preprocessing impacts, evaluation metrics, and error taxonomy. The protocol defines dataset usage and splits, prompting and context settings, scoring and selection rules, and reporting guidelines (results by project, language, and commit type), along with an error taxonomy to guide qualitative analysis. Importantly, this work emphasizes methodology and design rather than presenting new empirical benchmarking results. The blueprint is intended to support reproducibility and comparability in future studies.
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
C
IF:
4.2
Papers:
1.5K
Citations:
3.3K
Organization
No organization information available
Cited Papers
CoreGen: Contextualized Code Representation Learning for Commit Message Generation.png)
NEUROCOMPUTINGOAAI
NEUROCOMPUTINGIF6.5
OAAIShareSave
ShareSave
View more