.png){kind=icon}
Return
Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions
.png){kind=icon}
PRE
AI
DOI：10.1109/tse.2026.3688237.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
AI-based code-review tools automatically review and comment on pull requests to improve code quality. Despite their growing presence, little is known about their actual impact. We present a large-scale empirical study of 16 popular AI-based code-review actions for GitHub workflows, analyzing more than 22,000 review comments in 178 repositories. We investigate (1) how these tools are adopted and configured, (2) whether their comments lead to code changes, and (3) which factors influence their effectiveness. We develop a two-stage LLM-assisted framework to determine whether review comments are addressed. We then use interpretable machine learning to identify the influencing factors. We found that while adoption is growing, its effectiveness varies widely. Comments that are concise, contain code snippets, and are manually triggered, particularly those from hunk-level review tools, are more likely to result in code changes. These results highlight the importance of tool design and suggest directions for improving AI-based code review systems.
Keywords:
Code review
GitHub actions
large language models
empirical software engineering
Journal
IF:
5.6
Papers:
2.9K
Citations:
1.1W
Organization
View more organizations
Cited Papers
.png){kind=icon}
ShareSaveShareSaveShareSave View more