arrow
Return

Why Do GitHub Actions Workflows Fail? An Empirical Study

delete2025-01-01
delete0
delete
OA
AI
L
Lianyu Zheng
S
Shuang Li
X
Xi Huang
J
Jiangnan Huang
B
Bin Lin
J
Jinfu Chen
J
Jifeng Xuan
DOI:10.1145/3749371delete
deleteOriginal
deleteShare
deleteSave
Abstract

Abstract

En 中文
GitHub actions (GHA), a built-in continuous integration and continuous delivery (CI/CD) service of GitHub, has been widely adopted by developers, streamlining the automation of software development workflows. Despite its popularity, failures frequently occur during GHA workflow executions. Fixing these failures often requires significant human effort, and unsuccessful workflow executions waste computing resources. Understanding the reasons behind workflow failures could provide valuable insights for troubleshooting the existing issues of CI/CD and further improving the development process. In this article, we present an empirical study to reveal the reasons behind GHA workflow failures. By manually analyzing 375 failed workflow executions across 260 open-source Java projects, we built a comprehensive taxonomy categorizing the common failure types. The taxonomy was further validated by surveying 151 developers. This study is the first empirical work to analyze GHA workflow failures, bringing valuable knowledge to the field of continuous integration in software engineering. Moreover, our taxonomy and survey results not only underscore the critical need for better tools and practices to mitigate these failures but also indicate the directions to enhance the efficiency and reliability of CI/CD pipelines.

Journal

A
ACM Transactions on Software Engineering and Methodology
IF:
6.2
Papers:
1.4K
Citations:
4.5K

Organization

R
radboud university nijmegen
Scholars:
759
Papers: 255
Citations: 2.4W
Hangzhou Dianzi University cover
Hangzhou Dianzi University
Scholars:
1.9K
Papers: 617
Citations: 6.0K
wuhan university cover
wuhan university
Scholars:
7.0K
Papers: 2.1K
Citations: 4.9W
researcher View more organizations
Cited Papers

Cited Papers

err
IF0
err
err0
PREAI
err
errShare
errSave
The Chi-Square Test
err2011-11-08
err0
PREAI
errTodd Michael Franke; Timothy Ho; Christina A. Christie
errShare
errSave
Work practices and challenges in continuous integration: A survey with Travis CI users
err2018-09-11
err0
PREAI
errGustavo Pinto; Fernando Castor; Rodrigo Bonifacio; Marcel Rebouças
errShare
errSave
A statistical analysis of syntax errors
err1978-01-01
err0
PREAI
errG.David Ripley; Frederick C. Druseikis
errShare
errSave
Predicting continuous integration build failures using evolutionary search
err2020-12-01
err26
PREAI
errSaidani, Islem; Ouni, Ali; Chouchen, Moataz; Mkaouer, Mohamed Wiem
errShare
errSave
researcher View more