Mutation testing in the wild: findings from GitHub

doi:10.1007/s10664-022-10177-8

Return

Mutation testing in the wild: findings from GitHub

2022-07-18

Ana B. Sánchez *

Pedro Delgado‐Pérez

Sergio Segura

DOI：10.1007/s10664-022-10177-8

Original

Original request for help

Save

Abstract

En 中文

Mutation testing exploits artificial faults to measure the adequacy of test suites and guide their improvement. It has become an extremely popular testing technique as evidenced by the vast literature, numerous tools, and research events on the topic. Previous survey papers have successfully compiled the state of research, its evolution, problems, and challenges. However, the use of mutation testing in practice is still largely unexplored. In this paper, we report the results of a thorough study on the use of mutation testing in GitHub projects. Specifically, we first performed a search for mutation testing tools, 127 in total, and we automatically searched the GitHub repositories including evidence of their use. Then, we focused on the top ten most widely used tools, based on the previous results, and manually revised and classified over 3.5K GitHub active repositories importing them. Among other findings, we observed a recent upturn in interest and activity, with Infection (PHP), PIT (Java) and Humbug (PHP) being the most widely used mutation tools in recent years. The predominant use of mutation testing is development, followed by teaching and learning, and research projects, although with significant differences among mutation tools found in the literature-less adopted and largely used in teaching and research-and those found in GitHub only-more popular and more widely used in development. Our work provides a new and encouraging perspective on the state of practice of mutation testing.

Keywords:

Mutation testing

Mutation tools

Data mining

GitHub