Bayesian Data Analysis in Empirical Software Engineering Research

doi:10.1109/TSE.2019.2935974

Return

Bayesian Data Analysis in Empirical Software Engineering Research

2019-01-01

Carlo A. Furia *

Robert Feldt

Richard Torkar

DOI：10.1109/TSE.2019.2935974

Original

Save

View PDF

Abstract

En 中文

Statistics comes in two main flavors: frequentist and Bayesian. For historical and technical reasons, frequentist statistics have traditionally dominated empirical data analysis, and certainly remain prevalent in empirical software engineering. This situation is unfortunate because frequentist statistics suffer from a number of shortcomings-such as lack of flexibility and results that are unintuitive and hard to interpret-that curtail their effectiveness when dealing with the heterogeneous data that is increasingly available for empirical analysis of software engineering practice. In this paper, we pinpoint these shortcomings, and present Bayesian data analysis techniques that provide tangible benefits-as they can provide clearer results that are simultaneously robust and nuanced. After a short, high-level introduction to the basic tools of Bayesian statistics, we present the reanalysis of two empirical studies on the effectiveness of automatically generated tests and the performance of programming languages. By contrasting the original frequentist analyses with our new Bayesian analyses, we demonstrate the concrete advantages of the latter. To conclude we advocate a more prominent role for Bayesian statistical techniques in empirical software engineering research and practice.

Keywords:

Bayes methods

Software engineering

Testing

Data analysis

Machine learning

Statistical analysis

Software

Bayesian data analysis

statistical analysis

statistical hypothesis testing

empirical software engineering

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.