.png){kind=icon}
返回
Empirically evaluating flaky test detection techniques combining test case rerunning and machine learning models
.png){kind=icon}
OA
AI
DOI：10.1007/s10664-023-10307-w.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
A flaky test is a test case whose outcome changes without modification to the code of the test case or the program under test. These tests disrupt continuous integration, cause a loss of developer productivity, and limit the efficiency of testing. Many flaky test detection techniques are rerunning-based, meaning they require repeated test case executions at a considerable time cost, or are machine learning-based, and thus they are fast but offer only an approximate solution with variable detection performance. These two extremes leave developers with a stark choice. This paper introduces CANNIER, an approach for reducing the time cost of rerunning-based detection techniques by combining them with machine learning models. The empirical evaluation involving 89,668 test cases from 30 Python projects demonstrates that CANNIER can reduce the time cost of existing rerunning-based techniques by an order of magnitude while maintaining a detection performance that is significantly better than machine learning models alone. Furthermore, the comprehensive study extends existing work on machine learning-based detection and reveals a number of additional findings, including (1) the performance of machine learning models for detecting polluter test cases; (2) using the mean values of dynamic test case features from repeated measurements can slightly improve the detection performance of machine learning models; and (3) correlations between various test case features and the probability of the test case being flaky.
Keyword:
Software testing
Flaky tests
Machine learning
期刊
IF:
3.6
论文数:
2.0K
被引数:
5.3K
机构
引用论文
Isolation and characterization of two types of actinophage infectingStreptomyces scabies感染疥疮链霉菌的两种类型的放线噬菌体的隔离和表征
PREAI
分享收藏Photoacoustics, thermoacoustics, and acousto-optics for biomedical imaging.png)
Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine
Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in MedicineIF0
PREAI
分享收藏Smells in software test code: A survey of knowledge in industry and academia软件测试代码中的气味: 工业界和学术界的知识调查
PREAI
分享收藏The ratio of free to total prostate-specific antigen in serum is correlated to the prostate volume
OAAI
OAAI分享收藏
查看更多内容