Using mutation analysis for assessing and comparing testing coverage criteria

doi:10.1109/TSE.2006.83

2006-08-01

345

PRE

DOI：10.1109/TSE.2006.83

原文链接

原文求助

摘要

En 中文

The empirical assessment of test techniques plays an important role in software testing research. One common practice is to seed faults in subject software, either manually or by using a program that generates all possible mutants based on a set of mutation operators. The latter allows the systematic, repeatable seeding of large numbers of faults, thus facilitating the statistical analysis of fault detection effectiveness of test suites; however, we do not know whether empirical results obtained this way lead to valid, representative conclusions. Focusing on four common control and data flow criteria ( Block, Decision, C-Use, and P-Use), this paper investigates this important issue based on a middle size industrial program with a comprehensive pool of test cases and known faults. Based on the data available thus far, the results are very consistent across the investigated criteria as they show that the use of mutation operators is yielding trustworthy results: Generated mutants can be used to predict the detection effectiveness of real faults. Applying such a mutation analysis, we then investigate the relative cost and effectiveness of the above-mentioned criteria by revisiting fundamental questions regarding the relationships between fault detection, test suite size, and control/data flow coverage. Although such questions have been partially investigated in previous studies, we can use a large number of mutants, which helps decrease the impact of random variation in our analysis and allows us to use a different analysis approach. Our results are then compared with published studies, plausible reasons for the differences are provided, and the research leads us to suggest a way to tune the mutation analysis process to possible differences in fault detection probabilities in a specific environment.

Keyword:

testing and debugging

testing strategies

test coverage of code

experimental design

AI总结

对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。

期刊

IEEE Transactions on Software Engineering

IF:

5.6

论文数:

2.9K

被引数:

1.1W

机构

暂无机构信息

引用论文

Tongue thickness measured by ultrasonography is associated with tongue pressure in the Japanese elderly

PLOS ONE

IF0

2020-08-07

OAAI

Masahiro Nakamori; Eiji Imamura; Masako Fukuta; Keisuke Tachiyama; Teppei Kamimura; Yuki Hayashi; Hayato Matsushima; Kanami Ogawa; Masami Nishino; Akiko Hirata; Tatsuya Mizoue; Shinichi Wakabayashi

Prioritizing test cases for regression testing 确定回归测试的测试用例的优先级

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING

IF5.6

2001-10-01

828

PREAI

Rothermel, G; Untch, RH; Harrold, MJ

The clinical limits of myasthenia gravis and differential diagnosis

Neurology

IF0

1997-04-01

PREAI