Performance metamorphic testing: A Proof of concept

doi:10.1016/j.infsof.2018.01.013

Performance metamorphic testing: A Proof of concept

2018-06-01

18

OA

AI

DOI：10.1016/j.infsof.2018.01.013

原文链接

查看原文

摘要

En 中文

Context: Performance testing is a challenging task mainly due to the lack of test oracles, i.e. mechanisms to decide whether the performance of a program is acceptable or not because of a bug. Metamorphic testing enables the generation of test cases in the absence of an oracle by exploiting the so-called metamorphic relations between the inputs and outputs of multiple executions of the program under test. In the last two decades, metamorphic testing has been successfully used to detect functional faults in different domains. However, its applicability to performance testing remains unexplored. Objective: We propose the application of metamorphic testing to reveal performance failures. Method: We define Performance Metamorphic Relations (PMRs) as expected relations between performance measurements of multiple executions of the program under test. These relations can be turned into assertions for the automated detection of performance bugs, removing the need for complex benchmarks and domain experts guidance. As a further benefit, PMRs can be turned into fitness functions to guide search-based techniques on the generation of test data. Results: The feasibility of the approach is illustrated through an experimental proof of concept in the context of the automated analysis of feature models. Conclusion: The results confirm the potential of metamorphic testing, in combination with search-based techniques, to automate the detection of performance bugs.

Keyword:

Metamorphic testing

Performance testing

Search-based testing

Performance metamorphic testing: A Proof of concept

摘要

AI总结

期刊

机构

引用论文