Augmented testing to support manual GUI-based regression testing: An empirical study

Andreas Bauer

doi:10.1007/s10664-024-10522-z

2024-08-17

Andreas Bauer *

Julian Frattini

Emil Alégroth

DOI：10.1007/s10664-024-10522-z

原文链接

原文求助

摘要

En 中文

ContextManual graphical user interface (GUI) software testing presents a substantial part of the overall practiced testing efforts, despite various research efforts to further increase test automation. Augmented Testing (AT), a novel approach for GUI testing, aims to aid manual GUI-based testing through a tool-supported approach where an intermediary visual layer is rendered between the system under test (SUT) and the tester, superimposing relevant test information.ObjectiveThe primary objective of this study is to gather empirical evidence regarding AT's efficiency compared to manual GUI-based regression testing. Existing studies involving testing approaches under the AT definition primarily focus on exploratory GUI testing, leaving a gap in the context of regression testing. As a secondary objective, we investigate AT's benefits, drawbacks, and usability issues when deployed with the demonstrator tool, Scout.MethodWe conducted an experiment involving 13 industry professionals, from six companies, comparing AT to manual GUI-based regression testing. These results were complemented by interviews and Bayesian data analysis (BDA) of the study's quantitative results.ResultsThe results of the Bayesian data analysis revealed that the use of AT shortens test durations in 70% of the cases on average, concluding that AT is more efficient. When comparing the means of the total duration to perform all tests, AT reduced the test duration by 36% in total. Participant interviews highlighted nine benefits and eleven drawbacks of AT, while observations revealed four usability issues.ConclusionThis study presents empirical evidence of improved efficiency using AT in the context of manual GUI-based regression testing. We further report AT's benefits, drawbacks, and usability issues. The majority of identified usability issues and drawbacks can be attributed to the tool implementation of AT and, thus, can serve as valuable input for future tool development.

Keyword:

GUI-based testing

GUI testing

Augmented testing

Manual testing

Bayesian data analysis