Excluding code from test coverage: practices, motivations, and impact

André Hora

doi:10.1007/s10664-022-10259-7

2022-12-07

PRE

André Hora *

DOI：10.1007/s10664-022-10259-7

原文链接

原文求助

摘要

En 中文

Test coverage measures the percentage of code that is covered (and uncovered) by tests. In practice, not all code is equally important for coverage analysis, like code that will not be executed during tests. Some coverage tools provide support for code exclusion from coverage reports, however, we are not yet aware of what code tends to be excluded, the reasons behind it, and the impact on coverage analysis. In this paper, we provide an empirical study to understand code exclusion practices, motivations, and impact on test coverage. We first mine popular Python projects that adopt test coverage to assess code exclusion practices. We find that (1) over 1/3 of the projects perform coverage exclusion, (2) 75% of the code is already created using the exclusion feature, and (3) developers exclude non-runnable, debug-only, and defensive code, but also platform-specific and conditional importing. Next, we explore the motivations behind the exclusions and the importance of test coverage in current days: (4) most code is excluded because it is already untested, low-level, or complex and (5) distinct test coverage recommendations are available for developers, such as covering the change, exploring coverage reports, and increasing coverage. Lastly, we assess the impact of code exclusion on test coverage. We detect that (6) code exclusion may impact test coverage, decreasing the number of statements that should be covered by tests and (7) test coverage can be refined by following code exclusion recommendations. Based on our findings, we discuss implications for both practitioners and researchers to improve coverage analysis, tools, and documentation as well as inspire novel research on test coverage.

Keyword:

Software testing

Test coverage

Software evolution

Software maintenance

Software repository mining