.png){kind=icon}
返回
Operationalizing validity of empirical software engineering studies
.png){kind=icon}
PRE
AI
DOI：10.1007/s10664-023-10370-3.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Empirical Software Engineering studies apply methods, like linear regression, statistic tests, or correlation analysis, to better understand software engineering scenarios. Assuring the validity of such methods and corresponding results is challenging but critical. This is also reflected by quality criteria on the validity that are part of the reviewing process for the corresponding research results. However, such criteria are often hard to define operationally and thus hard to judge by the reviewers. In this paper, we describe a new strategy to define and communicate the validity of methods and results. We conceptually decompose a study into an empirical scenario, a used method, and the produced results. Validity can only be described as the relationship between the three parts. To make the empirical scenario fully operational, we convert informal assumptions on it into executable simulation code that leverages artificial data to replace (or complement) our real data. We can then run the method on the artificial data and examine the impact of our assumptions on the quality of results. This may operationally i) support the validity of a method for a valid result, ii) threaten the validity of a method for an invalid result if assumptions are controversial, or iii) invalidate a method for an invalid result if assumptions are plausible. We encourage researchers to submit simulations as additional artifacts to the reviewing process to make such statements explicit. Rating if a simulated scenario is plausible or controversial is subjective and may benefit from involving a reviewer. We show that existing empirical software engineering studies can benefit from such additional validation artifacts.
Keyword:
Simulations
Empirical methods
Validity
Threats
Testing
期刊
IF:
3.6
论文数:
2.0K
被引数:
5.3K
机构
引用论文
分享收藏Differences between Patterns of Brain Activity Associated with Semantics and Those Linked with Phonological Processing Diminish with Age.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏
查看更多内容