.png){kind=icon}
.png){kind=icon}
Return
Case study identification: A trivial indicator outperforms human classifiers
.png){kind=icon}
OA
AI
DOI：10.1016/j.infsof.2023.107252.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Context: The definition and term case studyare not being applied consistently by software engineering researchers. We previously developed a trivial smell indicatorto help detect the misclassification of primary studies as case studies. Objective: To evaluate the performance of the indicator. Methods: We compare the performance of the indicator against human classifiers for three datasets, two datasets comprising classifications by both authors of systematic literature studies and primary studies, and one dataset comprising only primary-study author classifications. Results: The indicator outperforms the human classifiers for all datasets. Conclusions: The indicator is successful because human classifiers failto properly classify their own, and others', primary studies. Consequently, reviewers of primary studies and authors of systematic literature studies could use the classifier as a sanitycheck for primary studies. Moreover, authors might use the indicator to double-check how they classified a study, as part of their analysis, and prior to submitting their manuscript for publication. We challenge the research community to both beat the indicator, and to improve its ability to identify true case studies.
Keywords:
Case study
Evaluation
Systematic review
Primary study
Smell indicator
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
4.3
Papers:
3.7K
Citations:
7.7K