A multi-dimensional test case evaluation framework based on clustering and differential testing

doi:10.1016/j.jss.2026.112936

A multi-dimensional test case evaluation framework based on clustering and differential testing

2026-05-10

0

PRE

AI

DOI：10.1016/j.jss.2026.112936

Original

Original request for help

Save

Abstract

En 中文

• We propose a novel four-dimensional evaluation framework that comprehensively assesses test case quality in OJ systems from code coverage, mutation score, test scale, and false positive rate perspectives. • Our framework reveals severe limitations in existing test case design: a substantial proportion of programs that pass all test cases actually contain logical errors, a critical issue that traditional metrics fail to capture. • We develop DFC-DT, an efficient false positive detection method that combines dynamic feature-based clustering with differential testing, achieving substantial program reduction and significant time cost reduction while maintaining comparable detection accuracy. • Extensive validation on two real-world OJ platforms (BuctOJ and AtCoder) with large-scale program submissions demonstrates the effectiveness of our approach, identifying numerous false positive programs with superior performance in Recall and F-measure. • We share our source code and empirical results on a Github repository, https://github.com/dfaskl/DFC-DT .

Keywords:

test case quality

code coverage

mutation score

false positive detection

differential testing

A multi-dimensional test case evaluation framework based on clustering and differential testing

Abstract

Journal

Organization