.png){kind=icon}
返回
PopArt: Ranked Testing Efficiency
.png){kind=icon}
OA
AI
DOI：10.1109/TSE.2022.3214796.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Too often, programmers are under pressure to maximize their confidence in the correctness of their code with a tight testing budget. Should they spend some of that budget on finding interesting inputs or spend their entire testing budget on test executions? Work on testing efficiency has explored two competing approaches to answer this question: systematic partition testing (ST), which defines a testing partition and tests its parts, and random testing (RT), which directly samples inputs with replacement. A consensus as to which is better when has yet to emerge. We present Probability Ordered Partition Testing (PopArt), a new systematic partition-based testing strategy that visits the parts of a testing partition in decreasing probability order and in doing so leverages any non-uniformity over that partition. We show how to construct a homogeneous testing partition, a requirement for systematic testing, by using an executable oracle and the path partition. A program's path partition is a naturally occurring testing partition that is usually skewed for the simple reason that some paths execute more frequently than others. To confirm this conventional wisdom, we instrument programs from the Codeflaws repository and find that 80% of them have a skewed path probability distribution. PopArt visits the parts of a testing partition in decreasing probability order. We then compare PopArt with RT to characterise the configuration space in which each is more efficient. We show that, when simulating Codeflaws, PopArt outperforms RT after $100{,}000$100,000 executions. Our results reaffirm RT's power for very small testing budgets but also show that for any application requiring high (above 90%) probability-weighted coverage PopArt should be preferred. In such cases, despite paying more for each test execution, we prove that PopArt outperforms RT: it traverses parts whose cumulative probability bounds that of random testing, showing that sampling without replacement pays for itself, given a nonuniform probability over a testing partition.
Keyword:
Software testing
randomness
systematic
efficiency
probability distribution
期刊
IF:
5.6
论文数:
2.8K
被引数:
1.1W
机构
引用论文
Calligonum polygonoides Linnaeus Extract: HPLC‐EC and Total Antioxidant Capacity Evaluation 刺瓜 刺瓜提取物: hplc ‐ ec和总抗氧化能力评价
PREAI
分享收藏Cysteine Dioxygenase 1 Is a Tumor Suppressor Gene Silenced by Promoter Methylation in Multiple Human Cancers半胱氨酸双加氧酶1是一种在多种人类癌症中被启动子甲基化沉默的抑癌基因.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏
查看更多内容