.png){kind=icon}
返回
QuoTe: Quality-oriented Testing for Deep Learning Systems
.png){kind=icon}
OA
AI
DOI：10.1145/3582573.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, there has been significant growth of interest in applying software engineering techniques for the quality assurance of deep learning (DL) systems. One popular direction is DL testing-that is, given a property of test, defects of DL systems are found either by fuzzing or guided search with the help of certain testing metrics. However, recent studies have revealed that the neuron coverage metrics, which are commonly used by most existing DL testing approaches, are not necessarily correlated with model quality (e.g., robustness, the most studied model property), and are also not an effective measurement on the confidence of the model quality after testing. In this work, we address this gap by proposing a novel testing framework calledQuoTe (i.e., Quality-oriented Testing). A key part of QuoTe is a quantitative measurement on (1) the value of each test case in enhancing the model property of interest (often via retraining) and (2) the convergence quality of the model property improvement. QuoTe utilizes the proposed metric to automatically select or generate valuable test cases for improving model quality. The proposedmetric is also a lightweight yet strong indicator of how well the improvement converged. Extensive experiments on both image and tabular datasets with a variety of model architectures confirm the effectiveness and efficiency of QuoTe in improving DL model quality-that is, robustness and fairness. As a generic quality-oriented testing framework, future adaptations can be made to other domains (e.g., text) as well as other model properties.
Keyword:
Deep learning
testing
robustness
fairness
期刊
A
IF:
6.2
论文数:
1.3K
被引数:
3.7K
机构
引用论文
.png){kind=icon}
分享收藏分享收藏
分享收藏Welding characteristics of aluminum, copper, nickel and aluminum alloy with alumina coating using ultrasonic complex vibration welding equipments铝、铜、镍及铝合金氧化铝涂层超声复合振动焊接特性研究
PREAI
分享收藏Thermal expansions of Ln2Zr2O7 (Ln = La, Nd, Sm, and Gd) pyrochloreLn2Zr2O7 (ln = La，Nd，Sm和Gd) 烧绿石的热膨胀
PREAI
分享收藏
查看更多内容