.png){kind=icon}
返回
Grammar Based Directed Testing of Machine Learning Systems
.png){kind=icon}
OA
AI
DOI：10.1109/TSE.2019.2953066.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The massive progress of machine learning has seen its application over a variety of domains in the past decade. But how do we develop a systematic, scalable and modular strategy to validate machine-learning systems? We present, to the best of our knowledge, the first approach, which provides a systematic test framework for machine-learning systems that accepts grammar-based inputs. Our Ogma approach automatically discovers erroneous behaviours in classifiers and leverages these erroneous behaviours to improve the respective models. Ogma leverages inherent robustness properties present in any well trained machine-learning model to direct test generation and thus, implementing a scalable test generation methodology. To evaluate our Ogma approach, we have tested it on three real world natural language processing (NLP) classifiers. We have found thousands of erroneous behaviours in these systems. We also compare Ogma with a random test generation approach and observe that Ogma is more effective than such random test generation by up to 489 percent.
Keyword:
Machine learning
Grammar
Robustness
Systematics
Test pattern generators
Natural language processing
Software testing
machine learning
natural language processing
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
5.6
论文数:
2.9K
被引数:
1.1W
机构
引用论文
.png){kind=icon}
Welding characteristics of aluminum, copper, nickel and aluminum alloy with alumina coating using ultrasonic complex vibration welding equipments铝、铜、镍及铝合金氧化铝涂层超声复合振动焊接特性研究
PREAI
分享收藏Thermal expansions of Ln2Zr2O7 (Ln = La, Nd, Sm, and Gd) pyrochloreLn2Zr2O7 (ln = La，Nd，Sm和Gd) 烧绿石的热膨胀
PREAI
分享收藏分享收藏
查看更多内容