arrow
Return

Automated robustness testing for LLM-based natural language processing software

delete2025-12-04
delete0
PRE
AI
M
Mingxuan Xiao
X
Xiao Yan
S
Shunhui Ji
H
Hanbo Cai
L
Lei Xue
P
Pengcheng Zhang
DOI:10.1016/j.eswa.2025.130642delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
• Propose AORTA, an automated robustness testing framework for LLM-based NLP software. • Develop ABS, an efficient testing method using adaptive beam search and backtracking. • Achieve an average test validity of 86.1. • Reduce test cost by up to 3442s and 219 queries per case over the strongest baseline.

Journal

Expert Systems with Applications cover
Expert Systems with Applications
IF:
7.5
Papers:
3.0W
Citations:
10.2W

Organization

H
Hohai University
Scholars:
2.3W
Papers: 1.8W
Citations: 2.1W
S
Suzhou Vocational Institute of Industrial Technology
Scholars:
13
Papers: 9
Citations: 224
Cited Papers

Cited Papers

Software Testing With Large Language Models: Survey, Landscape, and Vision
err2024-04-01
err31
errOAAI
errWang, Junjie; Huang, Yuchao; Chen, Chunyang; Liu, Zhe; Wang, Song; Wang, Qing
errShare
errSave
Keeper: Automated Testing and Fixing of Machine Learning Software
err2024-09-27
err0
errOAAI
errWan, Chengcheng; Li, Shicheng; Xie, Sophie; Liu, Yuhan; Hoffmann, Henry; Maire, Michael; Lu, Shan
errShare
errSave
errShare
errSave
errShare
errSave
errShare
errSave
Managing extreme AI risks amid rapid progress
err2024-05-24
err0
errOAAI
errYoshua Bengio; Geoffrey Hinton; Andrew Yao; Dawn Song; Pieter Abbeel; Trevor Darrell; Yuval Noah Harari; Ya-Qin Zhang; Lan Xue; Shai Shalev-Shwartz; Gillian Hadfield; Jeff Clune; Tegan Maharaj; Frank Hutter; Atılım Güneş Baydin; Sheila McIlraith; Qiqi Gao; Ashwin Acharya; David Krueger; Anca Dragan; Philip Torr; Stuart Russell; Daniel Kahneman; Jan Brauner; Sören Mindermann
errShare
errSave
Assessing the Robustness of Test Selection Methods for Deep Neural Networks
err2025-01-29
err0
PREAI
errQiang Hu; Yuejun Guo; Xiaofei Xie; Maxime Cordy; Wei Ma; Mike Papadakis; Lei Ma; Yves Le Traon
errShare
errSave
researcher View more