.png){kind=icon}
返回
Variable Renaming-Based Adversarial Test Generation for Code Model: Benchmark and Enhancement
.png){kind=icon}
PRE
AI
DOI：10.1145/3723353.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Robustness testing is essential for evaluating deep learning models, particularly under unforeseen circumstances. Adversarial test generation, a fundamental approach in robustness testing, is prevalent in computer vision and natural language processing, and it has gained considerable attention in code tasks recently. The Variable Renaming-Based Adversarial Test Generation (VRTG), which deceives models by altering variable names, is a key focus. VRTG involves substitution construction and variable name searching, but its systematic design remains a challenge due to the empirical nature of these components. This article introduces the first benchmark to examine the impact of various substitutions and search algorithms on VRTG effectiveness, exploring improvements for existing VRTGs. Our benchmark includes three substitution construction types, six substitution position rank ways and seven search algorithms. Analysis of four code understanding tasks and three pre-trained code models using our benchmark reveals that combining RNNS and Genetic Algorithm with code-based substitution is more effective for VRTG construction. Notably, this method outperforms the advanced black-box variable renaming test generation technique, ALERT, by up to 22.57%.
Keyword:
adversarial testing
deep code model
benchmark
期刊
A
IF:
6.2
论文数:
1.2K
被引数:
3.4K
机构
引用论文
Metamorphic Testing For Machine Learning: Applicability, Challenges, and Research Opportunities
.png){kind=icon}
PREAI
分享收藏
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and GenerationCodeT5: 用于代码理解和生成的标识符感知的统一预训练编码器-解码器模型
OAAI
OAAI分享收藏
查看更多内容