Adversarial attack classification and robustness testing for large language models for code

doi:10.1007/s10664-025-10693-3

Adversarial attack classification and robustness testing for large language models for code

2025-08-11

0

PRE

AI

DOI：10.1007/s10664-025-10693-3

Original

Original request for help

Save

Abstract

En 中文

In the rapidly evolving landscape of software development, Large Language models have become essential tools for code tasks such as code generation, completion, analysis, and suggestion. The growing integration of Large Language models for Code into development workflows highlights the critical need for ensuring robustness against potential vulnerabilities–specifically, weaknesses in how these models handle various inputs that could lead to incorrect or insecure code completion. In this context, vulnerabilities can manifest as the model’s susceptibility to producing flawed code when faced with adversarial inputs or subtle perturbations in task descriptions, code, or comments. Existing works have neglected the role of natural language inputs that guide code tasks. This study explores the impact of natural language, such as prompts, on adversarial attacks against Large Language models for Code. It assesses how perturbations at the level of character, word, and sentence affect model output. Identifying the most harmful vulnerabilities will inform the development of more resilient code-generation tools. We analyzed several key projects (including ReCode and Openattack) and datasets (including HumanEval and MBPP), to systematically examine how different types of attacks affect model performance. The first dimension of our taxonomy categorizes adversarial attacks based on the content type–whether the attack targets the code, natural language prompts, or comments. The second dimension classifies these attacks by their granularity, such as character-level, word-level, or sentence/statement-level perturbations. We employed a mixed-methods approach, combining quantitative analysis of model performance metrics with qualitative assessments to identify specific vulnerabilities across different types of inputs. Our analysis revealed that Large Language models for Code (LLM4Code) exhibit varying levels of robustness depending on the type and granularity of adversarial perturbations. Sentence-level perturbations were generally met with the highest resilience, indicating that LLM4Code can maintain code integrity when faced with broader contextual disruptions. However, word-level perturbations posed significant challenges, revealing vulnerabilities in the models’ ability to handle disruptions at this semantic level. Character-level perturbations showed mixed results, with models displaying both strengths and weaknesses in addressing minor syntactical deviations. These findings underscore the need for continued advancements in model robustness, particularly against semantic perturbations, to ensure reliable performance in diverse adversarial scenarios. This study presents a comprehensive framework for evaluating the robustness of Large Language models for Code. It highlights the significance of incorporating code and natural language inputs in adversarial testing, which is essential for improving security and reliability in real-world applications.

Keywords:

Large language models

Large language models for code

Model robustness

Adversarial attack classification

Perturbation

Adversarial attack classification and robustness testing for large language models for code

Abstract

Journal

Organization