.png){kind=icon}
Return
LevDetectCode: Zero-Shot Detection of AI-Generated Code using Levenshtein Distance
.png){kind=icon}
PRE
AI
DOI：10.1142/S0218194025500524.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
AI-generated code is spreading rapidly, making it harder to trace the true origins of code. Text-based tools such as DetectGPT perform well on texts but struggle with code. We propose a new approach: under masked reconstruction, code produced by LLMs exhibits a much wider spread in Levenshtein distance than human-written code. Our key innovation is to treat this Levenshtein distance dispersion as an indicator for token-probability differences, leading us to develop LevDetectCode. Our zero-shot detector relies solely on string-level Levenshtein distance, requiring neither GPUs nor access to model internals. Experiments on Python snippets from MBPP-train and HumanEval, as well as Java data from CodeContest_Java_test, using GPT-3.5, GPT-4, and Deepseek-V3, show a 5-10 point AUROC improvement over other zero-shot baselines, with an additional case study for the C++ dataset, which also demonstrates strong effectiveness. Furthermore, it runs nearly 3000 times faster and significantly reduces memory usage. These results demonstrate that Levenshtein distance can outperform other zero-shot methods that rely on model-based computations, offering a practical and portable solution for detecting AI-generated code.
Keywords:
Large language models
zero-shot
AI-generated code
Levenshtein distance
GPT-4
Deepseek-V3
Journal
I
IF:
0.6
Papers:
107
Citations:
543
Organization
No organization information available
Cited Papers
.png){kind=icon}
ShareSave
ShareSave
ShareSaveShareSave no more
no more