Code Language Models for Security Patch Management: How Far are We?

doi:10.1109/tsc.2026.3690594

Code Language Models for Security Patch Management: How Far are We?

2026-05-05

0

PRE

AI

DOI：10.1109/tsc.2026.3690594

Original

Original request for help

Save

Abstract

En 中文

The rapid expansion of open-source software has also brought significant security challenges to cloud infrastructure, particularly introducing and propagating vulnerabilities. In response, effective security patch management establishes a continuous, structured pipeline by systematically identifying, testing, and deploying security patches to fix vulnerabilities. However, manually managing a large number of security patches (i.e., any update is approved and installed by hand) is time-consuming, leading to a great motivation for automating this process. Although Code Language Models (CodeLMs) have shown potential in various code-centric tasks, there remains an open question as to how well CodeLMs perform within the context of security patch management. To bridge this gap, we performed the first comprehensive empirical study on fine-tuning or prompting nine state-of-the-art CodeLMs for three security-patch-related downstream tasks, including silent patch identification (distinguishing security patches from normal commits), record-patch linking (connecting authoritative vulnerability records, e.g., CVE, to the corresponding fixing commits), and vulnerability description generation (providing a piece of text summarizing the vulnerability fixed by the patch), covering classification, ranking, and generation problems. Our findings reveal that there is no “one-size-fits-all” model that can always perform the best. Furthermore, due to the lack of task-specific knowledge, naively prompting LLMs with the basic strategies is not consistently reliable and may even underperform smaller PTMs. Additionally, existing automated evaluation metrics cannot fully reflect the capability of LLMs in considered tasks. These findings underscore the considerable gap between current capabilities and the practical requirements for deploying CodeLMs in automating security patch management.

Keywords:

Automated patch management

large language models

OSS vulnerabilities

Code Language Models for Security Patch Management: How Far are We?

Abstract

Journal

Organization