Steer Your Model: Secure Code Generation With Contrastive Decoding

doi:10.1109/TSE.2025.3650127

Steer Your Model: Secure Code Generation With Contrastive Decoding

2026-01-05

0

PRE

AI

DOI：10.1109/TSE.2025.3650127

Original

Original request for help

Save

Abstract

En 中文

Large Language Models (LLMs) specialized in code have demonstrated impressive capabilities in various programming tasks such as code generation. However, these models often generate vulnerable code due to inherent flaws in training datasets derived from large-scale, unfiltered open-source repositories. Existing methods like SVEN (prefix tuning) and CoSec (supervised co-decoding) attempt to address these risks but face challenges with transferability or inflexible security constraints. To mitigate these issues, we propose <styled-content style="font-family: sans-serif;" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SCoDE</styled-content>, a two-stage approach for secure and functionally correct code generation. After an initial functional tuning phase, we integrate a plug-and-play security steering matrix at the model’s output embedding layer. This matrix can be transferred across models without modifying their original weights. During inference, we introduce a novel contrastive decoding mechanism that adaptively balances the base model’s functional logits with positive and negative security steering signals. Extensive experiments on 60 security scenarios and two standard benchmarks (HumanEval, MBPP) using StarCoder, Qwen2.5-Coder, and CodeLlama demonstrate that <styled-content style="font-family: sans-serif;" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SCoDE</styled-content> enhances security while maintaining functional correctness. On average, <styled-content style="font-family: sans-serif;" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SCoDE</styled-content> improves security by 28.09% over the original models, 12.07% over CoSec, and 4.82% over SVEN. For functional correctness, it achieves average gains of 55.03% on HumanEval and 41.81% on MBPP over the original models.

Keywords:

Large language models

code generation

code security

contrastive decoding

Steer Your Model: Secure Code Generation With Contrastive Decoding

Abstract

Journal

Organization