Self-Prompt: Gradient-guided discrete prompt optimization for code generation

doi:10.1016/j.neucom.2026.134963

Self-Prompt: Gradient-guided discrete prompt optimization for code generation

2026-08-31

0

PRE

AI

DOI：10.1016/j.neucom.2026.134963

Original

Original request for help

Save

Abstract

En 中文

While large language models demonstrate substantial capabilities in code generation, their execution reliability remains highly sensitive to prompt formulations. Automated prompt engineering techniques have alleviated the inefficiency of manual prompt design; nevertheless, existing discrete search methods primarily depend on heuristic linguistic priors, restricting optimization to suboptimal semantic boundaries. In this paper, we propose Self-Prompt, a gradient-driven optimization framework that bridges continuous gradient descent and discrete prompt synthesis. Rather than conducting heuristic searches within the discrete text space, our method decouples prompt generation into natural language contextual scaffolding and gradient-guided discrete trigger extraction. By formulating prompt extraction as a parameter-frozen pseudo-training procedure, Self-Prompt simulates the optimization dynamics of conventional fine-tuning while restricting all updates to a virtual embedding layer. The learned continuous representation is subsequently projected into the discrete vocabulary space, producing task-aligned trigger tokens that preserve the optimization signals acquired during gradient descent without modifying the underlying model parameters. Comprehensive evaluations on eleven diverse open-weight LLMs across the HumanEval, MBPP, and EvalPlus benchmarks demonstrate that Self-Prompt effectively circumvents the reliance on heuristic text search, achieving overall performance gains and favorable cross-model stability across the majority of evaluated configurations.

Self-Prompt: Gradient-guided discrete prompt optimization for code generation

Abstract

Journal

Organization