.png){kind=icon}
.png){kind=icon}
Return
LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation
.png){kind=icon}
PRE
AI
Z
Y
H
J
Y
Z
张
DOI：10.1109/tevc.2026.3656374.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recently, combining the strength of large language models (LLMs) and evolutionary computation (EC) has shown promising results for addressing optimization problems. It typically involves either iterative next-step solution seeking or directly prompting LLMs to generate critical optimization codes. However, these methods often suffer from low computational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. LLaMoCo features a comprehensive instruction set that includes code-style problem descriptions as input prompts and robust optimization codes from expert EC optimizers as target outputs. We then develop a novel two-phase learning strategy with a contrastive learning-based warm-up to enhance convergence during instruction tuning. Extensive experiments demonstrate that a CodeGen (350M) model tuned by our LLaMoCo yields a powerful domain-specific model for generating high-performance optimizers, achieving superior performance compared to GPT-4 family and other competitors on both synthetic and realistic problem sets.
Keywords:
Black-box optimization
code generation
evolutionary computation (EC)
large language model (LLM)
Journal
IF:
12
Papers:
1.8K
Citations:
2.4W
Organization
Cited Papers