arrow
Return

Large language models discover complementary heuristics for combinatorial optimization

delete2026-10-01
delete0
PRE
AI
H
Huatian Gong
S
Shuaian Wang
D
Dong‐Ping Song
J
Jiuh‐Biing Sheu
R
Ran Yan *
DOI:10.1038/s42256-026-01307-8delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Combinatorial optimization (CO) underpins decision-making across science, engineering and industry, especially manufacturing, logistics, healthcare operations and energy management. However, designing effective heuristics for each new problem variant requires deep domain knowledge and months of expert iteration. Directly prompting large language models (LLMs) can generate executable code, but single-pass synthesis of complete heuristics routinely fails on new CO problems. Here we introduce LLM-driven Algorithm Construction via Complementary Evolution (LACE), a framework that decomposes algorithm design into an input schema, output schema, tool library and heuristic portfolio (the $${\mathcal{I}}$$ – $${\mathcal{O}}$$ $${\mathcal{T}}$$ $${\mathcal{H}}$$ interface) and refines the portfolio through time-constrained complementary evolution. The interface specifies a verified problem contract under which the LLM operates, directing model capacity to high-level algorithmic reasoning rather than problem-specific implementation. Complementary evolution then iteratively generates and selects specialist heuristics capable of solving heterogeneous instances within strict runtime budgets. Across 36 classical CO-Bench problems, LACE attains an average score of 0.945, against 0.870 for the strongest existing LLM-based method and 0.571 for direct LLM prompting without any framework, locating the gain in the framework rather than the model. On four structurally new problems, LACE reaches 0.97–0.99, while five existing LLM-based baselines fail to produce any feasible algorithm. These results demonstrate that effective algorithm discovery for such problems can be automated to a substantial degree. Huatian Gong and colleagues developed LACE, a large language model-based framework that designs optimization algorithms. It builds a verified problem contract, then evolves a portfolio of complementary heuristics that together solve problems that no single method can handle.

Journal

Nature Machine Intelligence cover
Nature Machine Intelligence
IF:
23.9
Papers:
1.3K
Citations:
1.5W

Organization

N
nanyang technological university
Scholars:
792
Papers: 396
Citations: 0
U
University of Liverpool
Scholars:
473
Papers: 224
Citations: 0
T
the hong kong polytechnic university
Scholars:
573
Papers: 268
Citations: 0
N
national taiwan university
Scholars:
764
Papers: 280
Citations: 0
researcher View more organizations
Cited Papers

Cited Papers

Large language models surpass human experts in predicting neuroscience results
err2024-11-27
err0
errOAAI
errXiaoliang Luo; Akilles Rechardt; Guangzhi Sun; Kevin K. Nejad; Felipe Yáñez; Bati Yilmaz; Kangjoo Lee; Alexandra O. Cohen; Valentina Borghesani; Anton Pashkov; Daniele Marinazzo; Jonathan Nicholas; Alessandro Salatiello; Ilia Sucholutsky; Pasquale Minervini; Sepehr Razavi; Roberta Rocca; Elkhan Yusifov; Tereza Okalova; Nianlong Gu; Martin Ferianc; Mikail Khona; Kaustubh R. Patil; Pui-Shee Lee; Rui Mata; Nicholas E. Myers; Jennifer K. Bizley; Sebastian Musslick; Isil Poyraz Bilgin; Guiomar Niso; Justin M. Ales; Michael Gaebler; N. Apurva Ratan Murty; Leyla Loued-Khenissi; Anna Behler; Chloe M. Hall; Jessica Dafflon; Sherry Dongqi Bao; Bradley C. Love
errShare
errSave
Tool Learning with Foundation Models
err2024-12-24
err0
errOAAI
errQin, YJ; Hu, SD; Lin, YK; Chen, WZ; Ding, N; Cui, GQ; Zeng, ZN; Zhou, XH; Huang, YF; Xiao, CJ; Han, C; Fung, YR; Su, YS; Wang, HD; Qian, C; Tian, RC; Zhu, KL; Liang, SH; Shen, XY; Xu, BK; Zhang, Z; Ye, YN; Li, BW; Tang, ZW; Yi, J; Zhu, YZ; Dai, ZN; Yan, L; Cong, X; Lu, YX; Zhao, WL; Huang, YX; Yan, JX; Han, X; Sun, X; Li, DH; Phang, J; Yang, C; Wu, TS; Ji, H; Li, GL; Liu, ZY; Sun, MS
errShare
errSave
Large Language Models for Combinatorial Optimization: A Systematic Review
err2026-08-30
err0
PREAI
errDa Ros,Francesca; Soprano,Michael; Di Gaspero,Luca; Roitero,Kevin
errShare
errSave
Mathematical discoveries from program search with large language models
err2023-12-14
err0
errOAAI
errBernardino Romera-Paredes; Mohammadamin Barekatain; Alexander Novikov; Matej Balog; M. Pawan Kumar; Emilien Dupont; Francisco J. R. Ruiz; Jordan S. Ellenberg; Pengming Wang; Omar Fawzi; Pushmeet Kohli; Alhussein Fawzi
errShare
errSave
Toolformer: Language Models Can Teach Themselves to Use Tools
err2023-01-01
err0
PREAI
errCancedda,Nicola; Dessi,Roberto; Dwivedi-Yu,Jane; Hambro,Eric; Lomeli,Maria; Raileanu,Roberta; Schick,Timo; Scialom,Thomas; Zettlemoyer,Luke
errShare
errSave
Hyper-heuristics: a survey of the state of the art
err2017-12-21
err0
errOAAI
errEdmund K Burke; Michel Gendreau; Matthew Hyde; Graham Kendall; Gabriela Ochoa; Ender Özcan; Rong Qu
errShare
errSave
ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution
err2024-01-01
err0
PREAI
errBerto,Federico; Cao,Zhiguang; Hua,Chuanbo; Kim,Haeyeon; Park,Jinkyoo; Song,Guojie; Wang,Jiarui; Ye,Haoran
errShare
errSave
Deep active optimization for complex systems
err2025-08-25
err0
errOAAI
errYe Wei; Bo Peng; Ruiwen Xie; Yangtao Chen; Yu Qin; Peng Wen; Stefan Bauer; Po-Yen Tung; Dierk Raabe
errShare
errSave
Highly accurate protein structure prediction with AlphaFold
err2021-07-15
err0
errOAAI
errJohn Jumper; Richard Evans; Alexander Pritzel; Tim Green; Michael Figurnov; Olaf Ronneberger; Kathryn Tunyasuvunakool; Russ Bates; Augustin Žídek; Anna Potapenko; Alex Bridgland; Clemens Meyer; Simon A. A. Kohl; Andrew J. Ballard; Andrew Cowie; Bernardino Romera-Paredes; Stanislav Nikolov; Rishub Jain; Jonas Adler; Trevor Back; Stig Petersen; David Reiman; Ellen Clancy; Michal Zielinski; Martin Steinegger; Michalina Pacholska; Tamas Berghammer; Sebastian Bodenstein; David Silver; Oriol Vinyals; Andrew W. Senior; Koray Kavukcuoglu; Pushmeet Kohli; Demis Hassabis
errShare
errSave
researcher View more