Performance-Aligned LLMs for Generating Fast HPC Code

doi:10.1109/TPDS.2026.3675550

2026-03-18

PRE

DOI：10.1109/TPDS.2026.3675550

原文链接

原文求助

摘要

En 中文

优化科学软件是一项艰巨的任务，因为代码库通常规模庞大且复杂，性能可能取决于算法、实现方式、硬件等多种因素。性能不佳的原因可能源自不同来源，且难以诊断。近年来，大量研究利用大型语言模型（LLMs）辅助软件开发任务。然而，这些工具被训练用于建模代码的文本分布，并非专门设计用于理解代码的性能方面。在本工作中，我们提出了一种基于强化学习的方法论，使代码LLMs的输出与性能对齐。这使我们能够在现有LLMs的代码建模能力基础上进行扩展，生成性能更优的代码。我们证明了，我们的微调模型在一系列基准任务中，将生成的串行代码的预期加速比从0.9提高到1.6，将OpenMP并行代码的预期加速比从1.9提高到4.5。

Keyword:

Large language models (LLMs)

code generation

performance optimization

reinforcement learning

期刊

IEEE Transactions on Parallel and Distributed Systems

IF:

论文数:

5.2K

被引数:

1.1W

机构

lawrence livermore national laboratory

学者数:

653

论文数: 234

被引数: 0

university of maryland

学者数:

4.4K

论文数: 2.1K

被引数: 1

引用论文

HPC-Coder: Modeling Parallel Programs using Large Language Models

ISC High Performance 2024 Research Paper Proceedings (39th International Conference)

IF0

2024-05-01

OAAI

Daniel Nichols; Aniruddha Marathe; Harshitha Menon; Todd Gamblin; Abhinav Bhatele

A training algorithm for optimal margin classifiers

Proceedings of the fifth annual workshop on Computational learning theory

IF0

1992-07-01

OAAI

Bernhard E. Boser; Isabelle M. Guyon; Vladimir N. Vapnik

Attention Is All You Need 注意是你所需要的

IF0

OAAI

ChatGPT outperforms crowd workers for text-annotation tasks ChatGPT在文本注释任务方面胜过人群工作者

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA

IF9.1

2023-07-18

125

OAAI

Gilardi, Fabrizio; Alizadeh, Meysam; Kubli, Mael

Competition-level code generation with AlphaCode 使用字母码生成竞争级代码

Science

IF0

2022-12-09

OAAI

Yujia Li; David Choi; Junyoung Chung; Nate Kushman; Julian Schrittwieser; Rémi Leblond; Tom Eccles; James Keeling; Felix Gimeno; Agustin Dal Lago; Thomas Hubert; Peter Choy; Cyprien de Masson d’Autume; Igor Babuschkin; Xinyun Chen; Po-Sen Huang; Johannes Welbl; Sven Gowal; Alexey Cherepanov; James Molloy; Daniel J. Mankowitz; Esme Sutherland Robson; Pushmeet Kohli; Nando de Freitas; Koray Kavukcuoglu; Oriol Vinyals

LM4HPC: Towards Effective Language Model Application in High-Performance Computing LM4HPC：面向高性能计算中语言模型的有效应用

Lecture Notes in Computer Science

IF0

2023-09-01

PREAI

Le Chen; Pei-Hung Lin; Tristan Vanderbruggen; Chunhua Liao; Murali Emani; Bronis de Supinski

Faster sorting algorithms discovered using deep reinforcement learning

Nature

IF0

2023-06-07

OAAI

Daniel J. Mankowitz; Andrea Michi; Anton Zhernov; Marco Gelmi; Marco Selvi; Cosmin Paduraru; Edouard Leurent; Shariq Iqbal; Jean-Baptiste Lespiau; Alex Ahern; Thomas Köppe; Kevin Millikin; Stephen Gaffney; Sophie Elster; Jackson Broshear; Chris Gamble; Kieran Milan; Robert Tung; Minjae Hwang; Taylan Cemgil; Mohammadamin Barekatain; Yujia Li; Amol Mandhane; Thomas Hubert; Julian Schrittwieser; Demis Hassabis; Pushmeet Kohli; Martin Riedmiller; Oriol Vinyals; David Silver

A Comprehensive Overview of Large Language Models 大型语言模型的全面概述

ACM Transactions on Intelligent Systems and Technology

IF6.6

2025-08-19

OAAI

Humza Naveed; Asad Ullah Khan; Shi Qiu; Muhammad Saqib; Saeed Anwar; Muhammad Usman; Naveed Akhtar; Nick Barnes; Ajmal Mian

Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons

Biometrika

IF0

1952-12-01

PREAI

Ralph Allan Bradley; Milton E. Terry