Optimizing generative AI by backpropagating language model feedback

doi:10.1038/s41586-025-08661-4

2025-03-19

PRE

DOI：10.1038/s41586-025-08661-4

原文链接

原文求助

摘要

En 中文

近期人工智能（AI）领域的重大突破日益依赖于能够协调多个大型语言模型（LLMs）及其他专用工具（如搜索引擎和模拟器）的系统。目前，这些系统主要是由领域专家手工设计并通过启发式方法进行调整，而非自动优化，这给加速发展带来了巨大挑战。人工神经网络的发展曾面临类似困境，直到反向传播和自动微分的出现通过实现优化自动化而彻底改变了该领域。类似地，本文引入了TextGrad，这是一个能够通过反向传播LLM生成的反馈来优化AI系统的通用框架。通过利用自然语言反馈来评价并改进系统的任何部分——从提示词到分子或治疗方案等输出——TextGrad实现了跨任务生成式AI系统的自动优化。我们通过在解决博士水平科学问题、优化放射治疗计划、设计具有特定性质的分子、编码以及优化智能体系统等方面的研究，证明了TextGrad的通用性和有效性。TextGrad赋能科学家和工程师能够轻松开发具有影响力的生成式AI系统。

Keyword:

ALGORITHMS

期刊

Nature

IF:

48.5

论文数:

1.8W

被引数:

96.5W

机构

Chan Zuckerberg Biohub

学者数:

166

论文数: 90

被引数: 60

引用论文

High-dimensional automated radiation therapy treatment planning via Bayesian optimization 通过贝叶斯优化的高维自动放射治疗治疗计划

MEDICAL PHYSICS

IF3.2

2023-03-04

OAAI

Wang, Qingying; Wang, Ruoxi; Liu, Jiacheng; Jiang, Fan; Yue, Haizhen; Du, Yi; Wu, Hao

Automation in intensity modulated radiotherapy treatment planning—a review of recent innovations

The British Journal of Radiology

IF0

2018-09-04

OAAI

Mohammad Hussein; Ben J M Heijmen; Dirk Verellen; Andrew Nisbet

Competition-level code generation with AlphaCode 使用字母码生成竞争级代码

Science

IF0

2022-12-09

OAAI

Yujia Li; David Choi; Junyoung Chung; Nate Kushman; Julian Schrittwieser; Rémi Leblond; Tom Eccles; James Keeling; Felix Gimeno; Agustin Dal Lago; Thomas Hubert; Peter Choy; Cyprien de Masson d’Autume; Igor Babuschkin; Xinyun Chen; Po-Sen Huang; Johannes Welbl; Sven Gowal; Alexey Cherepanov; James Molloy; Daniel J. Mankowitz; Esme Sutherland Robson; Pushmeet Kohli; Nando de Freitas; Koray Kavukcuoglu; Oriol Vinyals

Faster sorting algorithms discovered using deep reinforcement learning

Nature

IF0

2023-06-07

OAAI

Daniel J. Mankowitz; Andrea Michi; Anton Zhernov; Marco Gelmi; Marco Selvi; Cosmin Paduraru; Edouard Leurent; Shariq Iqbal; Jean-Baptiste Lespiau; Alex Ahern; Thomas Köppe; Kevin Millikin; Stephen Gaffney; Sophie Elster; Jackson Broshear; Chris Gamble; Kieran Milan; Robert Tung; Minjae Hwang; Taylan Cemgil; Mohammadamin Barekatain; Yujia Li; Amol Mandhane; Thomas Hubert; Julian Schrittwieser; Demis Hassabis; Pushmeet Kohli; Martin Riedmiller; Oriol Vinyals; David Silver

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

Findings of the Association for Computational Linguistics: ACL 2023

IF0

2023-01-01

OAAI

Mirac Suzgun; Nathan Scales; Nathanael Schärli; Sebastian Gehrmann; Yi Tay; Hyung Won Chung; Aakanksha Chowdhery; Quoc Le; Ed Chi; Denny Zhou; Jason Wei

Scaling deep learning for materials discovery 扩展深度学习以实现材料发现

NATURE

IF48.5

2023-11-29

210

OAAI

Merchant, Amil; Batzner, Simon; Schoenholz, Samuel S.; Aykol, Muratahan; Cheon, Gowoon; Cubuk, Ekin Dogus

IF0

PREAI

Optuna

Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

IF0