Mitigating OOD overoptimism via in-sample value function in offline reinforcement learning

doi:10.1016/j.neunet.2026.108763

2026-02-23

PRE

DOI：10.1016/j.neunet.2026.108763

原文链接

原文求助

摘要

En 中文

• 提出了一种新颖的方法，即样本内分位数价值正则化（IEVR），该方法将样本内价值函数识别为一种有效的自然约束，用于限制离线强化学习中的分布外高估。 • 建立了一个连接样本内学习与价值正则化方法的实用框架，既增强了样本内学习方法的扩展性，又简化了价值正则化方法的实现难度。 • 通过理论分析和广泛的实证评估证明了IEVR的有效性，并在D4RL基准测试的广泛任务中实现了最先进的表现。

Keyword:

IEVR

offline reinforcement learning

out-of-distribution overestimation

value regularization

in-sample value function

期刊

Neural Networks

IF:

6.3

论文数:

8.2K

被引数:

3.0W

机构

east china normal university

学者数:

3.1W

论文数: 2.1W

被引数: 25

tsinghua university

学者数:

11.9W

论文数: 10.0W

被引数: 137

引用论文

Diffusion policy distillation for offline reinforcement learning 离线强化学习中的扩散策略蒸馏

Neural Networks

IF0

2025-10-01

PREAI

Zhang,Jiazhi; Cheng,Yuhu; Chen,C.L. Philip; Zhang,Hengrui; Wang,Xuesong

Are Expressive Models Truly Necessary for Offline RL?

Proceedings of the AAAI Conference on Artificial Intelligence

IF0

PREAI

Wang,Guan; Niu,Haoyi; Li,Jianxiong; Jiang,Li; Hu,Jianming; Zhan,Xianyuan

Adaptive pessimism via target Q-value for offline reinforcement learning 基于目标q值的自适应悲观离线强化学习

NEURAL NETWORKS

IF6.3

2024-12-01

OAAI

Liu, Jie; Zhang, Yinmin; Li, Chuming; Yang, Yaodong; Liu, Yu; Ouyang, Wanli

Kotb, M.; Weber, C.; Hafez, M.B.; Wermter, S. QT-TDM: Planning with transformer dynamics model and autoregressive q-learning. IEEE Robot. Autom. Lett. 2024, 10, 112–119. [Google Scholar] [CrossRef]Kotb, M.; Weber, C.; Hafez, M.B.; Wermter, S. QT-TDM: 基于Transformer动态模型和自回归Q学习的规划。IEEE Robot. Autom. Lett. 2024, 10, 112–119. [Google Scholar] [CrossRef]

IF0

2025-01-01

PREAI

Mostafa Kotb; Cornelius Weber; Muhammad Burhan Hafez; Stefan Wermter

Human-level control through deep reinforcement learning 通过深度强化学习实现人类层面的控制

Nature

IF0

2015-02-25

PREAI

Volodymyr Mnih; Koray Kavukcuoglu; David Silver; Andrei A. Rusu; Joel Veness; Marc G. Bellemare; Alex Graves; Martin Riedmiller; Andreas K. Fidjeland; Georg Ostrovski; Stig Petersen; Charles Beattie; Amir Sadik; Ioannis Antonoglou; Helen King; Dharshan Kumaran; Daan Wierstra; Shane Legg; Demis Hassabis

Grandmaster level in StarCraft II using multi-agent reinforcement learning 使用多智能体强化学习在星际争霸II中的大师水平

Nature

IF0

2019-10-30

PREAI

Oriol Vinyals; Igor Babuschkin; Wojciech M. Czarnecki; Michaël Mathieu; Andrew Dudzik; Junyoung Chung; David H. Choi; Richard Powell; Timo Ewalds; Petko Georgiev; Junhyuk Oh; Dan Horgan; Manuel Kroiss; Ivo Danihelka; Aja Huang; Laurent Sifre; Trevor Cai; John P. Agapiou; Max Jaderberg; Alexander S. Vezhnevets; Rémi Leblond; Tobias Pohlen; Valentin Dalibard; David Budden; Yury Sulsky; James Molloy; Tom L. Paine; Caglar Gulcehre; Ziyu Wang; Tobias Pfaff; Yuhuai Wu; Roman Ring; Dani Yogatama; Dario Wünsch; Katrina McKinney; Oliver Smith; Tom Schaul; Timothy Lillicrap; Koray Kavukcuoglu; Demis Hassabis; Chris Apps; David Silver

Model-Free Load Frequency Control of Nonlinear Power Systems Based on Deep Reinforcement Learning 基于深度强化学习的非线性电力系统无模型负荷频率控制

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

IF9.9

2024-04-01