.png){kind=icon}
返回
Incremental model-based reinforcement learning with model constraint
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2025.107245.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In model-based reinforcement learning (RL) approaches, the estimated model of areal environment is learned with limited data and then utilized for policy optimization. Asa result, the policy optimization process in model-based RL is influenced by both policy and estimated model updates. In practice, previous model-based RL methods only perform incremental policy constraint to policy updates, which cannot assure the complete incremental updates, thereby limiting the algorithm's performance. To address this issue, we propose an incremental model-based RL update scheme by analyzing the policy optimization procedure of model-based RL. This scheme includes both an incremental model constraint that guarantees incremental updates to the estimated model, and an incremental policy constraint that ensures incremental updates to the policy. Further, we establish a performance bound incorporating the incremental model-based RL update scheme between the real environment and the estimated model, which can assure non-decreasing policy performance improvement in the real environment. To implement the incremental model-based RL update scheme, we develop a simple and efficient model-based RL algorithm known as IMPO (Incremental Model-based Policy Optimization), which leverages previous knowledge to enhance stability during the learning process. Experimental results across various control benchmarks demonstrate that IMPO significantly outperforms previous state-of-the-art model-based RL methods in terms of overall performance and sample efficiency.
Keyword:
Model-based reinforcement learning
Monotonic performance improvement
Model constraint
Policy optimization
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
Parameterized MDPs and Reinforcement Learning Problems--A Maximum Entropy Principle-Based Framework
OAAI
OAAI分享收藏Causal inference multi-agent reinforcement learning for traffic signal control交通信号控制的因果推理多智能体强化学习.png)
INFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
分享收藏Dynamic sparse coding-based value estimation network for deep reinforcement learning基于动态稀疏编码的深度强化学习价值估计网络NEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
分享收藏Modeling Bellman-error with logistic distribution with applications in reinforcement learningNEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
分享收藏Study of bi-directional buck-boost converter topologies for application in electrical vehicle motor drives应用于电动汽车电机驱动的双向buck-boost变换器拓扑研究
PREAI
分享收藏分享收藏Deep reinforcement learning towards real-world dynamic thermal management of data centers面向真实世界的数据中心动态热管理的深度强化学习APPLIED ENERGY
APPLIED ENERGYIF11
PREAI
分享收藏Optimistic reinforcement learning by forward Kullback-Leibler divergence optimization基于前向Kullback-Leibler散度优化的乐观强化学习NEURAL NETWORKS
OAAI
NEURAL NETWORKSIF6.3
OAAI分享收藏
查看更多内容