.png){kind=icon}
返回
Shielded Planning Guided Data-Efficient and Safe Reinforcement Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2024.3359031.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Safe reinforcement learning (RL) has shown great potential for building safe general-purpose robotic systems. While many existing works have focused on post-training policy safety, it remains an open problem to ensure safety during training as well as to improve exploration efficiency. Motivated to address these challenges, this work develops shielded planning guided policy optimization (SPPO), a new model-based safe RL method that augments policy optimization algorithms with path planning and shielding mechanism. In particular, SPPO is equipped with shielded planning for guided exploration and efficient data collection via model predictive path integral (MPPI), along with an advantage-based shielding rule to keep the above processes safe. Based on the collected safe data, a task-oriented parameter optimization (TOPO) method is used for policy improvement, as well as the observation-independent latent dynamics enhancement. In addition, SPPO provides explicit theoretical guarantees, i.e., clear theoretical bounds for training safety, deployment safety, and the learned policy performance. Experiments demonstrate that SPPO outperforms baselines in terms of policy performance, learning efficiency, and safety performance during training.
Keyword:
Safety
Training
Planning
Optimization
Trajectory
Predictive models
Costs
Data-efficient exploration
model-based reinforcement learning (RL)
safe policy optimization
shielded planning
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
Carbohydrate metabolism of the cat 1. Activity of amylase in the gastrointestinal tract of the cat1
PREAI
分享收藏Safe exploration in model-based reinforcement learning using control barrier functions.png)
AUTOMATICA
OAAI
AUTOMATICAIF5.9
OAAI分享收藏Predictive Control Barrier Functions: Enhanced Safety Mechanisms for Learning-Based Control预测控制屏障功能: 基于学习的控制的增强安全机制
OAAI
OAAI分享收藏
查看更多内容