Data-Driven Safe Policy Optimization for Black-Box Dynamical Systems With Temporal Logic Specifications

doi:10.1109/TNNLS.2023.3339885

Data-Driven Safe Policy Optimization for Black-Box Dynamical Systems With Temporal Logic Specifications

2025-02-01

0

PRE

AI

DOI：10.1109/TNNLS.2023.3339885

Original

Original request for help

Save

Abstract

En 中文

Learning-based policy optimization methods have shown great potential for building general-purpose control systems. However, existing methods still struggle to achieve complex task objectives while ensuring policy safety during learning and execution phases for black box systems. To address these challenges, we develop data-driven safe policy optimization ((DSPO)-S-2), a novel reinforcement learning (RL)-based policy improvement method that jointly learns a control barrier function (CBF) for system safety and a linear temporal logic (LTL) guided RL algorithm for complex task objectives. Unlike many existing works that assume known system dynamics, by carefully constructing the data sets and redesigning the loss functions of (DSPO)-S-2, a provably safe CBF is learned for black-box dynamical systems, which continuously evolves for improved system safety as RL interacts with the environment. To deal with complex task objectives, we take advantage of the capability of LTL in representing the task progress and develop LTL-guided RL policy for efficient completion of various tasks with LTL objectives. Extensive numerical and experimental studies demonstrate that (DSPO)-S-2 outperforms most state-of-the-art (SOTA) baselines and can achieve over 95% safety rate and nearly 100% task completion rates. The experiment video is available at https://youtu.be/2RgaH-zcmkY.

Keywords:

Control barrier function (CBF)

linear temporal logic (LTL)

reinforcement learning (RL)

safe policy optimization

Data-Driven Safe Policy Optimization for Black-Box Dynamical Systems With Temporal Logic Specifications

Abstract

Journal

Organization