Computing control invariant set using reinforcement learning by leveraging default value function

doi:10.1016/j.compchemeng.2025.109249

Computing control invariant set using reinforcement learning by leveraging default value function

2025-06-30

0

OA

AI

Benjamin Decardi‐Nelson

DOI：10.1016/j.compchemeng.2025.109249

Original

Original request for help

Save

Abstract

En 中文

This paper presents a framework for computing the control invariant set (CIS) of a general nonlinear system using reinforcement learning (RL). By formulating the CIS problem as an RL task with a specially tailored reward design and value initialization, the proposed approach identifies safe states as those converging to a nonnegative value function, while states leading to constraint violation accumulate negative values. This distinction allows for a straightforward classification of the safe region without requiring complex set-based operations or assumptions on the shape of the invariant set. The paper provides a theoretical analysis showing that, in the absence of disturbances, states lying within the CIS converge to an optimal value of zero, whereas any trajectory that eventually violates the constraint attains strictly negative values. Leveraging standard RL algorithms, ranging from value iteration to Q-learning and deep Q-learning, this method is readily applicable to both low-dimensional and relatively high-dimensional systems with discrete or continuous state spaces. Numerical demonstrations on continuously stirred tank reactors illustrate that the proposed RL-based framework achieves results comparable to existing graph-based methods. These findings highlight the potential of RL, with only minimal modifications, as a data-driven tool to approximate and certify control invariant sets for complex process systems.

Keywords:

Control invariant set

Reinforcement learning

Computing control invariant set using reinforcement learning by leveraging default value function

Abstract

Journal

Organization