Data-Efficient Reinforcement Learning for Complex Nonlinear Systems

doi:10.1109/TCYB.2023.3324601

Data-Efficient Reinforcement Learning for Complex Nonlinear Systems

2024-03-01

5

PRE

AI

DOI：10.1109/TCYB.2023.3324601

Original

Original request for help

Save

Abstract

En 中文

This article proposes a data-efficient model-free reinforcement learning (RL) algorithm using Koopman operators for complex nonlinear systems. A high-dimensional data-driven optimal control of the nonlinear system is developed by lifting it into the linear system model. We use a data-driven model-based RL framework to derive an off-policy Bellman equation. Building upon this equation, we deduce the data-efficient RL algorithm, which does not need a Koopman-built linear system model. This algorithm preserves dynamic information while reducing the required data for optimal control learning. Numerical and theoretical analyses of the Koopman eigenfunctions for dataset truncation are discussed in the proposed model-free data-efficient RL algorithm. We validate our framework on the excitation control of the power system.

Keywords:

Data-driven control

Koopman theory

optimal control

reinforcement learning (RL)

Data-Efficient Reinforcement Learning for Complex Nonlinear Systems

Abstract

Journal

Organization