Data-Driven Inverse Reinforcement Learning Control for Linear Multiplayer Games

doi:10.1109/TNNLS.2022.3186229

Data-Driven Inverse Reinforcement Learning Control for Linear Multiplayer Games

2024-02-01

32

PRE

AI

DOI：10.1109/TNNLS.2022.3186229

Original

Original request for help

Save

Abstract

En 中文

This article proposes a data-driven inverse reinforcement learning (RL) control algorithm for nonzero-sum multiplayer games in linear continuous-time differential dynamical systems. The inverse RL problem in the games is solved by a learner reconstructing the unknown expert players' cost functions from demonstrated expert's optimal state and control input trajectories. The learner, thus, obtains the same control feedback gains and trajectories as the expert, only using data along system trajectories without knowing system dynamics. This article first proposes a model-based inverse RL policy iteration framework that has: 1) policy evaluation step for reconstructing cost matrices using Lyapunov functions; 2) state-reward weight improvement step using inverse optimal control (IOC); and 3) policy improvement step using optimal control. Based on the model-based policy iteration algorithm, this article further develops an online data-driven off-policy inverse RL algorithm without knowing any knowledge of system dynamics or expert control gains. Rigorous convergence and stability analysis of the algorithms are provided. It shows that the off-policy inverse RL algorithm guarantees unbiased solutions while probing noises are added to satisfy the persistence of excitation (PE) condition. Finally, two different simulation examples validate the effectiveness of the proposed algorithms.

Keywords:

Games

Cost function

Optimal control

Heuristic algorithms

Trajectory

System dynamics

Costs

Inverse optimal control (IOC)

inverse RL

nonzero-sum Nash games

off-policy

optimal control

Data-Driven Inverse Reinforcement Learning Control for Linear Multiplayer Games

Abstract

Journal

Organization