.png){kind=icon}
.png){kind=icon}
Return
Inverse Reinforcement Learning for Adversarial Apprentice Games
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2021.3114612.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This article proposes new inverse reinforcement learning (RL) algorithms to solve our defined Adversarial Apprentice Games for nonlinear learner and expert systems. The games are solved by extracting the unknown cost function of an expert by a learner using demonstrated expert's behaviors. We first develop a model-based inverse RL algorithm that consists of two learning stages: an optimal control learning and a second learning based on inverse optimal control. This algorithm also clarifies the relationships between inverse RL and inverse optimal control. Then, we propose a new model-free integral inverse RL algorithm to reconstruct the unknown expert cost function. The model-free algorithm only needs online demonstration of the expert and learner's trajectory data without knowing system dynamics of either the learner or the expert. These two algorithms are further implemented using neural networks (NNs). In Adversarial Apprentice Games, the learner and the expert are allowed to suffer from different adversarial attacks in the learning process. A two-player zero-sum game is formulated for each of these two agents and is solved as a subproblem for the learner in inverse RL. Furthermore, it is shown that the cost functions that the learner learns to mimic the expert's behavior are stabilizing and not unique. Finally, simulations and comparisons show the effectiveness and the superiority of the proposed algorithms.
Keywords:
Games
Cost function
Optimal control
Heuristic algorithms
Costs
Artificial neural networks
System dynamics
Adversarial games
apprentice games
inverse optimal control
inverse reinforcement learning (RL)
neural networks (NNs)
optimal control
Journal
IF:
8.9
Papers:
7.5K
Citations:
7.2W