Safe Adaptive Policy Transfer Reinforcement Learning for Distributed Multiagent Control

Bin Du

doi:10.1109/TNNLS.2023.3326867

Safe Adaptive Policy Transfer Reinforcement Learning for Distributed Multiagent Control

2025-01-01

4

PRE

AI

张卫东 (Weidong Zhang) *

DOI：10.1109/TNNLS.2023.3326867

Original

Original request for help

Save

Abstract

En 中文

Multiagent reinforcement learning (RL) training is usually difficult and time-consuming due to mutual interference among agents. Safety concerns make an already difficult training process even harder. This study proposes a safe adaptive policy transfer RL approach for multiagent cooperative control. Specifically, a pioneer and follower off-policy policy transfer learning (PFOPT) method is presented to help follower agents acquire knowledge and experience from a single well-trained pioneer agent. Notably, the designed approach can transfer both the policy representation and sample experience provided by the pioneer policy in the off-policy learning. More importantly, the proposed method can adaptively adjust the learning weight of prior experience and exploration according to the Wasserstein distance between the policy probability distributions of the pioneer and the follower. Case studies show that the distributed agents trained by the proposed method can complete a collaborative task and acquire the maximum rewards while minimizing the violation of constraints. Moreover, the proposed method can also achieve satisfactory performance in terms of learning speed and success rate.

Keywords:

Adaptive weight update

distributed multiagent system

policy transfer

safe reinforcement learning (RL)

transfer learning

Safe Adaptive Policy Transfer Reinforcement Learning for Distributed Multiagent Control

Abstract

Journal

Organization