.png){kind=icon}
Return
Multiagent Trust Region Policy Optimization
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2023.3265358.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We extend trust region policy optimization (TRPO) to cooperative multiagent reinforcement learning (MARL) for partially observable Markov games (POMGs). We show that the policy update rule in TRPO can be equivalently transformed into a distributed consensus optimization for networked agents when the agents' observation is sufficient. By using a local convexification and trust-region method, we propose a fully decentralized MARL algorithm based on a distributed alternating direction method of multipliers (ADMM). During training, agents only share local policy ratios with neighbors via a peer-to-peer communication network. Compared with traditional centralized training methods in MARL, the proposed algorithm does not need a control center to collect global information, such as global state, collective reward, or shared policy and value network parameters. Experiments on two cooperative environments demonstrate the effectiveness of the proposed method.
Keywords:
Optimization
Training
Approximation algorithms
Convergence
Games
Gradient methods
Observability
Decentralized learning
multiagent reinforcement learning (MARL)
partially observable
trust region policy optimization (TRPO)
Journal
IF:
8.9
Papers:
7.6K
Citations:
7.2W
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSaveShareSaveShareSaveShareSave
View more