.png){kind=icon}
返回
Distributed entropy-regularized multi-agent reinforcement learning with policy consensus
.png){kind=icon}
PRE
AI
DOI：10.1016/j.automatica.2024.111652.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Sample efficiency is a limiting factor for existing distributed multi -agent reinforcement learning (MARL) algorithms over networked multi -agent systems. In this paper, the sample efficiency problem is tackled by formally incorporating the entropy regularization into the distributed MARL algorithm design. Firstly, a new entropy -regularized MARL problem is formulated under the model of networked multi -agent Markov decision processes with observation -based policies and homogeneous agents, where the policy parameter sharing among the agents provably preserves the optimality. Secondly, an on -policy distributed actor-critic algorithm is proposed, where each agent shares its parameters of both the critic and actor for consensus update. Then, the convergence analysis of the proposed algorithm is provided based on the stochastic approximation theory under the assumption of linear function approximation of the critic. Furthermore, a practical off -policy version of the proposed algorithm is developed which possesses scalability, data efficiency and learning stability. Finally, the proposed distributed algorithm is compared against the solid baselines including two classic centralized training algorithms in the multi -agent particle environment, whose learning performance is empirically demonstrated through extensive simulation experiments. (c) 2024 Elsevier Ltd. All rights reserved.
Keyword:
Distributed actor-critic algorithm
Networked multi-agent system
Entropy regularization
Deep reinforcement learning
期刊
IF:
5.9
论文数:
1.2W
被引数:
5.2W
机构
引用论文
.png){kind=icon}
A Distributed Actor-Critic Algorithm and Applications to Mobile Sensor Network Coordination Problems
PREAI
分享收藏
查看更多内容