.png){kind=icon}
返回
Distributed Multiagent Reinforcement Learning Based on Graph-Induced Local Value Functions
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2024.3375248.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Achieving distributed reinforcement learning (RL) for large-scale cooperative multiagent systems (MASs) is challenging because: 1) each agent has access to only limited information and 2) issues on scalability and sample efficiency emerge due to the curse of dimensionality. In this article, we propose a general distributed framework for sample efficient cooperative multiagent reinforcement learning (MARL) by utilizing the structures of graphs involved in this problem. We introduce three coupling graphs describing three types of interagent couplings in MARL, namely, the state graph, observation graph, and reward graph. By further considering a communication graph, we propose two distributed RL approaches based on local value functions derived from the coupling graphs. The first approach is able to reduce sample complexity significantly under specific conditions on the aforementioned four graphs. The second approach provides an approximate solution and can be efficient even for problems with dense coupling graphs. Here there is a tradeoff between minimizing the approximation error and reducing the computational complexity. Simulations show that our RL algorithms have a significantly improved scalability to large-scale MASs compared with centralized and consensus-based distributed RL algorithms.
Keyword:
Couplings
Heuristic algorithms
Convergence
Approximation algorithms
Scalability
Reinforcement learning
Indexes
Distributed learning
Markov decision process
multiagent systems
optimal control
reinforcement learning
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
引用论文
Communication-Efficient Policy G ad en Methods for Distributed Reinforcement Learning用于分布式强化学习的通信高效策略和方法
OAAI
.png){kind=icon}
OAAI分享收藏Fabrication of porous hollow γ-Al2O3 nanofibers by facile electrospinning and its application for water remediation静电纺丝法制备多孔中空 γ-Al2O3纳米纤维及其在水体修复中的应用
PREAI
分享收藏
查看更多内容