.png){kind=icon}
返回
Regularization-Adapted Anderson Acceleration for multi-agent reinforcement learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2023.110709.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Originating from model-free reinforcement learning (RL), many modern multi-agent reinforcement learning (MARL) algorithms are usually armed with the paradigm of Centralized Training with De-centralized Execution (CTDE) to mitigate the non-stationary problem and make the training process stable. However, these methods still suffer from sample inefficiency and slow training convergence as in the single-agent reinforcement learning setting. Many common methods aiming to tackle these problems utilize the experience buffer and parallel training mechanism to speed up the training process, which would cost more computing resources and may still underuse the sampled experiences. In this paper, we propose Regularization -Adapted Anderson Acceleration (RA3) for model-free, off-policy MARL algorithms. Under the CTDE paradigm, this specific RA3 approach treats the joint action-value function update as a fixed-point iteration task and speeds up the training process with the same amount of sampled experiences as in the baseline algorithms. Furthermore, our RA3 employs an adaptive regularization strategy related to Bellman residuals to stabilize the update process and enhance the training performance. Experimental results demonstrate that the improved learning speed and superior performance of our proposed method are significantly improved on the predator-prey game and the challenging StarCraft II micromanagement benchmark tasks.& COPY; 2023 Elsevier B.V. All rights reserved.
Keyword:
Reinforcement learning
Decentralized partially observable Markov
decision process (Dec-POMDP)
Multi-agent reinforcement learning
Q-learning
Anderson acceleration
期刊
K
IF:
7.6
论文数:
1.2W
被引数:
4.5W
机构
暂无机构信息
引用论文
.png){kind=icon}
分享收藏Study of bi-directional buck-boost converter topologies for application in electrical vehicle motor drives应用于电动汽车电机驱动的双向buck-boost变换器拓扑研究
PREAI
分享收藏Tripodal schematic control architecture for integration of multi-functional indoor service robots
PREAI
分享收藏分享收藏Event-Driven Off-Policy Reinforcement Learning for Control of Interconnected Systems用于控制互连系统的事件驱动的非策略强化学习
OAAI
OAAI分享收藏
查看更多内容