Multiagent Continual Coordination via Progressive Task Contextualization

Lei Yuan

doi:10.1109/TNNLS.2024.3394513

Multiagent Continual Coordination via Progressive Task Contextualization

2024-01-01

0

OA

AI

DOI：10.1109/TNNLS.2024.3394513

原文链接

原文求助

摘要

En 中文

Cooperative multiagent reinforcement learning (MARL) has attracted significant attention and has the potential for many real-world applications. Previous arts mainly focus on facilitating the coordination ability from different aspects (e.g., nonstationarity and credit assignment) in single-task or multitask scenarios, ignoring the stream of tasks that appear in a continual manner. This ignorance makes the continual coordination an unexplored territory, neither in problem formulation nor efficient algorithms designed. Toward tackling the mentioned issue, this article proposes an approach, multiagent continual coordination via progressive task contextualization (MACPro). The key point lies in obtaining a factorized policy, using shared feature extraction layers but separated independent task heads, each specializing in a specific class of tasks. The task heads can be progressively expanded based on the learned task contextualization. Moreover, to cater to the popular centralized training with decentralized execution (CTDE) paradigm in MARL, each agent learns to predict and adopt the most relevant policy head based on local information in a decentralized manner. We show in multiple multiagent benchmarks that existing continual learning methods fail, while MACPro is able to achieve close-to-optimal performance. More results also disclose the effectiveness of MACPro from multiple aspects, such as high generalization ability.

Keyword:

Task analysis

Training

Reinforcement learning

Trajectory

Heuristic algorithms

Feature extraction

Continuing education

Continual learning

cooperation and coordination

multiagent system

reinforcement learning

Multiagent Continual Coordination via Progressive Task Contextualization

摘要

期刊

机构

引用论文