Heuristically-Accelerated Multiagent Reinforcement Learning

Reinaldo A. C. Bianchi

doi:10.1109/TCYB.2013.2253094

Heuristically-Accelerated Multiagent Reinforcement Learning

2014-02-01

57

PRE

AI

R

Reinaldo A. C. Bianchi *

M

Murilo Fernandes Martins

C

Carlos H. C. Ribeiro

A

Anna Helena Reali Costa

DOI：10.1109/TCYB.2013.2253094

Original

Original request for help

Save

Abstract

En 中文

This paper presents a novel class of algorithms, called Heuristically-Accelerated Multiagent Reinforcement Learning (HAMRL), which allows the use of heuristics to speed up well-known multiagent reinforcement learning (RL) algorithms such as the Minimax-Q. Such HAMRL algorithms are characterized by a heuristic function, which suggests the selection of particular actions over others. This function represents an initial action selection policy, which can be handcrafted, extracted from previous experience in distinct domains, or learnt from observation. To validate the proposal, a thorough theoretical analysis proving the convergence of four algorithms from the HAMRL class (HAMMQ, HAMQ(lambda), HAMQS, and HAMS) is presented. In addition, a comprehensive systematical evaluation was conducted in two distinct adversarial domains. The results show that even the most straightforward heuristics can produce virtually optimal action selection policies in much fewer episodes, significantly improving the performance of the HAMRL over vanilla RL algorithms.

Keywords:

Artificial intelligence

heuristic algorithms

machine learning

multiagent systems

Heuristically-Accelerated Multiagent Reinforcement Learning

Abstract

AI Summary

Journal

Organization