Model-based average reward reinforcement learning

doi:10.1016/S0004-3702(98)00002-2

Model-based average reward reinforcement learning

1998-04-01

52

PRE

AI

DOI：10.1016/S0004-3702(98)00002-2

Original

Original request for help

Save

Abstract

En 中文

Reinforcement Learning (RL) is the study of programs that improve their performance by receiving rewards and punishments from the environment. Most RL methods optimize the discounted total reward received by an agent, while, in many domains, the natural criterion is to optimize the average reward per time step. In this paper, we introduce a model-based Average-reward Reinforcement Learning method called H-learning and show that it converges more quickly and robustly than its discounted counterpart in the domain of scheduling a simulated Automatic Guided Vehicle (AGV). We also introduce a version of H-learning that automatically explores the unexplored parts of the state space, while always choosing greedy actions with respect to the current value function. We show that this Auto-exploratory H-Learning performs better than the previously studied exploration strategies. To scale H-learning to larger state spaces, we extend it to learn action models and reward functions in the form of dynamic Bayesian networks, and approximate its value function using local linear regression. We show that both of these extensions are effective in significantly reducing the space requirement of H-learning and making it converge faster in some AGV scheduling tasks. (C) 1998 Published by Elsevier Science B.V.

Keywords:

machine learning

Reinforcement Learning

average reward

model-based

exploration

Bayesian networks

linear regression

AGV scheduling

Model-based average reward reinforcement learning

Abstract

AI Summary

Journal

Organization