Active reward learning with a novel acquisition function

doi:10.1007/s10514-015-9454-z

Active reward learning with a novel acquisition function

2015-07-16

39

PRE

AI

DOI：10.1007/s10514-015-9454-z

Original

Original request for help

Save

Abstract

En 中文

Reward functions are an essential component of many robot learning methods. Defining such functions, however, remains hard in many practical applications. For tasks such as grasping, there are no reliable success measures available. Defining reward functions by hand requires extensive task knowledge and often leads to undesired emergent behavior. We introduce a framework, wherein the robot simultaneously learns an action policy and a model of the reward function by actively querying a human expert for ratings. We represent the reward model using a Gaussian process and evaluate several classical acquisition functions (AFs) from the Bayesian optimization literature in this context. Furthermore, we present a novel AF, expected policy divergence. We demonstrate results of our method for a robot grasping task and show that the learned reward function generalizes to a similar task. Additionally, we evaluate the proposed novel AF on a real robot pendulum swing-up task.

Keywords:

Reinforcement learning

Active learning

Bayesian optimization

Preference learning

Inverse reinforcement learning

Reward functions

Acquisition functions

Active reward learning with a novel acquisition function

Abstract

AI Summary

Journal

Organization