Geodesic Gaussian kernels for value function approximation

Masashi Sugiyama

doi:10.1007/s10514-008-9095-6

Geodesic Gaussian kernels for value function approximation

2008-07-09

21

PRE

AI

DOI：10.1007/s10514-008-9095-6

Original

Original request for help

Save

Abstract

En 中文

The least-squares policy iteration approach works efficiently in value function approximation, given appropriate basis functions. Because of its smoothness, the Gaussian kernel is a popular and useful choice as a basis function. However, it does not allow for discontinuity which typically arises in real-world reinforcement learning tasks. In this paper, we propose a new basis function based on geodesic Gaussian kernels, which exploits the non-linear manifold structure induced by the Markov decision processes. The usefulness of the proposed method is successfully demonstrated in simulated robot arm control and Khepera robot navigation.

Keywords:

reinforcement learning

value function approximation

Markov decision process

least-squares policy iteration

Gaussian kernel

Geodesic Gaussian kernels for value function approximation

Abstract

Journal

Organization