Kernel-Based Decentralized Policy Evaluation for Reinforcement Learning

doi:10.1109/TNNLS.2024.3453036

Kernel-Based Decentralized Policy Evaluation for Reinforcement Learning

2024-01-01

0

PRE

AI

DOI：10.1109/TNNLS.2024.3453036

Original

Original request for help

Save

Abstract

En 中文

We investigate the decentralized nonparametric policy evaluation problem within reinforcement learning (RL), focusing on scenarios where multiple agents collaborate to learn the state-value function using sampled state transitions and privately observed rewards. Our approach centers on a regression-based multistage iteration technique employing infinite-dimensional gradient descent (GD) within a reproducing kernel Hilbert space (RKHS). To make computation and communication more feasible, we employ Nystrom approximation to project this space into a finite-dimensional one. We establish statistical error bounds to describe the convergence of value function estimation, marking the first instance of such analysis within a fully decentralized nonparametric framework. We compare the regression-based method to the kernel temporal difference (TD) method in some numerical studies.

Keywords:

Estimation

Kernel

Convergence

Hilbert space

Function approximation

Approximation algorithms

Vectors

Gradient descent (GD)

multiagent reinforcement learning (MARL)

policy iteration

reinforcement learning (RL)

reproducing kernel Hilbert space (RKHS)

state-value function

Kernel-Based Decentralized Policy Evaluation for Reinforcement Learning

Abstract

Journal

Organization

Cited Papers