Sample efficient reinforcement learning via low-rank regularization

doi:10.1016/j.knosys.2025.114176

Sample efficient reinforcement learning via low-rank regularization

2025-07-27

0

PRE

AI

DOI：10.1016/j.knosys.2025.114176

Original

Original request for help

Save

Abstract

En 中文

In this paper, the usefulness of low-rankness in state-action value function estimation is demonstrated using a simplified setup that is amenable to theoretical analysis. First, the concept of low-rank functions is defined motivated by standard functional analysis results. Subsequently, a specific procedure is proposed based on nuclear-norm penalized series estimation, in which the estimation of the low-rank function naturally leads to estimation of a low-rank matrix. Risk bounds are established for the estimator, which shows faster convergence rates compared to the standard estimator without using low-rankness. Several simulated toy examples are used as proof of concept to demonstrate the performances in simulations.

Keywords:

low-rankness

state-action value function

nuclear-norm penalized estimation

risk bounds

convergence rates

Sample efficient reinforcement learning via low-rank regularization

Abstract

Journal

Organization

Cited Papers