.png){kind=icon}
Return
Sample Complexity and Overparameterization Bounds for Temporal-Difference Learning With Neural Network Approximation
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2023.3234234.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In this article, we study the dynamics of temporal-difference (TD) learning with neural network-based value function approximation over a general state space, namely, neural TD learning. We consider two practically used algorithms, projection-free and max-norm regularized neural TD learning, and establish the first convergence bounds for these algorithms. An interesting observation from our results is that max-norm regularization can dramatically improve the performance of TD learning algorithms in terms of sample complexity and overparameterization. The results in this work rely on a Lyapunov drift analysis of the network parameters as a stopped and controlled random process.
Keywords:
Neural networks
Approximation algorithms
Markov processes
Convergence
Complexity theory
Reinforcement learning
Kernel
reinforcement learning (RL)
stochastic approximation
temporal-difference (TD) learning
Journal
IF:
7
Papers:
1.3W
Citations:
6.7W
Organization
View more organizations
Cited Papers
Comparison of purified psoralen-inactivated and formalin-inactivated dengue vaccines in mice and nonhuman primates.png)
Vaccine
OAAI
VaccineIF0
OAAIShareSaveShareSaveShareSaveInteractions of a model block copolymer drug delivery system with two serum proteins and myoglobin
PREAI
ShareSaveFrequent Activation of c-
kis
as a Transforming Gene in Fibrosarcomas Induced by MethylcholanthreneScience
ScienceIF0
PREAI
ShareSaveShareSave View more