.png){kind=icon}
Return
A Universal Empirical Dynamic Programming Algorithm for Continuous State MDPs
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2019.2907414.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We propose universal randomized function approximation-based empirical value learning (EVL) algorithms for Markov decision processes. The empirical nature comes from each iteration being done empirically from samples available from simulations of the next state. This makes the Bellman operator a random operator. A parametric and a nonparametric method for function approximation using a parametric function space and a reproducing kernel Hilbert space respectively are then combined with EVL. Both function spaces have the universal function approximation property. Basis functions are picked randomly. Convergence analysis is performed using a random operator framework with techniques from the theory of stochastic dominance. Finite time sample complexity bounds are derived for both universal approximate dynamic programming algorithms. Numerical experiments support the versatility and computational tractability of this approach.
Keywords:
Approximation algorithms
Heuristic algorithms
Probabilistic logic
Dynamic programming
Function approximation
Convergence
Complexity theory
Continuous state-space Markov decision processes (MDPs)
dynamic programming (DP)
reinforcement learning (RL)
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
7
Papers:
1.3W
Citations:
6.7W
Organization
Cited Papers
ShareSaveShareSaveShareSaveSimulation-based optimization of Markov decision processes: An empirical process theory approach.png)
AUTOMATICA
AUTOMATICAIF5.9
PREAI
ShareSaveShareSave
ShareSaveShareSave
View more