.png){kind=icon}
返回
F-Discrepancy for Efficient Sampling in Approximate Dynamic Programming
.png){kind=icon}
PRE
AI
DOI：10.1109/TCYB.2015.2453123.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this paper, we address the problem of generating efficient state sample points for the solution of continuous-state finite-horizon Markovian decision problems through approximate dynamic programming. It is known that the selection of sampling points at which the value function is observed is a key factor when such function is approximated by a model based on a finite number of evaluations. A standard approach consists in generating these points through a random or deterministic procedure, aiming at a balanced covering of the state space. Yet, this solution may not be efficient if the state trajectories are not uniformly distributed. Here, we propose to exploit F-discrepancy, a quantity that measures how closely a set of random points represents a probability distribution, and introduce an example of an algorithm based on such concept to automatically select point sets that are efficient with respect to the underlying Markovian process. An error analysis of the approximate solution is provided, showing how the proposed algorithm enables convergence under suitable regularity hypotheses. Then, simulation results are provided concerning an inventory forecasting test problem. The tests confirm in general the important role of F-discrepancy, and show how the proposed algorithm is able to yield better results than uniform sampling, using sets even 50 times smaller.
Keyword:
Approximate dynamic programming (ADP)
F-discrepancy
Markovian decision problem (MDP)
state sampling
value function approximation
期刊
IF:
10.5
论文数:
1.1W
被引数:
5.0W
机构
引用论文
Critical Evaluation and Thermodynamic Optimization of the Li2O-Al2O3 and Li2O-MgO-Al2O3 Systems
.png){kind=icon}
PREAI
分享收藏Acceleration of Reinforcement Learning by Policy Evaluation Using Nonstationary Iterative Method使用非平稳迭代方法通过策略评估加速强化学习
OAAI
OAAI分享收藏A Clustering-Based Graph Laplacian Framework for Value Function Approximation in Reinforcement Learning基于聚类的图拉普拉斯框架，用于强化学习中的值函数逼近
PREAI
分享收藏Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming基于有限近似误差的离散时间迭代自适应动态规划
PREAI
分享收藏分享收藏
查看更多内容