.png){kind=icon}
返回
An analysis of temporal-difference learning with function approximation
.png){kind=icon}
OA
AI
DOI：10.1109/9.580874.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
We discuss the temporal-difference learning algorithm, as applied to approximating the cost-to-go function of an infinite-horizon discounted Markov chain, The algorithm we analyze updates parameters of a linear function approximator online during a single endless trajectory of an irreducible aperiodic Markov chain with a finite or infinite state space, We present a proof of convergence (with probability one), a characterization of the limit of convergence, and a bound on the resulting approximation error, Furthermore, our analysis is based on a new line of reasoning that provides new intuition about the dynamics of temporal difference learning. In addition to proving new and stronger positive results than those previously available, we identify the significance of online updating and potential hazards associated with the use of nonlinear function approximators, First, we prove that divergence may occur when updates are not based on trajectories of the Markov chain, This bet reconciles positive and negative results that have been discussed in the literature, regarding the soundness of temporal-difference learning, Second, we present an example illustrating the possibility of divergence when temporal-difference learning is used in the presence of a nonlinear function approximator.
Keyword:
dynamic programming
function approximation
Markov chains
neuro-dynamic programming
reinforcement learning
temporal-difference learning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
暂无机构信息
引用论文
.png){kind=icon}
分享收藏A process dissociation framework: Separating automatic from intentional uses of memory过程分离框架: 将自动与有意使用的内存分开
PREAI
分享收藏An Integrated Scheduling and Retransmission Proposal for Firm Real-Time Traffic in IEEE 802.11e
PREAI
分享收藏
查看更多内容