.png){kind=icon}
.png){kind=icon}
Return
On average versus discounted reward temporal-difference learning
.png){kind=icon}
OA
AI
DOI：10.1023/A:1017980312899.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We provide an analytical comparison between discounted and average reward temporal-difference (TD) learning with linearly parameterized approximations. We first consider the asymptotic behavior of the two algorithms. We show that as the discount factor approaches 1, the value function produced by discounted TD approaches the differential value function generated by average reward TD. We further argue that if the constant function-which is typically used as one of the basis functions in discounted TD-is appropriately scaled, the transient behaviors of the two algorithms are also similar. Our analysis suggests that the computational advantages of average reward TD that have been observed in some prior empirical work may have been caused by inappropriate basis function scaling rather than fundamental differences in problem formulations or algorithms.
Keywords:
average reward
dynamic programming
function approximation
temporal-difference learning
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
2.9
Papers:
2.6K
Citations:
3.4W
Organization
No organization information available