.png){kind=icon}
.png){kind=icon}
Return
Asynchronous Stochastic Approximations With Asymptotically Biased Errors and Deep Multiagent Learning
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2020.3026269.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Asynchronous stochastic approximations (SAs) are an important class of model-free algorithms, tools, and techniques that are popular in multiagent and distributed control scenarios. To counter Bellman's curse of dimensionality, such algorithms are coupled with function approximations. Although the learning/control problem becomes more tractable, function approximations affect stability and convergence. In this article, we present verifiable sufficient conditions for stability and convergence of asynchronous SAs with biased approximation errors. The theory developed herein is used to analyze policy gradient methods and noisy value iteration schemes. Specifically, we analyze the asynchronous approximate counterparts of the policy gradient (A2PG) and value iteration (A2VI) schemes. It is shown that the stability of these algorithms is unaffected by biased approximation errors, provided that they are asymptotically bounded. With respect to convergence (of A2VI and A2PG), a relationship between the limiting set and the approximation errors is established. Finally, experimental results that support the theory are presented.
Keywords:
Approximation algorithms
Approximation error
Stability analysis
Function approximation
Stochastic processes
Asymptotic stability
Convergence
Almost sure boundedness (stability)
asymptotically biased approximation errors
asynchronous stochastic approximations
deep function approximations
deep reinforcement learning
distributed control
multi agent learning
networked control systems
neuro-dynamic programming
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
7
Papers:
1.3W
Citations:
6.7W