.png){kind=icon}
返回
Parallel and Flexible Dynamic Programming via the Mini-Batch Bellman Operator
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2023.3270060.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The Bellman operator constitutes the foundation of dynamic programming (DP). An alternative is presented by the Gauss-Seidel operator, whose evaluation, differently from that of the Bellman operator where the states are all processed at once, updates one state at a time while incorporating into the computation the interim results. The provably better convergence rate of DP methods based on the Gauss-Seidel operator comes at the price of an inherent sequentiality, which prevents the exploitation of modern multicore systems. In this work, we propose a new operator for DP, namely, the mini-batch Bellman operator, which aims at realizing the tradeoff between the better convergence rate of the methods based on the Gauss-Seidel operator and the parallelization capability offered by the Bellman operator. After the introduction of the new operator, a theoretical analysis for validating its fundamental properties is conducted. Such properties allow one to successfully deploy the new operator in the main DP schemes, such as value iteration and modified policy iteration. We compare the convergence of the DP algorithm based on the new operator with its earlier counterparts, shedding light on the algorithmic advantages of the new formulation and the impact of the batch-size parameter on the convergence. Finally, an extensive numerical evaluation of the newly introduced operator is conducted. In accordance with the theoretical derivations, the numerical results show the competitive performance of the proposed operator and its superior flexibility, which allows one to adapt the efficiency of its iterations to different structures of MDPs and hardware setups.
Keyword:
Convergence
Costs
Dynamic programming
Optimal control
Cost function
Standards
Process control
Algorithms
dynamic programming (DP)
parallel programming
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
引用论文
.png){kind=icon}
A computational avenue towards understanding and design of zwitterionic anti-biofouling materials
PREAI
分享收藏分享收藏分享收藏
查看更多内容