.png){kind=icon}
Return
Beyond backpropagate through time: Efficient model-based training through time-splitting
.png){kind=icon}
PRE
AI
DOI：10.1002/int.22928.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Model-based policy gradient (MBPG) has been employed to seek an approximate solution to the optimal control problem. However, there is coupling between adjacent states due to temporal dependencies, making the training time grow linearly with the time horizon. This paper reshapes the training process of MBPG with the time-splitting technique to establish a time-independent algorithm called Training Through Time-Splitting (T3S). First, copy the coupled variables to obtain two independent variables. Meanwhile, an extra variable together with an equivalence constraint is introduced for problem consistency. Then, the transformed problem divides into subproblems with carefully derived loss functions. Subproblems own decoupled variables and shared policy networks, which means they can be optimized concurrently. Guided by the algorithm design, this paper further proposes an asynchronous parallel training scheme to accelerate training efficiency. Numerical simulation shows that the T3S algorithm outperforms the MBPG algorithm by 83.6% in wall-clock time with a trajectory tracking task.
Keywords:
model-based policy gradient
optimal control
parallel training
reinforcement learning
time-splitting
Journal
IF:
3.7
Papers:
3.1K
Citations:
8.1K
Organization
Cited Papers
The effect of solvents on the radiation‐induced polymerization of ethyl and isopropyl vinyl ethers
.png){kind=icon}
PREAI
ShareSaveShareSave
ShareSaveRecurrent Model Predictive Control: Learning an Explicit Recurrent Controller for Nonlinear Systems
OAAI
OAAIShareSaveShareSaveShareSave
View more