.png){kind=icon}
Return
Entropy Regularized Variational Dynamic Programming for Stochastic Optimal Control
.png){kind=icon}
PRE
AI
DOI：10.1109/tac.2026.3741797.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This paper addresses stochastic optimal control, where the state-feedback control policies are probability distributions, with an additional entropy penalty on the control policy. We interpret the cost function as a Kullback-Leibler (KL) divergence between two joint distributions, enabling the use of variational inference techniques. This approach leads to a dynamic programming principle, also defined in terms of KL divergence. In the linear case, we show the entropic penalty leads to Gaussian control policies, whose mean coincides with the linear quadratic regulator (LQR). Furthermore, when the state is not directly measured, we may prove a separation principle along the lines of linear quadratic Gaussian (LQG) control. In the case of nonlinear control systems being linear in the control inputs, with quadratic costs, we utilize our variational framework to approximate the optimal solution with Gaussian distributions. This yields closed-form recursive updates that extend traditional LQR control. We demonstrate the effectiveness of this new method in simulations.
Keywords:
Stochastic optimal control
Maximum entropy
Separation principle
Gaussian graphical models
Journal
IF:
7
Papers:
1.3W
Citations:
6.7W
Organization
No organization information available
Cited Papers
No cited papers available