.png){kind=icon}
.png){kind=icon}
Return
Reinforcement learning based constrained optimal control: An interpretable reward design
.png){kind=icon}
PRE
AI
DOI：10.1016/j.automatica.2026.112995.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This paper presents an interpretable reward design framework for reinforcement-learning-based optimal control problems with state and terminal constraints. The problem is formalized within a standard partially observable Markov decision process framework. The reward function is constructed from four weighted components: a terminal constraint reward, a guidance reward, a penalty for state constraint violations, and a cost reduction incentive reward. A theoretically justified reward design is then presented, which establishes bounds on the weights of the components. This approach guarantees the equivalence between maximizing expected returns within our reward framework and solving the original constrained optimal control problem, while also reducing numerical instability. Acknowledging the importance of prior knowledge in reward design, we sequentially solve two subproblems, using each solution to inform the reward design for the subsequent problem. Subsequently, we integrate reinforcement learning with curriculum learning, utilizing policies derived from simpler subproblems to assist in tackling more complex challenges, thereby facilitating convergence. The framework is evaluated against original and randomly weighted reward designs in a multi-agent particle environment. Experimental results demonstrate that the proposed approach enhances the satisfaction of terminal and state constraints and optimization of control cost. (c) 2026 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
Keywords:
Constrained optimal control
Reinforcement learning
Reward design
Penalty function
Multi-agent
Journal
IF:
5.9
Papers:
1.2W
Citations:
5.2W